System-Level Performance Evaluation of Cloud-based Deep Learning Training using AWS Sage-maker

Artificial intelligence systems have become a key enabler for the modern computational thinking technologies, especially for training the deep learning models such as Convolutional Neural Networks (CNNs), which are computationally intensive models. This paper introduces a benchmarking study of CNN training of Image Classification on AWS Sage Maker. The experiment is based on the popular Cats vs. Dogs dataset that has 25,000 labelled images. Two configurations of the SageMaker instance, that is, large and extra-large instances were evaluated for binary image classification problems and to monitor several system-level performance indicators. The metrics analysed are CPU-utilization, memory, training time and total cost of execution. Experimental results lead to meaningful information on the im-pact of cloud resources on deep learning workloads. In particular, compute-optimized in-stances substantially reduced the time for training, although the classification accuracy was comparable for the two configurations. In addition to this, in these instances, improved cost efficiency without sacrificing model performance also. The results of the above experiment can provide standardization to any small-medium businesses or explorers in the field of cog-nitive computing, assisting in decision making on cloud resources to scale and to cost-optimize deep learning applications.