Building scalable end-to-end deep learning pipelines in the cloud

In this talk, I’ll demonstrate serverless deep learning on AWS, focusing on AWS Batch, Fargate, SageMaker, Lambda, and Step Functions for scalable pipelines.

Machine and deep learning are critical for many companies, both internally and externally. A major challenge is the effective training and operationalization of models within a company's framework. Adopting a serverless approach offers a simplified, scalable, cost-effective, and reliable architecture for deep learning deployments. This presentation will explore how to implement such an approach within the AWS ecosystem, highlighting the shift away from traditional concerns like cluster management and scalability towards a more model-centric development process. However, it's important to consider certain limitations and organizational strategies for model training and deployment.

The session will detail the use of AWS services, including AWS Batch, AWS Fargate, Amazon SageMaker, AWS Lambda, and AWS Step Functions to create scalable deep learning pipelines. In doing so, I’ll demonstrate how serverless architecture can revolutionize deep learning projects by focusing on model development and operational efficiency.