
Engineering MLOps for scalable machine learning requires a structured approach to ensure seamless integration of machine learning into production environments.
Model serving is a crucial component of MLOps, allowing models to be deployed in real-time and handling a large volume of requests without impacting performance.
To achieve this, model serving platforms like TensorFlow Serving and AWS SageMaker support model deployment and management, making it easier to integrate models into production environments.
A key challenge in MLOps is model monitoring, which involves tracking model performance and detecting drift in real-time.
Related reading: Financial Data Enrichment
Why Is Necessary?
Machine learning models aren't built once and forgotten; they require continuous training so that they improve over time. MLOps provides the ongoing training and constant monitoring needed to ensure ML models operate successfully.
MLOps documents reliable processes and governance strategies to prevent problems, reduce development time and create better models. It uses repeatable processes in the same way businesses use workflows for organization and consistency.
MLOps automation ensures time isn't wasted on tasks that are repeated each time new models are built. This automation is essential for efficient ML development and pipeline management.
Here are the key benefits of MLOps automation:
- Speed and efficiency: Automating repetitive tasks reduces development time and cuts down on human error.
- Scalability: MLOps uses technology such as containerized software and data pipelines to handle large amounts of data efficiently.
- Reliability: MLOps model testing and validation fix problems in the development phase, increasing reliability early on.
Course Overview
The Machine Learning in Production course covers how to conceptualize integrated systems that continuously operate in production as well as solve common challenges unique to the production environment. This involves designing ML production systems end-to-end, including project scoping, data needs, modeling strategies, and deployment requirements.
You'll learn to establish a model baseline, address concept drift, and prototype how to develop, deploy, and continuously improve a productionized ML application. This process includes building data pipelines by gathering, cleaning, and validating datasets, and implementing feature engineering, transformation, and selection with TensorFlow Extended.
The course will also cover topics such as model monitoring and operations, including the importance of monitoring, troubleshooting pipelines, and detecting data drifts in the underlying input data. You'll learn to use analytics to address model fairness, explainability issues, and mitigate bottlenecks, and to deliver deployment pipelines for model serving that require different infrastructures.
Here's an interesting read: Feature Engineering Course
Machine Learning in Production Course Overview
The Machine Learning in Production Course is designed to help you build integrated systems that operate continuously in production, solving common challenges unique to this environment. This course covers how to design an ML production system end-to-end, from project scoping to deployment requirements.
You'll learn how to establish a model baseline, address concept drift, and develop, deploy, and continuously improve a productionized ML application. The course also focuses on building data pipelines by gathering, cleaning, and validating datasets, and implementing feature engineering, transformation, and selection with TensorFlow Extended.
One of the key aspects of the course is MLOps, or machine learning operations. This involves managing modeling resources, addressing model fairness and explainability issues, and mitigating bottlenecks. You'll also learn how to deliver deployment pipelines for model serving that require different infrastructures.
The course is divided into four modules: Introduction to MLOps, MLOps Development, MLOps Deployment, and Model Monitoring and Operations. In each module, you'll gain practical skills and knowledge to help you deploy machine learning models in production.
Here are some of the key topics you'll cover in the course:
- Machine learning operations (MLOps)
- MLOps development, including build, train, and evaluate machine learning models
- MLOps deployment, including model packaging, inference, and deployment strategies
- Model monitoring and operations, including troubleshooting, monitoring, and human-in-the-loop reviews
By the end of this course, you'll have a solid understanding of how to design, develop, and deploy machine learning models in production, and you'll be equipped with the skills and knowledge to tackle common challenges in this environment.
On a similar theme: Mlops Courses
Learner Reviews
The learner reviews for this course are overwhelmingly positive, with 84.45% of reviewers giving it 5 stars. The course is well-paced and introduces you to the basics of MLOps in a clear and concise manner.
Students have praised the course for its engaging teaching style, with one reviewer noting that they "actually are interested in what [Andrew Ng] is telling you and learn more." Another reviewer appreciated the course's focus on practical points that can be applied in real-world projects.
Here are the details of the learner reviews:
- 5 stars: 84.45%
- 4 stars: 12.79%
- 3 stars: 1.87%
- 2 stars: 0.58%
- 1 star: 0.29%
The majority of reviewers have praised the course for its engaging teaching style and practical application, making it a great choice for those looking to learn about MLOps.
Who Should Attend?
If you're looking to productionize machine learning models in the AWS Cloud, there are specific roles that should attend an MLOps engineering course. DevOps engineers are a key group to consider, as they play a crucial role in ensuring the smooth operation of machine learning models in production.
ML engineers are also a natural fit for such a course, given their expertise in developing and deploying machine learning models.
Developers and operations teams with responsibility for operationalizing ML models will also find the course highly relevant.
Course Outline
The Machine Learning in Production course is divided into 4 modules that cover the key aspects of MLOps. Module 1 focuses on Introduction to MLOps, covering topics such as machine learning operations, goals of MLOps, and communication.
In Module 2, MLOps Development is explored in-depth, including intro to build, train, and evaluate machine learning models, MLOps security, and automating with Apache Airflow and Kubernetes integration for MLOps.
Module 3 delves into MLOps Deployment, discussing introduction to deployment operations, model packaging, inference, and deployment strategies.
Module 4, Model Monitoring and Operations, covers the importance of monitoring, monitoring by design, and troubleshooting pipelines, as well as using Amazon SageMaker Model Monitor and human-in-the-loop reviews of model results.
Here's a breakdown of the key topics covered in each module:
MLOps Concepts
A data pipeline is a series of transformations that we apply to data between its source and destination, and it's a core concept of data engineering.
In MLOps, we can think of ML training as a data transformation, which is why it's natural to include the specific ML steps in the data pipeline itself, turning it into an ML pipeline.
Most ML models will need two versions of the pipeline: one for training and one for serving, because the data formats and way to access them are very different between each moment.
Here are some key differences between the training and serving pipelines:
- Training pipeline runs over batch files with all features, while the serving pipeline runs online and receives only part of the features in the requests.
- Training pipeline usually calculates statistics like averages and standard deviations for normalization, which can be ensured to be consistent using transformation frameworks like TensorFlow Transform.
Reliable Batch Operations
Dataiku automation nodes are dedicated production servers that execute scenarios for everyday production tasks like updating data, refreshing pipelines, and MLOps monitoring.
These nodes enable data scientists and ML engineers to deploy API services created in Dataiku on platforms beyond Dataiku API nodes, including AWS Sagemaker, Azure Machine Learning, and Google Vertex.
With these dedicated execution servers, multiple AI projects can run smoothly in a reliable and isolated production environment, ensuring seamless integration with various external platforms.
This capability extends the reach and flexibility of API deployment, providing a more efficient way to manage and deploy AI models in production.
Model Retraining and Comparisons
Model retraining is a crucial step in keeping your machine learning (ML) models accurate and up-to-date. Production models need to be updated periodically based on newer data or shifting conditions.
Teams can either manually refactor a model or set up automated retraining based on a schedule or specific triggers, such as significant data or performance drift.
Automated retraining saves time and ensures consistency in the model updates. In Dataiku, data scientists and ML engineers can set up automated retraining based on a schedule or specific triggers.
Comprehensive model comparisons allow teams to perform champion/challenger analysis on candidate models to make informed decisions about the best model to deploy in production.
A different take: Automated Feature Engineering
Model Validation
Model validation is a crucial step in the MLOps life cycle. It's not enough to just test a model's performance on a single metric for the entire validation set. Instead, we need to validate models individually for relevant segments of the data, known as slices.
For example, if gender is a relevant feature of a model, we should track separate metrics for male, female, and other genders. This is because a model could have equity issues or under-perform in important segments if it's not validated properly.
Model validation tests need to be statistical in nature, rather than having a binary pass/fail status. This means we need to decide on the right metrics to track and the threshold of their acceptable values, usually empirically, and often by comparison with previous models or benchmarks.
Here are some key metrics to track during model validation:
By tracking these metrics and validating our models for relevant slices, we can ensure that our models are fair, reliable, and perform well in all segments of the data. This is especially important when it comes to models that have a direct impact on people's lives, such as medical diagnosis or credit scoring models.
ML vs. Engineering
ML vs. Engineering is a crucial distinction in the world of MLOps. MLOps encompasses all processes in the lifecycle of an ML model, including predevelopment data aggregation, data preparation, and post-deployment upkeep and retraining.
In contrast, ML engineering is focused on the stages of developing and testing a model for production, similar to what software engineers do. This means ML engineers handle the training, deployment, and testing stages of the MLOps lifecycle.
MLOps teams often designate ML engineers to handle these tasks, and these professionals possess the same skills as typical software developers. Others on the operations team may have data analytics skills and perform predevelopment tasks related to data.
Here's a comparison of MLOps and ML engineering:
MLOps teams perform continual maintenance and adapt to changing end-user needs, which might call for retraining the model with new data. This is a key difference between MLOps and ML engineering.
MLOps Tools and Practices
Using an application programming interface from an existing AI service can simplify or expedite MLOps in various ways. APIs can be used to retrieve data from external data sources and for automated testing of ML models.
MLOps professionals often run parallel model development processes so that, if one model fails, they still have others in progress.
Pretrained models are used to show proof of concept, while generalized algorithms showing some successes are further trained for a specific task.
Publicly available data sources are used to bridge gaps in model training data, provide new data, and prevent model drift.
Here are some key MLOps best practices summarized:
- An API from an existing AI service can simplify or expedite MLOps
- Run parallel model development processes
- Use pretrained models for proof of concept
- Further train generalized algorithms for specific tasks
- Use publicly available data sources to bridge gaps
CI/CD with APIs
CI/CD with APIs is a game-changer for MLOps teams. It allows IT and ML engineers to programmatically perform Dataiku operations from external orchestration systems, incorporating MLOps tasks into existing data workflows.
Robust APIs enable this seamless integration, making it possible to bring DevOps tools into the MLOps world. Dataiku, as an open MLOps platform, integrates with popular DevOps tools like Jenkins, GitLabCI, Travis CI, or Azure Pipelines.
CI/CD pipelines can be automated using APIs, ensuring that MLOps tasks are executed consistently and reliably. This is especially useful for teams that want to streamline their MLOps workflows and reduce manual errors.
By leveraging APIs, teams can create a more efficient and scalable MLOps process, allowing them to focus on more strategic tasks.
Model Stress Tests and Documentation
Data preparation and data discovery are crucial in the MLOps life cycle, and successful MLOps use cases prioritize these aspects.
Stress tests simulating real-world data quality issues can reduce risk by assessing model robustness and behavior under adverse conditions before deployment.
Automatically-generated, customizable documentation for models and pipelines is essential for retaining critical project context for reproducibility and compliance purposes, while also reducing the burden of manual documentation.
Unified Monitoring in Dataiku acts as a central cockpit for all MLOps activity in one place, aggregating multiple types of monitoring and providing a one-stop solution for tracking the health of AI models across diverse origins.
Dataiku's automated documentation feature helps teams retain critical project context for reproducibility and compliance purposes.
Dataiku for Data Architects
Dataiku for Data Architects is a powerful tool that enables data architects to streamline their workflows. From AI orchestration to smooth operationalization, Dataiku helps data architects achieve their goals.
Dataiku's capabilities include AI orchestration, which allows data architects to manage and automate complex AI workflows. This saves time and reduces errors.
Dataiku's smooth operationalization feature ensures that models are deployed and maintained with ease, making it a valuable asset for data architects.
Frequently Asked Questions
What do MLOps engineers do?
MLOps engineers build and maintain platforms that standardize, automate, and monitor machine learning model development and deployment. They enable ML engineers to focus on model development, making the process more efficient and scalable.
What is the difference between ML engineer and MLOps engineer?
ML Engineers focus on building and training models, while MLOps Engineers deploy and maintain them in production environments, ensuring reliability and scalability. In essence, ML Engineers create, and MLOps Engineers operate, model-driven applications.
Are MLOps engineers in demand?
Yes, MLOps engineers are in high demand due to the increasing reliance on data-driven solutions in businesses. Their expertise in deploying and managing machine learning models is crucial for organizations seeking to leverage AI effectively.
Featured Images: pexels.com





