3 steps to transform an ML prototype into a real-world ML app

Training Machine Learning models inside notebooks is just one step to building real-world ML services. And as exciting as it is, it brings no business value unless you deploy and operationalize the model, as a real-world ML app.

In this article, you will learn how to transform an all-in-one Jupyter notebook with data preparation and ML model training, into a fully working batch-scoring system using a serverless ML stack.

For that, we will follow the 3-pipeline design.

Let’s get started!

The starting point

The starting point is this one Jupyter notebook where you:

  1. Loaded data from a CSV file
  2. Engineered features and targets
  3. Trained and validated an ML model.
  4. Generated predictions on the test set.

Let’s turn this notebook into a real-world ML app for batch prediction.

The 3-pipeline architecture

A batch-prediction service ingests raw data and outputs model predictions on a schedule (e.g. every 1 hour).

You can build one using this 3-pipeline architecture

real-world ML app
Batch-scoring service (Image by the author)

Step 1. Build the feature pipeline

Break your initial all-in-one notebook into 2 smaller ones:

  • Notebook A , reads raw data and generates features and targets.
  • Notebook B, takes in features and outputs a trained model.
Feature pipeline (Image by the author)

Notebook A is your feature pipeline. Let’s put it now to work.

You need 2 things:

  1. Automation. You want the feature pipeline to run every hour. You can do this with a GitHub action.
  2. Persistence. You need a place to store the features generated by the script. For that, use a managed Feature Store.

Step 2. Model training pipeline

Remember the 2 sub-notebooks (A and B) you created in step #1?

Notebook B is your model training pipeline.

real-world ML app

Well, almost. You only need to change 2 things…

  1. Read the features from the Feature Store, and not CSV files.
  2. Save the trained model (e.g. pickle) in the model registry, not locally on disk, so the batch-inference pipeline can later use it to generate predictions.

Step 3. Build the batch-prediction pipeline

Build a new notebook that:

  1. Loads the model from the model registry
  2. Loads the most recent feature batch.
  3. Generates model predictions and saves them somewhere (e.g. S3, database…) where downstream services can use them.
real-world ML app

Finally, you create another GitHub action to run the batch-inference pipeline on a schedule.


Wanna learn to design, develop and deploy a real-world ML system yourself?

Join the Real-World ML Tutorial + Community and get LIFETIME ACCESS to

β†’ 3 hours of video lectures 🎬
β†’ Full source code implementation πŸ‘¨β€πŸ’»
β†’ Discord private community, to connect with 100+ students and me πŸ‘¨β€πŸ‘©β€πŸ‘¦