Machine Learning models are as good as the input features you feed at training and inference time. And for many real-world applications, like financial trading, these features must be generated and served as fast as possible, so the ML system produces the best predictions possible. Generating and serving features fast is what a real-time feature pipeline does. Can …
Build a real-time feature pipeline in Python, step-by-step Read More »
Imagine you work as an ML engineer at a Fintech startup, and you are tasked to build a new credit card fraud detection system. Imagine you have already defined the problem with relevant stakeholders built a training dataset trained a good model to detect fraud (for example, a binary classifier) inside a notebook created a REST API around your model …
To solve a real-world business problem using a Large Language Model (LLM) you need 3 things: A base LLM, for example, a proprietary model like GPT-3 or a fully open-source one like Dolly. An algorithm that fine-tunes the model parameters using a private dataset, so the model performs the task you want it to perform. A private dataset for the …
Training Machine Learning models inside notebooks is just one step to building real-world ML services. And as exciting as it is, it brings no business value unless you deploy and operationalize the model, as a real-world ML app. In this article, you will learn how to transform an all-in-one Jupyter notebook with data preparation and …
3 steps to transform an ML prototype into a real-world ML app Read More »
Real-time ML is a fascinating topic, which I wanna go deeper into in the following weeks and months. Because of this, I started creating a sequence of small projects where I build real-time products, beginning with a real-time feature engineering pipeline. Real-time feature engineering A real-time feature pipeline is a program that ingests real-time raw data (e.g. …
No more reading blog posts. It is time to forecast for real. 😎 Here is a project you can build. “Let’s predict taxi demand in NYC” Let’s create a predictive model to forecast the number of taxi rides that will happen in Manhattan (New York City) per hour (e.g. tomorrow between 5 PM and 6 …
Reading blog posts about multi-billion-parameter Language Models is very cool. However, building real-world NLP products from these models is where the real business value is. And this is what companies look for in the job market. So, here is a PRO project you can build. “An app that recommends what ML paper to read” Imagine …
Let’s build a Machine Learning service to predict the Air Quality Index (AQI) in your city in the next 3 days, using a 100% serverless stack. You will learn a lot, AND you will build something useful for society. Win-win These are steps to build this ↓ Step 1 – Feature generation script 1 → …
After one month of working on my first real-world ML project, my manager told me: “Pau, we need to stop the project. This is not going anywhere.” I felt like a failure. However, with time I understood the 3 crucial mistakes I made, and transformed them into 3 learnings. 💎 Tip 1: Estimate the Return …
3 things you need to know before your first real-world ML project Read More »
Every aspiring data scientist I talk to thinks their job starts when someone else gives them: a dataset, and a clearly defined metric to optimize for, e.g. accuracy But it doesn’t. It starts with a business problem you need to understand, frame, and solve. This is the key data science skill that separates senior from …
