This post will show how MLFlow can be used for managing your machine learning life cycle. I will walk through the major components of MLFLow and how they can be used. I have installed MLFlow via pip and am using the SQLite backend in order to perform the model registration.

IRIS classification with MLFlow


This post is in continuation of my previous post where I showed how an Airflow DAG can be developed to schedule multiple activities in a typical data pipeline along with running Spark jobs. This post aims to serve as an independent guide on how you would go about writing a…


In this series, you will learn to use Airflow and Spark to create a simple word counter application for Wikipedia articles which can also be used to solve real-world ‘big-data’ problems such as:

1. Ingesting data from multiple sources to a staging area, and

2. Performing scheduled ETL (Extract, Transform…

Anurag Chatterjee

An experienced professional who likes to learn new technologies and use them to solve real-world problems. Exploring data engineering and ML technologies.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store