Pipelining in Python

It is used to chain multiple estimators into one and hence, automate the machine learning process. This is extremely useful as there are often a fixed sequence of steps in processing the data.

Some codes are meant to transform features- normalise numericals or turn text into vectors, or fill up missing data, they are transformers. Other codes are meant to predict variables by fitting an algorithm such as a random forest or support vector machine (SVM), they are estimators.

So, in a pipeline, we first sequentially apply a list of transformers (data modelling) and then a final estimator (ML model).

The transform steps must implement fit() and transform().

The final step, estimator, should implement fit() and predict().

The estimator must implement fit() however, not necessarily implement predict.

In short, pipelines are set up with the fit/transform/predict functionality, so that we can fit the whole pipeline to the training data and transform to the test data without having to do it individually for everything you do.

Implementation of a very basic pipeline-

Related posts:

Virtual reality is the next big thing. The fact is, right now virtual reality is still a commitment. You still have to get to your computer, you have to put on your headset, you have to align your…

Everyone would accept the fact that when it comes to fast food, Pizza is the choice of the lion’s share. Despite the COVID-19 situation that affected the economy the last year, sales for pizzerias…

Social media has been a source for journalism since its inception, good examples of journalism in social media is becoming harder and harder to find. Journalism is an ever-changing part of life. In a…