Written in Livemark
(2022-06-18 05:35)
(2022-06-18 05:35)
What is the Data Pipeline?
The Data Pipeline is an approach developed by the School of Data network to work with data from beginning to end. Aside from being a flexible guide for doing data-driven projects, it is also a wonderful tool for teaching how to work with data to beginners and experienced data practitioners alike as it divides the process into understandable and manageable steps. It is simple enough for beginners to grasp yet open enough for experienced practitioners to play around with.
The Data Pipeline is an ever-improving, dynamic tool that has been utilized, extended, and improved upon by countless data practioners over the years. Its current steps are:
- Define
- Find
- Get
- Verify
- Clean
- Analyze
- Present
Although the Pipeline is a template, it should not be thought of as a rigid one. It is designed to be easily adaptable to different contexts and you shouldn't be afraid to experiment with it.
It is also important to note that the process of using the Data Pipeline is not always linear—you may need to do the find and get steps multiple times or repeat the verify and clean steps if you find mistakes in the outcome of your analysis.
The Data Pipeline (2022)