14 - Document Your Pipeline with a Data Lineage Graph

📖 Task Introduction

Over the next three tasks, you’ll now add a connecting build schedule for the new elements of your pipeline, a README file in your repository, and a Data Lineage graph resource you can place at the root of the project (Datasource Project: Passengers) as a form of documentation just as you did for Datasource Project: Flight Alerts.

These are steps you completed in the previous tutorial on Scheduling Data Pipelines, so the instructions here are more brief to provide you an opportunity to practice what you learned there.

Be sure your new clean outputs have been built on the Master branch before proceeding.

🔨 Task Instructions

1. From the Master branch of your passengers_logic repository, ctrl+click the Explore lineage button in the top right of your application screen.
2. In the top left of your screen, click the Align icon to enable a snap-to-grid behavior.
3. Highlight the two *_raw datasets at the left of your graph and Expand nodes...
4. Click the << button and Add 2 nodes.
5. Change the Node color options to Folder to more clearly identify the pipeline stages.
6. Save your Data Lineage graph as Passengers Pipeline in your /Datasource Project: Passengers folder.