5 - Document the passengers pipeline

This content is also available at learn.palantir.com ↗ and is presented here for accessibility purposes.

📖 Task Introduction

Using the same pattern from previous tutorials, add a Notepad document as a “README” file in your .../Datasource Project: Passengers/documentation/ folder.

🔨 Task Instructions

  1. Open your .../Datasource Project: Passengers/documentation/ folder.

  2. Add a new Notepad document called Passengers Datasource Project Pipeline Documentation.

  3. Add the following text to the document and use the options in the upper-left of the application screen to adjust heading and sub-heading sizes, add bullets and numbering, and add any other discretionary styling desired:

    **Passengers Pipeline**
    
    Ownership
        * **Project team: Aviation Data Development**
        * **Project owner: [yourName@yourOrganization.com](mailto:yourName@yourOrganization.com)**
    
    Overview
    This pipeline takes raw data ingested from the Passengers datasource and:
        1. Parses raw and JSONs into a dataset
        2. Normalizes column names and data types across all raw datasets
    
  4. Open your passengers_datasource_clean Pipeline Builder artifact.

  5. Ctrl+click the View Lineage button at the top of the Pipeline outputs window in Pipeline Builder.

  6. Expand all upstream ancestor nodes.

  7. Click shift and drag a selection box around the left-most “raw” nodes on your graph.

  8. Right click on the selection and choose Group nodes....

  9. Name your group "Simulated External Data Source."

  10. Save your graph as Passengers Pipeline in your .../Datasource Project: Passengers/documentation/ folder.