This content is also available at learn.palantir.com ↗ and is presented here for accessibility purposes.
For those following the “Builder” path of the Data Engineering (DATAENG) learning path, this tutorial offers additional practice implementing project and transform best practices you’ve learned up to this point.
You’ve set up a Datasource project and pipeline for your flights alerts data, and in this tutorial you’ll be creating one for passengers. You’ll then move on to the next stage of your pipeline by creating a Transform project and generating a series of outputs that enable specific downstream workflows. Along the way, you’ll also get a feel for how Pipeline Builder processes non-linear data formats (JSON in this case).
⚠️ Course Prerequisites
DATAENG 04: Scheduling Data Pipelines: If you have not completed the previous course in this track, do so now.
🥅 Learning Objectives
Gain additional practice with Pipeline Builder and project structure primitives.
Process non-linear data formats in Pipeline Builder.
Create a Transform project and associated outputs.
💪 Foundry Skills
Use Pipeline Builder’s JSON parser transform.
Generate multiple outputs from a Pipeline Builder transform.
Generate a Data Lineage graph as documentation for the Datasource project segment of your production pipeline.