This content is also available at learn.palantir.com ↗ and is presented here for accessibility purposes.
Foundry applications and project structures that support data pipelines provide ample opportunities for your to let your current and future team know the relevant facts about your data transformations. Having preprocessed your data, it’s time to clean it and prepare it for use downstream. This means airtight transform logic and documenting the scope every step of the way.
In this tutorial, you’ll create “clean” outputs for your project to be consumed by downstream pipelines and use cases. In doing so, you’ll get additional practice within a recommended pipeline project structure.
⚠️ Course Prerequisites
DATAENG 02 (Builder): If you have not completed the previous course in this track, do so now.
🥅 Learning Objectives
Distinguish between preprocessing and cleaning steps.
Document the datasource stage of your pipeline.
Gain additional practice transforming data in Pipeline Builder.
💪 Foundry Skills
Generate multiple outputs from a Pipeline Builder transform.
Implement branching and pipeline documentation best practices.
Generate a Data Lineage graph as documentation for the Datasource project segment of your production pipeline.