15 - Configure a Connecting Build Schedule

📖 Task Introduction

A dataset should only belong to a single schedule.

As you connect the datasource and transform stages of your pipeline, we’ll want to make sure we avoid building datasets unnecessarily. The three clean outputs from your two datasource projects will serve as inputs to your transform stage pipeline, but as targets of the schedule you previously configured, they don’t need to be built again in the new schedule. Instead, they’ll serve as inputs in a connecting build schedule. Recall the logic of a connecting build attempts to build everything between the inputs and the targets, excluding the inputs but including the targets.

🔨 Task Instructions

1. Open the Data Lineage graph you saved in the previous task.

2. Click the Manage schedules button on the right side of your screen.

3. Your other two schedules appear in the Scheduler UI. Mouse over them alternately to see that the “clean” datasets on your graph are already the targets of a schedule.

4. Create a new schedule and name it *yourName Alert Metrics Schedule (e.g., Jmeier Alert Metrics Schedule). Add a description: Build schedule for Transform Project: Alert Metrics.

5. Switch to a connecting build and then define “what” will build “when”:

   • Set all of your country and priority datasets as the targets of your schedule.
   • Set the three “clean” datasets as input triggers.

6. Change the Build scope to Project Scoped.

7. Open the Advanced options and set the schedule to abort on failure but retry three times with a minute in between retries.

8. Save your schedule.