16 - Add a Schedule to Your Pipeline

📖 Task Introduction

In this task, you’ll define a new connecting build schedule — the “what,” “when,” and “how” the passengers datasets in your pipeline will build together. The instructions here are abridged, reflecting the fact you’ve already completed one schedule with your flight alerts project pipeline.

🔨 Task Instructions

1. Open your Passengers Pipeline Data Lineage graph and click the Manage schedules button on the right side of your screen.

2. Create a new schedule and name it yourName Passengers Schedule (e.g., Jmeier Passenger Schedule). Add a description: Build schedule for Datasource Project: Passengers.

3. Switch to a connecting build and then define “what” will build:

   • Set your passengers_clean and passenger_flight_alerts_clean as the targets of your schedule.
   • Set the raw/passengers_json_raw and raw/passenger_flight_alerts_csv_raw datasets as input triggers

4. Now define “when” to build. Configure your schedule to execute when both your trigger inputs have new data OR every Monday at 8:00 AM Central European Time (GMT +2). Use the clickable image below as a guide for implementing this logic (use the Add: Time button to add a temporal component).

   

5. Change the Build scope to Project Scoped.

6. Open the Advanced options and set the schedule to abort on failure but retry three times with a minute in between retries.

7. Save your schedule.