8. [Builder] Ontology Data Pipelines32. Destructive Backing Dataset Changes Part 2

32 - Destructive Backing Dataset Changes: Part 2

This content is also available at learn.palantir.com ↗ and is presented here for accessibility purposes.

📖 Task Introduction

How do we repair the registered table and index?

When a destructive change to a backing dataset is made involving a column removal or type change, you'll need to remove the property from the mapping and update the Phonograph table registration, which will initiate a reindex of the new table. Put succinctly, the corrective workflow is:

  1. Unmap the deleted column.
  2. Save your Ontology.
  3. Update the table registration.

⚠️ WARNING: "Destructive" schema changes to your backing datasets may cause you to lose edits made through Actions (or otherwise). To avoid data loss, consult the recommended reading at the bottom of this task.

🔨 Task Instructions

  1. In OMA, proceed to the property mapping interface for your flight alert object type.

  2. Delete the rule_name property from the right side of the mapping.

  3. Save your Ontology.

  4. Return to your object type and click into the Datasources menu item on the left-hand panel.

  5. Scroll down to the Phonograph section and click the Update button. This refreshes the Phonograph table with the new backing dataset schema.

    • The sync between the backing dataset and the object storage service (Phonograph) is now repaired.

Although we will not step through an example here, note that if the destructive change is a type change in a backing dataset column (for example, timestamp to date), the process is slightly different. In that case you would:

  1. Update the Base Format for that property from the Properties menu (the image below shows the updating to a date type).

  2. Save your Ontology.

  3. Update your table registration.

There's more to explore around the notion of backing dataset changes, especially when they are on columns that have received edits through the writeback process previously discussed. To learn more, review this documentation entry on breaking changes.