Use OData for SAP SLT syncs

This guide explains how to configure SAP SLT replication syncs to use the OData API, and how to migrate existing syncs from the legacy ODP API to OData. It covers batch and streaming syncs on both the SAP SLT source and the SAP (Custom source) connector when used for SLT replication.

Background

SAP SLT syncs read change data from the Operational Delta Queue (ODQ) on the SLT Replication Server. Foundry can reach the ODQ through one of two APIs on the Palantir Foundry Connector 2.0 for SAP Applications ("Connector"):

• ODP: The legacy API, which uses SAP's ODP Data Replication API over RFC (ODP-RFC).
• OData: The replacement API, which uses SAP's ODP-based data extraction via OData ↗. The PALODATA component of the Connector provides this API.

Migration criteria

This guide applies to SLT replication syncs only:

• Batch and streaming syncs on an SAP SLT source that use the SLT Replication Server object type.
• Batch syncs on an SAP (Custom source) source that use the SLT SAP object type.
• Streaming syncs on an SAP (Custom source) source.

Local table syncs on an SAP SLT source and other SAP object types (such as ERP tables, InfoProviders, or extractors) do not use the API selection described in this guide. Contact Palantir Support for guidance on object types that are not listed above.

Existing syncs are migrated in place. The sync keeps its source and its output dataset or stream.

Differences between the OData and ODP APIs

Review these differences before you migrate a sync:

• One subscription per table and SAP user: The OData API allows one active subscription per table for the SAP user that the source authenticates as. Two syncs that use the same source credentials cannot both ingest the same table through OData, even with different filters or schedules. With the ODP API, multiple syncs can subscribe to the same table independently.
• No remote context or parallel paging: OData syncs do not use the Remote context initial-load bypass or Parallel paging threads. Pages are retrieved sequentially, which can increase the duration of large initial loads.
• A new initial load is required: An OData sync cannot continue from an ODP delta pointer. Migrating a sync from ODP to OData requires one full initial load through OData, as described in the procedures below.

Compatibility and prerequisites

The OData API has requirements in the SAP system and in Foundry. All four requirement groups in the table must be met before you configure a sync to use OData.

SAP prerequisites

The following SAP-side requirements apply to all four source and sync types. They are SAP add-on versions, and they are separate from the Foundry connector requirements in the table above.

• SAP_BASIS 7.50 SP09 or later on the SLT Replication Server where the Connector is installed.
• Connector version 2.34 (SP34) or later. The PALANTIR and PALCONN components must be on SP34 or later. We recommend upgrading to SP39 Fix Pack 4 or later before you migrate, to pick up the latest OData functionality.
• The PALODATA component installed on the SLT system. Learn more about installing the Connector for SLT.
• The SAP Notes listed in Use ODP via OData, read and implemented where applicable.
• The Connector's SICF services active, as described in Activate SICF services.
• An existing SLT configuration for the source system. Its queue alias is the context used by the sync.

Check SAP component versions

Ask your SAP Basis team to confirm the installed component versions on the SLT Replication Server:

1. Log in to the SLT system with SAP GUI.
2. From the menu, select System > Status.
3. Under SAP System data, open Component information.
4. Confirm the following rows:
   • SAP_BASIS: Release 750 with support package SAPK-75009INSAPBASIS or later, or a higher release.
   • PALANTIR and PALCONN: Support package level SP34 or later. SP39 Fix Pack 4 or later is recommended.
   • PALODATA: Present in the list.

Foundry also checks the Connector version. In the API version card, the OData option displays one of the following messages if Foundry determines that the add-on cannot support OData:

• This source's add-on is older than version 2.34 (SP34), which OData requires. Upgrade the Connector before you select OData.
• We couldn't confirm this source's add-on is version 2.34 (SP34) or newer, so OData might not work. Confirm the component versions in SAP as described above.

Check Foundry support

Foundry connector and user interface updates are delivered by Palantir and do not have a version number that you need to manage. To confirm that your Foundry environment supports OData for a given sync type, open an existing sync of that type or start creating a new one. Then check for the controls listed in the compatibility table.

If the API version card does not appear for an SLT sync, OData selection is not yet available for that sync type on your enrollment. Do not attempt to configure OData by other means. Contact Palantir Support and include the source type, sync type, and SAP component versions.

Batch syncs

Batch SLT syncs that use OData follow the standard Foundry transaction types:

• APPEND: Incremental sync. Each run requests the changes recorded in the ODQ since the previous successful run and appends them to the dataset.
• SNAPSHOT: Each run replaces the contents of the dataset with the run's output.

Create a new batch sync using OData

1. Confirm that the SAP prerequisites are met and that the API version card is available for your source and sync type. See Check Foundry support.
2. Create the sync:
   • SAP SLT source: From the source overview page, select + New next to batch sync and choose the SLT Replication Server object type.
   • SAP (Custom source): From the source overview page, select + New next to batch sync and choose the SLT SAP object type.
3. Under API version, select OData. On an SAP SLT source, new syncs default to OData unless Foundry determines that the add-on is older than 2.34 (SP34). On an SAP (Custom source) sync, new syncs default to ODP, so you must select OData.
4. Configure the table and context:
   • SAP SLT source: Enter the Table name. The context comes from the source's Remote context setting.
   • SAP (Custom source): Select the Context (the SLT queue alias) and enter the Object name.
5. Configure incremental settings:
   • For incremental ingestion, set Transaction type to APPEND. On an SAP (Custom source) sync, set Incremental field to pointer. OData syncs do not support other incremental fields.
   • To read the full table on every run without a subscription pointer, set Transaction type to SNAPSHOT.
   • Optionally, set Max rows per sync (SAP SLT source) or maxRowsPerSync (SAP (Custom source) Advanced tab) to split a large initial load across several runs. See Initial loads split across multiple runs.
6. Save the sync. Do not add a schedule yet.
7. If your SAP landscape requires approval for new SLT initial loads or new ODQ subscriptions, obtain it from your SAP Basis team before you continue. This approval is an organizational process, not a Connector requirement, and many landscapes do not require it.
8. Run the sync manually. The first run of an APPEND sync performs the initial load of the table.
9. Validate the initial load. Compare the dataset row count with the number of rows in the SAP table, accounting for any filters or dropped columns. If you split the initial load across several runs, continue running the sync until the initial load is complete before you compare counts.
10. Add a schedule and confirm that later runs are incremental. Each APPEND transaction contains only changed records. Subsequent incremental runs do not usually require any further approval from SAP Basis.

Migrate an existing batch sync from ODP to OData

Existing batch syncs keep their source, output dataset, and transaction type when migrated.

• SNAPSHOT syncs: Confirm the SAP prerequisites and Foundry support. For SAP (Custom source) syncs, also meet the configuration requirements. Then open the sync configuration, select OData under API version, and save. No incremental-state reset is required.
• APPEND syncs: Follow Migrate an incremental batch sync, which reloads the table once through OData and then resumes incremental ingestion.

Migrate an incremental batch sync

Use this procedure for existing batch syncs whose Transaction type is APPEND. It performs one full reload of the table through OData and then resumes incremental ingestion.

1. Plan the migration. Complete the following checks:
   • Confirm that the SAP prerequisites are met and that the API version card and Reset incremental state optional property are available for the sync. See Check Foundry support.
   • Agree on a migration window with your SAP Basis team. The reload consumes SLT and ODQ resources in the same way as an initial load. Review resource sizing.
   • Check the SAP (Custom source) configuration requirements if you are migrating an SAP (Custom source) sync.
2. Prevent overlapping runs. Pause every schedule that includes the sync, and wait for any in-progress run to finish.
3. Select OData. Open the sync configuration and, under API version, select OData.
4. Enable Reset incremental state. Add the Reset incremental state optional property and turn it on. Foundry sets Transaction type to SNAPSHOT and locks the incremental settings. Confirm the following before you save:
   • Transaction type is SNAPSHOT.
   • The incremental settings are still present. On an SAP (Custom source) sync, Incremental field remains pointer. Do not remove the incremental settings; the sync needs them to keep its subscription after the reload.
5. Save the sync configuration.
6. Complete SAP-side subscription checks, if needed. OData allows one active subscription per table and SAP user. If another OData subscription already exists for the same table and SAP user, the run fails with an error stating that only one active incremental sync is permitted per table per technical user. See Subscription conflicts for how to resolve this before you run the sync.
7. Trigger the reload. If your SAP landscape requires approval for new initial loads, obtain it first. Then run the sync manually. The run performs a full initial load through OData and replaces the dataset contents.
8. Verify the initial load. Confirm that the run succeeded and that the dataset row count matches the SAP table, accounting for any filters or dropped columns. If you configured Max rows per sync, see Initial loads split across multiple runs before you compare counts.
9. Disable Reset incremental state. Turn off Reset incremental state. Foundry sets Transaction type back to APPEND and unlocks the incremental settings. Confirm that Transaction type is APPEND, then save.
10. Verify incremental ingestion. Run the sync manually again and confirm that the new transaction is an APPEND containing only changes made since the reload. An empty transaction is expected if no records changed.
11. Resume schedules.
12. Clean up the old ODP subscription. After you have verified the migrated sync, ask your SAP Basis team to review the legacy ODP subscription for the table in transaction ODQMON. The Connector no longer reads from that subscription. Terminating an unused subscription removes only queue and subscription data; it does not delete data from the SAP business table. Do not terminate subscriptions that belong to other consumers. Learn more about housekeeping for SLT subscriptions.

Initial loads split across multiple runs

This section applies to incremental syncs, including the incremental batch migration procedure, whose steps are referenced below. If an incremental sync uses Max rows per sync (or maxRowsPerSync), the initial load is delivered across several runs:

1. The reset run (step 7) writes the first portion of the table as a SNAPSHOT.
2. Turn off Reset incremental state after the first successful run, as described in step 9. Leaving it on restarts the initial load on every run, so the load never completes.
3. Run the sync manually as APPEND until the remaining portions have been delivered.
4. Validate completeness (step 8) only after the initial load has finished, then continue with step 10.

SAP (Custom source) configuration requirements

SAP (Custom source) batch syncs must meet the following requirements before you select OData. If a requirement is not met, the run fails with an error that names the setting.

• SLT object type only: OData is available only for syncs that use the SLT SAP object type.
• Context required: The sync must specify a Context.
• Incremental field: If set, the Incremental field must be pointer.
• JSON page format: If the source uses TSV serialization, add useKernelJsonSerialization: true to the sync in the Advanced tab. This changes how pages are encoded for transfer, not which rows are extracted.
• Boolean flags: The timestamp, resourceCheck, and debug settings must be true, false, ON, or OFF.
• Unsupported settings: Remove the following settings from the sync YAML, even if their value is empty: filter, incrementalType, initialIncrementalValue, twinTable, twinMap, twinJoinConfigItems, incrementalSubTypes, incrementalComparator, trace, and continuousResourceCheck. Settings that apply only to other object types, such as BEx, function, or TSV-specific settings, must also be removed.

If a sync uses filter, removing it changes which rows are extracted. Do not migrate that sync until you have an equivalent filter that the OData API supports. Contact Palantir Support for help expressing the filter.

The Parallel paging threads setting is ignored for OData syncs.

Streaming syncs

Streaming syncs do not use the SNAPSHOT and APPEND transaction types. A streaming sync holds its own SAP subscription, identified by the stream, and polls it for changes while the stream is running. Use Force snapshot to control how a stream starts:

• Force snapshot: Requests a new full initial load of the table on the first request after the stream starts, then polls for changes. The setting applies each time the stream starts, including automatic restarts, but not to later polls of a running stream.

Keep Reset queue registration turned off for streaming syncs. The setting only resets the stream's own OData subscription, which Force snapshot already does. It cannot take over a subscription held by another sync. While a stream runs, the setting is sent with every polling request rather than once, so the stream can repeatedly re-initialize and re-send the full table.

Streaming configuration reference

The following settings determine how a streaming sync connects to SAP. The SLT context is configured on the source, and all other settings are configured in the streaming sync editor.

SAP (Custom source) streaming syncs require a source with an explicit SLT connection type. Open the source configuration and confirm that the YAML contains a connectionType block like the following, where <context> is the SLT queue alias:

Copied!
1
2
3
4
5
6
7
type: magritte-sap-source
url: https://<host>:<port>/sap/palantir
usernamePassword: <username>:{{password}}
connectionType:
  type: slt
  slt:
    context: <context>

If the block is missing, add it as described in Enable streaming syncs before you create or migrate a stream. Every streaming sync on the source uses this context. No other source setup is required for OData.

Create a new streaming sync using OData

1. Confirm that the SAP prerequisites are met and that the API version card is available in the streaming sync editor. See Check Foundry support.
2. From the source overview page, create a new streaming sync and enter the Table name.
3. Under API version, select OData. New streams default to OData unless Foundry determines that the add-on is older than 2.34 (SP34).
4. Leave Force snapshot and Reset queue registration off. A new subscription performs an initial load automatically. If another sync already holds the OData subscription for the table and SAP user, resolve the subscription conflict before you start the stream.
5. Choose the output stream location and create the streaming sync.
6. If your SAP landscape requires approval for new SLT initial loads or new ODQ subscriptions, obtain it before you start the stream.
7. Start the stream. The stream first receives the table's initial load, then continues with changes.
8. Validate the stream. Confirm that the number of distinct keys in the stream matches the SAP table once the initial load has been received, then make a test change in SAP and confirm that it arrives in the stream.

Migrate an existing streaming sync from ODP to OData

Use this procedure to switch an existing SAP SLT or SAP (Custom source) streaming sync to OData while keeping its source and output stream. Migrating a stream re-sends the full table into the existing stream, so plan for the replay before you start.

Migration stages

Move the stream through the following stages during migration. Use this table to confirm the configuration at each stage.

The stream does not carry an ODP checkpoint over to OData. On its first start with Force snapshot turned on, the stream requests a new initial load through its OData subscription. Later starts with Force snapshot off resume from the OData delta pointer that SAP holds for the stream.

Procedure

1. Plan the migration. Complete the following checks:
   • Confirm that the SAP prerequisites are met and that the API version card is available in the streaming sync editor.
   • For SAP (Custom source) streams, confirm the source's SLT connection type.
   • Agree on a migration window with your SAP Basis team.
   • Identify downstream pipelines that consume the stream. After migration, every current row of the table is emitted again. Consumers must tolerate duplicate records, for example by removing duplicates based on the table's primary key and the /PALANTIR/TIMESTAMP column that streaming syncs always add.
   • The initial load reflects the current state of the table. Records deleted in SAP while the stream is stopped are not emitted as deletions. If downstream consumers depend on delete records, plan to reconcile them after the initial load.
2. Stop the stream. Stop the streaming sync and pause any schedule that restarts it automatically.
3. Select OData. Open the streaming sync configuration and, under API version, select OData.
4. Enable Force snapshot. Turn on Force snapshot so that the first OData start performs a new initial load instead of relying on the ODP pointer.
5. Check for subscription conflicts. If another sync already holds the OData subscription for this table and SAP user, resolve the conflict in SAP before you continue, as described in Subscription conflicts. Confirm that Reset queue registration is off.
6. Save the configuration. Before you start the stream, confirm that it matches the reconfigure stage in Migration stages: API version is OData and Force snapshot is on.
7. Start the stream. If your SAP landscape requires approval for new initial loads, obtain it first. Then select Start.
8. Validate the replay. Wait until the initial load has been received. Confirm that the number of distinct keys matches the SAP table.
9. Turn off Force snapshot, then save. The stream is now in the steady state shown in Migration stages. Saved changes do not affect the running stream; they take effect the next time it starts. With Force snapshot off, later restarts, including automatic restarts after downtime, poll only for changes.
10. Validate deltas. Make or wait for a change in SAP and confirm that it arrives in the stream.
11. Resume any schedule that restarts the stream.
12. Clean up the old ODP subscription. After you have verified the stream, ask your SAP Basis team to review the legacy ODP subscription for the table in transaction ODQMON, as described for incremental batch syncs.

Subscription conflicts

The OData API allows one active subscription per table for each SAP user. A conflict occurs when another sync, using the same SAP user, already holds the OData subscription for the table. Common causes include:

• Another SAP SLT or SAP (Custom source) sync with the same credentials that already ingests the table through OData.
• A sync that was deleted or replaced in Foundry, whose subscription remains active in SAP.
• An earlier test of OData for the same table and SAP user.

The error message includes the incremental ID, latest run date, and creator of the sync that holds the subscription. Use these details to identify the owning sync.

Reset queue registration does not resolve a conflict. It resets only the current sync's own subscription. If a different sync holds the subscription, the run fails with You cannot reset this sync, an active sync is already configured for this table.

To resolve a conflict, choose one of the following options:

• Release the subscription in SAP: Coordinate with the owner of the conflicting sync and stop it first. Then ask your SAP Basis team to terminate the conflicting OData subscription for the table and SAP user in transaction ODQMON. This removes the subscription only; it does not delete data from the SAP business table. The sync that held the subscription can no longer continue incrementally. If that sync is still needed, it requires a new initial load and the conflict returns, so use a different SAP user for one of the syncs instead.
• Use a different SAP user: If two syncs must ingest the same table independently, use sources that authenticate as different SAP users.

Troubleshooting

The API version card does not appear

• Confirm that the sync uses the SLT Replication Server object type (SAP SLT source) or the SLT SAP object type (SAP (Custom source)). The card does not appear for local tables or other object types.
• For SAP (Custom source) streaming syncs, confirm that the source has an explicit SLT connection type.
• If the object type is correct and the card still does not appear, OData selection is not yet available for that sync type on your enrollment. Contact Palantir Support.

The add-on version is not supported

The API version card can report that the add-on is older than 2.34 (SP34). A run can also fail with an error stating that OData requires SAP add-on 2.34.0 (SP34) or later. In either case, upgrade the Connector and install PALODATA as described in Install a Support Package and Installation. Then check the component versions.

The OData service is unavailable

If an OData run fails while connecting to SAP, while ODP runs on the same source succeed:

• Confirm that PALODATA is installed and that SAP_BASIS is 7.50 SP09 or later.
• Confirm that the SAP Notes listed in Use ODP via OData are implemented.
• Confirm that the Connector's SICF services are active.
• Review the Connector logs in the Add-on Cockpit, then contact Palantir Support with the error message and the SAP component versions.

A run reports an active subscription for the table

Errors such as Only one active incremental sync is permitted per table per technical user or You cannot reset this sync, an active sync is already configured for this table indicate a subscription conflict. The error message includes details of the sync that holds the subscription.

Every run reloads the full table

If a batch sync replaces the whole dataset on each run, or a stream re-sends the full table, check for settings that were left on:

• Reset incremental state (batch). Turn it off; Foundry sets the transaction type back to APPEND.
• Reset queue registration (batch or streaming). Turn it off. On a stream, this setting can re-initialize the subscription repeatedly while the stream runs.
• Force snapshot (streaming). Turn it off.

A running stream keeps the configuration it started with. To apply the change to a stream, stop the stream, turn the setting off, save, and start the stream again.

The dataset is partial or empty after a SNAPSHOT run

An incremental OData sync that runs as SNAPSHOT without Reset incremental state writes only the latest delta and replaces the dataset. To recover, follow the incremental batch migration procedure from step 2 to reload the full table, then return to APPEND.