Assign timestamps and watermarks

Supported in: Streaming

Assigns timestamps and watermarks to the input, filtering out records where the timestamp is null.

Transform categories: Other

Declared arguments

• Dataset - Dataset to assign timestamps and watermarks.
  Table
• Timestamp expression - Expression evaluating to timestamp to assign.
  Expression<Timestamp>
• optional Emit watermark on every record - If true, the watermark will be propagated for every record in the stream. This is generally inefficient and will add performance overhead but gives deterministic behaviour if the stream is replayed. It is recommended to set this value to false in most cases.
  Literal<Boolean>

Examples

Example 1: Base case

Argument values:

• Dataset: ri.foundry.main.dataset.a
• Timestamp expression: timestamp
• Emit watermark on every record: null

Input:

Output:

---

Example 2: Null case

Argument values:

• Dataset: ri.foundry.main.dataset.a
• Timestamp expression: timestamp
• Emit watermark on every record: null

Input:

Output:

---