Data connectivity & integrationPipeline Builder TransformsFilter

Filter

Supported in: Batch, Streaming

Filters the input dataset based on the specified filter condition.

Transform categories: Popular

Declared arguments

  • Dataset - Dataset to filter.
    Table
  • Filter condition - Condition to filter on. Values that return true are kept, others are removed.
    Expression<Boolean>

Examples

Example 1: Base case

Argument values:

  • Dataset: ri.foundry.main.dataset.a
  • Filter condition: recently_serviced

Input:

recently_servicedtail_number
trueKK-150
falseXB-120
trueMT-190

Output:

recently_servicedtail_number
trueKK-150
trueMT-190

Example 2: Base case

Description: Nulls are treated as false Argument values:

  • Dataset: ri.foundry.main.dataset.a
  • Filter condition: recently_serviced

Input:

recently_servicedtail_number
nullKK-150
trueXB-120

Output:

recently_servicedtail_number
trueXB-120