Coalesce data

Supported in: Batch

Operation to reduce the number of partitions. If say you have 1000 partitions andyou coalesce to 100 there will not be a shuffle, instead each of the 100 new partitions will claim 10 of the current partitions. If a larger number of partitions is requested, it will stay at the current number of partitions.

Transform categories: Other

Declared arguments

  • Dataset - Dataset to perform coalesce on.
    Table
  • Number of partitions - Number of partitions to coalesce to.
    Literal<Integer>