Flatten struct

Supported in: Batch, Streaming

Take all fields in a struct and turn them into columns in the output dataset.

Transform categories: Struct

Declared arguments

Dataset - Dataset containing struct column.
Table
Expression - Expression evaluating to a struct column that will be flattened.
Expression<Struct>
Max depth - The depth level specifying how deep a nested struct will be flattened.
Literal<Integer>
optional Column prefix - Add a prefix to all columns created during the flatten.
Literal<String>
optional Separator - Separate field names coming from nested structs.
Literal<String>

Argument values:

Input:

raw
{ airline: { id: NA, name: new air, }, tail_no: NA-123, }
{ airline: { id: FA, name: foundry airways, }, tail_no: FA-123, }

Output:

new_airline_name	new_airline_id	new_tail_no	raw
new air	NA	NA-123	{ airline: { id: NA, name: new air, }, tail_no: NA-123, }
foundry airways	FA	FA-123	{ airline: { id: FA, name: foundry airways, }, tail_no: FA-123, }

Argument values:

Input:

raw
{ airline: { id: NA, name: new air, }, tail_no: NA-123, }
{ airline: { id: FA, name: foundry airways, }, tail_no: FA-123, }

Output:

new_airline#SEPARATOR#name	new_airline#SEPARATOR#id	new_tail_no	raw
new air	NA	NA-123	{ airline: { id: NA, name: new air, }, tail_no: NA-123, }
foundry airways	FA	FA-123	{ airline: { id: FA, name: foundry airways, }, tail_no: FA-123, }

Argument values:

Input:

raw
null
{ airline: null, tail_no: NA-123, }
{ airline: { id: FA, name: null, }, tail_no: FA-123, }

Output:

new_airline_name	new_airline_id	new_tail_no	raw
null	null	null	null
null	null	NA-123	{ airline: null, tail_no: NA-123, }
null	FA	FA-123	{ airline: { id: FA, name: null, }, tail_no: FA-123, }