Transformstransforms-pythontransforms.apiContainerTransform

transforms.api.ContainerTransform

class transforms.api.ContainerTransform(transform, *, cpu_cores=None, memory_mb=None, memory_gb=None, gpu_type=None, container_image=None, container_tag=None, container_shell_command=None, incremental_override=None, identifier_override=None)

A callable object that describes a single step of a lightweight, single-node computation.

A ContainerTransform consists of a number of parameters that subclass the Param class and a compute function.

It is idiomatic to construct a Transform object using the provided decorator: transform.using().

Note that the original compute function is exposed via the ContainerTransform’s __call__() method.

For backward compatibility, you can also reference this class as transforms.api.ContainerTransformsConfiguration.

A ContainerTransform can be executed incrementally by wrapping it with the incremental decorator. For non-Catalog incremental inputs and outputs, set v2_semantics=True on the incremental decorator.

Copied!1
2
3
4
5
6
7
8
@incremental(v2_semantics=True)
@transform.using(
    my_input=Input('/input'),
    my_output=Output('/output')
)
def compute(my_input, my_output):
    # lightweight compute; incremental behavior controlled by @incremental
    my_output.write_pandas(my_input.pandas())

property reference

A reference to this transform, unique in the pipeline. This field is recomputed at each call to account for compute function renaming.

←

PREVIOUSconfigure

NEXTContainerTransformsConfiguration

→