transforms.api.IncrementalLightweightInput

class transforms.api.IncrementalLightweightInput(alias, rid, branch=None)

The input object passed into incremental ContainerTransform objects at runtime.

Its aim is to mimic a subset of the transforms.api.IncrementalTransformInput API, while providing access to the underlying foundry.transforms.Dataset.

property alias

The alias of the dataset this parameter is associated with.

arrow(mode='added')

A PyArrow table containing the full view of the dataset.

• Parameters: mode (str ↗) – The read mode, one of current, previous, or added. Defaults to added.

property branch

The branch of the dataset this parameter is associated with.

dataframe(mode='added')

A pandas DataFrame containing the full view of the dataset.

• Parameters: mode (str ↗) – The read mode, one of current, previous, or added. Defaults to added.

property end_transaction_rid

The ending transaction of the input dataset.

• Type: str ↗

filesystem(mode='added')

Access the filesystem in read-only mode.

Construct a FoundryDataSidecarFileSystem object for accessing the dataset’s files directly.

• Parameters: mode (str ↗) – The read mode, one of current, previous, or added. Defaults to added.

pandas(mode='added')

A pandas DataFrame containing the full view of the dataset.

• Parameters: mode (str ↗) – The read mode, one of current, previous, or added. Defaults to added.

path(mode='added')

Download the dataset’s underlying files and return a path to them.

• Parameters: mode (str ↗) – The read mode, one of current, previous, or added. Defaults to added. This argument is only applicable when @incremental is added and v2_semantics is True.

polars(lazy=False, mode='added')

A Polars DataFrame or LazyFrame containing the full view of the dataset.

• Parameters:
  • lazy (bool ↗ , optional) – Whether to return a LazyFrame or a DataFrame. Defaults to False.
  • mode (str ↗) – The read mode, one of current, previous, or added. Defaults to added.

property rid

The unique resource identifier of the dataset this parameter is associated with.

property start_transaction_rid

the starting transaction of the input dataset.

• Type: str ↗