Experiments

An experiment is an artifact representing a collection of metrics produced during a model training job.

Integrate experiments into model training code

Experiments can be created from any environment used to create models in Foundry. The ModelOutput class in Jupyter® Code Workspaces and Code Repositories provides hooks for creating and writing to experiments. These experiments can then be published alongside the model and are instantly viewable in the model page.

Code Workspaces

The below code snippet demonstrates experiment usage in Jupyter® Code Workspaces:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from palantir_models.code_workspaces import ModelOutput
# `my-alias` is an alias to a model in the current workspace
model_output = ModelOutput("my-alias")

experiment = model_output.create_experiment(name="my-experiment")

# log parameters
experiment.log_param("learning_rate", 1e-4)

# log metrics
experiment.log_metric("train/loss", loss)
experiment.log_metric("train/loss", loss, step=step)

# publish alongside model to persist in the models page
model_output.publish(ModelAdapter(model), experiment=experiment)

Code Repositories

The below code snippet demonstrates experiment usage in Code Repositories:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from transforms.api import configure, transform, Input
from palantir_models.transforms import ModelOutput

@transform(
    input_data=Input("..."),
    model_output=ModelOutput("..."),
)
def compute(input_data, model_output):
    experiment = model_output.create_experiment(name="my-experiment")

    # log parameters
    experiment.log_param("learning_rate", 1e-4)

    # log metrics
    experiment.log_metric("train/loss", loss)
    # can also provide optional step value
    experiment.log_metric("train/loss", loss, step=step)

    # publish alongside model to persist in the models page
    model_output.publish(ModelAdapter(model), experiment=experiment)

Learn more about creating and visualizing experiments.