• Documentation
    • Search documentation
    karat

    +

    K

    API Reference ↗
    Data connectivity & integrationPythonSet up local development

    Set up Python local development

    It is possible to carry out local development of transforms Python repositories, allowing for high-speed iterative development.

    Prerequisites

    • Ensure Java 17 is installed and that the environment variable JAVA_HOME points to the right Java installation. Java 17 can be downloaded from the Oracle website ↗.

    Set the JAVA_HOME environment variable based on your operating system:

    • Windows: Run SETX JAVA_HOME -m "<java-home-dir>" in PowerShell. This modifies the system environment variable, so you will need to restart the shell for changes to take effect. Alternatively, you can run [System.Environment]::SetEnvironmentVariable("JAVA_HOME", "<java-home-dir>") to set JAVA_HOME in the running process.
    • Linux or macOS: Run export JAVA_HOME=<java-home-dir>.

    Work with your Python transforms repository locally

    There are two options for developing locally in your Python transforms repository:

    Option 1: Use the Palantir extension for Visual Studio Code [Recommended]

    Beta

    The Palantir extension for Visual Studio Code is in the beta phase of development and may not be available on your enrollment. To enable the extension and local preview capabilities, contact your platform administrator to modify Code Repositories settings in Control Panel. The Palantir extension for Visual Studio Code is enabled by default when using VS Code workspaces.

    Prerequisites

    The "Work locally inside Visual Studio Code" dialog.

    Benefits

    • Preview your Python transforms directly from your local Visual Studio Code environment. The Palantir extension for Visual Studio Code supports full datasets (sample-less preview), so you can preview full datasets without losing precision. Note that to run previews locally, you will need your platform administrators to enable local preview through Control Panel.
    • Initiate a build directly from your own code editor.
    • Debug your code and run tests directly from the editor.
    • Leverage the library panel to add libraries.

    Troubleshooting

    If you experience unexpected language server errors such as broken imports, the Python interpreter's automatic set up may have failed. To fix this, you can manually set up the interpreter by navigating to the Command Palette ↗, then typing Python: Select interpreter and choosing a Python interpreter matching the path ./maestro/bin/python. The .maestro folder should be present if the Palantir extension successfully ran the Palantir: Install Python environment command.

    Option 2: Clone the repository and use Gradle commands

    Prerequisites

    • Ensure your repository is upgraded to the latest template version by following the steps outlined here.
    • Ensure that the environment variables CI, JEMMA, and CA are not set.
    • If running on an Apple silicon Mac, ensure that Rosetta 2 ↗ is installed. You can install Rosetta 2 by running /usr/sbin/softwareupdate --install-rosetta --agree-to-license in the terminal.

    Clone the repository locally

    1. In the menu bar of your repository, select the desktop icon to the right of Data lineage to open the dialog and copy the given repository URL.

    The top menu bar of a repository with the "Work locally" option to the right.

    The "Work locally" dialog.

    1. Using the command line, run git clone <URL> on your local machine in a directory of your choice. Then, use the cd command to navigate to the repository.

    Limitations

    • The token granted for cloning is short-lived and needs to be refreshed every 7 days.
    • You will still need to push your changes to Foundry to publish job specs or artifacts, or if you wish to run checks or builds.

    Preview

    Dataset previews are supported in local development. Refer to local preview for more details.