Snowflake

Connect Foundry to Snowflake to read and sync data between Snowflake and Foundry.

Supported capabilities

Capability	Status
Exploration	🟢 Generally available
Bulk import	🟢 Generally available
Incremental	🟢 Generally available
Virtual tables	🟢 Generally available
Export tasks	🟡 Sunset

Setup

Open the Data Connection application and select + New Source in the upper right corner of the screen.
Select Snowflake from the available connector types.
Choose to use a direct connection over the Internet or to connect through an intermediary agent.
Follow the additional configuration prompts to continue the setup of your connector using the information in the sections below.

Learn more about setting up a connector in Foundry.

Connection details

Option	Required?	Description
`Account identifier`	Yes	This is the identifier that precedes ".snowflakecomputing.com". See Snowflake's official documentation ↗ for more details.
`Roles`	No	This is the default role to be used by the connection in case the credentials provided have access to multiple roles.
`Database`	Yes	Specify a default database to use once connected.
`Schema`	No	Option to specify a default schema to use once connected. If not specified, all schemas will be available that are in-scope of the credentials.
`Warehouse`	No*	The virtual warehouse to use once connected. In the case of registered virtual tables, this will be used for any source-side compute.
`Credentials`	Yes	Option 1: Username and password Provide a username and password. We recommend the use of service credentials rather than individual user credentials. Option 2: Key pair authentication Provide a username and private key. See Snowflake's official documentation ↗ for details on configuring key pair authentication. Option 3: External OAuth (OIDC) Follow the displayed source system configuration instructions to set up External OAuth. See Snowflake’s official documentation ↗ for details on External OAuth and our documentation for details on how OIDC works with Foundry. For all credential options, ensure that the provided user and role has usage privileges on the target database(s) and schema(s), as well as select privileges on the target table(s). When registering virtual tables, the user and their role should also have usage privileges on the warehouse.
`Network Connectivity`	Yes**	Run `SELECT SYSTEM$ALLOWLIST()` in Snowflake and ensure that at least the entries for `SNOWFLAKE_DEPLOYMENT` and `STAGE` are added as egress policies in Foundry. Refer to the networking section below for more details.

* Warehouse details are optional for syncing Foundry datasets, but required for registering virtual tables.
** Network egress policies are required for direct connections, but not for agent-based connections.

Networking

To enable direct connections between Snowflake and Foundry, the appropriate egress policies must be added when setting up the source in the Data Connection application.

To identify the hostnames and port numbers of your Snowflake account to be allowlisted, you can run the following command in your Snowflake console. Ensure that at least the entries for SNOWFLAKE_DEPLOYMENT and STAGE are added as egress policies in Foundry.

Copied!1
2
3
4
SELECT t.VALUE:type::VARCHAR as type,
       t.VALUE:host::VARCHAR as host,
       t.VALUE:port as port
FROM TABLE(FLATTEN(input => PARSE_JSON(SYSTEM$ALLOWLIST()))) AS t;

See Snowflake's official documentation ↗ for additional information on identifying hostnames and port numbers to allowlist.

In a limited number of cases (depending on your Foundry and Snowflake environments) it may be necessary to establish a connection via PrivateLink. This is typically the case where both Foundry and Snowflake are hosted by the same CSP (for example, AWS-AWS or Azure-Azure.) If you believe this applies to your setup, contact your Palantir representative for additional guidance.

For egress policies that depend on an S3 bucket in the same region as your Foundry instance, ensure you have completed the additional configuration steps detailed in our Amazon S3 bucket policy documentation for the affected bucket(s).

Virtual tables

This section provides additional details around using virtual tables with a Snowflake source. This section is not applicable when syncing to Foundry datasets.

Virtual tables capability	Status
Source formats	🟢 Generally available: tables, views, and materialized views
Manual registration	🟢 Generally available
Automatic registration	🟢 Generally available
Pushdown compute	🟢 Generally available; available via the Snowflake Spark connector ↗
Incremental	🟢 Generally available: `APPEND` only [1]

When using virtual tables, remember the following source configuration requirements:

You must set up the source as a direct connection. Virtual tables do not support use of intermediary agents.
Ensure that bi-directional connectivity and allowlisting is established as described in the Networking section of this documentation.
If using virtual tables in Code Repositories, refer to the Virtual Tables documentation for details of additional source configuration required.
You must specify a warehouse in the connection details.
The credentials provided must have usage privileges on the warehouse.

See the Connection Details section above for more details.

[1] To enable incremental support for pipelines backed by Snowflake virtual tables, ensure that Change Tracking ↗ and Time Travel ↗ are enabled for the appropriate retention period. This functionality relies on CHANGES ↗ The current and added read modes in Python Transforms are supported. These will expose the relevant rows of the change feed based on the METADATA$ACTION column. The METADATA$ACTION, METADATA$ISUPDATE, METADATA$ROW_ID columns will be made available in Python Transforms.

Data model

Note that columns of type array ↗, object ↗, and variant ↗ will be parsed by Foundry as type string. This is due to the source's variable typing.

For example, the Snowflake array [ 1, 2, 3 ] would be interpreted by Foundry as the string "[1,2,3]".

See Snowflake's official documentation ↗for more details.