Configure network egress
Network egress refers to any network traffic originating from within Foundry that attempts to connect to an external system. This page outlines how network egress is configured and managed in Control Panel and how those configurations are consumed by user workloads in Foundry.
Network ingress refers to network traffic originating outside of Foundry that attempts to establish a connection to Foundry. Network ingress is also managed in Control Panel.
Network egress overview
Foundry provides strict network firewalls to protect customer data. Customer-managed network egress policies are used to apply network firewall rules to individual workloads using container networking technology (Cilium ↗, eBPF ↗). In addition to these customer-specific rules, Palantir's Information Security team maintains network firewall rules at the infrastructure proxy level to provide another layer of security. Together, these rules govern the network egress execution of user workloads in Foundry, including the following:
- Data connection sources using a cloud runtime
- External transforms
- External functions
- External streaming user-defined functions (UDFs)
- External model adapters (for modeling live deployments)
- Code workbooks
Network traffic that egresses from Foundry may still cause data to flow in either direction. This means that workloads within Foundry that are allowed to connect externally may be used to both sync and export data.
Opening a network egress route is always a security risk. Information security officers for a Foundry enrollment should ensure that they only open network routes to trusted destinations, and limit access to those routes to a trusted group of developers. Even a trusted external system can be abused by malicious actors to bypass security controls. Information security officers should leverage Foundry change management tools to ensure changes to egress logic are reviewed by a trusted group, and establish audit processes to ensure egress logic remains secure.
Network egress policies
Network egress policies are configured in Control Panel and represent the set of external destinations that user workloads may egress to from a Foundry enrollment. Egress policies may be addressed by domain name, IP, or CIDR block.
The destination of a network egress policy cannot be modified once the policy has been created. Network egress policies cannot be deleted after creation, they may only be revoked.
The following table summarizes the options that may be used when configuring network egress policies.
| Option | Description |
|---|---|
| Address | Option 1: DNS A domain name in the format subdomain.domain.com. Only the specific domain name will be allowed, and wildcards are not supported. For example, to allow traffic to both foo.mycompany.com and bar.mycompany.com, two separate domain name policies must be created. Option 2: IP A single IPv4 address in the format x.x.x.x Option 3: CIDR An IPv4 CIDR address block in the format x.x.x.x/x For more information, review the sections below about handling domain name addresses versus IP and CIDR addresses |
| Port(s) | The port or port range that should be allowed for the specified domain. Port values must be in the range of 1 - 65535 inclusive. Option 1: Single port When using a DNS address, you must specify a single port. Only HTTP, HTTPS, and traffic from certain source types (PostgreSQL, MS SQL Server, and FTP) will be allowed at the configured domain name and port. If port 80 is specified, HTTP traffic is allowed. Otherwise, only HTTPS traffic is allowed. Option 2: Port range When using this option, you must provide a starting and ending port, where the starting port is less than or equal to the ending port. |
| Global | Defaults to false. Enabling this option makes the specified domain and port a global policy and is strongly not recommended. |
Domain name addresses
A network egress policy that specifies the external destination using a domain name allows connections to HTTP/HTTPS traffic and some source types (PostgreSQL, MS SQL Server, FTP) without the need for additional IP policies.
For other non-HTTP/HTTPS traffic, domain name policies must be used in conjunction with IP policies. This is required because Palantir network proxies perform TLS inspection for all domain-based network egress traffic; for other non-HTTP/HTTPS traffic, some required metadata is not available.
Resolve the domain to an IP address and use an egress policy with an IP or CIDR address. Then, configure your Data Connection source in Foundry to connect through both domain name and IP policies.
If the domain name to IP address DNS resolution changes in the future, then the egress policy and destination will both need to be updated.
IP and CIDR addresses
A network egress policy that specifies the external destination using an IPv4 address or CIDR block may be used to allow any network traffic at the specified address(es) and port(s). These policies support any TCP protocol, including HTTP/HTTPS, FTP, or TCP-based SQL database protocols. IPv6 addresses are not supported.
Foundry attempts to perform TLS inspection by default for all network connections, including those allowed via an IP address or CIDR block. However, this cannot be applied to some protocols like FTP(S), SFTP, and most TCP-based database connections. When Palantir's network infrastructure attempts to perform TLS inspection for traffic that does not support it, you may encounter hanging connections and/or timeout errors.
By default, TLS inspection is disabled for traffic to the following ports for commonly used TCP protocols:
| Port(s) | Protocol |
|---|---|
20-21 | FTP |
22 | SSH, SFTP |
25 | SMTP |
990 | FTPS |
3306 | MySQL |
5432 | PostgreSQL |
TLS inspection may additionally be manually disabled on a per-policy basis by an administrator. If you believe this is required for your use case and is not covered by the list above, open a support issue. Note that disabling TLS inspection is not possible for DNS-based policies.
Additionally, note that a policy specifying a port range that includes any ports not in the above list will enable TLS inspection on all traffic, including traffic to the above ports.
Permissions
The below table shows a summary of the operations that are relevant for network egress policies and the default configuration of these permissions.
| Operation | Description |
|---|---|
| Propose | Any Foundry user may propose a network egress policy. Proposed policies only become active and usable once they are approved. Policies pending approval are visible in the Approvals inbox in Control Panel. |
| Approve/Create | Granted via the Manage network egress configuration workflow. By default, this workflow is then granted to both Enrollment Administrator and Information Security Officer roles. |
| Update metadata | Granted via the Manage network egress configuration workflow. By default, this workflow is then granted to both Enrollment Administrator and Information Security Officer roles. |
| Revoke | Granted via the Manage network egress configuration workflow. By default, this workflow is then granted to both Enrollment Administrator and Information Security Officer roles. |
| Pause | Palantir's Information Security team has the ability to block network egress that are deemed a security risk or threat. Any network egress policies blocked by this will appear as Paused in Control Panel. If you believe a legitimate network egress policy for your enrollment has been incorrectly paused, file a support ticket with the relevant details. |
| View | Allows assigned users and groups to see the existence of the policy, but does not allow them to import and use it in any workloads. In general, we recommend making anyone who might ever need to use the policy a viewer, even if they are not currently an importer. |
| Import | Allows assigned users and groups to see the existence of the policy and to import and use within workloads. |
The primary management interface for network egress lives at the enrollment level in Control Panel. Users with access to the Manage network egress configuration workflow, associated with the Enrollment administrator role, will be able to access the Network Egress page to perform any administrative actions on egress policies.
Enrollment Administrators and Information Security Officers are additionally always able to see which projects an egress policy has been imported into. This is possible regardless of the individual access of the administrative user's access to the project(s). This metadata is available to ensure that Information Security Officers have sufficient visibility into policy usage to take governance decisions on possibly revoking or otherwise restricting the ability for a policy to be used.
When approving a policy, the Information Security Officer must decide which users or groups should be granted the ability to either View or Import the network egress policy. This may also be managed later in Control Panel using the Manage Sharing button on the policy details page.
Users with Import access to a network egress policy can run arbitrary code in Foundry that talks to the destination address. Ensure that any users granted Import permissions are trusted and trained on your organization's information security policies to avoid misuse of this access.
Policy states
Throughout the lifecycle of a network egress policy, it may be in one of the various states described below:
| Policy state | Description |
|---|---|
Pending approval | Pending approval is the default state for new network egress policies. Policies in this state can be attached to workloads, but workloads attempting to use a pending approval policy will fail to run. |
Active | Once a policy is approved, it becomes active. Importers of an active policy are able to attach it to Foundry workloads to allow that workload to egress to the specified external address. |
Paused | Palantir's Information Security team has blocked egress to the specified address. Workloads attempting to use a paused policy will fail to run. |
Revoked | Information Security Officers for a Foundry enrollment may revoke a policy from Control Panel. Workloads attempting to use a revoked policy will fail to run. |
Common edge cases
What happens when there are two policies that are identical; one is paused/revoked and the other is active.
Any workloads using the active policy will run successfully, while workloads attempting to use the paused/revoked policy will fail to run.
What happens if there are overlapping policies and one is paused/revoked and the other is active?
This may happen if a policy using a single IP address and a policy specifying a CIDR block of IP addresses overlap.
In this case, as with identical policies, any workloads using the active policy will run successfully, while workloads attempting to use the paused/revoked policy will fail to run.
What happens if I propose a policy that already exists?
Duplicate policies are allowed, and the information security officer reviewing the proposal may choose to do one of the following:
- Deny the proposal entirely
- Deny the proposal and grant the proposer Importer access to an existing identical or overlapping policy
- Approve the proposal and allow duplicate or overlapping policies (not recommended)
Using a policy
Two steps must be taken for a network policy to take effect.
First, create the policy in Control Panel. If your Foundry installation's policy management is supported by Palantir, contact your Palantir representative with the details of your egress policy. This allows Palantir to assess security risks before expanding network access. If your Foundry installation's policy management is fully self-service, the policy will automatically be applied to the firewalls.
Second, the policy must also be assigned to a user workload:
- Data Connection sources using a direct connection runtime: Specify the network policy when creating a new source.
- External transforms: Reference the network policy in a Python transform declaration.
- Code Workbook: Granular network management is not yet available, and administrators must instead create a global policy. A global policy is applied indiscriminately to all workloads and should only be configured for highly trusted network routes.
To assign a policy to a user workload, the Importer permission is required for the specific egress policy. This permission is granted on a per-policy basis through the Manage Sharing setting on the respective egress policy.
Opt-in versus global
By default, policies are "opt-in" and must be attached to a workload in Foundry. For example, when creating a data connection source, policies for that source should be explicitly attached to the source by a user with Importer permission on the policy. This follows the principle of least privilege ↗, ensuring that workloads are only granted egress rules that are strictly required for that workload to run successfully.
Global policies are automatically applied to all workloads for the entire Foundry instance. Global policies are significantly riskier than opt-in policies and should only be used in cases where an opt-in policy cannot be used. As of 2024, the only workload type that still requires global policies is Code Workbook.
Support for global policies will be removed once all workloads support opt-in policy assignment.
Which IPs do connections from Foundry come from?
When connections are initiated from Foundry to external destinations, they come from certain IP ranges. Sometimes those IPs need to be added to an allowlist before connections from Foundry will be accepted.
When Foundry is hosted in Palantir's cloud infrastructure, the egress IPs where Foundry traffic originates will be displayed on the network egress management page in Control Panel. You should copy the CIDR ranges displayed there when adding to an allowlist in the destination system.
Note that connections through agents are handled differently. When using an agent, network traffic to the source system will always come from the agent host on your infrastrucutre; it is your responsibility to ensure network connectivity between the agent and target systems.
Amazon S3 bucket policies
For Foundry instances hosted in AWS, additional configuration is required when connecting to S3 buckets in the same region. This is due to how AWS handles networking for requests to S3 from a VPC gateway endpoint in the same region.
To check if your connection require extra configuration, navigate to the Network egress page in Control Panel. If you find an additional tab called S3 bucket policies, then your instance is hosted in AWS and you must explicitly allow traffic from Palantir's VPC endpoint to any S3 buckets in the same region.
If present, the S3 bucket policies tab will also display the region where your instance is hosted, along with the Amazon Reference Number (ARN) of the VPC endpoint used to route traffic from Foundry to any same-region S3 buckets.
To successfully connect to a bucket in the same region as Foundry, you must complete the following:
- Configure your bucket policy in the AWS console to allow traffic from the VPC endpoint for your Foundry instance. Learn more about AWS bucket policies ↗.
- Ensure that your desired bucket is listed under Network egress > S3 bucket policies > AWS S3 Bucket Policies in Control Panel.
- Create an appropriate network egress policy as described above, and add it to the configuration details of the Data Connection source you are using to connect.
Allow traffic to S3 from the VPC endpoint for your Foundry instance
In the AWS console, you must ensure that the bucket policy ↗ for your S3 bucket allows inbound traffic from the VPC endpoint displayed in the Network egress > S3 bucket policies tab, as shown below:
The Amazon S3 documentation contains an example ↗ of how a bucket policy may be used to restrict traffic to an S3 bucket to a specific VPC endpoint. To learn more about managing inbound traffic to S3 from VPC endpoints, review the official AWS documentation ↗.
You can configure AWS bucket policies to gate traffic based on the VPC endpoint where the traffic originates. However, you should always use some form of access authentication to provide security for your bucket, as multiple Foundry instances may share a single VPC endpoint. A bucket policy that allows access purely based on the VPC endpoint may allow unauthorized access by users outside your organization.
Add an S3 bucket policy
To add a same-region bucket, select Add bucket policy and follow the prompts to enter your desired bucket name, as shown below:
The form will automatically check for a valid bucket name, as well as the region. If a valid bucket name matches the region where your Foundry instance is hosted, you will be able to save. In addition to the bucket name, you must provide a policy of READ ONLY or READ WRITE. This dictates what level of access you would like Palantir to request when establishing a connection to S3.
The addition of same-region S3 buckets to the allowlist, as well as the desired bucket access policy, should be treated as a minimum level of access guaranteed to be available over the VPC endpoint for your Foundry instance.
Data in S3 should be secured with appropriate authentication and authorization in AWS, as well as network egress policy governance within Foundry.
Palantir may expand the provided list to allow access to additional same-region buckets and may also bump up the access policy for a given bucket from READ ONLY to READ WRITE. Proper authentication and authorization when using the S3 source type will ensure that you do not accidentally allow unauthorized Foundry users to access your S3 bucket(s).
Use an S3 bucket policy
Once added, same-region bucket policies take effect immediately and apply to all workloads attempting to egress directly from Foundry to that bucket. Unlike network egress policies, they do not need to be attached to specific sources or other workloads in Foundry.
When creating an S3 source, the configuration interface automatically checks if the bucket is in the same region as Foundry and if the bucket has already been added in Control Panel. Users configuring the source will see either a green check indicating that the bucket is added correctly, or a red check indicating that action is required by an administrator with access to add an S3 bucket policy in Control Panel.
Limits
You can add up to a maximum of 10 same-region buckets per enrollment. Contact Palantir Support If you require access to more than 10 same-region S3 buckets from your Foundry instance.
Microsoft Azure Storage policies
For Foundry instances hosted in Azure, additional configuration is required when connecting to Azure Storage resources as traffic is routed over Azure service endpoints.
To check if your connection require extra configuration, navigate to the Network egress page in Control Panel. If you find an additional selectable tab called Azure Storage policies, then your instance is hosted in Azure and you must explicitly allow traffic from Palantir's Azure subnets to any Azure Storage account with connected resources.
If present, the Azure Storage policies tab will display the subnet IDs that are used to route traffic from Foundry to any Azure Storage resource.
To successfully connect to an Azure Storage resource, you must complete the following:
- Configure your Azure Storage account's virtual network rules in Azure to allow traffic from the subnets for your Foundry instance. Learn more about managing Azure virtual network rules↗.
- Ensure that your desired Azure Storage account is listed under Network egress > Azure Storage policies in Control Panel.
- Create an appropriate network egress policy as described above, and add it to the configuration details of the Data Connection source you are using to connect.
Allow traffic to Azure Storage account from the subnets of your Foundry instance
In your Azure account, you must ensure that the virtual network rules ↗ for your Storage account allows inbound traffic from the subnet IDs displayed in the Network egress > Azure Storage policies tab, as shown below:
You can configure Azure Storage policies to gate traffic based on subnet IDs. However, you should always use some form of access authentication to provide security for your Azure Storage account, as multiple Foundry instances may share the same subnets. An Azure Storage policy that allows access purely based on the subnet IDs may allow unauthorized access by users outside your Organization.
Add an Azure Storage policy
To add a Azure Storage egress policy, select Add Azure Storage policy, and follow the prompts to enter your desired Storage account resource ID, as shown below:
The form will automatically check if the provided Azure Storage account resource ID is valid. You can find more information on how to find an Azure Storage account resource ID in the Azure documentation ↗.
Use an Azure Storage policy
Once added, Azure Storage policies take effect immediately and apply to all workloads attempting to egress directly from Foundry to that Storage account. Unlike network egress policies, they do not need to be attached to specific sources or other workloads in Foundry.
When creating an ABFS source, the configuration interface automatically checks if the Foundry is deployed on Azure and if there is a valid Storage account that has already been added in Control Panel. Users configuring the source will see a warning indicating that action is required by an administrator with access to add an Azure Storage policy in Control Panel, as shown below.
Following the creation of a network egress policy, if an Azure Storage policy is also required to egress, a warning is also displayed as shown below:
