Architecture examples

Foundry worker with direct connection policies

Foundry worker with direct connection policies is the recommended default for accessing external systems that accept inbound traffic from Foundry, such as REST APIs and cloud/SaaS systems.

In this scenario, data connections run in an isolated compute container called a Foundry worker, which handles authenticated encrypted network requests. The external system must accept inbound traffic from Foundry. Network egress from Foundry is managed and administered in-platform via direct connection egress policies.

External system credentials are stored with AES-256-GCM server-side encryption and can only be decrypted by containers triggered by authorized users.

To create a direct connection, follow instructions about setting up a source.

Foundry worker with agent proxy policy

Foundry worker with agent proxy policies is the recommended architecture to access external systems hosted on private networks. It requires the use of a data connection agent.

In this scenario, the agent acts as a simple network tunnel without performing any data processing itself.

The agent initiates a network request with Foundry to establish a websocket connection. Learn more about network controls available on agents.

All data connection and computation capabilities are executed by a Foundry worker, an isolated container with scalable compute resources that processes data and communicates with external systems via the provided websocket.

The ability to use the agent as a proxy is granularly managed and administered in-platform via agent proxy egress policies.

Multiple agents can be used to load balance workloads across multiple external systems.

Like direct connections, external system credentials are stored using AES-256-GCM server-side encryption and can only be decrypted by the container triggered by authorized users.

To create an agent-proxy connection, you will need to:

• Set up an agent.
• Create one or many agent proxy policies.
• Set up a Foundry worker source using the policies above.

Agent worker

Agent worker connection is the historical architecture used to access external systems hosted on private networks from Foundry. It requires the use of a data connection agent. Except for specific file-based syncs filtering large amount of data or micro-batching workflows, agent proxy connections are recommended for accessing on-premise or privately hosted systems. Learn more about agent worker known limitations.

In this scenario, the agent constantly polls Foundry via unidirectional outbound connections secured by HTTPS for new tasks to execute. Once received, the task is executed by the agent itself, also called agent worker and results are sent to Foundry over that same unidirectional connection. All the networking configuration required for the agent to be able to communicate with internal source systems is configured on the agent host itself.

External system credentials are stored on the platform using AES-128-GCM encryption with keys stored on the agent. During capability execution, the agent retrieves encrypted credentials from Foundry, decrypts them locally, and uses them for queries. Decrypted credentials are automatically deleted from memory after execution.

To create an agent worker connection, you will need to:

• Set up an agent.
• Configure the agent worker.
• Set up an agent worker source and assign it that agent.

Summary