All major concepts in Data Connection — Agents, Sources, and Syncs — are managed in Foundry as resources. This enables flexibly organizing these resources across Projects and folders, and allows security primitives such as markings to be applied to Data Connection resources.
Platform owners should be careful to not over-share Data Connection resources. Users with view and edit permissions on Sources and Agents should be trusted pipeline developers. The datasets produced by the Syncs can be imported to downstream Transforms projects as necessary, thereby granting access to the data without sharing access to the Agents or Sources.
This page documents how permissions work for all Data Connection resources, including:
The permissions listed below are defaults. In some environments, the default permission behavior may have been changed using custom roles.
Owner
An owner of an Agent is usually its creator and gets the following permissions:
Editor
An editor of an Agent can:
A user must be the editor of a Project to create an Agent in that Project, but must be an owner of the Project to administer the Agents within that Project. That means that a user may create an Agent and then be unable to generate download links or perform other administrative tasks on the Agent.
Viewer
A Viewer of an Agent can:
Owner
Editor
An editor of a Source can:
Keep in mind that Syncs can change the Source system if the Source credentials allow it (e.g. delete files from a directory or S3 Source, or drop data from a database via arbitrary SQL). You should only grant Edit access to the Source (or the Project containing it) to users whom you would also grant full access to the account the Source uses to access the data.
Viewer
Sync permissions are derived from the relevant Source and output Dataset.
Owner
Editor
Viewer
A Webhook’s permissions are inherited entirely from the Source with which it is associated.
By default, only the user who executes a Webhook may view the response using the history tab in the Data Connection application. The webhooks:read-privileged-data permission may be added to a custom or default role to allow users with that role to view the full request and response history for all webhooks that they can access. The standard recommended setup is to add a new role called "Webhook Privileged Data Viewer" with this permission, and ensure that users who need to view full Webhook history have this role on the relevant Sources.
As noted above, resources can have Organizations and Markings applied as an additional level of access control. Organizations and Markings are applied to the Source will propagate to the datasets produced from Syncs on that Source. This means users will not be able to view data in any of the datasets produced from a Source, unless they have access to all of the Organizations and Markings applied to that Source.
Since it is likely for Sources to contain data of varying levels of sensitivity, we recommend that you mark the Source with all of the Organizations and Markings that apply to the data available in that Source, and then use transforms to clean and unmark the data using the stop_propagating and/or stop_requiring functionality. To learn more about unmarking, see the guides on removing Markings and removing inherited Markings.
Note that applying Organizations and Markings to the data after it has been synced should not be considered sufficient; as access to the Source is effectively access to the data, the Source should be marked with the Organizations and Markings that apply to anything it contains.
As a quick reminder, the primary Data Connection resources are Agents and Sources. Plugins and drivers are deployed to Agents to expand functionality.
There are two recommended permission setups for Agents: all Agents in a single Project per Organization, or each Agent in its own Project. The diagram above illustrates the former option; the guidance below describes why you might choose each option.
Option 1: Place all Agents in a single Project per Organization.
Placing all Agents in a single Project is the cleanest approach, and is recommended if there is one group of users managing all Agents. This allows you to control access to the Agents at the Project level, and to permission Sources and Syncs separately, downstream from the Agent.
Choose this option only if it is acceptable for every Agent Manager and every Source editor to have access to all Agents. Remember that to deploy a Source on an Agent, a user must have Editor permission on that Agent.
Option 2: Place each Agent in its own Project.
This approach allows for full granularity of Agent permissions. It also guarantees that the structure will never drift from the “groupings” at any time during the evolution of the Source management. For example, you can safely assign / unassign Agents from a Source without needing to grant users access to an entire group of Agents, and there would be no need to restructure Projects to account for group permission changes in the future.
This approach is recommended if you have multiple groups managing Agents, and you would like to permission their access to Agents separately.
Editors of the Project containing the Agent will be able to create new Sources and Syncs that run on that Agent. Make sure you do not grant Editor role on the Project to users who should not be able to do this.
Each Source should be in its own Project in the datasource layer. Each Sync defined on a Source should output to a dataset within the same Project as the Source. If the Source contains sensitive data such as PII, you can apply a Marking to the Source which will propagate to the output datasets.
Editors of the Project containing the Source will be able to create new Syncs and edit existing Syncs for that Source, as well as create Webhooks and configure them for use elsewhere in Foundry.
Keep in mind that Syncs can change the Source system if the Source credentials allow it; for instance, deleting files from a Directory or S3 Source, or dropping data from a database via arbitrary SQL.
You should only grant Edit access to the Source (or the Project containing it) to users whom you would also grant full access to the account the Source uses to access the data.
There are two recommended permission setups for plugins and JDBC drivers: all resources in a single Project per Organization, or each JDBC / driver in a Project with one or more Agents that require it.
Option 1: Place all plugins and drivers in a single Project per Organization.
This is recommended if there is one group of users managing all Agents. This allows Agent Managers to have access to all plugins and drivers that have been uploaded. This approach should only be chosen if there should be equal access to plugins and drivers among all Agent Managers.
The recommended process for implementing this approach is:
Option 2: Place each plugin and driver type in its own Project.
This approach allows for certain "sensitive" plugins or drivers to be permissioned separately from others. If this approach is taken, you must ensure to give the required Agent owners adequate permissions on each Project so they can use them on Agents.