Sharepoint Online

Connect to SharePoint Online to import files from specified SharePoint libraries into Foundry.

Supported capabilities

Data model

The connector can transfer files of any type into Foundry datasets. File formats are preserved, and no schemas are applied during or after the transfer. Apply any necessary schema to the output dataset, or write a downstream transformation to access the data.

Performance and limitations

There is no limit to the size of transferable files. However, network issues can result in failures of large-scale transfers. In particular, direct cloud syncs that take more than two days to run will be interrupted. To avoid network issues, we recommend using smaller file sizes and limiting the number of files that are ingested in every execution of the sync. Syncs can be scheduled to run frequently.

Setup

1. Open the Data Connection application and select + New Source in the upper right corner of the screen.
2. Select SharePoint Online from the available connector types.
3. Choose to use a direct connection over the Internet or to connect through an intermediary agent.
4. Follow the additional configuration prompts to continue the setup of your connector using the information in the sections below.

Learn more about setting up a connector in Foundry.

Authentication

Follow the initial steps below to access Azure application credentials:

1. Create an application registration in Azure by following the instructions in the Microsoft documentation ↗.
   • At Step 5, select Accounts in this organizational directory only and skip Redirect URL (optional).
2. Note the client ID and tenant ID once registration is complete.

Then, choose between two available authentication method:

• Client credentials: Recommended when a wide range of access is required for every SharePoint site.
• Username/password: Recommended for limiting access to one or a few SharePoint sites.

Client credentials

In your Microsoft Entra admin center, complete the following steps:

1. Go to API Permissions in the left sidebar.

2. Select Add a Permission.

3. Select Microsoft Graph.

4. Select Application Permissions.

   • If you would like your application to read all SharePoint sites add Sites.Read.All.
     • If you plan to configure export tasks, use Sites.ReadWrite.All instead.
   • If you would like your application to read selected SharePoint sites add Sites.Selected.

5. If you are an Entra Administrator, select Grant admin consent for [tenant].

6. If you added Sites.Selected above, add your application to specific sites ↗.

   • The available options for the "roles" array parameter are "write" and/or "read". The "read" option is sufficient to ingest files from the SharePoint site.
   • To easily send a POST with proper authentication, use the Graph Explorer ↗.
   • You can receive metadata about a site by sending a GET to https://graph.microsoft.com/v1.0/sites/[tenantName]:/sites/[siteName] (for example: https://graph.microsoft.com/v1.0/sites/contoso.sharepoint.com:/sites/mySite). This request will return an ID that is a composite of several values: Site collection hostname, Site collection unique ID, and Site unique ID where the middle value is the siteId needed to run the permissions POST.

7. Generate a client secret. ↗.

Set the following source configurations in Data Connection:

Username/password

The username/password flow involves creating a user account that can sign in to Microsoft 365. The Graph API does not support two-factor authentication for the username/password authentication method. Because of this, we strongly recommend creating a randomly generated password of at least 32 characters in length.

In your Entra admin center, complete the following steps:

1. Go to API Permissions in the left sidebar.
2. Select Add a Permission.
3. Select Microsoft Graph.
4. Select Delegated Permissions.
5. Add the Sites.Read.All permission;.
   • If you plan to configure export tasks, use Sites.ReadWrite.All instead.
6. If you are an Azure Administrator, select Grant admin consent for [tenant].
7. Go to Authentication in the left sidebar.
8. Change Allow public client flows to Yes.
9. Create a user in Microsoft Entra ID with a randomly generated password of at least 32 characters.
10. Add that user to any SharePoint sites that you would like it to read or write.

Set the following source configurations in Data Connection:

XML-based permissioning for SharePoint Add-ins

If you are using SharePoint Add-ins for authorization and authentication ↗, and your SharePoint Add-in uses XML for permission management, you must ensure that the correct scope is set in the scope URI to avoid access issues when connecting to SharePoint.

Follow the steps below to verify and configure the correct scope:

1. Locate the AppManifest.xml file containing the permission settings for your SharePoint Add-in.
2. In the AppManifest.xml file, identify the scope URI within the XML file, which should look similar to this:

<AppPermissionRequests AllowAppOnlyPolicy="true"> <AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="FullControl" /> </AppPermissionRequests>.

3. Verify that the scope value (in this example, http://sharepoint/content/sitecollection/web) matches the SharePoint site to which you are connecting; if the scope value does not match, adjust the scope value accordingly.

Networking

The SharePoint Online connector requires network access to the following domains on port 443:

• login.microsoftonline.com
• graph.microsoft.com
• Your SharePoint URL; for example, contoso.sharepoint.com

If you are using a GovCloud Sharepoint instance, use the following domains on port 443 instead:

• login.microsoftonline.us
• graph.microsoft.us
• Your SharePoint URL; for example, contoso.sharepoint.us

Configuration options

The following configuration options are available for the SharePoint Online connector:

Sync data from Sharepoint Online

The SharePoint Online connector uses the file-based sync interface.

Export data to SharePoint Online

To export to a SharePoint site, first enable exports for your SharePoint Online connector. Then, create a new export.

Export configuration options