Connect Foundry to Microsoft SQL Server to read and sync data between SQL Server databases and Foundry.
Capability | Status |
---|---|
Exploration | 🟢 Generally available |
Batch syncs | 🟢 Generally available |
Incremental | 🟢 Generally available |
Change data capture syncs | 🟢 Generally available |
Learn more about setting up a connector in Foundry.
You can authenticate with SQL Server in the following ways:
authentication=ActiveDirectoryMSI
JDBC setting. An msiClientId
may optionally be provided.authentication=ActiveDirectoryPassword
JDBC setting. A username and password for an Active Directory user must be provided to use this setting.authentication=ActiveDirectoryServicePrincipal
JDBC setting. A principal ID (sometimes referred to as an application or client ID) must be specified, along with a secret for that principal ID.For more information on these authentication modes, see the official documentation ↗. For all authentication options, ensure that the provided user and role has the necessary privileges on the target database, as well as permission to read from or write to the target table(s).
* Note that Azure Active Directory is now called Microsoft Entra ID ↗; however, the JDBC options on the SQL Server driver published by Microsoft retain the original names referring to Active Directory.
The Microsoft SQL Server connector requires network access to the SQL Server instance that you wish to connect to. SQL Server connections will normally use a hostname to connect on port 1433.
To enable direct connections out of Foundry to SQL Server, the appropriate egress policies must be added when setting up the source in the Data Connection application.
For SQL Server instances hosted on a cloud service like Azure SQL or AWS RDS, you must add an egress policy for the hostname retrieved from your cloud provider’s console.
If Azure redirect mode is used, then egress policies for all resolved IP addresses must also be added. The resolved IP may occasionally change, and you must update the egress policies to allow the new IP. If your instance of SQL Server is hosted on Azure, for example, then you can find more information on public IP addresses for Azure SQL instances in the Azure SQL documentation ↗.
<your-database-name>.database.windows.net (port 1433)
nslookup <your-database-name>.database.windows.net
from the command line. The final result will be the IP address that this hostname resolves to in Azure. The following is an example IPv4-based egress policy for Azure SQL: x.x.x.x (port 1433)
. Azure SQL does load balancing across multiple hosts, so you may need to run the nslookup
command several times and add all of the resolved IP addresses.If you are connecting to an Azure SQL instance from a Foundry instance also hosted in Azure, you will need to use the Proxy connection policy option. For traffic originating within Azure, the connection policy defaults to Redirect. Using the redirect option to connection for Azure-Azure connections would require configuring egress policies for all Azure SQL IP addresses on all ports in the range of 11000 to 11999. This is possible but not recommended as it is overly permissive. For details on Azure SQL connection policies, see the official Azure SQL documentation ↗.
If you are connecting with an agent runtime, you must ensure that the agent host has firewalls open to the host names, IP addresses, and ports required to connect to your MS SQL Server database.
Option | Required? | Description |
---|---|---|
Host type | Yes | Specify how Foundry should connect with your SQL Server database. Option 1: Hostname Provide a hostname. This is the recommended option for all SQL Server connections, and should always be used when connecting to an Azure SQL ↗ instance. Option 2: IPv4 Provide an IPv4 address. If you normally connect using an IPv4 address, either within a corporate network or over the Internet, you can use this option. Option 3: IPv6 Provide an IPv6 address. Use this option if you normally connect using an IPv6 address. |
Port | Yes | Specify a port to use when connecting. The default port for most SQL Server instances will be 1433 . For more information on ports, see the official documentation for the version of SQL Server you are connecting to. |
Database name | Yes | The name of the database you're connecting to within your instance of MS SQL Server. |
Authentication | Yes | Configure using the Authentication guidance shown above. |
Require encryption | Yes | Defaults to enabled. For more details, see Microsoft's documentation for the encrypt setting on the SQL Server JDBC driver: Connection properties reference ↗ Encryption support examples ↗ |
Trust server certificate | Yes | Defaults to disabled. For more details, see Microsoft's documentation for the trustServerCertificate setting on the SQL Server JDBC driver: Connection properties reference ↗ |
Network Connectivity | Yes | You must provide a runtime that is networked to your MS SQL Server instance. For instances behind a corporate firewall, you will normally need to use an agent runtime. For cloud-hosted instances, refer to the Networking section for more details. |
The Microsoft SQL Server source supports change data capture syncs.
To enable change data capture for Microsoft SQL Server, you must run a command like the one below to enable CDC on the database.
USE <database>
GO
EXEC sys.sp_cdc_enable_db
GO
Then, run another command on each table that should be recording changelogs:
EXEC sys.sp_cdc_enable_table
@source_schema = N'<schema>'
, @source_name = N'<table_name>'
, @role_name = NULL
, @capture_instance = NULL
, @supports_net_changes = 0
, @filegroup_name = N'PRIMARY';
GO
Once change data capture is enabled for the table(s) you wish to sync to Foundry, you can navigate to the Overview page and select + Create CDC sync to start creating a new change data capture sync.
The exploration runtime must be working in order to create a change data capture sync. If the runtime is still initializing, you may need to wait a few seconds and refresh the page to proceed with creating a change data capture sync.
For more information on these commands and using change data capture (CDC) with Microsoft SQL Server, see the official documentation ↗ for the version of SQL Server in use.