The NFS connector is in the beta phase of development and may not be available on your enrollment. Functionality may change during active development.
Connect to Network File System (NFS) exports to sync data between folders and Foundry datasets. The connector supports NFS versions 3 and 4.1.
| Capability | Status |
|---|---|
| Exploration | 🟡 Beta |
| Bulk import | 🟡 Beta |
| Incremental | 🟡 Beta |
| File exports | 🔴 Not available |
The connector can transfer files of any type into Foundry datasets. File formats are preserved, and no schemas are applied during or after the transfer. Apply any necessary schema to the output dataset, or write a downstream transformation to access the data.
There is no limit to the size of transferable files. However, network issues can result in failures of large-scale transfers. In particular, direct cloud syncs that take more than two days to run will be interrupted. To avoid network issues, we recommend using smaller file sizes and limiting the number of files that are ingested in every execution of the sync. Syncs can be scheduled to run frequently.
To access on-premises NFS servers, we recommend using an agent proxy connection. Alternatively, you can use secure tunnels (currently still in beta).
Learn more about setting up a connector in Foundry.
The following configuration options are available for the NFS connector:
| Option | Required? | Default | Description |
|---|---|---|---|
Host | Yes | The domain name pointing to the server or the IP address of the server. | |
Export path | Yes | The exported filesystem path on the NFS server to connect to, for example /exports/data. | |
NFS version | Yes | The NFS protocol version to use. Review the NFSv4.1 settings and NFSv3 settings sections below for version-specific options. |
We recommend using NFSv4.1 whenever the NFS server supports it. NFSv4.1 supports RPC over TLS and Kerberos on a single port, which allows the whole connection to be secured. NFSv3 spreads its work across several RPC services that are secured separately and cannot protect the whole connection.
| Option | Required? | Default | Description |
|---|---|---|---|
NFS port | No | 2049 | The port on which the NFS server is running. |
Enable RPC over TLS | No | false | Secures all traffic using RPC-over-TLS (RFC 9289 ↗). |
Authentication | Yes | The RPC security flavor used to authenticate the connection, either Kerberos or AUTH_SYS. Review the Authentication section below for more information. |
If neither RPC over TLS nor the krb5p Kerberos protection level is enabled, file contents are sent over the network unencrypted. This is only safe if the full network path is trusted.
NFSv3 splits its work across four RPC services, each reachable on its own port and secured independently. Enabling TLS on one service leaves the others unencrypted.
| Option | Required? | Default | Description |
|---|---|---|---|
Access UID | Yes | The numeric user ID presented to the NFS server. | |
Access GID | Yes | The numeric group ID presented to the NFS server, also sent as the sole supplementary group. | |
Portmapper port | No | 111 | The TCP port of the portmapper (rpcbind) service, used to discover the port of any service that is not given a fixed port. Portmapper traffic is never encrypted. |
NFS service | Yes | The NFS program, which carries all file data. | |
Mount service | Yes | The MOUNT program, used to obtain the root file handle for the export. | |
Lock service | Yes | The NLM (network lock manager) program. | |
Status service | Yes | The NSM (network status monitor) program, used for lock recovery notifications. |
The following options are available for each of the four NFSv3 services:
| Option | Description |
|---|---|
Discover the port using the portmapper | When enabled, the connector asks the portmapper for the service's port when connecting instead of using a fixed port. |
Port | The fixed port the service is bound to on the NFS server. Only used if the port is not discovered through the portmapper. |
Wrap this service's traffic in TLS | Requires the NFS server to accept RPC over TLS on this service. |
Without TLS, NFSv3 sends file contents and the AUTH_SYS identity over the network unencrypted, and the MOUNT service transmits the root file handle and export metadata unencrypted. Ensure the network is fully trusted, enable TLS on the services your NFS server supports, or use NFSv4.1.
NFSv3 sources always authenticate with AUTH_SYS. NFSv4.1 sources can authenticate with either Kerberos or AUTH_SYS.
AUTH_SYS presents a numeric user ID (Access UID) and group ID (Access GID) to the NFS server. The server does not verify this identity, so access is governed entirely by the permissions of the export. The default identity 0/0 only grants root access on exports that are not configured to squash root access.
| Option | Required? | Description |
|---|---|---|
Access UID | Yes | The numeric user ID presented to the NFS server, between 0 and 4294967295. |
Access GID | Yes | The numeric group ID presented to the NFS server, between 0 and 4294967295. |
Kerberos authentication is only available for NFSv4.1 and uses a password to authenticate the principal.
| Option | Required? | Default | Description |
|---|---|---|---|
Realm | Yes | The Kerberos realm, for example EXAMPLE.COM. Realms are conventionally written in upper case. | |
KDC host | Yes | The hostname or IP address of the Kerberos key distribution center (KDC). | |
KDC port | No | 88 | The TCP port of the KDC. |
Principal | Yes | The Kerberos principal to authenticate as. The principal is combined with the realm to authenticate, for example nfsclient@EXAMPLE.COM. | |
Kerberos password | Yes | The password for the principal. | |
Protection level | Yes | The RPCSEC_GSS protection level. Review the table below for more information. |
The following protection levels are available:
| Protection level | Description |
|---|---|
Authentication (krb5) | Authenticates the connection only. Requests and file contents are neither integrity protected nor encrypted. |
Integrity (krb5i) | Authenticates the connection and integrity protects each request, but file contents are still sent unencrypted. |
Encryption (krb5p) | Authenticates the connection and encrypts the RPC payload, so file contents are protected in transit. |
Only Encryption (krb5p) is supported when RPC over TLS is enabled.
TLS requires the NFS server to support RPC over TLS as described by RFC 9289 ↗. If you enable TLS on a source whose NFS server does not support RPC over TLS, the connection will fail.
When TLS is enabled, the connector validates the NFS server's certificate and checks that the certificate's subject alternative name matches the configured Host. If the NFS server uses a self-signed certificate or a certificate issued by a private certificate authority, add the server certificate to the source. If the NFS server requires mutual TLS (mTLS), add a client certificate to the source.
You must add a network egress policy to allowlist the connection. The required egress policies depend on the NFS version:
Host on NFS port (2049 by default). When using Kerberos authentication, the connector must also be able to reach KDC host on KDC port (88 by default).Host on Portmapper port (111 by default) and on the port of each NFSv3 service. Services that discover their port through the portmapper are assigned a port by the NFS server when connecting, so their egress policies cannot be determined ahead of time. We recommend assigning a fixed port to each service on the NFS server and in the source configuration.Data Connection suggests the required egress policies based on the source configuration.
The NFS connector syncs files from the export into a Foundry dataset. Each sync walks the selected subfolder and every folder nested below it. Learn more about file-based ingestion modes.
| Option | Required? | Default | Description |
|---|---|---|---|
Subfolder | No | The directory below the export root to sync. Leave empty to sync the whole export. | |
Include patterns | No | Glob patterns matched against each file's path relative to the export root, for example *.csv. A file is synced only if it matches at least one of these patterns. Leave empty to include every file. | |
Exclude patterns | No | Glob patterns matched against each file's path relative to the export root, for example archive/**. A file matching any of these patterns is skipped, even when an include pattern also matches it. | |
Follow symbolic links | No | false | When disabled, symbolic links are skipped entirely. When enabled, links are resolved against the export root, and any target outside the export is skipped. |
Maximum depth | No | How many directory levels below the subfolder to descend. A value of 0 syncs only the files in the subfolder itself. Leave empty to descend without limit. | |
Minimum file size | No | Only sync files of at least this many bytes. | |
Maximum file size | No | Only sync files of at most this many bytes. | |
Modified after | No | Only sync files modified after this time. This filter is applied on every run, independently of incremental syncs. | |
Modified before | No | Only sync files modified before this time. | |
Order by | No | Traversal order | The order in which files that passed filtering are sorted before applying Maximum files per run. Options are Traversal order, Path, Size, and Last modified. |
Direction | No | Ascending | The sort direction for Order by. |
Maximum files per run | No | After filtering and sorting, keep at most this many files. Together with Order by, you can express conditions such as "the 100 oldest files" or "the 10 largest files". |
When incremental sync is enabled, each run records the modification time of the newest file it synced, and the next run only syncs files modified after that time. The first run syncs all files that match the sync configuration. Modification times are reported by the NFS server, so incremental syncs do not depend on the clocks of the connector and the NFS server being in agreement.
When an incremental sync uses Maximum files per run, files are always synced oldest first, and each run picks up where the previous run stopped. No files are skipped, including when the limit falls between files with identical modification times. Incremental syncs with a limit must leave Order by empty or sort by Last modified in Ascending order; other orderings cause the sync to fail.
Incremental syncs should use the APPEND or UPDATE transaction type. With the default SNAPSHOT transaction type, each run replaces the dataset with only the files found in that run.
The NFS connector does not support media set syncs. Files synced from NFS land in a regular dataset.