Configure logging

Applications across Foundry emit logs that provide visibility into their operation, including transform runs, build failures, and other system events.

Foundry logs contain information about:

• Application and function execution events
• Transform and pipeline runs
• Build and deployment activities
• System errors and warnings

Additionally, any user-added logs written in Code Repositories will also be emitted.

These logs can be exported to a Foundry streaming dataset for real-time monitoring and analysis. This feature is only available to Organization Administrators, who can configure a destination where logs from Foundry applications will flow into a Foundry streaming dataset.

Export Foundry logs

Foundry logs can be exported, per-organization, directly into a Foundry streaming dataset.

After defining an export configuration, log entries will be continuously written to the streaming dataset from the time the configuration is created. You can process and analyze the data in real-time using Foundry's extensive suite of data transformation and visualization tools or export the logs to external monitoring systems.

Export permissions

To export Foundry logs, a user must be an Organization Administrator. This role provides the necessary permissions to configure log exporting destinations for the organization. See organization permissions for more details about organization-level permissions.

Additionally, sensitive details in logs are obfuscated and/or removed to protect user privacy and security.

Define an export configuration

Foundry logs can be exported from resources belonging to a set of projects, or from all resources in the organization. Some limitations apply when defining an export configuration:

• Each project can be included in at most three export configurations.
  • This limit includes configurations that export logs from all projects.
• When selecting a set of projects for a configuration, you can include up to 25 projects per configuration.

Follow the steps below to define an export configuration:

1. Navigate to the Control Panel application in Foundry.
2. In the top banner, select the relevant organization from the dropdown.
3. In the left-hand sidebar, select Log observability settings under the Organization section. Alternatively, in the home page, search for Log observability settings and select the relevant page in the search results.

4. Select Create log export, which will open a dialog to define a configuration.

5. You will see an option to switch between Filter by projects or All projects across spaces. When selecting projects to export relevant logs from, you can filter the search results by project name and space name.

6. Specify the name and location of the streaming dataset where logs will be exported. The location must be a location within the Foundry file system and cannot be a user-created folder.

7. Select Create configuration after reviewing the access requirements.

8. After the configuration is created, it will appear in the list of existing configurations.

Note that when first configuring a log export, it may take some time for logs to begin streaming into the dataset at the export location.

Disable log exporting

To disable log exporting, delete any log storage configurations listed by selecting the trash icon on the right side of the configuration. The resulting dataset will continue to exist, but no new logs will be exported to it.

Analyze log data

Log storage datasets can contain high volumes of streaming data, so we recommend filtering the dataset appropriately before performing analysis. Consider time-based filtering to focus on relevant log entries for your monitoring and troubleshooting needs.

Foundry provides many powerful tools for performing log analysis, such as Transforms and Pipeline Builder.

Log schema

Foundry application logs are structured logs that follow a consistent schema, making them suitable for programmatic analysis and monitoring.

Specifically, Foundry logs contain information about various application events, including transform executions, build processes, system errors, and other operational activities. The log structure varies depending on the type of application or service that generates the log entry.

Application-specific information is captured within structured fields that provide context about the operation being performed, including execution details, error information, and performance metrics.

Log attribution

Foundry application logs are generated by various services and applications within the platform. When logs are streamed to the configured dataset, they are filtered to include only logs relevant to the organization that configured the log storage configuration.

Log exports configured for a given organization will receive logs from applications and services used by that organization. This includes transform executions, builds, and other activities performed within the organization's context.

The RID of the resource that generated the log is included in the log entry and can be used to trace the log back to the resource.

Log guarantees

Foundry logs are not audit logs. There is no guarantee of 100% reliable log delivery.

Log contents produced by Foundry are subject to change without notice.