Announcements

Announcing REST API Source with support for Webhooks

Date published: 2023-05-26

REST API webhooks are now generally available. You can now build webhooks using a direct cloud connection and with a dedicated user interface for building requests. Additionally, agents are also supported for REST API sources, and separately, you may specify multiple domains if you wish to construct chained webhook requests that are executed consecutively. The new source shows up as "REST API" on the Source Creation page and an in-platform tutorial explains how to use webhooks with this source type. The webhooks tutorial has been updated, along with a disambiguation page covering all options for REST connections.

Note that direct-connection for the REST API source requires network egress management via control panel to be enabled on your environment. Contact your administrator for any questions regarding egress management for your environment.

Use REST API webhooks

To use a REST API webhook, start by creating a “REST API” source in Data Connection. Then, provide the base URL and auth mechanism, along with the egress or agent configuration. In addition to taking advantage of the new support for cloud runtime and direct connection, you may also consider using agents or as necessary, specify multiple domains to construct chained webhook requests that are executed consecutively.

After creating a source, you can use the new request builder to construct webhooks, which can then be wired up to Actions or Workshop. The request builder supports:

• Parameterization of relative URL path, headers, query parameters, and body.
• Body provided as form input, form-url-encoded, raw JSON, or binary attachment.
• Test connections from the Data Connection UI.
• Chained requests, including extracting values from the response of a request to be used in subsequent requests.
• OAuth2 authorization using outbound apps. Contact your Palantir representative to enable.

Syncs, exports, and other capabilities that involve REST APIs

Currently, the REST API source type only supports webhooks. If you are using webhooks, you should use the new REST API source.

• For conducting syncs or exports to an internet-accessible source, you should use External Transforms.
• Otherwise, for syncs or exports to an on-premise source, continue using magritte-rest-v2.

Code Workspaces: The power of Foundry, now accessible with third-party IDEs

Date published: 2023-05-26

With Code Workspaces, data scientists, analysts, statisticians, and ML engineers can use the JupyterLab® and RStudio® Workbench IDEs in combination with the benefits of the Palantir Foundry platform. Code Workspaces brings the power of the Foundry Ontology and Palantir's first-class security, branching, and resource management capabilities to data science and machine-learning workflows.

Foundry security with familiar third-party IDEs

Code Workspaces creates an experience that is familiar for users while maintaining Foundry’s robust security features. Instead of downloading data to a local notebook or manually building connectors from a data lake to your HPC or SaaS ML platform, you can now work on your data directly in Foundry and decrease data transfer complexity and the risk of data spillage. The third-party IDEs supported by Foundry are bound to the core components of Foundry security: restricting access to a dataset in Foundry will restrict access for Code Workspaces IDEs as well, ensuring consistent permissions across tools.

Foundry users can now read and write data from Foundry using JupyterLab® or RStudio® Workbench, install packages, and leverage version control. This integration enables data lineage and branching for dashboards along with purpose-built enterprise-level privacy and security features. Production and development workflows can now be performed in the same platform in isolation, accelerating operationalization: for instance, you can build a dashboard on one branch using that branch's data, and merge into the main branch once the dashboard has been properly validated.

Data science with the power of Foundry

Foundry users can use Code Workspaces to collaborate, iterate, and build faster to get to workflow operationalization. Code Workspaces also supports Python- and R-related dashboard and workflow building.

By hosting everything within Foundry, the collaboration experience is smooth for teams that may be working on one code base at any time. Similar to the Git workflow, users with various technical backgrounds can propose changes on branches based on permissions. All users can benefit from the data lineage for dashboards view and a polished edit-concurrency experience. Furthermore, data scientists can benefit from intuitive package and resource profile management in the unified Code Workspaces interface.

When should I use Code Workspaces?

Code Workspaces gives platform administrators an easily-deployed, fully-managed, secure, and production-ready way to provide JupyterLab® and RStudio® Workbench to users with Foundry’s data governance and compliance with FedRAMP, GxP, and other standards built in. With Code Workspaces, users can securely connect to existing internal systems and build analyses, models, dashboards, or entire workflows on data with Foundry’s access controls and data permissioning.

With the introduction of Code Workspaces, operationalization is just a step away. Using familiar IDEs, you can now consume and publish models to the Ontology, and build governance-compliant dashboards developed on branches that can be automatically refreshed when new data is available.

To start building, review the Code Workspaces documentation.

---

RStudio® and Shiny® are trademarks of Posit™.

Jupyter®, JupyterLab®, and the Jupyter® logos are trademarks or registered trademarks of NumFOCUS.

All third-party trademarks (including logos and icons) referenced remain the property of their respective owners. No affiliation or endorsement is implied.

Object Storage v2 is Generally Available

Date published: 2023-05-26

Palantir is proud to announce the general availability of Foundry's next generation ontology architecture. Object Storage v2 (OSv2) is a total re-architecting of the backend infrastructure that powers the Foundry Ontology. OSv2 is meant to replace the Object Storage V1 (Phonograph) architecture in the long term and provides flexibility to back object storage with multiple data storage types in parallel to unlock query patterns, performance, or scale considerations as Foundry evolves.

OSv2 brings many new features to Foundry along with significant improvements in scale and performance and an architecture to support ongoing growth and investment in the Foundry Ontology.

Scale, performance, and latency improvements

Object Storage v2 comes with a number of improvements to scale, performance, and latency in Foundry:

• Significantly improved performance for Ontology data indexing through incremental object indexing for all object types by default.
• Increased indexing scale on the order of tens of billions of object instances for a single object type.
• Increased user edit throughput, enabling >10,000 objects to be edited in a single Action.
• Reduced user edit latency, faster observation of user edits in frontend applications, and near real-time automated propagation of user edits to materialized object datasets.

New capabilities

New capabilities in OSv2 include:

• The ability to migrate existing user edits after a breaking schema change in an object type definition.

• More granular object permissions with multi-datasource object types, supporting column/property level permissions.

Other new capabilities include:

• Automated module refresh support in Workshop when explicitly configured object types receive new data or user edits.
• Limitless paging endpoints to enable Ontology consumers to load more than 10K objects, particularly valuable for application developers interacting with Foundry Ontology through APIs.

Migrating to OSv2

Foundry offers a robust self-service migration tooling in Ontology Manager for selectively migrating object types from Object Storage V1 (Phonograph) to OSv2 without any downtime. Object Storage V1 (Phonograph) is not deprecated or removed, though OSv2 is recommended for any new object types due to its superior feature set and long-term product roadmap.

Read more in the Object Storage v2 documentation.

Consumer-only mode for Foundry access

Date published: 2023-05-26

Configure a simplified experience for single-application users with Foundry's new consumer-only mode. Foundry now supports a robust configuration to constrain the experience of certain user groups to only the custom applications to which they've been given permission. In this restricted view, the navigational sidebar is not shown because those users are not meant to leave those particular applications.

In the Control Panel Foundry Suite configuration, there is now a configuration to Restrict platform access. By default, this configuration is set to "Everyone in this organization", which maintains the current behavior.

Alternatively, platform access can now be given only to specific groups leaving other users with access to only the consumer-facing applications built in Slate or Workshop to which they have explicitly been granted access. Without platform access, these users will no longer see a Foundry sidebar nor will they be able to navigate to other parts of Foundry.

This configuration makes it possible to provide a truly stand-alone experience for custom applications for users who do not need to access the entirety of Foundry. For more information and guidance to configure this type of restricted access, review the Control Panel Workspaces documentation.

Introducing Quicksearch: Fast search and navigation

Date published: 2023-05-11

Quicksearch is a new tool for search, navigation, and discovery, built to make finding resources and navigating throughout Foundry fast and delightful. There are two new modes for Quicksearch interactions:

1. Jump-to mode provides a short list of personalized results based on recent history to directly navigate Foundry and switch between platform applications, custom applications, objects, datasets, and other resources.
2. Full results mode helps find content and discover what exists in Foundry. Perform searches of Foundry apps, objects, datasets, and other resources with advanced filters, rich metadata, and improved result ranking to highlight the most relevant results.

Jump-to mode

Navigate Foundry at the speed of thought without ever reaching for the mouse using Quicksearch keyboard shortcuts and jump-mode:

• Easy to access: Open Quicksearch by selecting Search... in the Foundry sidebar or with the keyboard shortcut *Ctrl+J (Windows) or Cmd + J (macOS).
• Easy to access: To open Quicksearch, select Search... in the Foundry sidebar, or use Ctrl + J on Windows or ⌘ + J on macOS.
• Navigation-focused: The initial dialog and dropdown are navigation-oriented. Start typing to receive suggestions. Use the keyboard to jump to any Foundry page in seconds.
• Search only on titles: Jump-to mode only searches on titles of apps, resources, and objects. To search additional fields and metadata, use the full results mode.
• Single click to full results: From the dialog and dropdown, press Enter/Return or filter by Apps, Objects, Datasets, or Files to move to advanced search mode.
• Personalized results: Results in Quicksearch are personalized by your recently accessed or favorited resources, and more.

Full results view

Explore Foundry data, applications, and resources to uncover relevant work products:

• Discovery-focused: Designed for searching and discovering, users can compare multiple search results to find the resources most useful to them.
• Search on various fields: Search for descriptions, column names, file paths, and more. Additionally, filter by creator, tags, Projects, folders, and more.
• Tabs: The interface includes a top results tab and four tabs to filter by Apps, Objects, Datasets, or Files
• Metadata: Users can view information on each result to evaluate whether the resource is relevant to them. Search by keyword highlights, file path, view count, last updated time, key security metadata, and more.
• Ranking: Results are ranked based on an algorithm that contains text matching and other boost parameters.
• Filters: Filters are provided for advanced searches. Additionally, by selecting the filter tabs from the initial search dropdown, users can perform complex filter searches ( for example, “all reports under project X that were created by me”).
• Permissions: Quicksearch respects all existing permissions in Foundry. Content with Discover permissions would open a Request access message. Users will not see content to which they do not have access.

Introducing Applications Portal

Date published: 2023-05-11

Applications Portal is a new tool for discovery and quick access to all apps in Foundry, including core Foundry platform apps and trusted custom apps promoted to the portal by platform administrators.

The Application Portal is launched from the Foundry sidebar and opens to a view of all applications, with additional categories for navigating both platform apps and promoted custom applications. Favoriting an application with a ⭐ will pin the application to the sidebar for quick access.

Curate the Applications Portal

The Applications Portal has two engagement modes:

• Basic (out-of-the-box): A tool to discover and learn about all Foundry platform applications. No admin work is required. The applications portal is an integral tool in the left sidebar, even without any promoted custom apps.
• Advanced (curation-based): An admin, Palantir representative, or customer governance team promotes all trusted custom apps to the Applications Portal.

For more details on the permissions and workflow for promoting applications, review the Applications Portal documentation.

Introducing Approvals: Platform change management

Date published: 2023-05-04

Approvals introduces new compliance, governance, and peer-review workflows in Foundry

Manage requests, approvals, and platform changes with the Approvals application. With a focus on increasing time-to-value, Approvals facilitates compliance, governance, and peer-review workflows for your organization by allowing you to set up a review process while providing an easily-accessible audit history.

Manage requests for permission changes in one inbox

The Approvals application allows administrators to set up user requests consisting of one or multiple tasks (individual actions) in Foundry for review by respective approvers, all from a common inbox. Users can select from a list of tasks for approval by the appropriate reviewers. Additionally, users are encouraged to select an existing group to join when making a Project access request, optimizing for an administrator’s ability to centrally organize permissions management through Group settings under Platform settings.

Once all tasks under a request are approved, Foundry completes the request automatically, saving administrators the need to perform a manual adjustment to apply the decision and the effort required to verify the correct setting was completed.

Approvals not only addresses the specific issue of collecting and managing Project access requests, but also lays a foundation for centralizing other approval workflows for access and change management processes currently distributed throughout Foundry.

Contextual decisions from rich UI

Using Approvals, Organizations obtain greater insight into the lineage for data access decisioning for audit purposes. Furthermore, reviewers, such as administrators and Project owners, can be assigned to specific tasks that relate to their area of oversight and responsibility.

When a request is filed by a user, in-platform notification and emails are automatically dispatched to the responsible reviewer. Requests are considered directly from within the Approvals user interface, and where necessary, reviewers can leave a comment or ask for clarification on specific tasks for a user’s response. Upon the approval of a request, Foundry enacts the change automatically without needing additional manual changes.

When should I use Approvals?

Approvals is designed for Organizations that require a clear process for users to obtain approval to resources with the permissions being automatically granted upon administrator or owner review. Additional Approvals request types such as Ontology proposals and reference additions are slated for future release.

The Approvals application is enabled on all stacks by default. Where custom workflows exist, we highly recommend adapting them to the Approvals application to take advantage of its improved workflow implementation.

Review the documentation on setting up approvals to get started.

---

Additional highlights

Analytics | Contour

Board titles | You can now give Contour boards a title to help identify them.

Data Integration | Pipeline Builder

Introducing approvals in proposals | Users can now choose to protect production critical Main branches in Pipeline Builder by requiring user approval before adopting a change. To do so, pipeline owners can enable the setting in Branches setting for the selected pipeline. Following standard best practices, only editors that did not contribute to the proposal may approve the change.