Foundry provides proxy endpoints for popular LLM providers, accepting requests in the same format as the providers' native APIs. This enables use of open-source SDKs and tooling while benefiting from Foundry capabilities such as rate limiting, data governance, and usage tracking.
The currently supported provider APIs and corresponding Foundry endpoints are as follows:
/api/v2/llm/proxy/anthropic/v1/messages/api/v2/llm/proxy/openai/v1/chat/completions/api/v2/llm/proxy/openai/v1/responses/api/v2/llm/proxy/openai/v1/embeddings[Beta] xAI chat completions ↗: /api/v2/llm/proxy/xai/v1/chat/completions[Beta] xAI responses ↗: /api/v2/llm/proxy/xai/v1/responses[Beta] Google generateContent ↗: /api/v2/llm/proxy/google/v1/models/{model}:generateContent[Beta] Google streamGenerateContent ↗: /api/v2/llm/proxy/google/v1/models/{model}:streamGenerateContent?alt=sseThe xAI and Google (Gemini) endpoints are currently in beta and actively being developed. Not all features or fields may be supported yet.
To use Palantir-provided language models, ensure that:
Authentication is sent using the following bearer token header:
Authorization: Bearer {FOUNDRY_TOKEN}
Requests to these endpoints should have the same shape as the corresponding provider endpoint. Refer to the provider’s documentation for the expected request format.
The Google streamGenerateContent proxy endpoint currently only supports the SSE response format, so you must provide the alt=sse query parameter.
Some providers, for example, Anthropic, use a non-standard authentication header. When using their SDKs, you may need to configure the authentication method to use a bearer token instead. Providers that already use bearer token authentication, such as OpenAI, require no special configuration.
These endpoints enforce the same data governance as other AIP usage, such as zero data retention (ZDR) and georestriction requirements. We selectively enable provider API features that are compatible with these requirements.
Only models and providers that have been enabled on your enrollment will be available through these endpoints. For models served by multiple providers, requests will only be routed to enabled providers. Endpoint usage is visible in the Resource Management application, and is subject to rate limiting.