Retrieval context

An AIP Agent with retrieval context will deterministically fetch information from data sources with every new user message to pass into the LLM. This is compatible with a single-completion prompting strategy which has a faster time to first token.

You may configure your agent with one of three types of retrieval context:

• Ontology context
• Document context
• Function-backed context

Ontology context

Ontology context allows users to perform semantic search over an object set with each message sent and provides the results of the search as additional context for the LLM to use in its answer. Ontology context differs from the ontology semantic search tool as it is not applied conditionally (the tool may not be used if the LLM decides it is not relevant) and the tool offers an additional ability to call a function to return the semantically similar objects instead of configuring a vector property.

After selecting Ontology context, configure the object set you would like to perform a semantic search across.

We currently only support an object set of one object type and that object type must contain a vector property on which to perform the semantic search. That object set could be an entire object type ("Static input") or an object set application variable ("Application variable").

The max number of objects passed into the LLM as context can be configured with a value ranging from 1 to 100 with a default value of 5. You can also configure from 1 to 5 content properties to pass into the LLM as additional context.

During a session, we render the content properties with links to the object in Object Explorer. This is visible in the collapsible context message between the sent message and received message.

Document context

Document context allows users to include relevant text from documents with each message sent to the LLM. Documents can be selected and included in the configuration of an AIP Agent in the same way they are added to a conversation in AIP Threads. Select Document context > Select documents or pick from Recently uploaded documents.

Function-backed context

Function-backed context enables users to perform their own retrieval on each query. This is ideal for situations where the retrieval methods provided out-of-the-box through Ontology context or Document context do not satisfy a given use case. For example, if a user wants to run a semantic search over multiple Ontology object types, they can write a function to do this, since it is not currently supported in Agent Studio.

Users can write these functions in TypeScript in the Code Repositories application. To do so, navigate to a TypeScript repository and import the AipAgentsContextRetrieval function interface.

Then, write a function that satisfies the interface as shown below. Note that the function must have messages as the only required input in order to satisfy the contract.  

Copied!
1
2
3
4
5
6
7
8
9
10
@AipAgentsContextRetrieval()
public exampleRetrievalFunction(messages: MessageList): RetrievedContext {
    let combinedText: string[] = [];
    messages.forEach((message) => {
        ...
    })
    return {
        retrievedPrompt: "..."
    }
}

The retrieval function must output a retrievedPrompt string, which will be pasted into the LLM system prompt by AIP Agents to answer user queries.

After publishing your function, choose Function-backed context in Agent Studio under the Retrieval context panel to select a function for retrieval.

Application variables in retrieval functions

Retrieval functions can also take in the values of application variables on the agent as input. To configure this, add optional arguments to the function definition. Currently, only string and object set application variables are supported, so the function input must be one of these types. 

Copied!
1
2
3
4
@AipAgentsContextRetrieval()
public movieRetrievalFunction(messages: MessageList, movieTitle?: string, movieSet?: ObjectSet<Movie>): RetrievedContext {
    ...
}

Use the API name for object types. This can be found in Ontology Manager. You can then configure a mapping between the application variables on the agent and the function inputs that match their respective types.

To create application variables, navigate to the Application variables panel in Agent Studio.

Write retrieval functions in AIP Logic

Users will soon be able to write these functions in AIP Logic, which offers a walk-up usable interface for developing no-code LLM-powered functions. To leverage retrieval functions in the meantime, we recommend writing a TypeScript function that satisfies the interface and calls the Logic function under the hood.

Custom citations

The AIP Agent interface renders citation bubbles if the LLM responds with citations in specified formats. With Function-backed context, users can take advantage of this in their functions by returning a string that prompts the LLM to write citations in a given format. The following formats are currently provided:

• Ontology object citations should be returned in the following format:

  <citation><key>ri.phonograph2-objects.main.object....</key></citation>
  

  where the <key></key> tags encapsulate the object RID. Selecting the citation bubble will link to the Object Explorer view for that object.

• Document (PDF) citations should be returned in the following format:

  <citation><mediaSetKey>ri.mio.main.media-set...</mediaSetKey><mediaItemKey>ri.mio.main.media-item...</mediaItemKey></citation>
  

  Selecting the citation bubble will open a dialog that displays the first page of the document.

  • If you would like to specify a page of the document in your citation, you can provide a page tag like so:
    
    

  <citation><mediaSetKey>ri.mio.main.media-set...</mediaSetKey><mediaItemKey>ri.mio.main.media-item...</mediaItemKey><page>12</page></citation>
  

  The document dialog will then display that page of the document.

  • You may also specify multiple pages in your citation by providing a value for page, like 5, 8 (pages 5 and 8), 5-12 (pages 5 through 12), or 5-8, 12 (pages 5 through 8 and page 12). The document preview dialog will show the first page provided in the citation; in each of these examples, this would be page 5.

• External URL citations should be returned in the following format:

  <citation><name>My Website</name><href>www.mywebsite.com</href></citation>
  

  The citation bubble will display the provided name, such as My Website, and selecting it will link to the provided URL, such as www.mywebsite.com.

In this example, the function accepts a set of document chunks that are represented as Ontology objects. It then conducts a semantic search on these objects and returns the five most relevant ones, formatted according to the citation style mentioned above.