Federated sources basics

If there are third party data sources (federated source systems) configured, you can query them through the Gotham API. To do this, you can first query the metadata of the available federated sources. Each federated source has a name, which is its identity, and one or more namespaces. A namespace represents a single way the data source can be queried. Its metadata contains a query shape, and queries against the datasource must use one of these query shapes.

The query shapes contain at least a term filter. The exact semantics of this filter depend on the source system: it can be an exact equality filter, or a fuzzier search. One or more of these term components can be combined with not, and and not filters.

As an example, when you list the federated source metadata by issuing a GET request to api/gotham/v1/federatedSources, you could get a response like this:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
{
    "data": [
        {
            "name": "wiki",
            "displayName": "Wikipedia System",
            "namespaces": [
                {
                    "name": "pages",
                    "displayName": "Wikipedia Pages searcher",
                    "queryShape": {
                        "type": "term",
                        "field": "query",
                        "value": "<Query>"
                    }
                }
            ]
        }
    ]
}

This tells us that we have a source system called wiki, which has a namespace called pages. The pages namespace can be queried using a term filter on the query field. The value of the query field is a string, which is the query string to search for.

To perform the search, you issue a POST request to federatedSources/wiki/namespaces/pages/objects/search with a body like this:

Copied!
1
2
3
4
5
6
7
{
  "query": {
    "type": "term",
    "field": "query",
    "value": "Palantir"
  }
}

The result would look similar to this:

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
{
    "data": [
        {
            "primaryKey": "ri.gotham.1234567890-0.object-external.wiki.pages.The_5FFellowship_5Fof_5Fthe_5FRing",
            "title": "The Fellowship of the Ring",
            "objectType": "com.palantir.object.wikipediaarticle",
            "properties": {
                "com.palantir.property.url": [
                    "https://en.wikipedia.org/wiki/The_Fellowship_of_the_Ring"
                ],
                "com.palantir.property.documenttitle": [
                    "The Fellowship of the Ring"
                ],
                "com.palantir.property.revision": [
                    "1168257867"
                ],
                "com.palantir.property.description": [
                    "wikipedia article"
                ]
            }
        },
        ...
    ],
  "nextPageToken": "+%21%7B%22totalHits%22%3A434%2C%22cursor%22%3A%7B%22pagingToken%22%3A%22101%22%2C%22offset%22%3A0%7D%2C%22checksum%22%3A%22kblv_-HCl3nfXrmy%22%7D"
}

You can treat this object similar to a native RevDB object in the API: you can list its properties and links, for example. However, since this is a federated object, you cannot perform any updates on it.