Search documentation
karat

+

K

User Documentation ↗

Create Streaming Dataset

POST/api/v2/streams/datasets/create
Warning

This endpoint is in preview and may be modified or removed at any time. To use this endpoint, add preview=true to the request query parameters.

Creates a streaming dataset with a stream on the specified branch, or if no branch is specified, on the default branch ('master' for most enrollments). For more information on streaming datasets, refer to the streams user documentation.

Third-party applications using this endpoint via OAuth2 must request the following operation scope: api:streams-write.

Query parameters

preview
boolean
optional

Enables the use of preview functionality.

Request body

CreateStreamingDatasetRequest
object
Hide child attributes

Hide child attributes

name
string
parentFolderRid
string

The unique resource identifier (RID) of a Folder.

schema
object

The Foundry schema to apply to the new stream.

Show child attributes

Show child attributes

branchName
string
optional

The branch to create the initial stream on. If not specified, the default branch will be used ('master' for most enrollments).

partitionsCount
integer
optional

The number of partitions for the Foundry stream.

Generally, each partition can handle about 5 mb/s of data, so for higher volume streams, more partitions are recommended.

If not specified, 1 partition is used.

This value cannot be changed later.

streamType
string (enum)
optional

A conceptual representation of the expected shape of the data for a stream. HIGH_THROUGHPUT and LOW_LATENCY are not compatible with each other. Defaults to LOW_LATENCY.

Enum values: LOW_LATENCY, HIGH_THROUGHPUT

compressed
boolean
optional

Whether or not compression is enabled for the stream. Defaults to false.

Response body

Dataset
object
Hide child attributes

Hide child attributes

rid
string

The Resource Identifier (RID) of a Dataset.

name
string
parentFolderRid
string

The unique resource identifier (RID) of a Folder.

Examples

Request

Copied!
1 2 3 4 5 curl -X POST \ -H "Content-type: application/json" \ -H "Authorization: Bearer $TOKEN" \ "https://$HOSTNAME/api/v2/streams/datasets/create?preview=true" \ -d '{"parentFolderRid":"ri.compass.main.folder.c410f510-2937-420e-8ea3-8c9bcb3c1791","partitionsCount":1,"streamType":"LOW_LATENCY","name":"My Dataset","branchName":"master","compressed":false}'

Response

Copied!
1 2 3 4 5 { "parentFolderRid": "ri.compass.main.folder.c410f510-2937-420e-8ea3-8c9bcb3c1791", "name": "My Dataset", "rid": "ri.foundry.main.dataset.c26f11c8-cdb3-4f44-9f5d-9816ea1c82da" }