Dataset Broker

The dataset_broker provides the endpoints register-state, send-state, register-dataset, update-datasets, request-state and status. The structure of the json request and reply data is as follows.

/register-state (POST)

Receives a datasetState ID to be registered.

request:

hash:

state_id_t. The datasetState ID.

reply:

result:

String. Contains success or a description of the error.

request:

String (optional). Has the value get_state which states that the dataset_broker doesn’t know this state and requests it to be sent to the endpoint /send-state (POST).

hash:

dset_id_t (optional). The datasetState ID of the requested state.

/send-state (POST)

Receives a datasetState.

request:

hash:

dset_id_t. The datasetState ID.

state:

json. The state.

reply:

result:

String. Contains success or a description of the error.

/register-dataset (POST)

Receives a dataset to be registered.

request:

hash:

dset_id_t. The dataset ID.

ds:

json. The dataset.

reply:

result:

String. Contains success or a description of the error.

request:

String (optional). Has the value get_state which states that the dataset_broker doesn’t know this state and requests it to be sent to the endpoint /send-state (POST).

hash:

dset_id_t (optional). The datasetState ID of the requested state.

/request-state (POST)

Replies with a datasetState.

request:

id:

state_id_t. The datasetState ID.

reply:

result:

String. Contains success or a description of the error.

state:

json. The requested state.

/update-datasets (POST)

Replies with an update on the dataset topology: all datasets that are older than the given timestamp and have one of the given roots. If The given dataset has a root that is not among the list of given roots, all datasets with the same root as the given dataset are included in the update additionally.

request:

ds_id:

dset_id_t. The dataset ID whose ancestors are requested.

ts:

json. Timestamp of the last update, generated by the dataset_broker only.

roots:

List of dset_id_t A list of all root datasets known by the requesting instance.

reply:

result:

String. Contains success or a description of the error.

ts:

json. Timestamp of this update. The recipient should store this for the next update.

datasets:

json. A dictionary with keys of type dset_id_t and a json-serialization of a dataset as values.

/status (GET)

Replies with the status of the dataset_broker.

reply:

states:

List of state_id_t describing the registered states.

datasets:

List of dset_id_t describing the registered datasets.

JSON serializations of dataset and datasetState

dataset

base_dset:

dset_id_t (optional). The ID of the base dataset. Only necessary for non-root datasets.

state:

state_id_t. The ID of the datasetState that describes the difference to the base dataset.

is_root:

bool. True if this is a root dataset, otherwise False.

state

type:

String. Typeid name of the datasetState.

data:

json. JSON serialization of the data of the state.

inner:

json. JSON serialization of any inner state.

Synchronization example

This shows an example of the synchronization between datasetManagers of two kotekan instances and a dataset_broker.