.. _dataset-broker:

****************************
Dataset Broker
****************************

The dataset_broker provides the endpoints ``register-state``, ``send-state``,
``register-dataset``, ``update-datasets``, ``request-state`` and ``status``. The
structure of the json request and reply data is as follows.

*/register-state* (POST)
========================
Receives a *datasetState* ID to be registered.

request:
    :hash:      ``state_id_t``. The *datasetState* ID.
reply:
    :result:    ``String``. Contains ``success`` or a description of the error.
    :request:   ``String`` (optional). Has the value ``get_state`` which states
                that the *dataset_broker* doesn't know this state and requests
                it to be sent to the endpoint :ref:`send-state`.
    :hash:      ``dset_id_t`` (optional). The *datasetState* ID of the requested
                state.

.. _send-state:

*/send-state* (POST)
======================
Receives a *datasetState*.

request:
    :hash:      ``dset_id_t``. The *datasetState* ID.
    :state:     ``json``. The :ref:`dataset-state`.
reply:
    :result:    ``String``. Contains ``success`` or a description of the error.

*/register-dataset* (POST)
============================
Receives a *dataset* to be registered.

request:
    :hash:      ``dset_id_t``. The *dataset* ID.
    :ds:        ``json``. The :ref:`dataset`.
reply:
    :result:    ``String``. Contains ``success`` or a description of the error.
    :request:   ``String`` (optional). Has the value ``get_state`` which states
                that the *dataset_broker* doesn't know this state and requests
                it to be sent to the endpoint :ref:`send-state`.
    :hash:      ``dset_id_t`` (optional). The *datasetState* ID of the requested
                state.

*/request-state* (POST)
========================
Replies with a *datasetState*.

request:
    :id:        ``state_id_t``. The *datasetState* ID.
reply:
    :result:    ``String``. Contains ``success`` or a description of the error.
    :state:     ``json``. The requested :ref:`dataset-state`.

*/update-datasets* (POST)
===========================
Replies with an update on the *dataset* topology: all *datasets* that are older
than the given timestamp and have one of the given roots. If The given *dataset*
has a root that is not among the list of given roots, all *datasets* with the
same root as the given *dataset* are included in the update additionally.

request:
    :ds_id:     ``dset_id_t``.  The *dataset* ID whose ancestors are requested.
    :ts:        ``json``.       Timestamp of the last update, generated by the
                                *dataset_broker* only.
    :roots:     ``List of
                  dset_id_t``   A list of all root *datasets* known by the
                  requesting instance.
reply:
    :result:    ``String``. Contains ``success`` or a description of the error.
    :ts:        ``json``.   Timestamp of this update. The recipient should store
                            this for the next update.
    :datasets:  ``json``.   A dictionary with keys of type ``dset_id_t`` and
                            a json-serialization of a :ref:`dataset` as values.

*/status* (GET)
===================
Replies with the status of the *dataset_broker*.

reply:
    :states:    List of ``state_id_t`` describing the registered states.
    :datasets:  List of ``dset_id_t`` describing the registered datasets.


JSON serializations of *dataset* and *datasetState*
====================================================

.. _dataset:

*dataset*
------------
:base_dset:     ``dset_id_t`` (optional). The ID of the *base dataset*. Only
                necessary for non-root datasets.
:state:         ``state_id_t``. The ID of the *datasetState* that describes the
                difference to the *base dataset*.
:is_root:       ``bool``. ``True`` if this is a root *dataset*, otherwise
                ``False``.

.. _dataset-state:

*state*
---------
:type:           ``String``. Typeid name of the *datasetState*.
:data:           ``json``. JSON serialization of the data of the state.
:inner:          ``json``. JSON serialization of any inner state.

Synchronization example
====================================================

This shows an example of the synchronization between *datasetManagers* of two
kotekan instances and a *dataset_broker*.

.. uml::

    @startuml
    skinparam BoxPadding 10

    box "kotekan instance"
    participant datasetManager1
    end box

    box "broker"
    participant dataset_broker
    end box

    box "kotekan instance"
    participant datasetManager2
    end box

    box "someone using curl"
    participant curl
    end box

    group Register a new state.

        datasetManager1 --> dataset_broker: register-state(request)
        note over dataset_broker
            Doesn't know this state,
            requests it.
        end note
        datasetManager1 <-- dataset_broker: reply
        datasetManager1 --> dataset_broker: send-state(request)
        datasetManager1 <-- dataset_broker: reply
    end

    group Register a new root dataset
        datasetManager1 --> dataset_broker: register_dataset(request)
        datasetManager1 <-- dataset_broker: reply
    end


    group Register the same state.
        datasetManager2 --> dataset_broker: register_state(request)
        datasetManager2 <-- dataset_broker: reply
    end
    group Register the same dataset.
        datasetManager2 --> dataset_broker: register_dataset(request)
        datasetManager2 <-- dataset_broker: reply
    end

    group Register a new state.
        datasetManager1 --> dataset_broker: register-state(request)
        note over dataset_broker
            Doesn't know this state,
            requests it.
        end note
        datasetManager1 <-- dataset_broker: reply
        datasetManager1 --> dataset_broker: send-state(request)
        datasetManager1 <-- dataset_broker: reply
    end

    group Register a new dataset.
        datasetManager1 --> dataset_broker: register_dataset(request)
        datasetManager1 <-- dataset_broker: reply
    end

    note over datasetManager2
        Finds an unknown dataset.
    end note

    group Request update on dataset topology.
        datasetManager2 --> dataset_broker: update-datasets(request)
        datasetManager2 <-- dataset_broker: reply
    end

    group Request state.
        datasetManager2 --> dataset_broker: request-state(request)
        datasetManager2 <-- dataset_broker: reply
    end

    group Request another state.
        datasetManager2 --> dataset_broker: request-state(request)
        datasetManager2 <-- dataset_broker: reply
    end

    group Status request.
        curl --> dataset_broker: status(request)
        curl <-- dataset_broker: reply
    end

    @enduml