Arvados

h1. Events API

(draft)

{{toc}}

See also: [[Websocket server]]

h1. Purpose

The Events API serves to notify processes about events that interest them _as soon as possible after those events happen_.

(The history of events that have happened in the past is also interesting, but that's addressed by the Logs API, not the Events API.)

h1. Conceptual model

An event reports a change to the state of an object.

The fact that an object's state has changed is meaningful only when its previous state is known. For example, if a client asks "tell me the next time object X changes" at nearly the same time X changes, the response depends on whether the request arrives before or after the change occurs.

Therefore, the Events API should support operations like:

* "tell me the current state of X, and then notify me next time it changes"

* "tell me as soon as X differs from my cached copy that has Etag E"

An "event stream" is a sequence of events about an object, starting from an implicit or explicit known state.

h1. Essential features

h2. Multiple streams

The Events API supports multiplexing event streams on a single connection. The cost of setting up and maintaining an event channel can be non-trivial, and the sequence of events concerning multiple related objects may be significant.

It is possible to add and remove event streams on an existing connection, without interrupting other streams.

It is permitted to hold a connection open with no event streams, but the server may close such connections after some time threshold.

h2. Delivery guarantees

In general, the Events API cannot guarantee that every event will be delivered.

However, there are specific cases where it is possible to detect missed events and notify the client.

If some events are dropped but the event stream is still open (for example, a server-side buffer overflows when a client is receiving data too slowly) the stream must indicate this to the client as soon as possible, and no later than the next event.

* The "missed events" signal does not necessarily specify the number of missed events.

* The server is permitted to send a "missed events" signal even if no events were missed.

Depending on the application, a client might respond to a "missed events" signal by

* restarting the affected streams immediately

* restarting the affected streams only if they stay silent for some timeout period

* doing nothing

* hanging up

h2. Event message content

Each event includes the UUID and Etag of the changed object.

If requested by the client, and supported by the server, events may also include other object attributes.

h1. Additional features

h2. Event sequence

With the current API server, it may be possible to update an object twice in quick succession such that the modification timestamps are out of order: i.e., the current state of object X has modification time T1, even though the same object previously had modification time T2>T1. If this occurs, the Events API must return the T2 update before the T1 update (or not return the T2 update at all).

In order to support delivery mechanisms where messages are re-ordered in transit, the Events API should assign a strictly increasing integer ID to each event sent over a given connection. Client pseudocode:

<pre>

receiveEvent(id, uuid, newEtag):

  if lastID[uuid] > id:

    # already received a newer update for this object

    return

  currentEtag[uuid] = newEtag

  lastID[uuid] = id

</pre>

Note these IDs are connection-specific: they cannot be used to reconnect and resume an event stream.

h2. Server-side event filters

Some clients will only be interested in a subset of possible changes. For example, a pipeline runner wants to know as soon as a container's "state" attribute changes, but might not care about other changes like "priority" or "progress".

Possible API features for reducing unnecessary work and network traffic:

# Allow clients to describe which attributes are interesting, e.g., @"select":["state"]@

# With each event, provide the list of changed attributes, e.g., @"changed":["state","output","log"]@, but not the attribute values themselves

These features might be tricky to implement efficiently for attributes that are computed on the fly. 

h2. Null stream

To simplify implementation of clients that subscribe to event streams but also retrieve some objects without listening for events, a client should be able to use the Events API to retrieve the current state of an object without subscribing to the object's event stream.

h2. Ownership-change events

Some clients need to know when an object is added or removed from a project.

When an object's owner_uuid changes, this event should be sent to:

# all clients subscribed to the object itself

# all clients subscribed to the old owner_uuid

# all clients subscribed to the new owner_uuid

Likewise, subscribing to stream X should cause clients to receive messages when a new object is created with owner_uuid=X, and when an object with owner_uuid=X is deleted.