Arvados

h1. Websocket server

(early draft)

{{toc}}

h2. Background

The Rails API server can function as a websocket server. Clients (notably Workbench, arv-mount, arv-ws) use it to listen for events without polling.

Problems with current implementation:

* Unreliable. See #9427, #8277

* Resource-heavy (one postgres connection per connected client, uses lots of memory)

* Logging is not very good

* Updates look like database records instead of API responses (e.g., computed fields are missing, collection manifest_text has no signatures)

* Offers an API for catching up on missed events after disconnecting/reconnecting, but this API (let alone the code) isn't enough to offer a "don't miss any events, don't send any events twice" guarantee. See #9388

#8460

h2. Desired features

Monotonically increasing event IDs, so clients can (meaningfully) request "all matching events since X"

h2. Design sketch

New server, written in Go.

One goroutine per connected client.

One database connection receiving notifications about new logs. (Possibly still N database connections serving "catch-up" messages to N clients.)

h2. Libraries

Websocket:

* https://godoc.org/golang.org/x/net/websocket

PostgreSQL:

* https://godoc.org/github.com/lib/pq via https://godoc.org/database/sql

* https://godoc.org/github.com/lib/pq#hdr-Notifications and https://godoc.org/github.com/lib/pq/listen_example

h2. Obstacles

#8565, #8566