Websocket server¶

(early draft)

• Table of contents
• Websocket server
  • Background
  • Desired features
  • Design sketch
  • Libraries
  • Obstacles
  • Related

Background¶

The Rails API server can function as a websocket server. Clients (notably Workbench, arv-mount, arv-ws) use it to listen for events without polling.

Problems with current implementation:

• Unreliable. See #9427, #8277
• Resource-heavy (one postgres connection per connected client, uses lots of memory)
• Logging is not very good
• Updates look like database records instead of API responses (e.g., computed fields are missing, collection manifest_text has no signatures)
• Offers an API for catching up on missed events after disconnecting/reconnecting, but this API (let alone the code) isn't enough to offer a "don't miss any events, don't send any events twice" guarantee. See #9388

#8460

Desired features¶

Monotonically increasing event IDs, so clients can (meaningfully) request "all matching events since X"

Design sketch¶

New server, written in Go.

One goroutine per connected client.

One database connection receiving notifications about new logs. (Possibly still N database connections serving "catch-up" messages to N clients.)

Libraries¶

Websocket:

• https://godoc.org/golang.org/x/net/websocket

PostgreSQL:

• https://godoc.org/github.com/lib/pq via https://godoc.org/database/sql
• https://godoc.org/github.com/lib/pq#hdr-Notifications and https://godoc.org/github.com/lib/pq/listen_example

Obstacles¶

#8565, #8566

Related¶

It might be expedient to offload synchronization to some existing software that does this well.

• Apache Zookeeper -- "Coordination services are notoriously hard to get right. They are especially prone to errors such as race conditions and deadlock. The motivation behind ZooKeeper is to relieve distributed applications the responsibility of implementing coordination services from scratch."
• Google Chubby paper