Skip to main content

Clients keep a replica, synced through endpoints of its own

Mission Control is a person's daily task manager (ADR 0029) and soon an Android app, which must stay useful on a phone that is often offline and feel as quick as a local app. The web app read every list from the server and polled it. We decided that a client keeps a replica: a local SQLite database of what a person needs day to day, the organizations they are in with their people, roles, projects, tasks, the links between them and comments, and their own conversations and messages. Screens read the replica and change it at once; each change also waits in an outbox as an operation, which reaches the server when it can. The server has sync endpoints of their own for this: one says what changed since a client last looked, one applies the operations a client pushes, and a stream says when to look again. The web app and the Android app share one client library for it (@ai/mc-store), so they share one data model and differ in their interaction and UI.

Considered Options​

  • A cache of API responses kept in the browser, as TanStack Query can persist: rejected. A cache of responses cannot be queried for a list it was not asked for, has no place for something created offline, and cannot tell from the endpoints what was deleted.
  • Syncing over the endpoints the API already has, such as tasks updated since a time: rejected. Deletions, a person losing access, and records that change through others (a comment's files, a message's task) are invisible to it, and every list would still be polled.
  • Pushing row changes, such as "set these fields on this task": rejected. A transition carries a reason and follows rules about who may move what; an operation is a request the API already takes, applied by the same service, so its checks (ADR 0001) and events (ADR 0003) apply as they do to the web app's own requests.
  • Recording changes from the services, or from ADR 0003's events: rejected. Those events are in-process and published after commit, so a crash loses a change, and every new write path would have to remember. Triggers record each change in the same transaction, whoever makes it: the services, the System, a migration.
  • A log of every change: rejected for one row per record holding its latest change, so the table grows with the records, not with their history.
  • Ids given by the server once an operation is applied: rejected. A client chooses the id of what it creates, so it can show it, link to it and change it again before the server has seen it.
  • Replicating everything a person can reach: rejected. Settings, runtimes, secrets, skills and routines change rarely and only matter online; they stay requests to the API.

Consequences​

  • An audience decides who sees a record. What every member of an organization sees has the audience o:<organization>; a person's conversations and their messages have p:<organization>:<user>. A client pulls with a cursor per audience. An audience it is no longer in is dropped from the replica, and a new one starts from the beginning, so joining an organization brings its history.
  • A record the caller can no longer see is sent as deleted, whether it was deleted, its project was, or the caller lost access. Deletions are kept as rows of their own and not pruned yet; pruning them would need a horizon past which a client pulls again from the beginning.
  • The stream carries no records. It says when to pull, and the server looks for changes twice a second only while someone follows it.
  • An operation is applied once. Each has an id the client chooses; the server keeps what became of it for 30 days and answers a retry with that. An id another person used is refused.
  • Clients may choose ids for the tasks, comments and messages they create, through the API too. An id in use, by anything anywhere, is refused the same way, so the answer reveals nothing.
  • What is still to push stays on top. Each pulled page is written with the outbox applied again over it, so the replica shows what was done here as the server will have it.
  • A refused operation is undone, and said so. The client asks for its records again with the next pull and drops it; the web app shows why.
  • The last write wins for one field changed in two places, as ADR 0029 already accepts for checklists. Operations change only the fields they name.
  • Agents and the CLI keep using the API. Their changes reach every replica like any other.
  • The web app still needs the server to start: signing in and the app's own files come from it. Once started it reads the replica, writes survive a dropped connection and a reload, and tabs share one replica.
  • A replica of another shape is pulled again. A change to its tables raises its version; a client that finds an older one drops it and pulls again, keeping its outbox.