diff --git a/docs/README.md b/docs/README.md index 22780cb..c9e4056 100644 --- a/docs/README.md +++ b/docs/README.md @@ -21,6 +21,8 @@ partial, and planned work. - [Client compatibility](client-compatibility.md) describes realistic client expectations. - [Storage and backup](storage.md) explains where mail and metadata live. +- [JMAP and the Store](jmap.md) defines the future canonical object model, + protocol mappings, and migration contract. - [Development](development.md) covers CMake, tests, sanitizers, and timezone maintenance. - [Bongo 0.7.0 release test matrix](release-testing-0.7.md) is the mandatory diff --git a/docs/architecture.md b/docs/architecture.md index fbcfa07..fc8b137 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -86,3 +86,9 @@ Web interface. The implementation is maintained Python 3 and JavaScript; no Ruby runtime or Ruby rewrite is planned. A later responsive redesign should preserve the simple integrated product rather than introduce a second server stack. + +For 0.9, JMAP becomes the canonical public object and synchronisation model, +while `bongostore` remains the only durable data owner. IMAP, POP3, +ManageSieve, CalDAV, CardDAV, SMTP/LMTP delivery, and the Web interface map to +the same internal objects and compatibility metadata; they do not maintain +parallel databases. See [JMAP and the Bongo Store](jmap.md). diff --git a/docs/jmap.md b/docs/jmap.md new file mode 100644 index 0000000..5f1f9a9 --- /dev/null +++ b/docs/jmap.md @@ -0,0 +1,134 @@ +# JMAP and the Bongo Store + +JMAP is planned for Bongo 0.9. It becomes the public object and +synchronisation API for modern clients, but it does not replace the Store with +a second database. `bongostore` remains the only durable owner of mail, +mailboxes, calendars, contacts, files, quotas, sharing state, and protocol +compatibility metadata. + +## One storage model + +The intended layering is: + +```text +SMTP/LMTP delivery Web and JMAP clients + | | + +--------+ JMAP object API + + | | + IMAP / POP3 / ManageSieve / CalDAV / CardDAV + | + internal Store/NMAP service + | + bongostore +``` + +JMAP is therefore the canonical object and change model, not a daemon which +owns a parallel copy of every message. Public protocol agents translate their +operations into the common Store API. Only `bongostore` commits durable state +and advances change identifiers. + +The existing private NMAP service on TCP 689 remains an internal transport. +It may be extended or versioned to carry JMAP-shaped operations, but it must +not be exposed as the public JMAP endpoint. + +## Protocol mappings + +The compatibility protocols remain supported and use the same objects: + +- IMAP `APPEND` imports an Email and its immutable MIME blob; flag changes + update keywords; `COPY` and `MOVE` update mailbox membership and the + corresponding compatibility state. +- POP3 `RETR` reads an Email and its blob. `DELE` is committed only when the + POP3 transaction successfully reaches `QUIT`. +- ManageSieve manages the same SieveScript objects exposed by JMAP Sieve. +- CalDAV and CardDAV translate resources and sync tokens to CalendarEvent, + Calendar, ContactCard, and AddressBook objects. +- SMTP and LMTP local delivery import through the same internal object API as + JMAP Email import. +- The Web interface uses the common service API instead of editing Store files + or maintaining a Web-only database. + +This design keeps a message, contact, or event consistent regardless of the +protocol which created or changed it. + +## Compatibility metadata + +JMAP IDs and state strings do not remove metadata required by older clients. +The Store must preserve and transactionally update at least: + +- IMAP UID, UIDVALIDITY, UIDNEXT, MODSEQ, subscriptions, flags, keywords, and + mailbox membership; +- POP3 stable UIDL values and transaction-safe deletion state; +- CalDAV/CardDAV href, ETag, collection identity, and sync token mappings; +- the active Sieve script and script revision state; +- JMAP object IDs, account capabilities, per-type state strings, change + journals, blob references, principals, shares, and quota roots. + +Opaque identifiers remain stable across restart, backup/restore, and protocol +access. A compatibility agent must never derive an identity from a mutable +display name or path. + +## Standards baseline + +The supported 0.9 baseline is the published JMAP standards applicable to +Bongo: + +- Core (RFC 8620), Mail (RFC 8621), WebSocket transport (RFC 8887), and MDNs + (RFC 9007); +- S/MIME signature verification (RFC 9219), Blob Management (RFC 9404), and + Quotas (RFC 9425); +- Contacts (RFC 9610), Sieve Scripts (RFC 9661), Sharing (RFC 9670), and VAPID + Web Push (RFC 9749). + +Calendar support follows the IETF JMAP Calendars work once its Standards Track +publication is stable enough to declare an interoperability contract. Active +drafts such as FileNode, mail sharing, Email push subscriptions, metadata, and +blob extensions may be implemented behind explicit experimental capabilities. +Their method names and persisted representations must not become an +unversioned database contract before the specifications stabilise. + +JMAP Submission complements SMTP submission; it does not replace SMTP server +delivery. Similarly, JMAP Sieve complements ManageSieve and does not remove +the latter from supported Thunderbird and legacy-client configurations. + +## Store migration + +Moving an existing installation to the JMAP-capable Store is an explicit, +restartable migration rather than an implicit conversion during normal +startup: + +1. validate the source Store, available disk space, owners, and configuration; +2. stop writers or enter a supported read-only maintenance mode; +3. create a verified snapshot and record the old Store generation; +4. scan mailboxes, calendars, contacts, scripts, quotas, and shares without + changing the original data; +5. preserve every original MIME message byte-for-byte and assign stable object, + blob, mailbox, and compatibility IDs; +6. build UID, UIDL, href/ETag, state, full-text, quota, and change-journal + indexes; +7. verify object counts, hashes, mailbox membership, protocol mappings, and + quota totals; +8. atomically select the new Store generation and retain the snapshot for + rollback. + +The migration records progress per account and object type so an interruption +does not duplicate messages or silently skip data. Rollback changes the active +generation; it does not attempt to reverse partially written files in place. + +## Release gates + +JMAP is not complete merely because a session document can be fetched. The +0.9 gate includes: + +- method, capability, authentication, access-control, pagination, error, and + rate-limit tests; +- concurrent JMAP/IMAP/POP3/DAV/Web changes against the same account; +- push reconnect, missed-event recovery, and change-journal compaction; +- byte-for-byte MIME and attachment round trips; +- UID/UIDL/ETag stability over restart, migration, backup, and restore; +- quota, sharing, Sieve, submission, and notification interoperability; +- malformed JSON, oversized requests, authorization-boundary, fuzzing, + sanitizer, and resource-exhaustion tests. + +The published specifications and current working-group status are tracked at +the [IETF JMAP documents page](https://datatracker.ietf.org/wg/jmap/documents/). diff --git a/docs/roadmap.md b/docs/roadmap.md index 9a2e41a..14eff15 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -102,6 +102,9 @@ fit Bongo; it is not a reason to rewrite working server code. - stable migration and compatibility APIs needed by the Web redesign and future clients. +The [JMAP Store design](jmap.md) defines the one-Store architecture, +compatibility metadata, migration, and release gates. + Native Exchange/MAPI or ActiveSync compatibility remains research after the open-protocol work. It is not a 0.9 or 1.0 release gate. diff --git a/docs/storage.md b/docs/storage.md index 020979e..733f0b9 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -74,3 +74,6 @@ Large-store and migration benchmarks are part of the 1.0 release work. The per-user metadata design avoids a single database bottleneck, but it does not remove the need to test filesystem inode usage, indexes, backup duration, search performance, and recovery with realistically large mailboxes. + +The future JMAP-capable generation and its restartable conversion procedure +are specified in [JMAP and the Bongo Store](jmap.md).