bongo/docs/storage.md

Storage and backup

Bongo does not put every message in one global SQLite database. The Store uses a per-user layout so a large account does not lock or enlarge the metadata database of every other account.

Store layout

Each account below /var/lib/bongo/users has its own store.db SQLite metadata database and Maildir-like collection directories containing the document payloads. Mail, calendar objects, and contacts use the Store's common object and collection model. The SQLite database tracks collections, UIDs, flags, properties, and indexes; message bodies remain separate files.

On first access a new account receives INBOX, Drafts, Archive, Sent, Trash and Junk mail collections with their standard IMAP SPECIAL-USE roles, plus a personal calendar and personal and collected address books. These objects and their payload directories belong only to that account's Store.

Optional per-user mail quotas are stored and enforced in each Store database. They count logical mail-message sizes across all mailboxes, but not calendar, contact, configuration, or authentication data. See Mail storage quotas for administration and protocol behaviour.

The system Store below /var/lib/bongo/system contains server configuration documents and other system-owned objects. The Queue spool below /var/lib/bongo/spool contains messages which have been accepted but not yet fully delivered.

Other SQLite databases serve separate purposes:

• /var/lib/bongo/dbf/userdb.sqlite stores local authentication records, Argon2id password hashes, encrypted TOTP secrets, recovery codes, app passwords, login aliases, and LDAP cache metadata.
• /var/lib/bongo/dbf/sieve.sqlite stores ManageSieve scripts and vacation state.
• /var/lib/bongo/dbf/external-accounts.sqlite stores external account schedules, identities, encrypted credentials, and import state.

Web tasks, sender identities, and signatures are ordinary user Store documents below /preferences. Login sessions exist only in bongoweb memory and are invalidated whenever that service restarts.

SQLite write-ahead-log and sidecar files are part of a live database state; copying only the main .db file while Bongo is writing is not a reliable backup.

Backup scope

A complete backup includes at least:

• /etc/bongo;
• all of /var/lib/bongo, including Store payloads, metadata, Queue spool, account databases and user preference objects;
• TLS, DKIM, SRS, external-account encryption, TOTP encryption, and ACME hook keys wherever the configuration points to them;
• distribution service overrides and reverse-proxy configuration needed to reproduce the deployment.

For a simple consistent filesystem backup, stop Bongo, confirm that its agents have exited, snapshot or copy the entire state and configuration tree, then restart it. A storage snapshot is acceptable only when the filesystem and SQLite state are captured atomically. Online backup tooling must use the SQLite backup API or checkpoint/locking rules and coordinate Store payload creation; a file-by-file copy is insufficient.

bongo-storetool can import and export mailbox data in mbox or Maildir form. That is useful for migration and recovery of mail content, but it is not a replacement for a complete server backup because it does not preserve every account, key, configuration, calendar, contact, task, and protocol state.

Restore testing

Restore into an isolated host with the same or a newer compatible Bongo build. Check database integrity, ownership and permissions, then validate the configuration before starting the agents. Test login, message counts, flags, folder hierarchy, calendar/contact access, queued delivery, app passwords, and a signed outbound message.

Large-store and migration benchmarks are part of the 1.0 release work. The per-user metadata design avoids a single database bottleneck, but it does not remove the need to test filesystem inode usage, indexes, backup duration, search performance, and recovery with realistically large mailboxes.

The future JMAP-capable generation and its restartable conversion procedure are specified in JMAP and the Bongo Store.