bongo/docs/queue.md

Queue, routing, and delivery

bongoqueue durably owns messages after SMTP or the external collector has accepted them and before every recipient has reached a final destination. It coordinates filtering, local Store delivery, remote SMTP, and LMTP; it is not a public submission protocol.

Message flow and durability

Queue data lives below /var/lib/bongo/spool. An accepted message remains there while a temporary scanner, Store, DNS, SMTP, LMTP, or quota-dependent delivery condition is retried. Permanent failures produce the configured delivery-status response or bounce. The Queue must not acknowledge a message to its upstream path before the payload and recipient state are durable.

Enabled filtering agents register for Queue stages. Antivirus, SpamAssassin, rules/Sieve, and final local or remote delivery therefore see a common envelope and message rather than independent copies. External POP3 or IMAP collection enters this same path and may delete a source message only after Bongo has committed it successfully.

The internal Queue protocol defaults to TCP 8670. Bind and firewall it as a private service interface. It does not authenticate arbitrary Internet clients and must never be exposed as an SMTP alternative.

Limits and failure handling

The suffixless queue document controls spool free-space reservation, message lifetime, retry interval, worker concurrency, trusted/hosted domains, and bounce policy. minimumfreespace prevents normal acceptance from consuming the filesystem's final space. Per-user Store quota is separate: a full mailbox defers or rejects that recipient without corrupting other recipient state.

Collector backpressure is also separate from the byte reserve. The collector document limits globally pending and per-user pending external messages. Queue counts only entries carrying its durable collected-message flag, including entries currently owned by a Queue worker. Once a limit is reached, the collector does not download or delete the next source message. bongo-queuetool collected-count [user] exposes the global count and, optionally, the selected user's count.

JSON integral configuration values and the generic agent loader preserve signed 64-bit values. In particular, a large minimumfreespace value is not truncated into a negative 32-bit number while Store configuration is mirrored to disk.

A scanner outage is a delivery-policy decision, not permission to silently bypass inspection. ClamAV and SpamAssassin failures leave the message queued for the configured retry path. Likewise, a per-recipient LMTP temporary error must preserve only that recipient for retry and must not redeliver recipients which already returned a final success.

Queue control-file rewrites use a same-directory work file and an atomic rename. The original is retained unless every buffered write and close succeeds. A failed submission flush or final rename is rejected and its partial entry discarded, so the submitting SMTP peer remains responsible for retrying the message instead of receiving a false success.

Administration and shutdown

Use bongo-queuetool to inspect and perform documented queue operations; do not edit spool files manually. A service stop first prevents new work and asks the managed agents to finish or abandon their current transaction safely. Queue and Store state must be included in a consistent backup.

bongo-queuetool message exports the exact Queue-owned bytes. The hold-local, deliver-local, and delete commands provide a bounded recovery and diagnostic path: target 999 is not part of normal routing, deliver-local retains its source entry, and delete must finish the operation explicitly. The STQ-02 fixture uses these commands to compare the Queue payload against the Store document, including the intentional Queue trace header.

The 0.7 release matrix exercises byte preservation, disk exhaustion, retry, expiry, bounce, hold/release, concurrency, invalid commands, quota mapping, and collector floods under STQ-02 through STQ-12. SMTP and LMTP rows add their protocol-specific recipient and retry cases.