Complete protocol and service documentation
This commit is contained in:
Mario Fetka
@@ -39,6 +39,11 @@ partial, and planned work.
|
||||
- [Authentication, TOTP, LDAP, and app passwords](authentication.md)
|
||||
- [SMTP, submission, LMTP, and trusted relay](smtp.md)
|
||||
- [IMAP compatibility](imap.md)
|
||||
- [POP3 and POP3S](pop3.md)
|
||||
- [Sieve filtering and ManageSieve](sieve.md)
|
||||
- [Webmail and user administration](web.md)
|
||||
- [CalDAV and CardDAV](dav.md)
|
||||
- [Queue, routing, and delivery](queue.md)
|
||||
- [Mail storage quotas](quotas.md)
|
||||
- [SPF, DKIM, DMARC, and SRS](mail-authentication.md)
|
||||
- [SpamAssassin and ClamAV](antispam.md)
|
||||
|
||||
@@ -42,6 +42,10 @@ handled by the authenticated outbound SMTP path.
|
||||
| Bongo Web | Python 3 Webmail, account settings, CalDAV, and CardDAV. |
|
||||
| `bongo-setup` / `bongo-admin` | Guided initial setup and validated administration. |
|
||||
|
||||
Protocol and component details are maintained in [smtp.md](smtp.md),
|
||||
[imap.md](imap.md), [pop3.md](pop3.md), [sieve.md](sieve.md),
|
||||
[queue.md](queue.md), [web.md](web.md), and [dav.md](dav.md).
|
||||
|
||||
Agents use common C libraries for configuration, logging, authentication,
|
||||
TLS, protocol I/O, Store access, and Queue access. GNU GSASL is shared by the
|
||||
mail protocols; its direct Bongo verification callback does not create or
|
||||
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
# CalDAV and CardDAV
|
||||
|
||||
Bongo exposes a user's calendar and address book through the Web service so
|
||||
Thunderbird and other DAV clients see the same objects as Bongo Webmail.
|
||||
Calendar resources are iCalendar documents; contact resources are vCards.
|
||||
|
||||
## Discovery and paths
|
||||
|
||||
Clients may use `/.well-known/caldav` and `/.well-known/carddav` discovery.
|
||||
Authenticated homes use these forms:
|
||||
|
||||
```text
|
||||
/calendars/USER/personal/
|
||||
/addressbooks/USER/personal/
|
||||
```
|
||||
|
||||
Calendar objects end in `.ics`; address-book objects end in `.vcf`. The
|
||||
authenticated user may access only that user's home. Resource names are
|
||||
decoded and validated before being mapped to Store paths, preventing encoded
|
||||
slashes, traversal, NUL bytes, or another account name from escaping the
|
||||
personal collection.
|
||||
|
||||
## HTTP and DAV behaviour
|
||||
|
||||
The service supports `OPTIONS`, `GET`, `HEAD`, `PUT`, `DELETE`, `PROPFIND`,
|
||||
and `REPORT` for the implemented calendar and address-book resources.
|
||||
Depth-aware multistatus responses describe the principal, home, personal
|
||||
collection, and objects. Calendar multiget/query and address-book multiget
|
||||
read the requested resources. ETags and `If-Match`/`If-None-Match` protect
|
||||
updates against accidental overwrite.
|
||||
|
||||
Incoming iCalendar and vCard payloads are parsed strictly and bounded by the
|
||||
Web request and object-size limits. Bongo preserves the canonical source data
|
||||
needed for lossless DAV round trips while also mapping useful fields into the
|
||||
Store model used by Webmail. Calendar events include recurrence and the
|
||||
standard date/time data supported by the converter; contacts include
|
||||
structured names, multiple email addresses and telephone numbers,
|
||||
organization, birthday/anniversary, URL, note, and postal address fields.
|
||||
|
||||
DAV authentication follows the Web master/app-password scope and external
|
||||
TLS policy. When a reverse proxy terminates TLS, it must pass DAV methods and
|
||||
bodies unchanged and use the same trusted-proxy configuration as Webmail.
|
||||
The release gate tests discovery, CRUD, reports, ETags, isolation, and
|
||||
TLS/proxy policy under DAV-01 through DAV-03.
|
||||
@@ -0,0 +1,66 @@
|
||||
# POP3 and POP3S
|
||||
|
||||
Bongo keeps POP3 for simple and older mail clients while using the same
|
||||
accounts, authentication policy, message files, flags, and per-user mail quota
|
||||
as IMAP and Webmail. POP3 is an access protocol only: SMTP remains responsible
|
||||
for message submission.
|
||||
|
||||
## Listeners and TLS
|
||||
|
||||
The suffixless `pop3` configuration document controls the two listeners:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"port": 110,
|
||||
"port_ssl": 995,
|
||||
"allow_legacy_tls": false,
|
||||
"proxy_protocol_enabled": false,
|
||||
"proxy_protocol_networks": []
|
||||
}
|
||||
```
|
||||
|
||||
Port 110 offers `STLS` before authentication. Port 995 uses implicit TLS.
|
||||
Password and GNU GSASL mechanisms are made available only on a protected
|
||||
connection under the secure default. Enabling `allow_legacy_tls` is an
|
||||
explicit compatibility exception for obsolete clients; current TLS is still
|
||||
negotiated when both peers support it.
|
||||
|
||||
PROXY protocol must remain disabled unless a trusted TCP proxy is actually in
|
||||
front of POP3. Enabling it requires at least one exact trusted IPv4 address or
|
||||
CIDR in `proxy_protocol_networks`; ordinary clients must never be included.
|
||||
|
||||
## Commands and capabilities
|
||||
|
||||
The server implements the normal authorization and transaction commands:
|
||||
`USER`, `PASS`, `AUTH`, `APOP`, `STAT`, `LIST`, `UIDL`, `RETR`, `TOP`, `DELE`,
|
||||
`RSET`, `NOOP`, `CAPA`, `UTF8`, `STLS`, and `QUIT`. Multiline replies apply
|
||||
POP3 dot transparency and canonical CRLF termination even when a stored
|
||||
message was read in chunks.
|
||||
|
||||
`CAPA` reports the implemented `TOP`, `RESP-CODES`, `AUTH-RESP-CODE`,
|
||||
`PIPELINING`, `EXPIRE NEVER`, `UIDL`, and `UTF8` features. `STLS`, `USER`, and
|
||||
the `SASL` mechanism list are state-dependent. After `UTF8` succeeds, `STLS`
|
||||
is no longer offered and is rejected, preventing already-buffered cleartext
|
||||
UTF-8 session state from crossing the TLS boundary.
|
||||
|
||||
## Deletion and quota semantics
|
||||
|
||||
`DELE` marks a message only for the current POP3 transaction. `RSET` or a
|
||||
disconnect leaves it in the mailbox; a successful `QUIT` commits the marked
|
||||
deletions. Storage is therefore released only after that commit. POP3 has no
|
||||
standard quota query command, but delivery to the same account is still
|
||||
limited by the shared Store quota described in [quotas.md](quotas.md).
|
||||
|
||||
Use IMAP or Webmail when a client needs server-side folders, flags, searching,
|
||||
or concurrent mailbox changes. POP3 clients which delete downloaded mail
|
||||
should be tested carefully before enabling that option for a large account.
|
||||
|
||||
## Operational checks
|
||||
|
||||
After changing the listener or TLS policy, validate and synchronize the
|
||||
configuration, restart the agent through `bongo-manager`, and test both the
|
||||
clear listener's `STLS` transition and a direct POP3S login. Also verify
|
||||
`STAT`, `LIST`, `RETR`, `TOP`, `UIDL`, deletion rollback on disconnect, and
|
||||
deletion commit on `QUIT`. The release gate is recorded under POP-01 through
|
||||
POP-06 in [release-testing-0.7.md](release-testing-0.7.md).
|
||||
+9
-1
@@ -43,7 +43,7 @@ Mail-storage limits are enforced by the shared Store for SMTP delivery, IMAP,
|
||||
POP3, Web access, and external collection. IMAP exposes RFC 9208 quota queries;
|
||||
POP3 has no standardized quota command and releases charged storage only when
|
||||
its `QUIT` update transaction successfully commits marked deletions. See
|
||||
[Mail storage quotas](quotas.md).
|
||||
[POP3 and POP3S](pop3.md) and [Mail storage quotas](quotas.md).
|
||||
|
||||
## Server-side filtering
|
||||
|
||||
@@ -59,6 +59,9 @@ without depending on the Web interface. Later interfaces, including a future
|
||||
JMAP implementation, must operate on the same user filter model rather than
|
||||
inventing incompatible rules.
|
||||
|
||||
See [Sieve filtering and ManageSieve](sieve.md) for listener, command,
|
||||
storage, and vacation behaviour.
|
||||
|
||||
## Web, calendars, and contacts
|
||||
|
||||
The Python 3 Web service provides Webmail and user settings. CalDAV exposes
|
||||
@@ -66,6 +69,9 @@ calendars using iCalendar data, and CardDAV exposes address books using vCard
|
||||
data. External HTTPS is normally supplied by a reverse proxy; the Bongo Web
|
||||
listener itself defaults to HTTP on loopback.
|
||||
|
||||
See [Webmail and user administration](web.md) and
|
||||
[CalDAV and CardDAV](dav.md) for the current 0.7 interfaces.
|
||||
|
||||
JMAP and Exchange/MAPI protocols are not part of Bongo 0.7. JMAP is planned
|
||||
for 0.9 after the mail protocols and Store API are stable. Native
|
||||
Exchange/Outlook compatibility remains research beyond that milestone.
|
||||
@@ -75,6 +81,8 @@ Exchange/Outlook compatibility remains research beyond that milestone.
|
||||
The Store and Queue protocols coordinate local Bongo agents. They are not
|
||||
public client protocols and have no Internet compatibility promise. Bind them
|
||||
to a private interface and firewall ports 689 and 8670 from clients.
|
||||
Queue durability and retry policy are described in
|
||||
[Queue, routing, and delivery](queue.md).
|
||||
|
||||
## Authentication and message security
|
||||
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Queue, routing, and delivery
|
||||
|
||||
`bongoqueue` durably owns messages after SMTP or the external collector has
|
||||
accepted them and before every recipient has reached a final destination. It
|
||||
coordinates filtering, local Store delivery, remote SMTP, and LMTP; it is not
|
||||
a public submission protocol.
|
||||
|
||||
## Message flow and durability
|
||||
|
||||
Queue data lives below `/var/lib/bongo/spool`. An accepted message remains
|
||||
there while a temporary scanner, Store, DNS, SMTP, LMTP, or quota-dependent
|
||||
delivery condition is retried. Permanent failures produce the configured
|
||||
delivery-status response or bounce. The Queue must not acknowledge a message
|
||||
to its upstream path before the payload and recipient state are durable.
|
||||
|
||||
Enabled filtering agents register for Queue stages. Antivirus,
|
||||
SpamAssassin, rules/Sieve, and final local or remote delivery therefore see a
|
||||
common envelope and message rather than independent copies. External POP3 or
|
||||
IMAP collection enters this same path and may delete a source message only
|
||||
after Bongo has committed it successfully.
|
||||
|
||||
The internal Queue protocol defaults to TCP 8670. Bind and firewall it as a
|
||||
private service interface. It does not authenticate arbitrary Internet
|
||||
clients and must never be exposed as an SMTP alternative.
|
||||
|
||||
## Limits and failure handling
|
||||
|
||||
The suffixless `queue` document controls spool free-space reservation,
|
||||
message lifetime, retry interval, worker concurrency, trusted/hosted domains,
|
||||
and bounce policy. `minimumfreespace` prevents normal acceptance from
|
||||
consuming the filesystem's final space. Per-user Store quota is separate: a
|
||||
full mailbox defers or rejects that recipient without corrupting other
|
||||
recipient state.
|
||||
|
||||
A scanner outage is a delivery-policy decision, not permission to silently
|
||||
bypass inspection. ClamAV and SpamAssassin failures leave the message queued
|
||||
for the configured retry path. Likewise, a per-recipient LMTP temporary error
|
||||
must preserve only that recipient for retry and must not redeliver recipients
|
||||
which already returned a final success.
|
||||
|
||||
## Administration and shutdown
|
||||
|
||||
Use `bongo-queuetool` to inspect and perform documented queue operations; do
|
||||
not edit spool files manually. A service stop first prevents new work and asks
|
||||
the managed agents to finish or abandon their current transaction safely.
|
||||
Queue and Store state must be included in a consistent backup.
|
||||
|
||||
The 0.7 release matrix exercises byte preservation, disk exhaustion, retry,
|
||||
expiry, bounce, hold/release, concurrency, invalid commands, quota mapping,
|
||||
and collector floods under STQ-02 through STQ-12. SMTP and LMTP rows add their
|
||||
protocol-specific recipient and retry cases.
|
||||
@@ -0,0 +1,61 @@
|
||||
# Sieve filtering and ManageSieve
|
||||
|
||||
Bongo stores each user's server-side mail rules independently from the mail
|
||||
payloads. `bongosieve` provides the ManageSieve management protocol, while
|
||||
`bongorules` evaluates the active script as mail passes through the Queue.
|
||||
Thunderbird and other ManageSieve clients can therefore maintain filters
|
||||
without editing server files.
|
||||
|
||||
## Listener and authentication
|
||||
|
||||
ManageSieve is disabled by the base template until it is required. Its
|
||||
suffixless `sieve` document has conservative loopback defaults:
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": false,
|
||||
"listen_address": "127.0.0.1",
|
||||
"port": 4190,
|
||||
"allow_legacy_tls": false,
|
||||
"proxy_protocol_enabled": false,
|
||||
"proxy_protocol_networks": [],
|
||||
"maximum_script_size": 1048576,
|
||||
"maximum_connections": 64
|
||||
}
|
||||
```
|
||||
|
||||
The greeting offers `STARTTLS`; GNU GSASL mechanisms are advertised only
|
||||
after TLS. Authentication uses the same Bongo user and app-password policy as
|
||||
the mail protocols, with the Sieve service scope. Cleartext received after a
|
||||
`STARTTLS` command is discarded before negotiation. PROXY protocol has the
|
||||
same source-restricted rules as the other mail listeners.
|
||||
|
||||
## Management protocol
|
||||
|
||||
The RFC 5804 service implements `CAPABILITY`, `STARTTLS`, `AUTHENTICATE`,
|
||||
`HAVESPACE`, `PUTSCRIPT`, `CHECKSCRIPT`, `LISTSCRIPTS`, `GETSCRIPT`,
|
||||
`SETACTIVE`, `RENAMESCRIPT`, `DELETESCRIPT`, `NOOP`, and `LOGOUT`. Script
|
||||
names and literals are bounded, syntax is validated before activation, an
|
||||
active script cannot be deleted accidentally, and scripts are isolated by
|
||||
authenticated user.
|
||||
|
||||
The implementation advertises `fileinto`, `envelope`, `body`, `variables`,
|
||||
`vacation`, and `vacation-seconds`. Actions include keep, discard, file into a
|
||||
folder, redirect, reject, and vacation response. Vacation history is durable
|
||||
so repeated delivery does not generate an immediate reply loop. Scripts and
|
||||
vacation state live in `/var/lib/bongo/dbf/sieve.sqlite`; include that database
|
||||
and its SQLite sidecars in backups.
|
||||
|
||||
## Filters in Webmail
|
||||
|
||||
The Web filter editor and ManageSieve are two interfaces to the same intended
|
||||
user rule model. A filter may match senders or other message properties and
|
||||
deliver matching messages to a folder such as `Family`. Mailing-list
|
||||
recognition is one possible filter condition, not a separate replacement for
|
||||
the filter system. Future JMAP support must map to the same rules rather than
|
||||
creating a third incompatible store.
|
||||
|
||||
Test script upload, syntax rejection, activation, rename, retrieval, deletion,
|
||||
folder delivery, redirect, reject, vacation deduplication, and quota limits
|
||||
before exposing the service. The 0.7 release cases are SIEVE-01 through
|
||||
SIEVE-05 in [release-testing-0.7.md](release-testing-0.7.md).
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
# Webmail and user administration
|
||||
|
||||
The Python 3 `bongo-web` service supplies the classic Bongo Webmail layout,
|
||||
user settings, tasks, contacts, calendars, and the restricted administrator
|
||||
view. It talks to the same Store used by IMAP and the delivery agents; it is
|
||||
not a second mailbox database.
|
||||
|
||||
## Deployment
|
||||
|
||||
The suffixless `web` configuration document is the only Web listener
|
||||
configuration. A normal deployment listens on loopback port 8080 behind
|
||||
Apache, nginx, nginx-proxy-manager, or HAProxy. Set `public_base_url`,
|
||||
`external_https`, `trusted_proxies`, and `client_ip_header` to describe the
|
||||
real external path. Forwarded client addresses are trusted only from the
|
||||
configured proxies.
|
||||
|
||||
Bongo can instead serve HTTPS directly. It loads the configured certificate
|
||||
and private key, binds the privileged HTTPS and redirect ports, then
|
||||
permanently changes to the `bongo` UID/GID before accepting requests. The HTTP
|
||||
listener redirects to the configured HTTPS base URL except for the narrowly
|
||||
scoped ACME HTTP-01 challenge path. Reverse-proxy examples are installed
|
||||
below `/usr/share/bongo/examples/reverse-proxy`.
|
||||
|
||||
## Sessions and security
|
||||
|
||||
Login creates a server-side session with an HttpOnly cookie. State-changing
|
||||
API requests require the session's CSRF token. Login attempt windows, session
|
||||
lifetime, second-factor lifetime, and second-factor attempt limits are
|
||||
configurable. TOTP fields are shown only when the account has enabled that
|
||||
optional feature; recovery codes and app passwords are handled by the shared
|
||||
authentication subsystem.
|
||||
|
||||
HTML mail is sanitized before display. Safe inline or explicitly permitted
|
||||
images can remain useful for signatures and normal content, while active
|
||||
markup and unwanted remote tracking are blocked. Administrator actions are
|
||||
both role-checked and limited to a fixed command set; selecting the
|
||||
administration view does not create a second login session.
|
||||
|
||||
## User functions
|
||||
|
||||
The current API and interface cover mailbox folders and conversations,
|
||||
reading and sending mail, contacts, calendar events, filters, actionable
|
||||
tasks, external account collection, sending identities, preferences, and
|
||||
signatures. A task is a workflow item: it can be opened for its details and
|
||||
then dismissed, retried, ignored, or completed where the task type permits.
|
||||
English, German, and every installed gettext catalogue use the same API
|
||||
instead of hard-coded interface text.
|
||||
|
||||
The administrator view exposes health and a restricted set of validated
|
||||
actions. Full server configuration remains the responsibility of
|
||||
`bongo-admin` and the configuration TUI; Webmail must not become an arbitrary
|
||||
root command interface.
|
||||
|
||||
The current classic interface is the 0.7 baseline. The responsive PWA redesign
|
||||
is scheduled for 0.8 and documented separately in [web-pwa.md](web-pwa.md).
|
||||
JMAP and background Web Push belong to 0.9.
|
||||
Reference in New Issue
Block a user