bongo/contrib/testing/README.md

Local mail protocol fixtures

These fixtures are intentionally bound to loopback and use non-standard ports. They are development tools, not production configuration.

Release and sanitizer harnesses

The scripts used for the 0.7 release evidence are kept here rather than as one-off files in /tmp:

• test-config-tls-material.sh generates disposable GnuTLS material and covers CFG-09/CFG-10 generation, permissions, reuse, and unsafe-input cases.
• web-direct-tls-smoke.sh covers the CFG-08 direct Web TLS 1.2/1.3 listener, security headers, plain-HTTP rejection, and a mismatched private key.
• generate-protocol-test-config.py creates the disposable local protocol configuration used by sanitizer sessions.
• protocol-smoke.py delivers through authenticated SMTP submission and reads the message through IMAP/POP over explicit and implicit TLS.
• smtp-basic-check.py exercises the port-25 greeting and the non-delivery HELO/EHLO, NOOP, RSET, HELP, VRFY-policy, and QUIT command surface.
• smtp-submission-auth-check.py exercises port 587 STARTTLS and port 465 implicit TLS directly, including the shared GSASL PLAIN and LOGIN mechanisms, pre-TLS AUTH rejection, invalid credentials, and repeated STARTTLS rejection. It authenticates but does not submit a message.
• smtp4dev-fixture.py manages two authenticated, loopback-only external SMTP provider fixtures for STARTTLS and implicit TLS. It validates credentials, captures messages through smtp4dev's supported REST API, and keeps both databases below /tmp.
• smtp-internal-relay-check.py temporarily lowers the trusted port 26 limits, verifies allowed and denied source addresses plus connection, recipient, message, and byte limits, then restores the exact SMTP configuration without committing a message.
• smtp-internal-normalization-check.py sends only to the reserved .invalid namespace through trusted port 26, inspects the real Queue routing envelope for ten sender-normalization cases after atomically moving each entry to the hold queue, deletes every test entry, and restores the exact SMTP configuration.
• smtp-local-delivery-check.py authenticates as one disposable local user over STARTTLS submission, sends to another local user, verifies the exact message in the recipient's Store INBOX, rejects duplicate delivery, and removes its Store and Queue fixtures.
• smtp-client-interoperability-check.py repeats real delivery with curl, swaks, Xeams Email Sender, GNU Mailutils, and paced Telnet input. It covers authenticated STARTTLS submission plus unauthenticated local delivery on the public and trusted-device listeners. Password-bearing temporary client files use mode 0600 and every delivered Store and Queue fixture is removed.
• smtp-telnet-timeout-check.py temporarily lowers the SMTP socket timeout, proves that idle Telnet connections close on both plain listeners, and proves that slow manual EHLO, NOOP, and RSET input resets the timeout after each complete command. It restores the SMTP configuration and active service before exiting.
• smtp-stress-timeout-check.py uses only two simultaneous connections to trigger the configurable load threshold. It proves that the first connection keeps the normal timeout while newly accepted inbound, trusted-device, and submission connections receive the shorter overload timeout, then restores the exact configuration and active service.
• sendmail-local-submission-check.py runs as an unprivileged Unix account and verifies both /usr/sbin/sendmail and GNU mail through Bongo's restricted, rate-limited local SMTP submission path. It confirms that no Bongo system credential is needed and that Bcc is removed.
• c-literal-length-check.py audits manually sized string literals passed to Bongo connection I/O helpers and rejects lengths that include a trailing NUL, truncate a protocol line, or read past the literal.
• sanitizer-protocol-session.sh runs that protocol path under ASan/UBSan/LSan or TSan and validates clean process shutdown.
• asan-protocol-session.sh and tsan-protocol-session.sh preserve the two explicit entry points used by the original release runs.
• sieve-validate-no-lsan.sh is the narrow helper used by the ASan session; the Sieve validator remains leak-tested directly by CTest.
• cpack-stage-smoke.sh checks imports and required executables from an extracted CPack staging tree in a private mount namespace.
• debian-source-probe.sh and debian-libspf2-package-build.sh preserve the disposable Debian container probes used before the complete bundle builder was available.
• setup-wizard-defaults.py, config-tui-edit.py, legacy-config-install.py, protocol-diagnostics.py, and store-diagnostics.py preserve the remaining interactive and diagnostic bring-up tests. Destructive live helpers require an explicit opt-in environment variable.
• mail-dns-live-check.py logs in through the real Web and IMAPS interfaces and verifies that setup produced both an actionable SPF/DKIM/DMARC task and a forwardable administrator message without private-key material.
• dkim-material-check.py compares a real OpenDKIM-generated RSA-2048 key with Bongo's GnuTLS-derived public value, imports it, checks permissions and DNS output, then verifies byte-identical reuse.
• manager-lifecycle-check.sh exercises systemd reload/restart/stop/start, one agent crash, and an unclean manager death which leaves a stale PID file. It requires explicit live-test opt-in and restores an active service on exit.
• agent-privilege-check.sh verifies the live manager/agent process tree, real/effective/saved UID and GID tuples, supplementary groups, and inherited, permitted, effective, and ambient capabilities.
• store-user-layout-check.sh idempotently provisions disposable Store, Queue, quota, and protocol users, then proves that two freshly accessed accounts have separate Store databases and the complete standard folder layout, including the IMAP \\Junk spam collection.
• store-backup-compare.py compares backups made before and after a restore. It requires byte-identical payloads, paths, GUIDs, flags, IMAP UIDs, custom properties, and calendar-link metadata while ignoring only Store-generated creation/modification timestamps.
• store-protocol-safety-check.py sends unknown, malformed, oversized, and unauthorised commands to a live Store listener. It verifies that maintenance/debug commands remain manager-only, a user cannot inspect a foreign Store, and the listener remains available after every rejection.
• store-quota-concurrency-check.py writes oversized contact and calendar fixtures without charging mail quota, then races two Store deliveries against one remaining quota slot. Exactly one delivery must succeed and the other must receive Store status 5220.

The complete mapping from the original one-off /tmp names to maintained scripts is recorded in tmp-script-map.md.

Every script which creates credentials requires disposable values through the environment; no test password or private key is stored in Git. For example:

export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
export BONGO_TEST_ROOT=/tmp/bongo-0.7-r1-asan-root/usr
export BONGO_SANITIZER=asan
unshare --user --map-root-user --net \
  ./contrib/testing/sanitizer-protocol-session.sh

The live user-layout check deliberately retains its clearly named accounts so later Queue, quota, and protocol matrix rows can reuse them:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/store-user-layout-check.sh

The Queue-to-Store byte-preservation check uses the retained stqqueue account. It creates a target-999 diagnostic entry, exports the exact Queue-owned bytes, delivers that entry through Queue's local Store path, and compares the resulting Store object byte-for-byte:

./contrib/testing/queue-store-byte-check.sh

The script deletes the held Queue entry even on failure. It deliberately retains the delivered Store object as release evidence. Set BONGO_QUEUE_TOOL, BONGO_STORE_TOOL, or BONGO_TEST_RUN_AS only when testing a non-standard installation.

The Queue disk-space check brackets QDSPC with filesystem samples and verifies that the suffixless queue configuration's minimumfreespace reserve was subtracted without 32-bit truncation:

./contrib/testing/queue-disk-space-check.sh

The bounded collector flood check seeds separate POP3 and IMAP Cyrus source accounts with 128-KiB messages, delivers them against a small Store quota, and then temporarily lowers the Collector per-user pending limit to one. It verifies unique delivery across Store and Queue, exact quota enforcement, and that the first over-limit source message remains at Cyrus:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_USER=stqquota
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/collector-flood-quota-check.py

The script restores quota and Collector configuration and removes its source, Queue, Store, and external-account fixtures. Use BONGO_TEST_CLEANUP_ONLY=1 to clean up a previously interrupted run.

The Queue failure-safety check retains one isolated message in an 8 MiB tmpfs fixture while it first fills that filesystem below the operative reserve and then overlays the spool with an unreadable directory. It verifies rejection/defer responses, restores all mounts through traps, and compares the retained message byte-for-byte after both recoveries:

./contrib/testing/queue-failure-safety-check.sh

It requires passwordless permission for Bongo service start/stop/restart, mount, umount, and stat on the dedicated test spool. Run it only on a disposable test installation.

The Store protocol safety check is non-destructive and requires the two disposable users retained by store-user-layout-check.sh:

export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/store-protocol-safety-check.py

The password is read only from the environment and is redacted by the Store's command logger. The test deliberately attempts REINDEX, REPAIR, RESET, and SHUTDOWN as a normal user; run it only against a build where those commands require manager privileges.

The live Store quota check uses the retained stqquota account. It reads and restores the account's original limit, removes every fixture document, and requires explicit destructive-test opt-in:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/store-quota-concurrency-check.py

The script invokes sudo -n bongo-admin user quota only to set and restore the temporary administrative limit. The password is never passed on a command line.

The partial Queue quota check creates two held messages and temporarily leaves space for exactly one of them. It verifies that the first reaches Store, the second remains byte-identical in Queue after Store returns quota exceeded, and the retained message is delivered exactly once after the first is deleted:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/queue-quota-partial-check.py

Every fixture message and Queue entry is removed and the original quota is restored, including after a failed assertion.

The normal SMTP companion check submits an authenticated message to the full test mailbox. It verifies that SMTP accepts the durable Queue transaction, Store receives no partial object, the sender receives exactly the standard 5.2.2 DSN, and that ordinary SMTP mail is not retained indefinitely:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/smtp-quota-dsn-check.py

The Collector companion creates a real Queue transaction carrying MSG_FLAG_COLLECTED_MESSAGE. It proves that a full Store keeps the sole durable copy in local-delivery queue 6, never hands it to the outgoing SMTP agent, and delivers it exactly once after the quota is released:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/collector-quota-retry-check.py

Set BONGO_TEST_TRACE=1 to print bounded per-stage timings. On failure, fixtures are normally removed and the original quota is restored. For local diagnosis only, BONGO_TEST_KEEP_FAILURE=1 preserves the matching Queue entry while still restoring the quota; remove that entry with bongo-queuetool delete after inspection.

The quota-state companion covers administrative transitions and portable Store restore semantics. It lowers a limit below current usage, proves that growth remains blocked, removes the limit, checks the signed 64-bit maximum and its overflow boundary, then verifies that PAX restore reconstructs exact logical usage without replacing the target account's administrative limit:

export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
./contrib/testing/store-quota-state-check.py

This test backs up the complete disposable account before modifying it and restores both the original Store content and quota in its cleanup path.

The Debian source probes and package-build commands used by BLD-14/BLD-15 are consolidated in contrib/debian/build-trixie-bundle.sh; they are not maintained as duplicate container-only snippets here.

Cyrus IMAP, POP3, and LMTP

Install net-mail/cyrus-imapd, then start the disposable fixture as root:

sudo ./contrib/testing/cyrus-fixture.py start

It creates no files outside /tmp/bongo-cyrus-fixture and listens on:

• IMAP: 127.0.0.1:18143
• POP3: 127.0.0.1:18110
• LMTP: 127.0.0.1:18024

The source mailboxes form a multi-account-per-user matrix:

Bongo owner	Cyrus source account	Protocol	Bongo destination
test1	test1-pop@cyrus.bongo.test	POP3	INBOX
test1	test1-imap@cyrus.bongo.test	IMAP	Cyrus IMAP
test2	test2-pop@cyrus.bongo.test	POP3	INBOX
test2	test2-imap@cyrus.bongo.test	IMAP	Cyrus IMAP

All source accounts use the test-only password BongoCollector-Test-2026. Insert a complete RFC 5322 message through Cyrus' real LMTP delivery path:

sudo ./contrib/testing/cyrus-fixture.py inject message.eml

Seed three different messages in each source mailbox and exercise Bongo's real Collector POP3 and IMAP transport implementation:

./contrib/testing/cyrus-fixture.py verify-collector \
  --probe ./build/src/agents/collector/collector-remote-transport-probe \
  --count 3

The probe lists and fetches every message, checks its fixture marker, deletes it through the selected protocol, and verifies that every Cyrus source mailbox is empty. A live end-to-end run creates all four Bongo external-account records. This proves that accounts, remote UID/UIDL values, retry state and deletion state remain scoped by account ID even when one Bongo user owns multiple sources. All four use after_import, so Cyrus deletion happens only after the Bongo Queue has durably accepted and committed the complete message. From that point the normal scanner, rules, quota, and Store delivery pipeline owns it.

The fixture can also seed deterministic scanner and quota cases. All ports remain loopback-only and the flood is deliberately bounded:

sudo ./contrib/testing/cyrus-fixture.py seed --profile gtube --count 1
sudo ./contrib/testing/cyrus-fixture.py seed --profile eicar --count 1
sudo ./contrib/testing/cyrus-fixture.py seed --profile flood --count 32 \
  --message-size 131072
sudo ./contrib/testing/cyrus-fixture.py seed-suite

gtube contains SpamAssassin's standard GTUBE string. eicar creates the harmless standard EICAR attachment at runtime; the complete signature is kept out of the Git checkout to avoid host antivirus false alarms. seed-suite places three normal messages, one GTUBE message, one EICAR message, and 32 128-KiB flood messages in every source mailbox (148 messages total). Set a small quota on disposable Bongo users before collection, then restore it after the test. The test passes only when clean messages are imported, scanner policy is applied, quota cannot be exceeded, and a message rejected before a durable import remains retrievable from Cyrus.

The same LMTP listener is the forwarding target for the exact transport-domain mapping below. This accepts *@cyrus.bongo.test without making the test domain a local Bongo domain:

"lmtp_transports": ["cyrus.bongo.test=127.0.0.1:18024"]

Cyrus lmtpd is pre-authorized only on this loopback-only fixture listener. That matches a trusted local MTA-to-store LMTP hop and must not be copied to a network-exposed Cyrus listener.

The generated short-lived test CA at /tmp/bongo-cyrus-fixture/tls/ca.crt is the fixture trust anchor. Set BONGO_COLLECTOR_CAINFO to that path for the Collector process during this test; normal installations continue to use the operating-system trust store. The packaged bongo.service uses PrivateTmp=true, so a live systemd test must make the fixture visible inside that service namespace. A runtime-only drop-in can bind precisely this fixture read-only:

[Service]
BindReadOnlyPaths=/tmp/bongo-cyrus-fixture

Install it with systemctl edit --runtime bongo.service, restart Bongo, and set the Collector ca_file to the path above. Do not weaken certificate or host-name verification for the test.

After a live collection cycle, show the source-mailbox counts with:

./contrib/testing/cyrus-fixture.py mailbox-status

Stop or remove all fixture state with:

sudo ./contrib/testing/cyrus-fixture.py stop
sudo ./contrib/testing/cyrus-fixture.py reset

Internal SMTP relay policy

The live port-26 check changes rate limits and restarts Bongo several times. It therefore requires an explicit opt-in and must only be run against a disposable test installation:

export BONGO_ALLOW_LIVE_SMTP_TEST=1
./contrib/testing/smtp-internal-relay-check.py

The default test topology binds the allowed client to 172.16.11.190 and the denied client to 127.0.0.1, both connecting to 172.16.11.190:26. Override BONGO_TEST_INTERNAL_HOST, BONGO_TEST_ALLOWED_SOURCE, or BONGO_TEST_DENIED_SOURCE when the fixture uses different addresses. The script verifies exact protocol responses, temporarily tests one connection, one recipient, 64 bytes, and one message per rate window, and restores the original SMTP document even after failure. After every service restart it allows a two-second agent startup grace period and then requires two successful SMTP readiness sessions. Override the grace period with BONGO_TEST_STARTUP_GRACE if slower test hardware needs it. Its DATA transactions are deliberately rejected before Queue commit.

The companion SMTP-05 check temporarily maps the allowed source IP to the device name matrix and points remote delivery at unreachable loopback 127.0.0.2. This safely leaves each accepted .invalid message available for bongo-queuetool show, where the script verifies the actual Queue envelope rather than log text:

export BONGO_ALLOW_LIVE_SMTP_TEST=1
./contrib/testing/smtp-internal-normalization-check.py

It covers root@localhost, another local user, case-insensitive localhost.localdomain, single and nested hosted subdomains converted to plus-addresses, a correct hosted-domain sender that remains unchanged, a lookalike domain-boundary attack, an unrelated domain that is canonicalized, and the null reverse path. Focused native tests cover explicit IP mapping, DNS suffix boundaries, forward-confirmed reverse DNS, invalid host labels, truncation, and the rule that either authenticated submission or a trusted internal connection may relay to a remote recipient. A source outside internal_relay_networks remains rejected before the SMTP greeting.

Authenticated external SMTP provider

Xeams DevNullSMTP is useful as a deliberately simple relay capture, but it cannot exercise an external user's SMTP authentication and TLS settings. smtp4dev-fixture.py starts two independent smtp4dev instances for those tests:

• STARTTLS SMTP: 127.0.0.1:12587, API: 127.0.0.1:15080
• implicit TLS SMTP: 127.0.0.1:12465, API: 127.0.0.1:15081
• username: provider
• disposable password: BongoProvider-Test-2026
• trust anchor: /tmp/bongo-smtp4dev-fixture/tls/ca.crt

Both SMTP listeners and both HTTP APIs bind only to loopback. AUTH is not offered before STARTTLS, only PLAIN and LOGIN are enabled after encryption, and smtp4dev's permissive SmtpAllowAnyCredentials development default is explicitly disabled.

The fixture can use an existing executable through SMTP4DEV_BIN. The following pinned upstream CI build was used for the 0.7 local test:

curl -fL -o /tmp/Rnwood.Smtp4dev-linux-x64-3.16.0-ci20260724100_pr2100.zip \
  https://github.com/rnwood/smtp4dev/releases/download/3.16.0-ci20260724100_pr2100/Rnwood.Smtp4dev-linux-x64-3.16.0-ci20260724100_pr2100.zip
./contrib/testing/smtp4dev-fixture.py prepare \
  --archive /tmp/Rnwood.Smtp4dev-linux-x64-3.16.0-ci20260724100_pr2100.zip

The helper requires the pinned SHA-256 4bcff7d9ecbb62d3abe44888ba38502674b9a634c6b87453e43f1af272bd4249. An archive with another name is accepted only with an explicit --sha256; every member is checked for path traversal and symbolic links before extraction. The downloaded binary is not redistributed by Bongo.

Start and validate the two instances:

./contrib/testing/smtp4dev-fixture.py start
./contrib/testing/smtp4dev-fixture.py verify

The verification performs valid and invalid authentication over both TLS modes, submits a unique local capture through each listener, checks its summary and raw RFC 5322 source through /api/messages, then deletes only those probe messages. Later relay and external-identity tests can query the same supported API to verify the exact envelope, headers, MIME body, DKIM signature, and authenticated provider account.

Remove captured messages without changing the process configuration, or stop and remove the entire disposable fixture:

./contrib/testing/smtp4dev-fixture.py clear
./contrib/testing/smtp4dev-fixture.py stop
./contrib/testing/smtp4dev-fixture.py reset

Outbound SMTP capture

Use net-mail/xeams-devnullsmtp-bin from the Bongo Gentoo overlay:

xeams-devnullsmtp -p 2526 -d bongo.test -s /tmp/bongo-devnull-mail

Configure 127.0.0.1:2526 as Bongo's relay host before testing any non-local recipient. Xeams DevNullSMTP has no listen-address option, so isolate it with a firewall or network namespace; do not expose it on a production network.

HAProxy and PROXY v2

haproxy-bongo-test.cfg exposes loopback-only high ports and forwards every connection to the corresponding normal Bongo listener with a PROXY v2 header. Before starting it, temporarily enable proxy_protocol_enabled and set proxy_protocol_networks to ["127.0.0.1/32"] in the smtp, imap, pop3, and sieve documents. Never trust an unbounded network for PROXY headers.

Validate and run the fixture in the foreground:

haproxy -c -f contrib/testing/haproxy-bongo-test.cfg
haproxy -db -f contrib/testing/haproxy-bongo-test.cfg

The test listeners are SMTP 11025, submission 11587, submissions 11465, IMAP 11143, IMAPS 11993, POP3 11110, POP3S 11995, and ManageSieve 14190. Exercise real protocol greetings, capabilities, authentication, and TLS handshakes through these ports, then restore the original Bongo documents.