master
293 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Mario Fetka
|
737103b716 |
Verify scanner-health fault detection, no SA-age false alarm (SCAN-05)
New live test against the shared live_instance fixture, named scanner-health-live to avoid colliding with the native worker- scanner-health unit test (src/agents/worker/CMakeLists.txt). worker config is re-read fresh on every scheduled job run (LoadConfiguration() at the top of BongoWorkerRunScannerHealth() itself), so unlike web config this needs no bongo-manager restart -- just bongo-testtool scheduler-request to force an immediate run. Proves the row's actual claim operationally: a nonexistent data directory faults for *both* ClamAV and SpamAssassin (availability is checked for both), but a SpamAssassin rule directory containing only a 400-day-old .cf file does *not* fault -- only ClamAV's freshness is ever judged by file mtime (clamav_database_max_age_seconds, scanner-health-support.c's RelevantFile()/CheckScanner()), SpamAssassin's CheckScanner() call always passes a hardcoded maximum_age of 0. Also confirms the fix path: ClamAV recovers to "completed" once pointed back at real data (with a generous max-age, same reasoning as the ADM-05 test -- this host's actual /var/lib/clamav freshness is irrelevant to what this row verifies). Found and fixed a real race while getting this to pass reliably: consecutive scheduler-request calls issued before a prior request had actually been picked up by the worker's own poll cycle could coalesce into the same run rather than trigger a fresh one against whatever config was just written -- silently leaving a later step observing an earlier step's stale result. Fixed by waiting for the scheduler to be genuinely idle (running=0, requested=0) both before requesting and after, via bongo-testtool scheduler-list, not just polling task state until it happened to match. Verified 2x pass against a fresh live instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
6cae406daa |
Verify collected POP3/IMAP mail traverses the SMTP scanner path (SCAN-03)
New live test against the shared live_instance fixture. Traced the code path before writing it, not assumed: CollectExternalAccount() (collector.c) hands each fetched message to CollectorStoreSubmitFile() (external_store.c), which does the same QCREA/QSTOR FLAGS/QSTOR RAW/QSTOR MESSAGE/QRUN sequence SMTP delivery uses to inject into Q_INCOMING. The flags it sets (MSG_FLAG_SOURCE_EXTERNAL|PROXIED_MESSAGE|ENCODING_8BITM|COLLECTED_MESSAGE) deliberately don't include MSG_FLAG_SPAM_CHECKED, so antispam.c's ProcessConnection() (which only ever skips already-checked mail) treats a collected message exactly like an SMTP one. Seeds a GTUBE message on a fake POP3 source and an EICAR message on a fake IMAP source (fake-pop3-source-fixture.py/fake-imap-source-fixture.py, the same machinery collector-flood-quota-live-check.py already established for this instance), creates disposable external accounts via bongo-web's REST API, and confirms the collected GTUBE message gets the same X-Spam-Flag: YES + numeric score smtp-scanner-check.py already verifies for SMTP, and the collected EICAR message never reaches the recipient. Deliberately doesn't re-test quota accounting for collected mail (collector-flood-quota-live-check.py already covers that exhaustively) or the virus bounce recipient (a collected message's "from" is the external source's own synthetic sender, not something this instance can observe delivery to -- the bounce mechanism itself is already proven by SCAN-01, same avirus.c code path regardless of a queue entry's origin). Verified 2x pass against a fresh live instance (~250s each, two real 60s-minimum collector poll cycles). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
78d270f3f6 |
Assert the scanner's numeric score round-trips (0.7 matrix SCAN-01/02)
smtp-scanner-check.py already verified the boolean X-Spam-Flag but never checked that fake-scanner-fixture.py's score/threshold value (spamd.c always replaces the queued message with the scanner's full response, not just a flag) actually makes it into the delivered X-Spam-Status header for either the clean or GTUBE case -- SCAN-02 explicitly calls out "score/header handling", not just the tag. Verified 2x pass against a fresh live instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
c3742a5bf5 |
docs: record CFG-17 and ADM-05 in the 0.7 test matrix
Adds the postmaster-alias and /health+/metrics live-test work from this
session as CFG-17 and ADM-05, with evidence entries in
docs/test-evidence/0.7-r1.md.
Also fixes a real numbering mistake: the postmaster-alias test and its
CMakeLists comment were originally written referencing "CFG-08", picked
without checking the matrix first -- CFG-08 was already a different,
completed row ("Reverse-proxy, HAProxy, direct HTTP, and direct HTTPS
validation paths"). Renumbered to CFG-17 (the next actually-free CFG
row) throughout the test script's docstring/print output/throwaway
domain names and the CMakeLists comment; re-verified the test still
passes after the rename.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
4d95128988 |
Verify /health and /metrics auth gating and real content (0.7 matrix ADM-05)
New live test against the shared live_instance fixture: sets
web.health_secret and web.admin_command (pointing at this instance's own
bongo-admin, not the /usr/bin/bongo-admin default) and restarts the
instance's own bongo-manager to apply them -- no Bongo agent has a live
config-reload path, same restart_manager() pattern
collector-flood-quota-live-check.py already established for this exact
instance (enumerate the whole child process tree, not just the manager
PID, since bongo-manager's own SIGTERM doesn't cascade to its children).
Covers all four /health auth cases (no credentials, wrong Bearer token,
correct Basic with the fixed "health" username, correct Bearer) plus the
same gate on /metrics, then checks /metrics for real content: manager
up, at least one agent's enabled state, queue depth, and collector
backlog all present on an idle instance; worker task state verified by
actually forcing a ClamAV-data-missing fault (scanner_health.
clamav_database_directories pointed at a directory that doesn't exist,
scanner-health run via bongo-testtool scheduler-request) and confirming
bongo_worker_task_open{job="scanner-health",key="clamav"} is 1, then
clearing the fault and confirming it flips back to 0.
The "clear the fault" step deliberately does not just restore the
original config: this host's real /var/lib/clamav signature age is
whatever freshclam last left it at (observed ~10 days during
development, past the 48h production default), which is irrelevant to
what this test verifies. Restores the real data directories but with a
large clamav_database_max_age_seconds, isolating "does the fix path
clear the task" from "is this host's ClamAV installation currently
fresh" -- the assertion must not depend on when the test happens to run
relative to the host's last signature update.
Verified 2x pass against a fresh live instance.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
3d115eccdb |
Fix restart_manager() readiness checks broken by /health now requiring auth
Both collector-flood-quota-live-check.py's restart_manager() (three call sites) and collector-flood-quota-check.py's restart_bongo() waited for readiness by polling GET /health for a 200 -- which now always 401s, since web.health_secret is empty by default and neither script has any other reason to know it. Every restart would have spun for the full 30s and then failed with "Bongo Web did not become ready after restart" even though the server was perfectly healthy. Replaced with a plain TCP-connect check on the same host/port (the web_target = urlsplit(WEB_URL) + wait_port() pattern this same function already uses for the Store/Queue ports), since both scripts only ever needed "is the HTTP server accepting connections yet", not anything /health-specific. Verified: collector-flood-quota (three restart_manager() calls per run) passes end to end against a fresh live instance, 165s runtime, no regression. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
97e15d67f6 |
Verify the default postmaster alias routes to admin (0.7 matrix CFG-08)
New live test (wired via bongo_add_live_test against the shared live_instance fixture, no dedicated instance needed): submits directly to postmaster@<the instance's own domain> from an external, unauthenticated sender -- matching how a real postmaster report/bounce/ abuse notice arrives -- and confirms delivery to admin's own Store INBOX, with zero alias configuration added by this script or by live-instance-fixture.py's bootstrap(). aliases.d/default_config's "postmaster": "admin" fragment has domainalias="" (applies globally, not to one specific domain), so this is a faithful test of the row's actual claim without needing a second, freshly-added domain -- the mechanism being verified doesn't distinguish domains in the first place. Verified 2x pass against a fresh live instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
7a654f478e |
Cover SRS encode/decode/tamper/expiry/bounce/loop handling (0.7 matrix MAILAUTH-07)
Debian Trixie package bundle / packages (push) Failing after 12m13s
Extends smtp-srs-forward-live-check.py (previously encode-only) with
live coverage of the rest of MAILAUTH-07, and enables srs_reverse on
the shared dkim_dmarc_instance so decode-side tests can run there too.
Two genuine smtpd.c bugs found and fixed along the way, both only
observable once srs_reverse was actually turned on:
1. The RCPT TO SRS-detection check only tested a bare 3-character "SRS"
prefix (strncasecmp(name, "SRS", 3) == 0), unlike libsrs2's own
SRS_IS_SRS_ADDRESS(x) macro, which also requires x[3] in {'0','1'}
and x[4] to be a real separator. Any real mailbox or alias merely
starting with those letters -- this instance's own pre-existing
"srsforward" alias localpart included -- was misdetected as an
SRS0/SRS1 address and rejected outright the moment srs_reverse was
enabled. Confirmed by first reproducing the break on the existing
encode test. Fixed to match libsrs2's own check exactly.
2. A successfully SRS-decoded recipient was still subject to the normal
SMTPRemoteRecipientAllowed(IsAuthed, IsTrusted) anti-relay gate. Since
srs_forward() (srs2.c) never wraps a sender whose domain matches
srs_domain, a decoded SRS address is *always* non-local -- and every
real bounce/DSN arrives from an arbitrary, unauthenticated external
MTA -- so this made reverse SRS decoding entirely unreachable for its
actual purpose: every real bounce to a Bongo-forwarded message would
have been rejected as relay-denied. Fixed by treating a successful
BongoMailAuthSRSReverse() as a bounded relay exception for that one
recipient.
Verified via bongo-queuetool (a decoded recipient is queued for relay to
the *original*, pre-SRS sender address, not just "accepted") and via a
synthetic prior-hop SRS0 sender for the loop case, since srs_forward()'s
own domain-match short-circuit means an address this instance minted
itself never reaches srs_compile_guarded()'s SRS1-compression branch --
only a second, different-domain hop does, the actual real-world scenario
that branch exists for. tamper/expiry both collapse to the same SMTP 550
(libsrs2 maps every srs_reverse() failure mode to one SRS_FAILURE); the
expiry case forges a timestamp exactly matching srs2.c's own 2-character
base32 encoding so it is genuinely exercising SRS_ETIMESTAMPOUTOFDATE
before the hash is ever checked, not just SRS_EHASHINVALID again.
Verified 2x pass against a fresh instance, plus a full regression run of
every other test sharing dkim_dmarc_instance (dkim-dmarc-check,
dane-mta-sts-check, smtp-opportunistic-tls-live-check,
smtp-required-tls-live-check, smtp-forward-metadata-live-check,
smtp-command-safety-live-check, smtp-forward-header-live-check,
smtp-proxy-protocol-live-check) plus smtp-open-relay/smtp-internal-relay/
smtp-internal-relay-live, given the relay-permission change: 18/18 green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
29f732bf9b |
Cover missing/mismatched/rotated DKIM key safety (0.7 matrix MAILAUTH-06)
Adds three cases exercising GenerateDKIMSignature()/ReadPrivateKey()
(src/libs/mailauth/dkim.c) key-failure handling on the outbound path:
- missing: the key file is unlinked before submission.
- mismatched: the key file is overwritten with content that isn't a
valid PEM private key at all.
- rotated: the key file is replaced with a genuinely different valid
keypair; delivery must succeed immediately (no manager restart)
and the signature must independently verify against the *new*
key, proving GenerateDKIMSignature() reads the key fresh from disk
per message rather than caching it.
For missing/mismatched, PrepareExternalMessage() (smtpc.c) requires
either a real signature or SRS rewriting before it will even open the
outbound connection; this authenticated submission qualifies for
neither, so the message must retain in Queue rather than leak out
unsigned, then recover and deliver normally once the key is restored.
Investigated but deliberately left out the matrix row's fourth case,
"unreadable" (a chmod(0o000) key file): confirmed via strace that
openat() on the key path still succeeds, because this whole instance
runs as netns-fixture.py's "fake root" (`unshare --user
--map-root-user`) -- a real Linux capability (CAP_DAC_OVERRIDE)
granted for any file the same real UID owns, not a Bongo behavior this
suite can turn off. The "missing" case already exercises the identical
fopen()-failure code path in ReadPrivateKey() regardless of whether
the underlying errno would be ENOENT or EACCES in a real deployment,
so C-level coverage of that failure mode isn't actually lost.
Verified 2x pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
3c329a8564 |
Independently cryptographically verify outbound DKIM (0.7 matrix MAILAUTH-05)
assert_dkim_signature() only ever checked the DKIM-Signature header's
d=/s= parameters superficially -- never that the signature actually
verifies. MAILAUTH-05 explicitly requires an independent verification
pass, not Bongo grading its own homework.
Adds verify_dkim_signature_independently(): re-derives the public key
from this instance's own self-provisioned ("temporary") private key
file and performs the full RFC 6376 relaxed/relaxed canonicalization
and RSA-SHA256 verification by hand, exactly as an unrelated verifier
would, rather than trusting Bongo's own dkim.c to grade its own
signature.
Writing this surfaced a real bug in the verifier itself, not Bongo:
naively using message.get(name) for each entry in a signature's h=
list returns the same (first) header instance for every repeated name,
but a two-hop relay's captured mail genuinely has two Received:
headers and h=Received:Received:..., so opendkim actually signed the
first and second Received field separately. RFC 6376 5.4.2 requires
consuming repeated header names from the *last* instance upward for
each h= occurrence, not the same instance repeatedly -- fixed by
pulling all instances via get_all() and popping from the end.
Verified 2x pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
612360927f |
Cover forged Authentication-Results stripping (0.7 matrix MAILAUTH-04)
Adds a case sending an unsigned message that arrives with an attacker-supplied Authentication-Results header falsely claiming dkim=pass/dmarc=pass. Confirms SMTPAuthenticationResultsSanitize() (src/agents/smtp/auth-results.c) strips it unconditionally and that Bongo's own freshly computed result (dkim=none for an unsigned message) is what actually survives -- checking both that the forged claim's distinguishing content is gone and that exactly one Authentication-Results header remains on the delivered copy, not two. Verified 2x pass. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
d04f9c9368 |
Cover DMARC alignment and none/quarantine/reject/temp policies (0.7 matrix MAILAUTH-03)
dkim-dmarc-check.py only ever asserted dmarc=pass as a side effect of its DKIM "valid" case; MAILAUTH-03 requires exercising all four policy dispositions and confirming which ones Bongo actually enforces at the SMTP level. Adds four throwaway domains, each with its own _dmarc TXT policy (none/quarantine/reject, plus a SERVFAIL domain for temperror), each sent a deliberately unaligned message with smtp.dmarc_enforce turned on. src/agents/smtp/smtpd.c only takes an SMTP-level action for BONGO_DMARC_DISPOSITION_REJECT (550) and DMARC temperror (451, regardless of disposition) -- p=none and p=quarantine are both accepted normally either way, matching real DMARC semantics where quarantine is a mailbox-side decision, never an SMTP-level one. Getting a genuinely unaligned probe message right needed one non- obvious fix: libspf2 hardcodes an unconditional SPF pass for a loopback client regardless of any DNS data, and this instance's bongosmtp never gets a BONGO_TEST_SPF_ZONE_FILE to substitute a non-loopback test address. Matching the probe's envelope-from domain to its header-from domain let that meaningless SPF pass count as *aligned*, satisfying DMARC through SPF no matter what DKIM did -- every probe silently passed regardless of policy. Using a mismatched envelope domain (still no DKIM signature at all) keeps that pass from ever aligning, so the result genuinely depends on DKIM alone. Also fixed a real bug this surfaced: the DMARC policy phase restarts bongo-manager (tearing down and respawning bongostore) while the script's own Store connection, opened once at the top of main(), was still held open across that restart and then used again in the top-level cleanup -- failing with "Broken pipe" once the policy phase was added. Closes and clears that connection before the restart phase begins instead of after. Verified 2x pass, plus the full set of other tests sharing the same dkim_dmarc_instance fixture, to confirm the extra manager restarts introduced here don't leave shared instance state broken for them. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
5a8d9ea987 |
Cover invalid/multiple/DNS-error DKIM cases (0.7 matrix MAILAUTH-02)
dkim-dmarc-check.py only ever exercised valid (dkim=pass) and modified
(dkim=fail via a body tampered after signing), leaving three of the
five MAILAUTH-02 cases untested against a live instance:
- invalid: a cryptographically well-formed signature referencing a
selector with no published key at all -- RFC 6376 6.1.2 treats
"no key found" as permerror, distinct from a wrong/tampered
signature.
- multiple: two DKIM-Signature headers on one message, one valid and
one referencing the no-key selector -- confirms the aggregate
result still passes (BongoMailAuthDKIMVerifierFinish() in
src/libs/mailauth/dkim.c prioritizes any passing signature).
- DNS error: the selector's _domainkey TXT lookup itself fails
(SERVFAIL), not merely absent -- must resolve to temperror, not
permerror.
The DNS-error case needed a fixture change: fake-dns-fixture.py could
only answer a listed record or NXDOMAIN for anything unlisted, with no
way to express "the lookup itself failed" at all. Adds a "SERVFAIL
<fqdn> <type>" zone-file directive; the DKIM test appends one for a
throwaway selector and reloads the running fixture over SIGHUP, the
same append-and-reload pattern smtp-opportunistic-tls-live-check.py's
own register_zone() already uses for MX-side test domains.
Verified 2x pass, plus the full set of other tests sharing the same
dkim_dmarc_instance fixture (fake-dns-fixture.py is shared state) to
confirm the zone-loader signature change didn't regress anything else.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
9e8d4b40da |
Cover all seven RFC 7208 SPF results in the live SPF test
Debian Trixie package bundle / packages (push) Failing after 10m3s
smtp-spf-check.py only ever exercised pass and fail, leaving softfail, neutral, none, temperror, and permerror completely untested against a live Bongo instance (0.7 release matrix row MAILAUTH-01 requires all seven). temperror specifically was structurally impossible to test before this change: BongoMailAuthTestZoneSPFServer() (src/libs/mailauth/spf.c) hardcoded NETDB_SUCCESS for every zone entry, so a DNS lookup could never itself fail -- only a malformed record could be simulated (permerror), never a temporary DNS failure. Adds an "ERROR:<name>" value convention to the zone file format (HOST_NOT_FOUND, TRY_AGAIN, NO_RECOVERY, or NO_DATA), letting a domain resolve as a failing DNS lookup instead of a TXT answer. softfail/neutral use "~all"/"?all" records; none relies on SPF_DNS_ZONE's own closed-world default (HOST_NOT_FOUND for anything not explicitly added, no fallback to real DNS) by simply never adding that domain; permerror uses a malformed IPv4 literal in the record itself. Verified 2x pass against a fresh live instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
1fdf31050a |
Harden restart_manager()'s readiness probe against a bongoqueue auth race
A full plain `ctest` run intermittently failed a live test's very first post-restart bongo-queuetool call with "Failed authentication with store" -- reproduced twice, each time on a different live test (smtp-dsn-live-check, then separately smtp-postfix-interop-live-check in an earlier run), never in isolated reruns. Every affected script's restart function only waited for the SMTP listener to answer before declaring the restart complete and immediately issuing queue-tool calls; nothing actually confirmed bongoqueue itself (a separate agent, started as one of several children in whatever order bongo-manager's supervisor loop happens to schedule them) had finished loading its own NMAP AUTH SYSTEM credential and Store-backed config first. Adds wait_queue_ready() to every already-wired live test that calls bongo-queuetool after a restart (smtp-dsn, smtp-alias-routing, smtp-postfix-interop, smtp-recipient, smtp-internal-normalization, collector-flood-quota, smtp-forward-metadata): retries a real, lightweight `queue list` round trip for up to 15s instead of assuming the SMTP listener answering is proof enough. Directly targets the observed failure mode rather than adding a guessed fixed settle delay. Verified each of the seven individually, then a full 210/210 plain `ctest` run with no failures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
18c14a103e |
Wire smtp-postfix-interop into ctest as a self-contained live test
smtp-postfix-interop-check.py depends on Postfix's smtp-source/ smtp-sink developer tools, unpackaged anywhere on this host and buildable only from a full Postfix source checkout. Investigated Postfix's own upstream test suite as a possible source of additional adaptable test ideas (per user request) and found it consists solely of implementation-internal C unit tests (src/ptest/) with no protocol-level or integration coverage -- nothing else worth porting. smtp-postfix-interop-live-check.py drops the external-tool dependency entirely instead: run_concurrent_load() is a small threaded SMTP client replacing smtp-source's -s/-m session/message fan-out, and LoadSinkServer is an in-process socketserver fixture (same pattern as smtp-dsn-live-check.py's DSNHandler) replacing smtp-sink, supporting LMTP and plain-SMTP framing plus a runtime-toggleable accept/reject/ disconnect mode. Covers the same scenarios as the original: concurrent load through both the authenticated submission and public inbound listeners, concurrent LMTP-transport and relay-host (DKIM-signed) delivery, and Queue retain-then-retry-then-deliver recovery after a hard 4xx rejection or an abrupt mid-transaction disconnect. Needs its own dedicated netns instance for the same use_relay_host exclusivity reason as smtp-dsn-live-check.py. Verified 4x pass in isolation; one transient timeout under a full 210-test serial ctest run led to bumping BONGO_TEST_TIMEOUT to 90s for headroom, then reproduced instead (once) on an unrelated, pre-existing test (smtp-dsn-live-check, itself clean in isolation) during a follow-up full run -- confirms general host-load-sensitive timing under a long serial run rather than a bug in this test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
abf6dd2bbc |
Add outbound_tlsrpt_ca_file and wire smtp-tlsrpt-delivery into ctest
Debian Trixie package bundle / packages (push) Successful in 17m17s
DeliverHttps() (src/agents/worker/jobs/tlsrpt-delivery.c) had no way to trust anything but curl's system CA bundle for a TLSRPT rua= HTTPS endpoint -- unlike the SMTP relay leg's own relay_tls_ca_file, there was no equivalent override, so a private-CA or self-signed endpoint had no path to being trusted short of installing it into the whole host's system-wide trust store. Adds outbound_tlsrpt_ca_file, mirroring relay_tls_ca_file's config template default, schema validation, and CURLOPT_CAINFO wiring. smtp-tlsrpt-delivery-live-check.py ports smtp-tlsrpt-delivery-check.py's RFC 8460 coverage (mail and HTTPS report delivery, DKIM requirement, aggregation, durable retry across a restart, and expiry) onto its own dedicated netns-instance-fixture.py instance, using the new CA-file override to trust its own disposable test PKI instead of needing real root for system-wide CA installation. Needs its own dedicated instance for the same use_relay_host exclusivity reason as smtp-dsn-live-check.py. Verified 2x pass against a fresh instance, plus a full 207/207 ctest regression with no other failures. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
e9ea832865 |
Wire smtp-smtputf8 and imap-reconnect-consistency into ctest
smtp-smtputf8-live-check.py: ports smtp-smtputf8-check.py's RFC 6531 coverage (SMTPUTF8 parameter/header validation on the inbound path, capability-gated relay of SMTPUTF8/BODY=8BITMIME, and 5.6.7 DSN status when relaying to a peer that can't accept it) onto its own dedicated netns-instance-fixture.py instance -- needs one of its own for the same use_relay_host exclusivity reason as smtp-dsn-live-check.py, and routes through the authenticated STARTTLS submission listener instead of the production internal_relay listener no netns instance provisions. imap-reconnect-consistency-live-check.py: ports imap-reconnect- consistency-check.py's concurrent-session and restart-consistency coverage (parallel APPEND UID uniqueness, cross-session flag/COPY/ EXPUNGE visibility, stale-connection severance, and full Store state survival across a manager restart) onto the shared live_instance fixture's own restart_manager() pattern instead of a real `sudo systemctl restart bongo.service`. Both verified 2x pass against a fresh instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
32b4ac5df7 |
Wire smtp-dsn into ctest as a dedicated netns live test
Ports smtp-dsn-check.py's RFC 3461 coverage (MAIL/RCPT DSN parameter syntax, RET/ENVID/NOTIFY/ORCPT propagation to a relay peer, DSN suppression against a peer that doesn't advertise it, and generated success/failure report content) onto a dedicated netns-instance- fixture.py instance rather than a real `sudo systemctl restart bongo.service`, following smtp-relayhost-live-check.py's precedent: smtp.use_relay_host is a global, all-outbound-mail toggle exclusive with every other netns test's direct-MX assumption, so it needs its own instance rather than the shared dkim_dmarc_instance. No netns instance provisions the production internal_relay listener smtp-dsn-check.py used, so this submits DSN-parameterized mail through the authenticated STARTTLS submission listener instead (the same path smtp-relayhost-live-check.py's submit() uses), and relays to an in-process capturing SMTP fixture the script starts itself rather than a separate fake-*-fixture.py file. Verified 2x pass against a fresh instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
91414e00a0 |
Wire smtp-alias-routing into ctest as a live_instance test
Debian Trixie package bundle / packages (push) Successful in 17m33s
Ports smtp-alias-routing-check.py's coverage (address aliases, domain aliases, chain resolution, alias-loop rejection, canonical-address deduplication) onto the shared live_instance fixture instead of a real `sudo systemctl restart bongo.service` against the production bongo.service, following the same restart_manager()/alias-fragment mirror pattern smtp-recipient-live-check.py already established. Verified 2x pass against a fresh instance. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
69270bba17 |
Wire smtp-recipient into ctest; fix two bugs found while verifying it
Debian Trixie package bundle / packages (push) Failing after 13m50s
smtp-recipient-check.py (kept as a standalone manual script) needs
real root to apply alias-fragments via a real systemctl restart, and
checks its temporary alias fragment's file mirror at the hardcoded
production path /etc/bongo/aliases.d/. smtp-recipient-live-check.py
does the same restart against just the live_instance scratch prefix's
own bongo-manager instead, and checks that scratch prefix's own mirror.
Applying it uncovered a real bug in bongo-manager itself:
WriteAliasSourceDirectory() (manager.c) unconditionally tries to
regain root via XplSetEffectiveUserId(0) before writing the alias
mirror files, then restore the caller's original EUID afterward. A
CTEST_BUILD_NOROOT instance never has real root to begin with (see
main()'s own getuid()==0 check), so the escalation call always fails,
which made WriteAliasSourceDirectory() fail, which made bongostore
itself exit at startup ("Couldn't apply local configuration") --
breaking the *entire* instance any time alias-fragments changes,
confirmed independently of this new test by reproducing against the
unmodified binary before applying the fix. Guarded the escalation call
behind #ifndef CTEST_BUILD_NOROOT, matching the same tolerance pattern
already used elsewhere (MsgMakePathChown(), spf.c) -- production
builds are byte-for-byte unaffected. The matching restore call needs
no guard: seteuid() to your own current EUID is always permitted
regardless of privilege, so it was already a safe no-op.
Also fixes a second, independent pre-existing bug found while
confirming the above wasn't a regression: smtp-pipelining-check.py's
run_queue() unconditionally used `sudo -n -u bongo <path>` with no
unprivileged-mode check and no --host/--port flags, unlike every other
script's established _bongo_queue_argv pattern -- broken against a
live_instance's own scratch-prefix bongo-queuetool (the passwordless
sudo rule only covers the real system path), silently failing (with a
sudo password prompt) whenever cleanup() found matching Queue entries.
This is a shared helper for smtp-8bitmime/smtp-bdat/smtp-binarymime
too, so all four were affected.
Verified: smtp-recipient passes 2x in a row; a full clean production
(non-CTEST_BUILD_NOROOT) build of bongo-manager still compiles
byte-identical-logic; the pipelining family (4 tests) passes cleanly
after its own fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
0986d07c19 |
Wire smtp-internal-relay into ctest as a live_instance test
smtp-internal-relay-check.py needs real root to apply the internal_relay rate-limit fields via a real systemctl restart. smtp-internal-relay-live-check.py does the same restart against just the live_instance scratch prefix's own bongo-manager instead, reusing the same child-tree-kill restart_manager() pattern established for smtp-stress-timeout-live-check.py. BONGO_TEST_DENIED_SOURCE overrides the script's own default (127.0.0.1, which collides with ALLOWED_SOURCE's own default in this loopback-only environment) with a distinct loopback alias (127.0.0.2). Named smtp-internal-relay-live, not smtp-internal-relay, to avoid colliding with src/agents/smtp's own C-level unit test of that name. Verified: passes 2x in a row, no new bugs found beyond the naming collision and source-address default. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
f9346de96a |
Wire smtp-internal-normalization into ctest as a live_instance test
smtp-internal-normalization-check.py needs real root to apply internal_relay_device_mappings via a real systemctl restart. smtp-internal-normalization-live-check.py does the same restart against just the live_instance scratch prefix's own bongo-manager instead, reusing the same child-tree-kill restart_manager() pattern established for smtp-stress-timeout-live-check.py. Unlike the netns-based smtp-22/23/24/25 family, live-instance-fixture.py's own bootstrap() already provisions a real internal_relay listener for the instance's own domain, matching this script's own precondition checks, so nothing needs enabling here beyond what already exists -- ported cleanly with no new bugs found. Verified: passes 2x in a row. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
c3d4be6fc9 |
Wire smtp-telnet-timeout into ctest as a live_instance test
smtp-telnet-timeout-check.py needs real root to apply socket_timeout
via a real systemctl restart. smtp-telnet-timeout-live-check.py does
the same restart against just the live_instance scratch prefix's own
bongo-manager instead, reusing the same child-tree-kill restart_manager()
pattern established for smtp-stress-timeout-live-check.py.
Also fixes a real config bug the original script never accounted for
(and this live version initially hit too): smtpd.c's config validation
requires stress_socket_timeout <= socket_timeout ("SMTP stress socket
timeout must be between 1 and the normal timeout"), or bongosmtp
rejects the whole config and exits at startup. The original only sets
socket_timeout, which happened to work in production because its
default stress_socket_timeout is already lower than a typical
production socket_timeout -- but this test's own deliberately low
socket_timeout (3s, to keep the test fast) ended up *lower* than the
live_instance's default stress_socket_timeout (10s), breaking it. Now
clamps stress_socket_timeout to stay within the constraint.
Verified: passes 2x in a row.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
a7f0fa1132 |
Wire smtp-proxy-protocol into ctest, on the dkim_dmarc_instance netns
smtp-proxy-protocol-check.py (kept as a standalone manual script) and its shared helper smtp-forward-metadata-check.py both need real root. smtp-proxy-protocol-live-check.py reuses smtp-forward-metadata-live- check.py's config/restart helpers, gated behind a real haproxy binary (find_program(HAPROXY_EXECUTABLE), test simply isn't registered if the host doesn't have one -- matching connio's own optional-openssl precedent) since this is the one script in the family that needs it, for the send-proxy-v2 public-listener case. Chased what looked like a genuine Bongo hang (bongosmtp never answering on any port after a restart, sockets shown LISTENing but every connection timing out) all the way to a gdb backtrace + a read of SMTPApplyProxyProtocol() (smtpd.c ~349) before concluding it wasn't a bug: proxy_protocol_networks=["127.0.0.1/32"] correctly makes Bongo require a PROXY line from any 127.0.0.1 connection before it will speak SMTP at all (ConnReadProxyProtocol(), 5s wait) -- and the shared module's own generic restart_manager() readiness probe connects from that exact address without ever sending one, so it was always going to time out on its own once this test's config took effect. Fixed by letting that probe fail silently and relying on this script's own PROXY-aware wait_ready() instead, which does send a header. Verified: passes 2x in a row standalone (350s timeout budgets in the generic probe's own wasted ~60s per restart), and together with the rest of the dkim_dmarc_instance test family (16/16 passing). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
de1126486e |
Wire smtp-forward-header into ctest, on the dkim_dmarc_instance netns
smtp-forward-header-check.py (kept as a standalone manual script) and
its shared helper smtp-forward-metadata-check.py both need real root.
smtp-forward-header-live-check.py reuses smtp-forward-metadata-live-
check.py's config/restart helpers and MetadataSMTPHandler fixture
directly, dropping the now-unnecessary port-stealing config ("port":
TEMPORARY_EXTERNAL_PORT) the original needed only because production
shares one loopback address with the fixture -- this netns instance's
fixture already binds its own dedicated 127.0.0.30. Also fixes the
same "trust list still says 127.0.0.1" leftover already found and
fixed in smtp-forward-metadata-live-check.py: the "fallback"/"xforward"
cases' proxy_header/xforward trust entries now reference FIXTURE_HOST
(127.0.0.30) instead of the original script's 127.0.0.1 assumption.
Verified: passes 2x in a row, first try after porting (no new bugs
found beyond the ones already fixed in the shared helper).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
038366c290 |
Wire smtp-command-safety into ctest, on the dkim_dmarc_instance netns
smtp-command-safety-check.py (kept as a standalone manual script) and
its shared helper smtp-forward-metadata-check.py both need real root.
smtp-command-safety-live-check.py reuses smtp-forward-metadata-live-
check.py's config/restart helpers instead, enabling internal_relay
itself the same way that module's own direct_configuration() does.
One assertion needed correcting, not just porting: the NUL-byte
injection case ("NOOP\x00INJECT") expected the connection to be force-
closed. SMTPParseCommandLine() (protocol.c ~89) compares strlen(line)
against the actual bytes read for the line -- an embedded NUL makes
those differ, so the whole line is safely rejected as
SMTP_COMMAND_UNKNOWN (500) without ever executing "INJECT" as a second
command. That's correct, memory-safe behavior (confirmed by reading the
parser, not just by the test failing) -- Bongo replies and keeps the
connection open for further commands, like any other malformed input in
this same test, rather than force-closing it. Adjusted the test to
match instead of asserting a disconnect that was never Bongo's actual
behavior.
Verified: passes 2x in a row.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
f18afd4a43 |
Wire smtp-forward-metadata into ctest, on a dedicated netns instance
smtp-forward-metadata-check.py (kept as a standalone manual script) was
never wired into ctest at all -- it needs real root to temporarily
steal 127.0.0.1:25 from production's own bongo.service, so the fixture
"external" server it drives can bind the exact address Bongo's own
listener normally owns. smtp-forward-metadata-live-check.py reuses the
shared dkim_dmarc_instance netns instead: extra loopback aliases are
free there (matching fake-dane-target-fixture.py's 127.0.0.2, smtp-
outbound-opportunistic-tls-check.py's 127.0.0.10-13, smtp-required-tls-
live-check.py's 127.0.0.20-25), so the fixture gets its own dedicated
127.0.0.30:25 and nothing needs stealing -- just a restart_manager()
child-tree-kill cycle to apply config, same pattern as the other netns
live tests.
internal_relay -- the trusted listener this test injects mail through
-- is not enabled by netns-instance-fixture.py's bootstrap() by
default (unlike the shared live_instance fixture), so this test enables
it itself as part of its own temporary config.
Two real bugs found and root-caused via strace while debugging this:
- smtpd.c's config validation rejects internal_relay_enabled unless
internal_relay_domain is non-empty ("Internal SMTP relay requires a
normalization domain") -- easy to miss since this instance's own
default is "".
- smtpc.c's PrepareExternalMessage() requires every outbound message
to be either DKIM-signed or SRS-rewritten, or delivery aborts
immediately after connect() with no SMTP dialog at all (looks
identical to "nothing is listening" from the receiving side).
internal_relay rewrites the envelope sender's domain to
internal_relay_domain, so an invented domain with no DKIM key
silently broke every delivery attempt; fixed by reusing this
instance's own already-DKIM-provisioned domain
(netns-instance-fixture.py's DEFAULT_DOMAIN) instead.
Also serves as the shared helper module for the (not yet wired)
smtp-command-safety-live-check.py, smtp-forward-header-live-check.py,
and smtp-proxy-protocol-live-check.py -- collapsing the original script
family's SMTP22-imports-SMTP07 layering into one file, since none of
those three need smtp-outbound-opportunistic-tls-check.py's much
larger DNS-fixture/resolver machinery.
Verified: passes 2x in a row standalone, and together with the rest of
the dkim_dmarc_instance test family (13/13 passing) to confirm no
state leaks between tests sharing that instance.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
85a27dbd64 |
Wire smtp-stress-timeout into ctest as a live_instance test
smtp-stress-timeout-check.py needs real root to apply socket_timeout/ stress_socket_timeout, which have no live-reload path, via a real `systemctl restart bongo.service`. smtp-stress-timeout-live-check.py does the same restart against just the shared live_instance scratch prefix's own bongo-manager instead, reusing the same child-tree-kill restart_manager() pattern established for collector-flood-quota-live- check.py and the netns-based live tests. Also corrects the CMakeLists.txt comment explaining why smtp-command- safety/smtp-forward-header stay unwired: smtp-proxy-protocol-check.py shares the same smtp-forward-metadata-check.py helper and its restart_bongo() call, and was missing from that comment's enumeration. Verified: smtp-stress-timeout passes 2x in a row. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
cb97907310 |
Wire smtp-required-tls and smtp-relayhost live tests into ctest
smtp-required-tls-live-check.py: new ctest-driven live test for both outbound_tls_required (global) and RFC 8689 REQUIRETLS (per-message), using the dkim_dmarc_instance netns and fake-dane-target-fixture.py's new "requiretls" mode (identical to "tls" but advertises REQUIRETLS post-STARTTLS). smtp-relayhost-live-check.py: reworked to use a new self-contained fixture, fake-relay-target-fixture.py, instead of smtp4dev -- the package should not need to compile/vendor a .NET runtime just to prove relay/AUTH/DKIM plumbing works, the same reasoning that already led to fake-pop3/imap-source-fixture.py replacing Cyrus for the collector tests. The fixture runs three simultaneous listeners (no-auth STARTTLS, auth-required STARTTLS, auth-required implicit TLS) and captures delivered mail to disk for the test to inspect directly, instead of smtp4dev's REST API. CMakeLists.txt drops the smtp4dev-fixture-prepare FIXTURES_SETUP and BONGO_SMTP4DEV_* variables entirely. Both scripts, plus collector-flood-quota-live-check.py, get a restart_manager() fix: bongo-manager's own SIGTERM handling does not cascade to its agent children (confirmed empirically while debugging the relayhost "unreachable" phase -- killing only the manager PID left bongostore/bongosmtp/bongosmtpc/etc. running as orphans, still bound to their ports and still serving the *old* pre-restart config; production only gets away with a bare kill because systemd's cgroup accounting tears down the whole unit). All three now enumerate and terminate the whole child tree via `ps --ppid` before spawning a fresh manager. collector-flood-quota-live-check.py also fixes a real, independently existing flake: fake-pop3-source-fixture.py's seed() writes messages to randomly named "<uuid4>.eml" files and sorts by that random filename, so which of the pending-limit phase's two messages the collector fetches first isn't deterministic. The test hardcoded acceptance of only one of the two possible markers, failing on roughly half of all runs whenever the other message won the race; now accepts either. Verified: smtp-required-tls-live-check and smtp-relayhost-live-check each pass 2x in a row; collector-flood-quota passes 3x in a row after the marker fix (previously flaked on ~half of runs). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
66139c1311 |
Wire smtp-required-tls into ctest; scaffold smtp-relayhost (WIP)
Debian Trixie package bundle / packages (push) Failing after 11m56s
smtp-required-tls-live-check.py reuses the dkim_dmarc_instance netns, same as smtp-opportunistic-tls-live-check.py. Two independent things under RFC 8689 REQUIRETLS and legacy outbound_tls_required: - Global smtp.outbound_tls_required=True is mutually exclusive with the shared instance's other tests (opportunistic-tls's broken-tls case needs plaintext fallback to actually fire), so this temporarily flips it and restarts just this netns's own bongo-manager for that phase, then restores it -- same pattern as collector-flood-quota-live- check.py's pending-limit phase. - Per-message REQUIRETLS needs no config change (requiretls_enabled defaults true) but does need smtpc.c's SMTPRequireTlsHasAuthenticated Policy() gate satisfied -- a REQUIRETLS message refuses to even attempt STARTTLS against an unauthenticated destination (protocol.c), confirmed via strace: an unsigned-zone target never got past DNS/DANE lookups before smtpc deferred it. fake-dane-target-fixture.py's "msg-requiretls"/"msg-tls-only" targets need real DANE (update_zone= True) to clear that gate; "msg-plain" deliberately stays unsigned as a distinct negative case. A REQUIRETLS message smtpc.c can't relay at all (no authenticated policy, or a TLS-capable-but-not-REQUIRETLS- capable next hop) turned out to bounce immediately (DELIVER_FAILURE), not defer -- confirmed via a direct StoreClient read of the sender's own INBOX, not the assumption the original manual script encoded. Also adds fake-dane-target-fixture.py's fourth mode, "requiretls": identical to "tls" except the post-STARTTLS EHLO also advertises REQUIRETLS, the one thing that distinguishes the positive REQUIRETLS case from "msg-tls-only"'s negative one. smtp-relayhost-live-check.py is scaffolding only, not yet passing: its "relay unreachable" phase assumed ECONNREFUSED defers for retry, but smtpc.c's DELIVER_REFUSED groups with the other retryable codes at the QMOD RAW/QRTS switch (smtpc.c ~2337) -- the bounce actually observed needs more strace-based root-causing before trusting either the test or concluding it's a real bug. It also currently depends on smtp4dev- fixture.py (a real, externally-downloaded relay/capture server); this needs to move to a lightweight, self-contained Python fixture instead, matching fake-pop3/imap-source-fixture.py's replacement of Cyrus for the collector tests, so the project doesn't need an external .NET runtime dependency for something libmailutils-collector-style fixtures already prove doesn't need one. Verified: smtp-required-tls-live-check passed twice in a row via ctest end to end. smtp-relayhost-live-check is wired but currently fails; do not treat it as working yet. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
9623321728 |
Wire collector external-account flood/quota test into ctest; fix bongo-web Store target
collector-flood-quota-live-check.py replaces the old real-Cyrus-rooted
collector-flood-quota-check.py: two new protocol-minimal fixtures,
fake-pop3-source-fixture.py and fake-imap-source-fixture.py, stand in for
Cyrus as external POP3/IMAP sources, satisfying libcurl's actual protocol
requirements (STLS/STARTTLS negotiation, stable POP3 UIDL, IMAP UID
FETCH/STORE with client-tag echoing) rather than a loose test double. The
test seeds a flood of messages across both protocols, verifies convergence
(no duplicates/loss, quota boundary respected), then reconfigures the
collector's per-user pending-message limit down to 1 and verifies the
enforcement boundary (exactly one message imported, one left at the
provider, the "pending-message limit" error surfaced through the web API).
Two real bugs found and fixed in the new fixtures during verification
against a live bongocollector (not guessed -- diagnosed via a direct,
non-cleanup-swept POP3 account against the real collector):
- fake-pop3/imap-source-fixture.py generated their self-signed TLS
certificate's SAN as DNSName("127.0.0.1") instead of an IPAddress SAN.
curl's CURLOPT_SSL_VERIFYHOST correctly rejected this, so every
fetch silently failed with an SSL hostname-mismatch error. Fixed to
emit an IPAddress SAN when the configured host parses as one.
- collector-flood-quota-live-check.py's own wait_flood() required both
store_ids and queue_ids to be non-empty before accepting a settled
flood, but CollectorStoreSubmitFile's QSTOR/QRUN commits and triggers
local delivery synchronously (external_store.c) -- nothing is ever
left in the transient committed-queue listing by the time a settled
flood is observed, so the condition could never be satisfied. Fixed to
require only store_ids.
Also found and fixed a real product bug, not a test artifact: every
bongo_web/*.py module that opens its own Store connection (auth.py,
mail_store.py, card_store.py, preference_store.py, calendar_store.py)
constructed StoreClient(...) without host/port, silently defaulting to
StoreClient's class default (host="localhost", port=689) -- correct for a
normal install where the Store is always on the compiled-in NMAP_PORT, but
with no way for pure-Python bongo-web to discover a differently-configured
Store (this scratch instance's own BONGO_CTEST_LIVE_INSTANCE build uses
port 16890). Every login against a scratch instance silently authenticated
against whatever real production Store happened to be listening on 689 on
the same host, always failing with a plain "authentication failed" --
previously misdiagnosed mid-session as admin-account-specific; it is not,
it affects every account. Fixed with a new src/www/bongo_web/store_target.py
helper reading BONGO_STORE_HOST/BONGO_STORE_PORT (empty when unset,
preserving production behavior exactly), spliced into all five call sites.
live-instance-fixture.py's start() now sets BONGO_STORE_HOST/PORT and
BONGO_COLLECTOR_CAINFO (the latter pointing bongocollector's libcurl
CAINFO at a CA bundle collector-flood-quota-live-check.py assembles from
the fixtures' certs before creating any external account -- read fresh on
every curl call, so the file doesn't need to exist yet at manager launch)
in bongo-manager's environment. That env construction is now a shared
manager_environment(prefix, values) helper, used by both start() and by
collector-flood-quota-live-check.py's own restart_manager() (needed for
its second phase's mutually-exclusive collector config) -- restart_manager()
previously hand-duplicated an older, incomplete copy of this env dict that
silently regressed bongo-web's Store target and bongocollector's TLS trust
every time phase two restarted the manager.
Verified: ctest -R collector-flood-quota passed twice in a row end to end
(setup, test, teardown), ~160s each.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
2d9df9e662 |
Add fake POP3/IMAP source fixtures, replacing Cyrus for collector tests
src/agents/collector/external_transport.c fetches external mail purely via libcurl's pop3(s):// and imap(s):// URL schemes (CollectorRemoteList/ Fetch/Delete: STLS/STARTTLS, USER+PASS or LOGIN, UIDL/UID SEARCH ALL, RETR/FETCH by (stable) UID, DELE/UID STORE+EXPUNGE) -- no hand-rolled client, so these fixtures need to satisfy curl's own protocol implementation, not just a loose test double. fake-pop3-source-fixture.py and fake-imap-source-fixture.py are the same "protocol-mimicking test double" concept as fake-scanner-fixture.py/ fake-dane-target-fixture.py, standing in for cyrus-fixture.py's real (root-requiring) Cyrus instance. Each user's mailbox is a plain directory of <uid>.eml files; a `seed --user --profile flood|normal --count --message-size` CLI matches cyrus-fixture.py's own shape, so this is a drop-in replacement for collector-flood-quota-check.py. Verified directly against real curl (not just Bongo) for both: STARTTLS negotiation, USER/PASS/UIDL/RETR/DELE for POP3, and quoted mailbox names (e.g. "Cyrus IMAP"), UID SEARCH/FETCH/STORE/EXPUNGE for IMAP -- including a real bug caught this way: UID comparisons using zero-padded stems against curl's unpadded UID values in URLs, silently no-opping FETCH/ STORE instead of matching. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
4c24f7e2f2 |
Wire pop3-capa-stls-check.py into ctest, no fixture needed
reference_cyrus_capa() is a soft, informational-only reference check (catches OSError, reports available=False) -- the actual PASS/FAIL assertions are entirely about Bongo's own POP3 listener (CAPA, STLS, pre-TLS credential gating). No Cyrus fixture dependency at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
528c896894 |
Add live SRS forwarding verification, reusing the DANE netns instance
smtp-srs-forward-live-check.py verifies SRS (Sender Rewriting Scheme) rewriting for untrusted mail forwarded externally through a locally hosted alias: submits from an unauthenticated external sender through the public inbound listener to a fixed "srsforward" alias, and confirms the outbound envelope sender Bongo actually sends on the wire is rewritten to SRS0=/SRS1=...@<hosted domain> rather than leaking the original sender unchanged. netns-instance-fixture.py's bootstrap() now enables SRS forwarding permanently (srs_forward/srs_domain/srs_secret_file) and writes a fixed alias fragment directly under aliases_dir -- discovered that ConfigurationStore.install() only ever touches its own "default_config" fragment there, and alias fragments are plain files, not a Store/ bongo-admin round trip, so this needs no live connection and takes effect before bongo-manager's first start (no restart, matching every other rework this session). Harmless to DANE/opportunistic-TLS/DKIM sharing this instance: SRS only activates on the alias-forwarding code path, which none of them exercise. smtp-srs-forward-check.py (the original real-root/systemd script) is left untouched. Confirmed: passes reliably standalone and as part of the full dkim-dmarc/dane/opportunistic-tls/srs chain (8 tests) in one ctest invocation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
6eb4b7003d |
Add live IDNA2008 SMTP routing verification via a dedicated instance
smtp-idna-live-check.py exercises real inbound/outbound SMTP paths for
IDNA2008 (non-transitional) domain identity, on its own dedicated netns
instance (--root /tmp/bongo-idna-instance) rather than the shared
dkim_dmarc_instance: it hosts a different domain (the A-label of
"bücher.smtp26.test") that would otherwise collide with dkim-dmarc-
check.py's own hosted domain. Verifies Unicode-form and A-label-form
RCPT TO for the same mailbox land in the same place, an invalid A-label
is rejected with 501, and outbound direct-MX delivery to a Unicode
external domain ("faß.smtp26.test") uses the canonical A-label on the
wire, not the Unicode form or a transitional mapping.
Root-caused two real bugs in fake-dane-target-fixture.py along the way:
- fake-dns-fixture.py sets the DNS AD bit unconditionally (correct for
the AD-bit-trust record types DKIM/DMARC/SPF/MX rely on), which also
makes src/agents/smtp/dane.c's Remote->mxDnssecAuthenticated true for
*any* domain it answers for -- triggering a real libunbound TLSA
lookup. An unsigned zone made that lookup inconclusive
(SMTP_DANE_TEMPFAIL) rather than cleanly "no DNSSEC", aborting
delivery before any SMTP exchange -- confirmed via strace against the
live bongosmtpc process. Fixed by properly signing the target's zone
(update_zone=True, matching dane-mta-sts-check.py) instead of
registering plain MX/A records.
- The fixture's EHLO response never advertised SMTPUTF8. A message
submitted with the SMTPUTF8 parameter must relay through an
SMTPUTF8-capable hop even once its own envelope has been canonicalized
to pure ASCII (RFC 6531) -- Bongo correctly refused to continue past
EHLO against a target that didn't advertise it.
Also split relay-routing verification out of scope entirely (documented
in the new script's docstring): use_relay_host is a global SMTP-wide
toggle with no live-reload, so testing it needs a config-swap restart
with no way around it, and relay_host's own IDNA canonicalization is
already covered by idna-canonicalization-check.py.
Confirmed: two full setup/check/teardown cycles pass cleanly back to
back, and the full dkim-dmarc/dane/opportunistic-tls/idna chain (11
tests) passes together in one ctest invocation.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
706fcab428 |
Wire IDNA2008 canonicalization as an unconditional, no-instance ctest
Split out of smtp-idna-check.py's verify_installed_python_paths(): pure source-tree Python (bongo.configuration.model.canonicalize_domains, bongo.domain.domain_to_ascii, bongo_web.compose.build_message) that never touched the network or needed a live instance in the first place, unlike the rest of that script. Registered unconditionally next to c-literal-length-check.py, so it runs even without BONGO_CTEST_LIVE_INSTANCE. smtp-idna-check.py's remaining live-routing assertions (local Unicode- domain delivery, direct-MX + relay routing to Unicode targets with canonical-A-label DNS/envelope verification, invalid-A-label rejection) still need real root + systemd, and don't fit either shared fixture cleanly: use_relay_host is a global SMTP-wide toggle, so permanently enabling it for a live_instance/netns_instance relay test would break every other already-wired test relying on direct MX delivery on that same instance. Left for a dedicated instance if pursued further. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
66d36d6353 |
Fix fake-dane-target-fixture.py rejecting a legitimate plaintext retry
Root-caused via strace against the live bongosmtpc process: after advertising STARTTLS, the fixture's handler treated any command other than literal "STARTTLS" as a protocol violation and dropped the connection -- but advertising STARTTLS never obligates a client to use it. Bongo's own outbound TLS logic (src/agents/smtp/smtpc.c, the plaintextFallback path around line 1656) does exactly this correctly: after a failed handshake it reconnects, sees STARTTLS offered again, deliberately skips it a second time, and sends MAIL FROM directly. The strace showed Bongo's own connect/EHLO/MAIL FROM sequence was correct throughout -- the fixture just closed the connection on the MAIL FROM it wasn't expecting. handle_transaction() now takes an optional pending_line: a command already read off the wire (the one that turned out not to be STARTTLS) gets processed as the transaction's first command instead of being discarded. Re-enables the broken-tls scenario in smtp-opportunistic-tls-live-check.py, previously left out with a "not yet understood to be correct" caveat -- now confirmed passing reliably across repeated runs, alongside the tls/plaintext/outage cases and the full dkim-dmarc/dane/opportunistic-tls chain. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
6bd9691ad0 |
Add live opportunistic-TLS verification via multi-mode fake-dane-target
fake-dane-target-fixture.py gains --mode (tls/plaintext/broken-tls),
per-connection and per-STARTTLS-attempt markers under
<root>/dane-connections and <root>/dane-starttls-attempts, and fixes two
bugs found while extending it: a missing None-check on read_line() that
could crash the handler on an empty first line, and (like
fake-scanner-fixture.py before it) no allow_reuse_address, which caused
real EADDRINUSE failures during repeated start/stop cycles.
smtp-opportunistic-tls-live-check.py reuses the same dkim_dmarc_instance
netns DANE's live test established (outbound DKIM signing is already on
by bootstrap() default), verifying:
- a "tls" target: delivered encrypted
- a "plaintext" target (never advertises STARTTLS): still delivered,
unencrypted -- proving TLS is opportunistic, not required
- an "outage" target (MX/A published, nothing listening at first): the
message queues and is automatically retried once the target comes
online, with no operator intervention
Every delivery also carries the instance's own DKIM signature, proving
direct MX delivery (not just STARTTLS) ran the normal outbound path.
netns-instance-fixture.py's bootstrap() now also bakes a fast queue retry
schedule (5s/15s vs. the 300s/4000s production default) -- confirmed the
outage case never got its first automatic retry within two minutes of
waiting otherwise. Safe here: this netns instance is never shared with
queue-bounce-expiry-check.py or queue-quota-partial-check.py, which run
against the separate live_instance fixture and keep its own timing.
smtp-outbound-opportunistic-tls-check.py (the original real-root/systemd
script) is left untouched -- several other still-manual scripts
(smtp-idna-check.py, smtp-srs-forward-check.py, smtp-relayhost-check.py,
smtp-required-tls-check.py, smtp-tlsrpt-delivery-check.py,
smtp-dane-mta-sts-check.py, smtp-forward-metadata-check.py) still import
it as a shared helper module.
Not covered: --mode broken-tls (advertise STARTTLS, drop the connection
mid-handshake). Manual investigation showed smtpc.c repeatedly
re-attempting STARTTLS on every queued retry rather than reaching
ReconnectRemotePlaintext()'s single-retry-in-plaintext path -- documented
in the test's docstring as a follow-up rather than asserted as understood
behavior.
Confirmed: two full setup/dkim-check/dane-check/opportunistic-tls/
teardown cycles pass cleanly back to back.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
Mario Fetka
|
4a04ca0889 |
Wire smtp-size-check.py into ctest via a baked-in message_size_limit
Originally needed a config-swap + `systemctl restart bongo.service` cycle to apply a reduced message_size_limit, then another to restore it. Baked a permanent 1 MiB limit into live-instance-fixture.py's bootstrap() instead (comfortably above the largest message any other wired live test sends, ~32KB) -- this script only verifies the already-configured boundary now, matching the smtp-lmtp-delivery-check.py rework's reasoning (no Bongo agent has a live config-reload path). Also fixed queue() to use the same unprivileged-safe invocation as queue-bounce-expiry-check.py/smtp-lmtp-delivery-check.py instead of unconditionally requiring sudo. Confirmed passing standalone and as part of the full smtp-* suite (the only failures there are four pre-existing, unrelated sudo-requirement bugs in smtp-pipelining-check.py's shared queue-cleanup helper). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
9de21fb53d |
Wire smtp-lmtp-delivery-check.py into ctest via a fake-lmtp-fixture.py
Originally needed a config-swap + `systemctl restart bongo.service` cycle just to point an lmtp_transports mapping at its own embedded LMTP receiver, plus a bogus relay to prove transport-mapping precedence. No Bongo agent has a live config-reload path (ConfigLock is never write-locked after startup), so that config can only take effect via a fresh bootstrap -- baked directly into live-instance-fixture.py's bootstrap() instead, matching the fake-scanner-fixture.py (spamd/clamd) concept: a real background fixture process started alongside the instance, not a server embedded in the test script's own process. fake-lmtp-fixture.py speaks the same protocol subset the previous embedded LMTPHandler did, but keeps delivery/attempt state on disk (<root>/lmtp-received, <root>/lmtp-attempts) since it now runs as a separate process from its test client -- cleared at the start of each start()/stop() cycle so back-to-back ctest runs sharing the fixed /tmp/bongo-live-instance/run root don't see stale attempts satisfy a fresh run's checks. Also fixes a latent EADDRINUSE flakiness in fake-scanner-fixture.py: its ThreadingTCPServer subclasses never set allow_reuse_address, so a rapid setup/teardown cycle could fail while the previous process's socket sat in TIME_WAIT. Confirmed: three back-to-back setup/test/teardown cycles pass cleanly, and the pre-existing shared live_instance suite shows no regressions (five unrelated pre-existing failures confirmed via isolated rerun -- smtp-pipelining/8bitmime/bdat/binarymime share a queue-cleanup helper that unconditionally requires sudo, and imap-idle passed cleanly in isolation, both untouched by this change). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
aa0f155d1f |
Vendor Dovecot ImapTest as a submodule and wire it into ctest
Adds contrib/imaptest (Dovecot's upstream IMAP compliance/stress tester) and contrib/dovecot-core (pinned to 2.4.4) as git submodules. ImapTest's own configure requires a built Dovecot tree's dovecot-config (--with-dovecot=DIR, see contrib/imaptest/m4/dovecot.m4's DC_DOVECOT) -- there is no lighter-weight way to satisfy that on a host without Dovecot installed, which is why dovecot-core is vendored too, built only far enough to produce libdovecot.la and friends (never installed). imaptest-fixture.py builds both in-tree, idempotently, wired into CMakeLists.txt as an imaptest-build FIXTURES_SETUP step ahead of upstream-imaptest-check.py's already-existing "scripted --suite core" mode, which needed nothing beyond the shared live_instance fixture and an imaptest binary/source path -- confirmed passing end to end. debian-trixie.yml's checkout now fetches submodules too, so a fresh CI checkout doesn't leave contrib/dovecot-core and contrib/imaptest empty. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
3a602ee82b |
Add live outbound DANE verification via fake-dane-target-fixture.py
netns-fixture.py now bind-mounts a writable /etc/dnssec/root-anchors.txt (the first path LoadTrustAnchors() in dane.c tries), and netns-instance-fixture.py's fake-dns-fixture.py invocation passes --anchor-file so it can write DS records for whichever zones it signs. fake-dane-target-fixture.py is a disposable STARTTLS SMTP receiver with a self-signed leaf cert generated via `cryptography`, standing in for a DANE-protected remote MX; it self-registers its MX/A/TLSA records into the shared netns instance's DNS zone and reloads fake-dns-fixture.py. netns-instance-fixture.py's own inbound SMTP port moves off 25 (to 2525): bongosmtp always binds 0.0.0.0 (MsgGetAgentBindIPAddress() never sets MSGAPI_FLAG_BOUND, so there's no configurable bind address), and a wildcard bind on 25 blocks any other process -- like the fake DANE target -- from binding a different loopback address on that same port. dane-mta-sts-check.py sends a message through the instance's own submission listener to a domain it doesn't host, exercising smtpc.c's real outbound MX/TLSA lookup and libunbound DNSSEC validation against fake-dns-fixture.py's signed zone. A second phase restarts the fake target with a mismatched certificate but a stale TLSA record, proving SMTPDaneVerify() actually blocks delivery rather than just attempting opportunistic TLS. Wired into CMakeLists.txt as dane-target-setup/-teardown fixtures plus the dane-mta-sts-check test, reusing the dkim_dmarc_instance netns. smtp-dane-mta-sts-check.py stays as a separate manual script -- its three real-DNS-resolver-stack coverage (BIND+Unbound, Technitium, PowerDNS) needs real root and a real systemd-managed bongo.service that this scratch instance deliberately has neither of. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Mario Fetka
|
f5261fc454 |
Add DNSSEC zone signing to fake-dns-fixture.py for live DANE tests
DANE (src/agents/smtp/dane.c) calls libunbound directly and does real
cryptographic DNSSEC validation, unlike everything else Bongo trusts
via the resolver's AD bit (src/libs/xpl/dns.c's dnssec_authenticated
is just `((HEADER *)answer_buffer)->ad != 0`) -- so unlike SPF/DKIM/
DMARC, exercising it live needs an actual signed zone, not just a
"controlled data" stub.
A `SIGN <zone-apex>` line in the zone file now marks every record at or
below that apex as DNSSEC-signed: a KSK is generated once per apex
(RSASHA256, via dnspython's dns.dnssec module), every marked RRset gets
a matching RRSIG, the apex answers DNSKEY queries, and the trust anchor
(a `<apex> IN DS ...` line) is written to a file in the exact format
libunbound's ub_ctx_add_ta_file() expects (see LoadTrustAnchors() in
dane.c, which reads one of three hardcoded paths -- meant to be
bind-mounted over one inside netns-fixture.py's isolated namespace, the
same trick already used for /etc/resolv.conf).
Verified against libunbound directly (the same API dane.c calls, not
just dnspython's own validator): TLSA/A/MX all resolve with
secure=1 bogus=0 against a self-signed test zone and its generated
trust anchor. Chose a non-reserved fake TLD after landing on RFC 6761
reserved names (.test, .invalid) first, which libunbound synthesizes
NXDOMAIN for locally without ever querying anything -- looked like the
signing was broken when the real problem was the zone name.
Also fixes two real bugs found while building this:
- dns.dnssec.sign() returns a bare RRSIG rdata, not an RRset --
appending that directly to a response breaks dnspython's wire
renderer with an opaque "unsupported operand type(s) for +=: 'int'
and 'NoneType'" (add_rrset() silently returns None instead of a
count). Every signature is now re-wrapped with rrset.from_rdata()
before being stored.
- The existing TXT-specific character-string quoting (needed so a long
value like a DKIM key splits into multiple 255-byte chunks) was being
applied to every record type. Fine for TXT, but it breaks a
structured type like TLSA outright ("expecting an identifier") since
usage/selector/matching-type octets plus a hex digest is not a
free-text blob -- now scoped to TXT only.
All answers also now set AD=1 unconditionally: this fixture represents
"the local, trusted resolver" Bongo's own AD-bit-trust model assumes,
and every answer it gives is test-controlled data. Confirmed no
regression against the existing DKIM/DMARC live test, which already
depends on this same fixture for its own (unsigned) TXT records.
|
||
|
Mario Fetka
|
7f4ea52af3 |
Wire setup-wizard-defaults.py and legacy-config-install.py into ctest
setup-wizard-defaults.py is pure Python driving bongo.configuration.tui directly against its own throwaway BONGO_TEST_CONFIG_ROOT -- no compiled binary, no running daemon, so the shared live_instance's already-installed Python package (found via BONGO_LIVE_TEST_ENVIRONMENT's PYTHONPATH) is safe to reuse as-is. legacy-config-install.py is different: it drives bongo-config's real compiled install prompt end-to-end. A compiled binary's XPL_DEFAULT_* filesystem paths are baked in at *build* time (CMAKE_INSTALL_PREFIX), not overridable via BONGO_PREFIX or any other env var at runtime -- confirmed the hard way, pointing this script at the shared instance's own bongo-config with only BONGO_PREFIX set silently targeted the shared instance's real prefix regardless (only bongo-config's own "existing TLS certificate does not cover this hostname" safety check stopped it from actually overwriting anything). So this test gets its own dedicated scratch build+prefix via legacy-install-setup (configure-legacy-install-test.cmake), distinct from both live_instance and dkim_dmarc_instance for the same "compiled paths are baked in" reason those two are distinct from each other. No running daemon afterward, so setup is just configure+build+install, no teardown/cleanup fixture needed. |
||
|
Mario Fetka
|
38eac7276c |
Fix bongo-web PYTHONPATH isolation and wire two more live tests
bongo-manager (and everything it launches, including bongo-web) never inherited the PYTHONPATH override that ctest's own client-side ENVIRONMENT already sets for every contrib/testing/*.py invocation -- that only covers ctest's own test processes, not this fixture's internally-launched daemon tree. bongo-web is a plain `#!/usr/bin/env python3` script, so `from bongo_web import security` / `from libbongo import _auth_security` silently resolved against whatever production bongo_web/libbongo happened to be on the system's default sys.path instead of this scratch instance's own freshly installed copy -- exactly the isolation-breach class already fixed for ctest's client-side processes, just missed here. Confirmed by a real "cannot open Bongo authentication database" 503 from bongo-web: it had loaded production's compiled libbongo._auth_security, whose baked-in XPL_DEFAULT_DBF_DIR pointed at production's /var/lib/bongo, not this scratch prefix. Also: - c-literal-length-check.py (a pure static scan of src/ and include/ for I/O calls whose literal length doesn't match what they actually send) is registered unconditionally now -- it needs no live instance and applies to every build, unlike everything else in this file. - protocol-smoke.py is registered as its own live test, not just used internally by the fixture's own readiness wait. - mail-dns-live-check.py was evaluated but left unwired: its "admin" account authenticates fine over IMAP with the exact same credentials the fixture provisions, but bongo-web's own login endpoint rejects it every time, even against a fully wiped Store. Real and reproduced, not yet root-caused -- documented rather than shipping a test that can never pass. Of the remaining unwired contrib/testing/*.py scripts, 21 (up from the 2 already known) turned out to hardcode `sudo -n systemctl restart bongo.service` either directly or via a shared imported helper -- appropriate for a real systemd-managed Bongo, not this ctest scratch instance, which has no systemd unit for them to restart in the first place. They stay as separate scripts against a real installation. The rest (smtp-dane-mta-sts-check.py, smtp-srs-forward-check.py) manage their own real DNS resolver setup with direct /etc/resolv.conf backup/restore, cyrus-fixture.py/smtp4dev-fixture.py/upstream ImapTest need their own external-tool preparation step, and several (c-literal-length-check.py aside) are diagnostic utilities rather than pass/fail tests -- none force-fit into this session's live/netns fixture split. |
||
|
Mario Fetka
|
58200ec331 |
Rename dkim-dmarc-instance-fixture.py to netns-instance-fixture.py
Not DKIM-specific: it runs a second, small, fully isolated Bongo instance inside netns-fixture.py's namespace, which any future test needing the same kind of isolation (e.g. ACME challenge validation, whenever that gets test coverage) can reuse the same way, not just DKIM/DMARC. Text-only rename; no behavior change. Also documents why smtp-alias-routing-check.py, smtp-dsn-check.py, smtp-internal-normalization-check.py, smtp-internal-relay-check.py, smtp-recipient-check.py, smtp-smtputf8-check.py, smtp-telnet-timeout-check.py, and imap-reconnect-consistency-check.py stay unwired: they hardcode `sudo -n systemctl restart bongo.service` to prove behavior survives a real daemon restart, against the real system-installed service -- appropriate for a real systemd-managed Bongo, not a ctest scratch instance with no systemd unit for them to restart in the first place. A live-instance-fixture.py restart() capability was prototyped for this and worked reliably in isolation (~15s, repeatable back-to-back), but something in these scripts' actual config-change-then-restart sequence destabilized the instance in ways not resolved by longer readiness waits -- reverted rather than ship something unreliable. If a future revision of one of these splits out assertions that do not depend on the restart itself, wire that part in normally. |
||
|
Mario Fetka
|
a125c806d7 |
Add live DKIM/DMARC verification via an isolated fake-DNS instance
DKIM key lookup and OpenDMARC's _dmarc.<domain> policy fetch go through their libraries' own DNS resolvers with no test-zone hook like SPF's (BongoMailAuthTestZoneSPFServer() in spf.c), so unlike SPF this needs a real DNS listener. Binding port 53 unprivileged requires a genuinely fresh network namespace (there is no other way to get CAP_NET_BIND_SERVICE without real root) -- but a fresh network namespace also means a fresh, isolated loopback, unreachable from and to the host's default namespace. netns-fixture.py provides a persistent, unprivileged network+mount namespace via the same "holder process" pattern container runtimes use for a pod's shared namespace: unshare starts a long-lived no-op process that owns the namespaces, and later commands join them with nsenter. fake-dns-fixture.py is a minimal authoritative-only TXT stub server that runs inside it, bound to the namespace's own /etc/resolv.conf (a private bind-mount, invisible outside the namespace -- the host's real DNS configuration is never touched). Because that isolated loopback can't reach or be reached by the shared live_instance fixture's ports, dkim-dmarc-instance-fixture.py runs a second, small, fully isolated Bongo instance -- and its live test client -- entirely inside the same namespace via nsenter, with its own CMake build (distinct BONGO_CTEST_NMAP_PORT/QUEUE_PORT so its bootstrap-time Store/Queue connections, which run outside the namespace, can never collide with the shared instance's identical defaults) and its own bootstrap()/provision_user() flow. Two things only show up when a process is namespace-scoped fake root rather than the invoking user directly, both fixed here: BONGO_USER must be compiled as "root" (not the real username) since --map-root-user only maps UID 0, leaving bongo-manager's attempt to drop to any other configured user with nowhere to go; and bongo-web's Python privilege-drop calls os.setgroups([]), restricted even to become one's own current identity inside an unprivileged user namespace, so it's disabled outright for this instance (unconditionally enabled by apply_scenario() with no scenario-level gate, and its crash was observed to abort the rest of bongo-manager's own startup sequence). Also fixed along the way: netns-fixture.py's stop() never reaped its killed holder process, and nsenter's setns() was observed to intermittently fail joining an already-running, already-confirmed-ready holder for reasons never conclusively root-caused -- wait_until_joinable() retries a cheap probe join before every real one now. And fake-dns-fixture.py's TXT responses crashed the handler outright for any value over 255 bytes (a DKIM public key routinely is one), silently timing out every query instead of erroring -- now split into properly chunked character-strings per RFC 1035, and any remaining answer-building bug now returns SERVFAIL instead of dropping the request. dkim-dmarc-check.py hand-signs a message with the instance's own generated key (RFC 6376 relaxed/relaxed canonicalization, rsa-sha256) rather than depending on Bongo's own outbound signing path, and confirms both dkim=pass/dmarc=pass for a correctly signed message and dkim=fail for a tampered one, verified against the fake DNS server's fabricated records -- not real DNS, and no shared machine DNS configuration touched anywhere in this chain. |
||
|
Mario Fetka
|
fa41d90218 |
Add EICAR/GTUBE fake spamd+clamd fixture and a live scanner test
fake-scanner-fixture.py implements exactly the two wire protocols src/agents/antispam/spamd.c and src/agents/avirus/avirus.c speak (spamd's PROCESS/Content-length/Spam: exchange; clamd's INSTREAM chunked stream and stream: OK / stream: ... FOUND reply), so a live-instance test host never needs a real SpamAssassin or ClamAV install. It flags only each scanner's own standard self-test string -- SpamAssassin's GTUBE and ClamAV's EICAR -- and passes every other message through unchanged, with a real X-Spam-Flag header added so a delivered message stays verifiable (a real spamd also prepends its own headers; echoing the message back unchanged would have made clean vs. tagged deliveries indistinguishable). live-instance-fixture.py now starts/stops it alongside bongo-manager and enables both scanner agents, pointed at the fake endpoints. smtp-scanner-check.py exercises all three live paths: a clean message delivered with X-Spam-Flag: NO, a GTUBE message still delivered but tagged X-Spam-Flag: YES (antispam only tags, per docs/antispam.md -- it never drops), and an EICAR message dropped from the queue with a "Virus Alert" bounce reaching the authenticated sender (BounceToSender() in avirus.c). |
||
|
Mario Fetka
|
e9e5f1de2c |
Add a no-DNS-server SPF test path and a live pass/fail SPF test
BongoMailAuthCheckSPF() (src/libs/mailauth/spf.c) always resolved SPF
through real DNS, so live inbound-SPF testing had no way to fabricate
sender-domain policies without either a real DNS server or editing
shared machine DNS configuration. BongoMailAuthTestZoneSPFServer(),
compiled only into CTEST_BUILD_NOROOT builds, uses libspf2's own zone
DNS layer -- the same mechanism its upstream test suite uses instead
of live queries -- populated from a file named by
BONGO_TEST_SPF_ZONE_FILE. A production build never defines
CTEST_BUILD_NOROOT, so this code and its getenv() read do not exist
there.
Found along the way: libspf2 hardcodes an unconditional PASS for
127.0.0.1 ("localhost is always allowed") regardless of what any DNS
layer returns. Every live test connects over loopback, so a
FAIL/SOFTFAIL zone record could never be observed without also
substituting a non-loopback address -- 203.0.113.42, RFC 5737
TEST-NET-3, reserved for exactly this -- for the real client IP
whenever the test zone is active.
live-instance-fixture.py's bootstrap() now starts bongosmtp with
BONGO_TEST_SPF_ZONE_FILE pointing at a fixed run-directory path; since
spf.c re-reads it on every check rather than caching, a test can
rewrite its content between SMTP transactions with no restart needed.
smtp-spf-check.py exercises both the pass and fail cases against
fabricated sender domains over the real inbound SMTP listener.
|