Commit Graph

200 Commits

Author SHA1 Message Date
Mario Fetka c7f50f8f62 Document the 5th CRLF bug, the now-permanent robustness test suite,
and the missed symbols-file entry in SIEVE-04/06 evidence

Corrects the earlier "no bugs found" malformed-mail robustness
write-up: turning one of those ad hoc checks into a deterministic
regression test surfaced a genuine 5th CRLF bug in mu_header_size(),
a different code path from the four multipart-specific fixes. Records
the fix, the now-permanent mimerobust.c/.at and hdrsize.c/.at test
additions, and the dh_makeshlibs symbols-file fix caught by
re-verifying with a real dpkg-buildpackage build.
2026-08-01 20:36:26 +02:00
Mario Fetka a139a297e5 Document the CRLF/MIME bug chain, upstream test additions, and
malformed-mail robustness pass in SIEVE-04/06 evidence

Records the four-bug root cause and fix for 0005-crlf-mime-fixes.patch,
the new upstream test coverage added to the earlier body.c and
envelope-recipient patches (sieve/tests/body.at, extended
envelope.at), the honestly-documented RFC 5173 5.2 gap found while
writing those tests, and a clean malformed-mail robustness pass (21
deliberately broken messages, no new bugs found).
2026-08-01 18:51:42 +02:00
Mario Fetka 02f56166d5 Add missing symbols to the Debian mailutils package's .symbols file
Debian Trixie package bundle / packages (push) Failing after 14m2s
dh_makeshlibs/dpkg-gensymbols failed the Trixie build: neither the
body.c patch's exports nor the new envelope-recipient accessors were
ever added to debian/libmailutils9t64.symbols, so both surfaced
together the first time the build actually completed dh_makeshlibs
against the patched source. Fixed by extending 0002-debian-trixie.patch
with the missing entries, each tagged 1:3.19-1+bongo1 (matching
dpkg-gensymbols' own suggestion) since none of these symbols exist in
any unpatched Debian mailutils build.

Verified with a full dpkg-buildpackage in a fresh Debian Trixie Docker
container against the real pinned source, not just a symbols-file diff.
2026-08-01 15:01:03 +02:00
Mario Fetka 7f9129598f Implement envelope :is "to" in the vendored Mailutils Sieve patch
Debian Trixie package bundle / packages (push) Failing after 12m21s
Mailutils' retrieve_envelope() only ever resolved "from". Add
mu_envelope_set_recipient()/mu_envelope_sget_recipient() to
libmailutils, mirroring the existing sender accessor pair 1:1 (same
struct field, callback slot, and header-derived fallback via a new
message_envelope_recipient()), so "to" now resolves the same way
"from" always did. Packaged as a separate patch (0004) for independent
upstream review, wired into both the Gentoo overlay ebuild and the
Debian Trixie build script.

Verified against Mailutils' own upstream Sieve test suite (107/107,
zero regressions), against a standalone program replicating Bongo's
exact message-construction path (positive and negative match), and
live through bongo.service (two consecutive SIEVE-04 passes).
2026-08-01 14:27:44 +02:00
Mario Fetka 1a3979ae20 Update SIEVE-04/06, IMAP-13, and BLD-15 evidence for the Mailutils and GMime fixes
Rewrite contrib/testing/sieve-execution-check.py: replace the now-stale
"envelope/body rejected at upload" check with real live execution
tests. envelope :is "from" is confirmed to match the real sender
through actual SMTP delivery; envelope :is "to" is confirmed to never
match anything (a genuine, separate limitation of Mailutils'
retrieve_envelope(), which only implements the "from" envelope-part,
not something the require.c patch touches); body :contains is
confirmed to match real message content and correctly not match
messages without it, through Bongo's real delivery pipeline rather
than just the standalone `sieve` CLI tool used earlier to validate the
patch in isolation.

Document all of this, plus the GMime 3 address-search segfault find
and fix, as dated updates in the evidence file rather than rewriting
history: SIEVE-04 and SIEVE-06 get the envelope/body reality-check,
IMAP-13 gets the GMime 3 segfault root cause and fix, and BLD-15 notes
both dependency fixes now being part of the standard Debian Trixie
build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-01 12:08:16 +02:00
Mario Fetka ba434bf79e Re-enable envelope/body now that the Mailutils Sieve engine supports them
The Gentoo mail-mta/bongo overlay's net-mail/mailutils ebuild now
carries a local patch (files/mailutils-3.21-sieve-envelope-body.patch)
that fixes require.c's registry lookup so require ["envelope"]; can
reach the standard test Mailutils already implements, and adds a new
body.c test module (RFC 5173, using only Mailutils' own MIME API) that
Mailutils never implemented in any form. Verified against Mailutils'
own upstream Sieve test suite (107/107, zero regressions, including
the real envelope.at case) plus live execution of both extensions.

Advertise both again in the ManageSieve CAPABILITY string and allow
them in safe_requirement(); update docs/sieve.md and drop the now-
resolved ROADMAP item about replacing the Sieve engine entirely.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-01 11:44:06 +02:00
Mario Fetka 3b133a825e Add SIEVE-06 Cassandane comparison test and full-block evidence
Verify Bongo deduplicates "keep; keep;" and "keep; fileinto \"INBOX\";"
to a single delivered copy, matching Cyrus Cassandane's
test_dup_keep_keep/test_dup_keep_fileinto regression tests, while two
genuinely distinct fileinto targets still each receive their own copy.
Documents the duplicate-delivery fix found via this comparison, and
synthesizes the Pigeonhole cross-references and all six real bugs
found and fixed across the whole SIEVE-01..06 block.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-01 09:56:12 +02:00
Mario Fetka ae56c525cd Add SIEVE-05 ManageSieve safety test and evidence
Verify invalid-syntax rejection, script size/quota boundaries,
duplicate-name overwrite semantics, script-name traversal/control-char
rejection (with ".." correctly accepted, since Bongo's SQL-backed
store has no filesystem traversal risk), cross-user isolation, and
pre-authentication command safety. Documents the ConnReadLine/
ConnReadAnswer NUL-byte hang fix and the ConnReadLine overlong-line
nul-termination fix, both found and fixed earlier in this block.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-01 08:05:49 +02:00
Mario Fetka 8dbc0719fb Add SIEVE-04 live Sieve execution test and evidence
Verify fileinto (variables-interpolated target, implicit-keep
suppression) and vacation (auto-reply delivery, dedup interval) end
to end through real SMTP delivery, and that envelope/body are rejected
cleanly at upload. Documents the fileinto/MOVE queue-parsing fix, the
vacation/reject bare-recipient address qualification fix, and the
envelope/body Sieve-engine limitation, all found and fixed earlier in
this block.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 21:02:19 +02:00
Mario Fetka cecb2de2b3 Stop advertising broken envelope/body Sieve extensions
Debian Trixie package bundle / packages (push) Failing after 12m54s
The installed GNU Mailutils Sieve engine cannot run either extension:
its require resolution only ever checks its action registry for a bare
extension name, so `require ["envelope"];` (the only syntax RFC 5228
scripts use) can never succeed no matter how a same-named test is
registered, and it does not implement `body` (RFC 5173) in any form.
Both are confirmed independently of Bongo against Mailutils' own
`sieve` reference tool. Stop claiming support in the ManageSieve
CAPABILITY string and the safe_requirement() allowlist so uploads
requiring them are rejected cleanly instead of silently accepted and
never actually usable. Document the limitation and follow-up options
(upstream patch, engine replacement, or a native Bongo Sieve engine)
in ROADMAP.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 17:15:46 +02:00
Mario Fetka 8e86f3745d Add SIEVE-03 ManageSieve lifecycle test and evidence
Verify SETACTIVE atomicity/idempotence, RENAMESCRIPT active-flag
preservation and collision handling, the DELETESCRIPT NONEXISTENT/ACTIVE
fix under a live session, LOGOUT pre-auth behavior, and activation
persistence across reconnects, cross-checked against Dovecot
Pigeonhole's managesieve command handlers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 16:17:52 +02:00
Mario Fetka 961d43b8a2 Add SIEVE-02 ManageSieve script-management test and evidence
Verify HAVESPACE/CHECKSCRIPT/PUTSCRIPT/LISTSCRIPTS/GETSCRIPT against
the live port 4190 listener, confirming the HAVESPACE and DELETESCRIPT
RFC 5804 fixes and comparing against Dovecot Pigeonhole's managesieve
command handlers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 16:13:47 +02:00
Mario Fetka 0a6cfea7ed Add SIEVE-01 ManageSieve auth test and evidence
Verify greeting/STARTTLS/pipeline-discard/SASL PLAIN+LOGIN behavior
against the live port 4190 listener, matching Dovecot Pigeonhole's
managesieve command set.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 15:49:20 +02:00
Mario Fetka 389e990e3d Renumber duplicate SCAN-03/SCAN-04 matrix rows to SCAN-08/SCAN-09
The scanner-outage-policy and update-channel rows duplicated the IDs
already used for the collector-scan-path and runtime-EICAR/GTUBE
rows. No evidence entries referenced the duplicated IDs yet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 15:31:47 +02:00
Mario Fetka 057f6996fc Extend SMTP-34 with concurrent relay-host delivery and DKIM check
Adds a sixth scenario: concurrent load through the generic
use_relay_host SMTP client (DeliverGlobalRelay, distinct from the
lmtp_transports path already covered), verifying every relayed
message carries a DKIM-Signature header. Raises the trusted internal
relay port's rate limit for the test's duration, since running
several deliberate load phases through it otherwise trips the
abuse-prevention limit that exists to bound exactly that in
production.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 15:27:36 +02:00
Mario Fetka 589e473526 Extend SMTP-34 with concurrent load against the real public port 25
Port 26 is the trusted internal relay; port 25 is what actually
receives concurrent load from the open internet, including the
antispam/antivirus scan path. Confirmed bongoantispam/bongoavirus and
their ClamAV/SpamAssassin daemons were active and not bypassed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 15:17:50 +02:00
Mario Fetka 49838c0d12 Add SMTP-34 Postfix smtp-source/smtp-sink interop live test
Verifies concurrent SMTP/LMTP load, soft-reject and disconnect
failure injection, and Queue retry/recovery using upstream Postfix's
own test tools, built locally since installing mail-mta/postfix
would remove mail-mta/bongo.

The investigation initially suspected a real concurrent-LMTP message
loss bug in Bongo; extensive isolated reproduction (see the evidence
entry) traced it entirely to two bugs in this test script: a
doubled sudo prefix that silently broke every Queue state query, and
a capture check that compared dump-file counts before bongosmtpc had
finished flushing the last LMTP session. No Bongo source change
resulted. Passed twice consecutively end to end.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 15:10:06 +02:00
Mario Fetka f3dfad01ba Add POP-06 shared Store quota reservation live test
Verifies DELE alone, and an abrupt disconnect right after DELE, leave
the account's quota Used bytes unchanged; DELE+RSET+QUIT also leaves
it unchanged; and only DELE+QUIT durably frees the space. Completes
the POP-01..07 block. No source change required.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 08:12:30 +02:00
Mario Fetka cbf5b80465 Add POP-05 UTF8/STLS-gating/PIPELINING/malformed-input live test
Verifies UTF8 mode (including a corrected idempotent-repeat
assumption), STLS rejection after UTF8, byte-correct UTF-8 RETR vs.
the ASCII-only gate without UTF8 mode, pipelined command ordering,
RFC 2449 response codes, oversized-line/unknown-command handling, and
documents the 900s idle timeout from source (no fast-test config knob
exists for it, unlike SMTP-25). No source change required.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 08:08:44 +02:00
Mario Fetka 9d73e71f12 Add POP-04 DELE/RSET/NOOP/QUIT commit and rollback live test
Verifies pre-login bad-state rejection, that DELE without a following
QUIT never reaches the durable Store DELETE path (confirmed via a
fresh IMAP check after an abrupt disconnect), RSET restoring deleted
flags, QUIT's durable commit, and DELE error handling. No source
change required.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 07:59:15 +02:00
Mario Fetka 69a11b5d20 Add POP-03 STAT/LIST/UIDL/RETR/TOP and dot-transparency live test
Seeds IMAP-appended fixtures covering plain, dot-stuffed, and bare-LF
bodies, then verifies STAT/LIST/UIDL consistency and byte-correct
RETR/TOP output against the live server, with UIDL/RETR semantics
cross-checked against Dovecot's pop3-commands.c. No source change
required.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 07:53:22 +02:00
Mario Fetka 3b92a6e859 Add POP-02 TLS USER/PASS and shared SASL authentication live test
Verifies port 995 TLS, USER/PASS login, malformed-argument and
bad-password handling, and PLAIN/LOGIN SASL mechanisms shared with
SMTP/IMAP. Cross-checked the repeated-USER-before-PASS semantics
against Dovecot's pop3-login and fixed a wrong test assumption rather
than the server, which already matches Dovecot's behavior.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 07:49:56 +02:00
Mario Fetka 097592191e Add POP-01 CAPA/STLS credential-gating live test
Verifies the cleartext CAPA advertisement, USER/AUTH/PASS rejection
before STLS, RFC 2595 pipelined-plaintext discard, and post-STLS
capability/state transitions, with a live Cyrus fixture reference for
the CAPA shape. No source defect found; documents the existing
STARTTLS-gated behavior against the matrix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 07:47:00 +02:00
Mario Fetka 1164b32183 Record POP-07 evidence for the fixed live POP3/Store path
Documents the Dovecot ImapTest POP3 profile run against the rebuilt
live service, including the SIGABRT this exact path previously hit
before the FlushOutStreamEx fix and the live SEARCH HEADER
reproduction against the crash fixture.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-31 07:36:29 +02:00
Mario Fetka ac7e3cf640 Complete upstream IMAP validation and bound header search
Debian Trixie package bundle / packages (push) Failing after 11m25s
2026-07-30 22:38:12 +02:00
Mario Fetka a1163d5ed7 Document IMAP over-quota verification 2026-07-30 08:13:48 +02:00
Mario Fetka 56cfee5e44 Verify RFC 9208 IMAP quota queries 2026-07-30 02:41:04 +02:00
Mario Fetka b88ce2a634 Verify IMAP restart consistency 2026-07-30 02:31:21 +02:00
Mario Fetka 6e17eff2f3 Document IMAP robustness verification 2026-07-30 02:14:23 +02:00
Mario Fetka 1f026e5bd1 Document legacy IMAP4 compatibility tests 2026-07-30 01:57:29 +02:00
Mario Fetka e4d3560180 Document completed IMAP special-use tests 2026-07-29 23:45:30 +02:00
Mario Fetka 579ce8e097 Document completed IMAP UTF-8 tests 2026-07-29 23:23:50 +02:00
Mario Fetka 212f0b803b Document completed IMAP IDLE tests 2026-07-29 23:01:20 +02:00
Mario Fetka cf0a7d9be8 Document completed IMAP extended search tests 2026-07-29 22:37:23 +02:00
Mario Fetka f7ffc3317b Document completed IMAP search coverage 2026-07-29 18:53:41 +02:00
Mario Fetka d0a838365b Document IMAP expunge release coverage 2026-07-29 14:41:11 +02:00
Mario Fetka 9b410c8e60 Document IMAP 01 through 11 ImapTest coverage 2026-07-29 14:06:35 +02:00
Mario Fetka 26b6a4b757 Record IMAP FETCH and STORE release coverage 2026-07-29 12:17:45 +02:00
Mario Fetka d0cf38c8a6 Harden upstream ImapTest adapter 2026-07-29 10:33:10 +02:00
Mario Fetka 6770651877 Add upstream protocol test adapters
Debian Trixie package bundle / packages (push) Successful in 22m26s
2026-07-29 10:11:44 +02:00
Mario Fetka 9fc7fc6f80 Document current-server IMAP source audit 2026-07-29 09:44:54 +02:00
Mario Fetka 03a0226808 Document live IMAP selected state 2026-07-29 08:17:07 +02:00
Mario Fetka cb6782d11f Document live IMAP mailbox lifecycle 2026-07-29 07:55:16 +02:00
Mario Fetka 15c86d49b8 Cover basic live IMAP commands 2026-07-29 07:38:34 +02:00
Mario Fetka a5fbb83b3b Validate live IMAP authentication and TLS 2026-07-29 07:36:55 +02:00
Mario Fetka 461a65b668 Record Store and TLSRPT aggregation coverage 2026-07-28 20:18:01 +02:00
Mario Fetka c874d0f35f Move external accounts into user Stores
Debian Trixie package bundle / packages (push) Successful in 22m4s
2026-07-28 18:25:47 +02:00
Mario Fetka 94cf0665c1 Record live Sieve Store verification 2026-07-28 17:56:10 +02:00
Mario Fetka b0cd6f35a2 Store Sieve state in user stores
Debian Trixie package bundle / packages (push) Successful in 22m11s
2026-07-28 17:48:55 +02:00
Mario Fetka f2dd078d62 Drop unreleased scheduler migration
Debian Trixie package bundle / packages (push) Successful in 22m18s
2026-07-28 17:32:35 +02:00