Replaces reliance on Mailutils' stock editheader.so (whose addheader/
deleteheader mutations were silently discarded by BongoSieveEvaluate,
since it only ever extracted a discrete action list, never the mutated
message) with a Bongo-authored module in BONGO_SIEVE_MODULE_DIR that
shadows Mailutils' own via the module search path. It still mutates the
message's mu_header_t directly (same well-tested approach as stock
editheader.c, for folding/matching correctness) but also reports a
BONGO_SIEVE_HEADER_EDITED marker through a new public
BongoSieveReportAction() ABI, so BongoSieveEvaluate() knows to
serialize the mutated header+body back out.
Persisting the edit required new plumbing since Bongo's queue only ever
referenced spooled messages by id (QMOD MAILBOX/RAW), never by value:
rules.c uploads Sieve-edited content per recipient via a new QMOD
MESSAGE verb, tagging the routing directive with a
" ~EDITED:<token>" marker; queue.c strips that marker before existing
directive parsing, and substitutes the recipient's own
e<entry>-<token>.msg for just that one DeliverToStore() call --
crucially never touching the shared d<entry>.msg other recipients of
the same queue entry still read unedited. Scoped to the immediate
delivery attempt only; a retry falls back to the unedited shared copy.
Also required generating a libtool .la wrapper for the module: Mailutils'
lt_dlopenext() tries .la files across the whole search path before ever
trying a bare .so in any single directory, so a .so-only module never
gets a chance once Mailutils' own moddir .la is found downstream,
regardless of search-path order.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BongoSieveEvaluate() ran editheader's actions correctly but only ever
reported the discrete action list (fileinto target, vacation params,
...) through its callback, then discarded the mutated mu_message_t.
Every routing directive rules.c sends (QMOD MAILBOX/TO/RAW) references
the already-spooled message by queue id, never by value, so the header
edits never reached delivery -- confirmed live via a real Sieve script
whose addheader never appeared on the stored message.
BongoSieveEvaluate() now takes modified_message/modified_message_length
out-parameters, populated only when the header set actually changed
(a before/after comparison, so scripts that never touch editheader are
byte-for-byte unaffected). The reconstruction stays entirely in Bongo's
own code rather than trusting Mailutils' own header+body combiner
(mu_message_get_streamref, the pattern from upstream's own
libmailutils/tests/modmesg.c): that path terminates the header block
with a hard-coded '\n' instead of '\r\n' on a CRLF message -- the same
bug class already patched locally for mu_header_size() (hdrsize.at)
but not this call path -- which would otherwise reformat every message
routed through Sieve, not just ones editheader actually mutated.
rules.c sends the reconstructed content via a new QMOD MESSAGE NMAP
verb (queue.c/CommandQmodMessage) that atomically rewrites the queue
entry's spooled d<id>.msg in place before any routing directive is
committed, mirroring the existing temp-file+rename pattern already
used for local queue submission.
Extends the live SIEVE-04 execution check with addheader/deleteheader
coverage through the real ManageSieve -> SMTP -> delivery pipeline.
Adds "editheader" to safe_requirement()'s allowlist, so
addheader/deleteheader work through Mailutils' own existing
libmu_sieve/extensions/editheader.so loadable module -- Bongo never
built or modified this module itself, it already ships as part of a
stock Mailutils install.
The actual safety boundary against Mailutils' other, genuinely unsafe
loadable modules ("pipe" and "moderator" run external programs,
"spamd" makes network connections) is safe_requirement()'s allowlist
itself: require.c never even attempts mu_sieve_load_ext() for a name
that isn't in it, and checks the action/test registry first regardless
(so a name Bongo has already registered directly, like "vacation",
never triggers a dlopen attempt at all). Also wires
mu_sieve_library_path_prefix to an additional, Bongo-owned module
directory (currently empty) alongside Mailutils' own unmodified,
system-wide one, so future Bongo-authored backend-aware modules (e.g.
a real imap4flags talking to bongostore over NMAP) have somewhere to
live without needing a Mailutils patch to be found -- kept deliberately
separate from any restriction concern, since the allowlist already
covers that.
and the missed symbols-file entry in SIEVE-04/06 evidence
Corrects the earlier "no bugs found" malformed-mail robustness
write-up: turning one of those ad hoc checks into a deterministic
regression test surfaced a genuine 5th CRLF bug in mu_header_size(),
a different code path from the four multipart-specific fixes. Records
the fix, the now-permanent mimerobust.c/.at and hdrsize.c/.at test
additions, and the dh_makeshlibs symbols-file fix caught by
re-verifying with a real dpkg-buildpackage build.
test suite, and fix a missed symbols-file entry
mu_header_size() (libmailutils/mailbox/header.c) hard-coded a 1-byte
blank-line separator ("+1") regardless of what was actually parsed;
for a CRLF-terminated message the real separator is 2 bytes, so every
non-multipart message's body picked up the blank line's own trailing
LF as a spurious leading byte. This is a different code path from the
four multipart-specific bugs already fixed here -- it's used by every
plain message, not just multipart ones. Fixed by having header_parse()
record the actual separator width it found (a new eol_size field) and
mu_header_size() use that instead of a hard-coded constant, falling
back to the original "+1" only when parsing never found a blank line
at all, to leave that untested edge case's behavior unchanged. New
regression tests (hdrsize.c/.at) confirmed to fail without the fix and
pass with it.
Also adds a permanent malformed-mail robustness test suite
(mimerobust.c/.at, 13 cases: missing/empty boundary, a boundary that
never appears, truncated input, no closing boundary, nested
multipart, boundary values with regex-special characters, and more)
verifying mu_message_is_multipart()/get_num_parts()/get_part() handle
all of them with a clean error or a sensible part count -- never a
crash or a hang -- so that stays true as the parser evolves, requested
explicitly rather than left as a one-off ad hoc check.
Separately: 0004's mu_message_default_envelope_recipient() export was
never added to debian/libmailutils9t64.symbols, the same class of
oversight already fixed once for the body.c/envelope-recipient
symbols. Caught by re-running the real dpkg-buildpackage verification
after this patch, which failed with the identical dh_makeshlibs error
as before. Fixed in 0002-debian-trixie.patch alongside the existing
entries.
Full stack (0001, 0003, 0004, extended 0005) re-verified clean: 777/777
libmailutils tests (Gentoo distfile), 756/756 (Debian 3.19, different
upstream baseline), 116/116 and 114/114 sieve tests respectively, zero
regressions, via both the cached Gentoo distfile and a genuine
dpkg-buildpackage build of Debian's source in a fresh Trixie container.
Deployed live, bongo.service active, SIEVE-04 passing.
malformed-mail robustness pass in SIEVE-04/06 evidence
Records the four-bug root cause and fix for 0005-crlf-mime-fixes.patch,
the new upstream test coverage added to the earlier body.c and
envelope-recipient patches (sieve/tests/body.at, extended
envelope.at), the honestly-documented RFC 5173 5.2 gap found while
writing those tests, and a clean malformed-mail robustness pass (21
deliberately broken messages, no new bugs found).
Mailutils patches
Neither patch had ever added regression tests to Mailutils' own
upstream test suite, only been verified against the existing one --
a real gap for upstream submission.
0003-sieve-envelope-body.patch: adds sieve/tests/body.at (new file,
Mailutils never had one), covering :contains, :raw vs :text
quoted-printable decoding, :content MIME-type restriction, and a
genuine multipart/alternative message exercising the boundary
traversal and per-part decoding the CRLF fixes (0005) also depend on.
Also documents one narrow, honestly-labeled RFC 5173 5.2 conformance
gap found while writing these (a header-only message still yields an
empty rather than absent body, so ":contains \"\"" trivially matches)
rather than silently omitting it.
0004-sieve-envelope-recipient.patch: extends sieve/tests/envelope.at
with a ":to" case. Writing it surfaced that mu_message_reconstruct_
envelope()'s lazy fallback -- the only place the previous patch wired
up a recipient callback -- is never reached for mbox-backed messages,
since libproto/mbox/message.c (a separate library) eagerly attaches
its own sender/date-only envelope first. Since mbox's own "From "
separator records no recipient at all (unlike sender), there is no
mbox-native value to prefer, so the same generic header-derived
default is now exported (mu_message_default_envelope_recipient(),
declared in mailutils/message.h) and wired into mbox's own envelope
construction too, closing the gap for both Mailutils' own sieve CLI
test infrastructure and any other mbox-backed caller.
Full upstream suite re-verified clean (762/762 libmailutils, 116/116
sieve, zero regressions) on both the Gentoo mailutils-3.21 distfile
and Debian's mailutils_3.19 source, and live against bongo.service.
libmailutils' MIME parser and header unfolding
Discovered while extending the body.c test matrix with a real
multipart/quoted-printable message (Cassandane-inspired): the
resulting message was silently empty/misparsed. Root-caused to four
distinct, previously-unknown bugs in vanilla Mailutils, all variants
of the same mistake -- hard-coding a 1-byte "\n" line terminator
instead of handling the 2-byte "\r\n" a real (CRLF-terminated) SMTP
message actually uses:
- mu_string_unfold() (libmailutils/string/unfold.c) copied a bare CR
through as ordinary content, corrupting the tail of every
CRLF-terminated header value, not just folded ones.
- match_boundary() (libmailutils/mime/mime.c) only stripped a
trailing LF from candidate boundary lines, so no boundary line in a
CRLF message ever matched and mu_message_get_num_parts() found zero
parts.
- The part header/body blank-line check only recognized a one-byte
blank line, so even a recognized part's body offset was never set
and its body came out empty.
- The body-length calculation subtracted a hard-coded 1 byte for the
CRLF "conceptually attached to the boundary" per RFC 1521, leaving
a stray trailing CR in every part's extracted body.
Fixed as a separate, upstream-submittable patch
(0005-crlf-mime-fixes.patch) with its own new regression tests
(hdrunfold.c/.at, mimeparts.c/.at) added to Mailutils' own test suite,
verified to fail without the fix and pass with it. Full upstream
suite re-run clean (zero regressions) on both the Gentoo mailutils-3.21
distfile and a real dpkg-buildpackage build of Debian's mailutils_3.19
in a fresh Trixie container.
sieve-execution-check.py's SIEVE-04 now also submits a genuine
multipart/alternative message with a quoted-printable text/plain part
and checks body :text :contains matches inside it -- the previous
body test never exercised multipart traversal, so a message shaped
like this would have silently broken despite otherwise passing.
dh_makeshlibs/dpkg-gensymbols failed the Trixie build: neither the
body.c patch's exports nor the new envelope-recipient accessors were
ever added to debian/libmailutils9t64.symbols, so both surfaced
together the first time the build actually completed dh_makeshlibs
against the patched source. Fixed by extending 0002-debian-trixie.patch
with the missing entries, each tagged 1:3.19-1+bongo1 (matching
dpkg-gensymbols' own suggestion) since none of these symbols exist in
any unpatched Debian mailutils build.
Verified with a full dpkg-buildpackage in a fresh Debian Trixie Docker
container against the real pinned source, not just a symbols-file diff.
Mailutils' retrieve_envelope() only ever resolved "from". Add
mu_envelope_set_recipient()/mu_envelope_sget_recipient() to
libmailutils, mirroring the existing sender accessor pair 1:1 (same
struct field, callback slot, and header-derived fallback via a new
message_envelope_recipient()), so "to" now resolves the same way
"from" always did. Packaged as a separate patch (0004) for independent
upstream review, wired into both the Gentoo overlay ebuild and the
Debian Trixie build script.
Verified against Mailutils' own upstream Sieve test suite (107/107,
zero regressions), against a standalone program replicating Bongo's
exact message-construction path (positive and negative match), and
live through bongo.service (two consecutive SIEVE-04 passes).
Rewrite contrib/testing/sieve-execution-check.py: replace the now-stale
"envelope/body rejected at upload" check with real live execution
tests. envelope :is "from" is confirmed to match the real sender
through actual SMTP delivery; envelope :is "to" is confirmed to never
match anything (a genuine, separate limitation of Mailutils'
retrieve_envelope(), which only implements the "from" envelope-part,
not something the require.c patch touches); body :contains is
confirmed to match real message content and correctly not match
messages without it, through Bongo's real delivery pipeline rather
than just the standalone `sieve` CLI tool used earlier to validate the
patch in isolation.
Document all of this, plus the GMime 3 address-search segfault find
and fix, as dated updates in the evidence file rather than rewriting
history: SIEVE-04 and SIEVE-06 get the envelope/body reality-check,
IMAP-13 gets the GMime 3 segfault root cause and fix, and BLD-15 notes
both dependency fixes now being part of the standard Debian Trixie
build.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same fix as the Gentoo overlay ebuild: patch require.c so
require ["envelope"]; can reach Mailutils' own standard envelope
test, and add a new body.c (RFC 5173) test module Mailutils never
implemented. Confirmed source-identical for the relevant files
between Debian's mailutils_3.19 and the Gentoo overlay's 3.21, and
regenerated as a proper git-diff-formatted quilt patch so it applies
cleanly via the existing apply_quilt_patch() helper.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Gentoo mail-mta/bongo overlay's net-mail/mailutils ebuild now
carries a local patch (files/mailutils-3.21-sieve-envelope-body.patch)
that fixes require.c's registry lookup so require ["envelope"]; can
reach the standard test Mailutils already implements, and adds a new
body.c test module (RFC 5173, using only Mailutils' own MIME API) that
Mailutils never implemented in any form. Verified against Mailutils'
own upstream Sieve test suite (107/107, zero regressions, including
the real envelope.at case) plus live execution of both extensions.
Advertise both again in the ManageSieve CAPABILITY string and allow
them in safe_requirement(); update docs/sieve.md and drop the now-
resolved ROADMAP item about replacing the Sieve engine entirely.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Passing NULL for "use default options" to internet_address_list_parse()
and internet_address_list_to_string() relies on GMime's parser/format
options singleton accessors (g_mime_parser_options_get_default(),
g_mime_format_options_get_default()) always returning a valid object.
At least the GMime 3.2.15 build shipped on Debian Trixie has a broken
singleton that returns NULL instead, and the parser/formatter then
dereferences that NULL unconditionally rather than falling back to
safe defaults -- a reliable segfault on any input, not just malformed
ones. Reproduced live in a Debian Trixie container (gdb backtrace
through g_mime_parser_options_get_warning_callback() and
g_mime_format_options_get_newline()), fixed by allocating our own
short-lived options objects instead of relying on NULL, and confirmed
against both the real store-address-search CTest under GMime 3.2.15 in
that container and the full local suite under GMime 2.6.23.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verify Bongo deduplicates "keep; keep;" and "keep; fileinto \"INBOX\";"
to a single delivered copy, matching Cyrus Cassandane's
test_dup_keep_keep/test_dup_keep_fileinto regression tests, while two
genuinely distinct fileinto targets still each receive their own copy.
Documents the duplicate-delivery fix found via this comparison, and
synthesizes the Pigeonhole cross-references and all six real bugs
found and fixed across the whole SIEVE-01..06 block.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A Sieve script producing more than one delivery action targeting the
same mailbox -- "keep; keep;" or "keep; fileinto \"INBOX\";" -- filed
one independent "QMOD RAW"/"QMOD MAILBOX" recipient line into the
queue's rewritten envelope per action, so the message was delivered
once per redundant action instead of once. Confirmed live: a "keep;
keep;" script produced two copies of the same message in INBOX.
Real Sieve deployments are expected to collapse this; Cyrus's
Cassandane test suite has explicit regression tests for exactly these
two cases (test_dup_keep_keep, test_dup_keep_fileinto). Track which
mailboxes this execution has already delivered into and skip issuing
a second QMOD command for a repeat target, while still counting it
toward "a delivery happened" so the implicit-keep fallback does not
add a third copy.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verify invalid-syntax rejection, script size/quota boundaries,
duplicate-name overwrite semantics, script-name traversal/control-char
rejection (with ".." correctly accepted, since Bongo's SQL-backed
store has no filesystem traversal risk), cross-user isolation, and
pre-authentication command safety. Documents the ConnReadLine/
ConnReadAnswer NUL-byte hang fix and the ConnReadLine overlong-line
nul-termination fix, both found and fixed earlier in this block.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Both functions bound their newline search with strchr(cur, '\n')
against a buffer that is only artificially nul-terminated at 'limit'
for that purpose. A client that sends a literal 0x00 byte anywhere
before the real line terminator (e.g. inside a ManageSieve quoted
string, which the grammar otherwise happily rejects with SYNTAX for
other control characters) makes strchr() stop at that embedded NUL
instead of the real '\n' sitting later in the already-fully-buffered
data. The read loop then believes no complete line is present yet and
blocks in ConnTcpRead() waiting for more bytes that will never come,
hanging the connection (and the thread and active_connections slot
behind it) instead of returning the SYNTAX error every other
malformed command gets. Reproduced live against bongosieve: PUTSCRIPT
with a NUL byte in the script name times out completely instead of
answering NO.
Use memchr() bounded by the real buffer extent instead, which finds
the '\n' regardless of embedded NULs, matching the original intent of
the artificial termination (bounding the search, not truncating the
data).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
When a line exceeds the caller's buffer before a '\n' is found,
ConnReadLine() filled the buffer completely and returned Length with
no room left for a nul terminator, unlike the normal found-'\n' path
which always leaves room and terminates. Any caller that does
strlen() on the result (managesieved.c's serve() loop does, on every
iteration) reads past the end of its stack buffer in this case.
Reachable live: a ManageSieve client using a non-synchronizing {N+}
literal for a script larger than the 1MB parser ceiling has its
command line rejected before the server knows to skip the announced
literal bytes, so the payload gets fed through the line parser
instead and manages to exceed managesieved.c's 8192-byte line buffer
with no embedded CRLF. Reserve one byte for the terminator in both
overlong-line branches, matching the found-'\n' path.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verify fileinto (variables-interpolated target, implicit-keep
suppression) and vacation (auto-reply delivery, dedup interval) end
to end through real SMTP delivery, and that envelope/body are rejected
cleanly at upload. Documents the fileinto/MOVE queue-parsing fix, the
vacation/reject bare-recipient address qualification fix, and the
envelope/body Sieve-engine limitation, all found and fixed earlier in
this block.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
client->recipient is Bongo's bare local Store identity (matching how
IMAP/POP3/ManageSieve auth and BongoSieveStoreOpenNmap() address
mailboxes), not an RFC 5321 address. SieveSendVacation()'s default
:from and its anti-spoofing check, and SieveSendReject()'s MDN
From:/Final-Recipient fields, all called SieveAddress() directly on
this bare value or compared a real address against it -- neither can
ever succeed, since SieveAddress() requires a local-part and a domain,
and no address can ever equal a bare username under strcasecmp(). Both
actions were completely non-functional: vacation silently no-op'd and
reject hard-failed back to a plain keep, confirmed live and diagnosed
with temporary tracing (reverted here).
Add QualifyRecipientAddress(), which reads this server's own hosted
domains live via NMAP (the same way accounts.c resolves alias
domains) and appends the first one to the bare recipient. Extend
SieveAddress() to optionally return the parsed local-part so the
vacation :from check can compare local-parts instead of comparing a
full address against a bare username.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Temporary instrumentation only, to be reverted once the vacation
delivery gap is diagnosed. Mirrors the SMTP-34 diagnostic pattern from
earlier this release cycle (see commit 26e29503).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BongoQueueParseMailboxRecipient() (envelope-format.c) requires the
5-field "MRecip ORecip flags MBox MsgFlags" format to parse a queued
mailbox-filing recipient line back out during local delivery. Both
Sieve fileinto (SieveAction()) and the classic pre-Sieve MOVE filing
rule sent only 4 fields, omitting MsgFlags. The queue silently kept
the malformed line unchanged forever instead of delivering it: the
resulting message never lands anywhere and never bounces, with the
only diagnostic a plain Log(LOG_WARNING, ...) call that (per the
SMTP-34 investigation earlier this cycle) never reaches journald after
privilege drop. Confirmed live: every fileinto/MOVE delivery was
completely non-functional. Append a message-flags field of 0 (unused
by DeliverToStore() today, but required for the line to parse) to
both call sites.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Capture the chosen direction: import Cyrus libsieve unmodified via git
submodule with minimal build-time compatibility patches, mirroring the
0.8 Bulwark Webmail integration plan, instead of patching Mailutils or
writing a native engine from scratch.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The installed GNU Mailutils Sieve engine cannot run either extension:
its require resolution only ever checks its action registry for a bare
extension name, so `require ["envelope"];` (the only syntax RFC 5228
scripts use) can never succeed no matter how a same-named test is
registered, and it does not implement `body` (RFC 5173) in any form.
Both are confirmed independently of Bongo against Mailutils' own
`sieve` reference tool. Stop claiming support in the ManageSieve
CAPABILITY string and the safe_requirement() allowlist so uploads
requiring them are rejected cleanly instead of silently accepted and
never actually usable. Document the limitation and follow-up options
(upstream patch, engine replacement, or a native Bongo Sieve engine)
in ROADMAP.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verify SETACTIVE atomicity/idempotence, RENAMESCRIPT active-flag
preservation and collision handling, the DELETESCRIPT NONEXISTENT/ACTIVE
fix under a live session, LOGOUT pre-auth behavior, and activation
persistence across reconnects, cross-checked against Dovecot
Pigeonhole's managesieve command handlers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verify HAVESPACE/CHECKSCRIPT/PUTSCRIPT/LISTSCRIPTS/GETSCRIPT against
the live port 4190 listener, confirming the HAVESPACE and DELETESCRIPT
RFC 5804 fixes and comparing against Dovecot Pigeonhole's managesieve
command handlers.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
RFC 5804 requires DELETESCRIPT to report NONEXISTENT and ACTIVE with
different response codes, but Bongo always answered ACTIVE for both.
Thread a was_active out-parameter through BongoSieveStoreDelete (SQLite
and NMAP backends) so managesieved.c can tell the client which one it
hit, matching Dovecot Pigeonhole's SIEVE_ERROR_NOT_FOUND/SIEVE_ERROR_ACTIVE
split.
Also reject HAVESPACE for a 0-byte script instead of answering OK, since
PUTSCRIPT already refuses to store an empty script; Pigeonhole's
cmd-havespace.c does the same.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verify greeting/STARTTLS/pipeline-discard/SASL PLAIN+LOGIN behavior
against the live port 4190 listener, matching Dovecot Pigeonhole's
managesieve command set.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
HAVESPACE parsed and validated the client's requested size but the
handler ignored it, always answering OK regardless of the configured
script size limit. A client could be told a script would fit and
then have PUTSCRIPT reject it for size moments later. Matches
Dovecot Pigeonhole's cmd_havespace(), which genuinely checks the
requested size against the storage's quota.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The scanner-outage-policy and update-channel rows duplicated the IDs
already used for the collector-scan-path and runtime-EICAR/GTUBE
rows. No evidence entries referenced the duplicated IDs yet.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a sixth scenario: concurrent load through the generic
use_relay_host SMTP client (DeliverGlobalRelay, distinct from the
lmtp_transports path already covered), verifying every relayed
message carries a DKIM-Signature header. Raises the trusted internal
relay port's rate limit for the test's duration, since running
several deliberate load phases through it otherwise trips the
abuse-prevention limit that exists to bound exactly that in
production.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Port 26 is the trusted internal relay; port 25 is what actually
receives concurrent load from the open internet, including the
antispam/antivirus scan path. Confirmed bongoantispam/bongoavirus and
their ClamAV/SpamAssassin daemons were active and not bypassed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verifies concurrent SMTP/LMTP load, soft-reject and disconnect
failure injection, and Queue retry/recovery using upstream Postfix's
own test tools, built locally since installing mail-mta/postfix
would remove mail-mta/bongo.
The investigation initially suspected a real concurrent-LMTP message
loss bug in Bongo; extensive isolated reproduction (see the evidence
entry) traced it entirely to two bugs in this test script: a
doubled sudo prefix that silently broke every Queue state query, and
a capture check that compared dump-file counts before bongosmtpc had
finished flushing the last LMTP session. No Bongo source change
resulted. Passed twice consecutively end to end.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The suspected concurrent-LMTP message loss was a false alarm caused
by leaked smtp-sink debug processes sharing one port via
SO_REUSEPORT (each Postfix inet_listen() enables it unconditionally),
which silently split incoming connections across several stale
listeners writing to different dump directories. Bongo's LMTP
delivery was never losing messages. Removes the diagnostic tracing
added while chasing this.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
bongo.service runs with PrivateTmp=true, so /tmp inside the service
is not the host's /tmp; the previous diagnostic file was invisible.
/var/lib/bongo is in ReadWritePaths and already readable via sudo.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
journald's 200-line window truncates before bongosmtpc's own trace
lines are ever reached once bongoqueue's higher-volume per-entry
trace fills it. Both agents now also append to
/tmp/bongo-smtp34-diag.log directly, avoiding any windowing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds stdout tracing around FindLMTPTransport/ConnectLMTP/
DeliverMessage to see whether a connection attempt and delivery
result actually occur for entries that bongoqueue later discards as
keep=FALSE without ever reaching the sink.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Log()/syslog output from bongoqueue never reaches journald in this
environment (likely no /dev/log after its privilege drop), so the
prior NOTICE-level tracing was invisible. XplConsolePrintf writes
directly to stdout, which the systemd unit does capture.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds LOG_NOTICE tracing to StartImportedQueueEntry and
ProcessQueueEntry to locate where queued remote-delivery entries
disappear under concurrent LMTP load with an aggressive retry
config. Will be removed or replaced with a real fix once the root
cause is confirmed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verifies DELE alone, and an abrupt disconnect right after DELE, leave
the account's quota Used bytes unchanged; DELE+RSET+QUIT also leaves
it unchanged; and only DELE+QUIT durably frees the space. Completes
the POP-01..07 block. No source change required.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verifies UTF8 mode (including a corrected idempotent-repeat
assumption), STLS rejection after UTF8, byte-correct UTF-8 RETR vs.
the ASCII-only gate without UTF8 mode, pipelined command ordering,
RFC 2449 response codes, oversized-line/unknown-command handling, and
documents the 900s idle timeout from source (no fast-test config knob
exists for it, unlike SMTP-25). No source change required.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verifies pre-login bad-state rejection, that DELE without a following
QUIT never reaches the durable Store DELETE path (confirmed via a
fresh IMAP check after an abrupt disconnect), RSET restoring deleted
flags, QUIT's durable commit, and DELE error handling. No source
change required.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Seeds IMAP-appended fixtures covering plain, dot-stuffed, and bare-LF
bodies, then verifies STAT/LIST/UIDL consistency and byte-correct
RETR/TOP output against the live server, with UIDL/RETR semantics
cross-checked against Dovecot's pop3-commands.c. No source change
required.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verifies port 995 TLS, USER/PASS login, malformed-argument and
bad-password handling, and PLAIN/LOGIN SASL mechanisms shared with
SMTP/IMAP. Cross-checked the repeated-USER-before-PASS semantics
against Dovecot's pop3-login and fixed a wrong test assumption rather
than the server, which already matches Dovecot's behavior.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verifies the cleartext CAPA advertisement, USER/AUTH/PASS rejection
before STLS, RFC 2595 pipelined-plaintext discard, and post-STLS
capability/state transitions, with a live Cyrus fixture reference for
the CAPA shape. No source defect found; documents the existing
STARTTLS-gated behavior against the matrix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Documents the Dovecot ImapTest POP3 profile run against the rebuilt
live service, including the SIGABRT this exact path previously hit
before the FlushOutStreamEx fix and the live SEARCH HEADER
reproduction against the crash fixture.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
FlushOutStreamEx correctly detects when a downstream codec reports
zero bytes processed, but every codec call site discards its FALSE
return and writes anyway, producing a heap buffer overflow at the
1024-byte codec boundary. A downstream codec can legitimately report
zero progress once while settling into a new stream state, so allow
one retry before treating it as a real failure.
Adds a regression test built from an actual crash-triggering flood
message that reproduces the overflow in RFC822_Fold_Decode under
ASan without the fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Mario Fetka