Record STQ-10 quota delivery evidence

2026-07-23 20:14:55 +02:00
parent 7f382d2777
commit 9ffff35e7e
3 changed files with 77 additions and 1 deletions
@@ -172,6 +172,23 @@ export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
 ./contrib/testing/smtp-quota-dsn-check.py
 ```

+The Collector companion creates a real Queue transaction carrying
+`MSG_FLAG_COLLECTED_MESSAGE`. It proves that a full Store keeps the sole
+durable copy in local-delivery queue 6, never hands it to the outgoing SMTP
+agent, and delivers it exactly once after the quota is released:
+
+```sh
+export BONGO_ALLOW_LIVE_USER_TEST=1
+export BONGO_TEST_PASSWORD='a-disposable-password-of-at-least-12-characters'
+./contrib/testing/collector-quota-retry-check.py
+```
+
+Set `BONGO_TEST_TRACE=1` to print bounded per-stage timings. On failure,
+fixtures are normally removed and the original quota is restored. For local
+diagnosis only, `BONGO_TEST_KEEP_FAILURE=1` preserves the matching Queue entry
+while still restoring the quota; remove that entry with `bongo-queuetool
+delete` after inspection.
+
 The Debian source probes and package-build commands used by BLD-14/BLD-15 are
 consolidated in `contrib/debian/build-trixie-bundle.sh`; they are not maintained
 as duplicate container-only snippets here.
@@ -115,7 +115,7 @@ inputs and outputs is absent from the ZIP.
 | STQ-07 | Backup/export and restore reproduce mail, metadata, contacts, calendars | [PASS](test-evidence/0.7-r1.md#stq-07) |  |  |
 | STQ-08 | Invalid commands, oversized lines, and unauthorised access fail safely | [PASS](test-evidence/0.7-r1.md#stq-08) |  |  |
 | STQ-09 | Per-user mail quota excludes contacts/calendars and atomically blocks concurrent growth | [PASS](test-evidence/0.7-r1.md#stq-09) |  |  |
-| STQ-10 | Queue/SMTP and external collection map Store 5220 to quota-exceeded delivery without data loss |  |  |  |
+| STQ-10 | Queue/SMTP and external collection map Store 5220 to quota-exceeded delivery without data loss | [PASS](test-evidence/0.7-r1.md#stq-10) |  |  |
 | STQ-11 | Lowered/unlimited limits, large sizes, upgrade migration, backup and restore preserve quota state |  |  |  |
 | STQ-12 | Bounded 128-KiB collector flood reaches a small test quota without exceeding it, duplicating mail, or deleting the first message that was not durably imported |  |  |  |

@@ -1837,3 +1837,62 @@ The script removes every mail, contact, and calendar fixture in a `finally`
 cleanup path, restores the original administrative limit, keeps the password
 out of process arguments, and refuses to run without explicit live-test
 opt-in. No external mail or network service was involved.
+
+## STQ-10
+
+Result: **PASS**
+
+The complete Store-5220 path was exercised separately for ordinary SMTP mail
+and for a message whose remote POP3/IMAP source has already been removed by
+Collector.
+
+The SMTP run first exposed a deterministic 180-second delay after `DATA`.
+GnuTLS had already decrypted the SMTP terminator into its internal record
+buffer, but `ConnTcpRead()` polled the kernel socket before consulting
+`gnutls_record_check_pending()`. Commit `332521b3` consumes pending plaintext
+first and covers the boundary with the TLS peer regression test. The same run
+also found that the DSN used the local Store username instead of the SMTP
+envelope recipient. Commits `2be3a0a1` and `f31214eb` preserve and format the
+RFC 3461 envelope address, including an independently supplied ORCPT value.
+
+The final authenticated SMTP transaction completed through `QUIT` in 0.401
+seconds. The complete test, including observing the generated DSN and restoring
+the fixture account, completed in 4.675 seconds:
+
+```text
+STQ-10 SMTP PASS sender=test1@bongo.test recipient=stqquota@bongo.test accepted=yes store-partial=no dsn=5.2.2 queued=no quota-restored=yes
+```
+
+Collector requires different failure semantics because Queue owns the sole
+durable copy after a configured `after_import` deletion. Commit `88182956`
+added a live fixture that creates such a transaction and verifies its bytes,
+flags, Store state, retry and cleanup. Commit `0f768de1` added a bounded
+administrative retry command.
+
+The first live Collector run proved that the complete message and
+`MSG_FLAG_COLLECTED_MESSAGE` survived Store status 5220, but also exposed a
+second timeout: after retaining the local mailbox recipient, Queue continued
+into the outgoing SMTP agent and held the entry lock until that agent timed
+out. Commit `7f382d27` now moves a quota-blocked Collector message to queue 6
+and returns immediately. It cannot enter remote SMTP, and the normal Queue
+monitor or an explicit retry can process it after space is available.
+
+All 88 CTest tests passed against that final source in 14.90 seconds. The
+commit was pushed to `master`, installed as `mail-mta/bongo-9999::bongo`, and
+the live test then reported:
+
+```text
+STQ-10 collector trace 0.126s: baseline documents=0 bytes=0
+STQ-10 collector trace 0.186s: quota constrained
+STQ-10 collector trace 0.284s: enqueued as 000-a62598a
+STQ-10 collector trace 1.347s: candidate retained queue 006-a62598b
+STQ-10 collector trace 2.656s: retained queue settled as 006-a62598b
+STQ-10 collector trace 2.934s: quota released
+STQ-10 collector trace 3.250s: retry 006-a62598b: exit=0 stdout='' stderr=''
+STQ-10 collector trace 3.502s: delivered as Store document 0000000000000023
+STQ-10 COLLECTOR PASS user=stqquota retained=006-a62598b bytes=34549 duplicate=no partial=no retry-delivered=yes quota-restored=yes elapsed=4.050s
+```
+
+Post-test checks showed an empty Queue, zero charged bytes, an unlimited
+restored quota, and `bongo.service` active/running with `NRestarts=0`. No
+message left the local test system.