geos_one/bongo
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases 1 Wiki Activity

Record bounded collector flood evidence

Browse Source
This commit is contained in:
Mario Fetka
2026-07-23 23:00:01 +02:00
parent b4f3a67ac4
commit 689ad14972
3 changed files with 74 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
contrib/testing/collector-flood-quota-check.py
+22 -15
View File
@@ -527,6 +527,8 @@ def main() -> int:
api.delete_account(account_id)
account_ids.remove(account_id)
delete_test_content(store)
store.Quit()
store = None
set_quota(0)
blocked_collector = dict(original_collector)
@@ -541,6 +543,7 @@ def main() -> int:
"collector", "maximum_pending_messages_per_user", 1
)
api = WebApi()
store = open_store()
seed("normal", 2, HOLD_SOURCE[0])
hold_id = api.create_account(HOLD_SOURCE)
account_ids.append(hold_id)
@@ -562,6 +565,9 @@ def main() -> int:
api.delete_account(hold_id)
account_ids.remove(hold_id)
delete_test_content(store)
store.Quit()
store = None
replace_configuration("collector", original_collector)
restart_bongo()
wait_local_configuration(
@@ -571,21 +577,6 @@ def main() -> int:
collector_changed = False
clear_remote(HOLD_SOURCE[0])
finally:
if collector_changed:
try:
replace_configuration("collector", original_collector)
restart_bongo()
wait_local_configuration(
"collector", "maximum_pending_messages_per_user",
original_collector.get(
"maximum_pending_messages_per_user"
),
)
except Exception as cleanup_error:
print(
f"STQ-12 cleanup warning (Collector): {cleanup_error}",
file=sys.stderr,
)
try:
cleanup_api = WebApi()
for account_id in account_ids:
@@ -604,11 +595,27 @@ def main() -> int:
store.Quit()
except Exception:
pass
store = None
for source in (*FLOOD_SOURCES, HOLD_SOURCE):
try:
clear_remote(source[0])
except Exception:
pass
if collector_changed:
try:
replace_configuration("collector", original_collector)
restart_bongo()
wait_local_configuration(
"collector", "maximum_pending_messages_per_user",
original_collector.get(
"maximum_pending_messages_per_user"
),
)
except Exception as cleanup_error:
print(
f"STQ-12 cleanup warning (Collector): {cleanup_error}",
file=sys.stderr,
)
try:
set_quota(original_quota)
except Exception as cleanup_error:
docs/release-testing-0.7.md
+1 -1
View File
@@ -117,7 +117,7 @@ inputs and outputs is absent from the ZIP.
| STQ-09 | Per-user mail quota excludes contacts/calendars and atomically blocks concurrent growth | [PASS](test-evidence/0.7-r1.md#stq-09) | | |
| STQ-10 | Queue/SMTP and external collection map Store 5220 to quota-exceeded delivery without data loss | [PASS](test-evidence/0.7-r1.md#stq-10) | | |
| STQ-11 | Lowered/unlimited limits, large sizes, upgrade migration, backup and restore preserve quota state | [PASS](test-evidence/0.7-r1.md#stq-11) | | |
| STQ-12 | Bounded 128-KiB collector flood reaches a small test quota without exceeding it, duplicating mail, or deleting the first message that was not durably imported | | | |
| STQ-12 | Bounded 128-KiB collector flood reaches a small test quota without exceeding it, duplicating mail, or deleting the first message that was not durably imported | [PASS](test-evidence/0.7-r1.md#stq-12) | | |
## SMTP receive, submission, and delivery
docs/test-evidence/0.7-r1.md
+51
View File
@@ -1942,3 +1942,54 @@ The cleanup path restored the original PAX backup and original quota. Final
checks showed zero charged bytes, an unlimited quota, and
`bongo.service` active/running with `NRestarts=0`. No message left the local
test system.
## STQ-12
Result: **PASS**
Commits `92e0ba11` and `b4f3a67a` bound external collection by both a global
pending-message count and a per-user count. Collector obtains the durable
count from authenticated Queue command `QCOUNT COLLECTED [user]`, increments
its local view after every successful commit, and stops before fetching the
next remote message when either configured limit is reached. External
accounts are processed serially: one POP3 or IMAP account completes its
bounded poll before Collector starts the next account.
The Queue counter accepts committed control files and in-progress replacement
controls, deduplicates their queue IDs, requires the collected-message flag
and matching local recipient, and now also requires the corresponding message
data file. This last condition prevents an administrative deletion or
completed delivery from leaving an orphaned work control that falsely holds
Collector at its cap. `QDELE` also removes the corresponding work control.
The reusable live check uses the local Cyrus fixture and a disposable Bongo
user:
```sh
export BONGO_ALLOW_LIVE_USER_TEST=1
export BONGO_TEST_PASSWORD='<disposable-test-password>'
./contrib/testing/collector-flood-quota-check.py
```
It seeded eight distinct 128-KiB messages into a Cyrus POP3 account and eight
into a Cyrus IMAP account. With `stqquota` limited to 400,000 bytes, exactly
two messages reached Store and the remaining fourteen stayed durably in
Queue 006. All sixteen unique markers were present exactly once, the charged
usage never exceeded the configured quota, and both remote mailboxes were
deleted only after the corresponding Queue commit.
The second phase temporarily set
`collector.maximum_pending_messages_per_user` to one, seeded two more POP3
messages, and verified that Collector committed exactly one. It recorded the
pending-message-limit diagnostic and left the first over-limit message at the
provider. The successful installed-service run reported:
```text
STQ-12 PASS pop3=8 imap=8 size=131072 quota-used=273887/400000 store=2 queue=14 duplicates=no overflow=no source-retained-at-limit=yes quota-restored=yes
```
The final GCC debug suite passed all 91 CTest tests in 11.48 seconds. The
live-test cleanup removed its external-account records, remote fixtures,
Store objects and Queue entries, restored the original Collector document and
unlimited quota, and left `bongo.service` active. No message left the local
test system.