3 Commits

Author SHA1 Message Date
Mario Fetka 55a9c77a00 Fix real bongo.__init__.py install collision and queuetool port default
Debian Trixie package bundle / packages (push) Failing after 16m43s
Two more genuine bugs, found chasing down the remaining failures in the
newly-wired live-instance ctest suite:

1. src/apps/storetool/CMakeLists.txt installed its own bongo/__init__.py
   (a namespace-package merge stub, same pattern -- and the same "should
   not be installed" caveat -- as src/apps/admin/bongo/__init__.py) to the
   exact same site-packages/bongo/ destination as the real one from
   src/libs/python/bongo, which does `from .BongoError import BongoError`
   / `from .version import VERSION`. Since storetool's subdirectory is
   add_subdirectory()'d after src/libs/python/bongo in the top-level
   CMakeLists.txt, its stub silently overwrote the real __init__.py on
   every install, breaking `bongo.BongoError` and `bongo.VERSION` for
   every consumer -- surfaced here as "TypeError: 'module' object is not
   callable" from bongo/store/StoreConnection.py's `raise
   bongo.BongoError(...)`. Excluded storetool's copy via a REGEX anchored
   on the full source path (a bare "^__init__\.py$" pattern, tried first,
   never matched anything: install()'s REGEX EXCLUDE matches the whole
   path, not a path relative to the DIRECTORY argument), scoped narrowly
   enough to leave bongo/storetool/__init__.py (a different, legitimate
   subpackage marker) alone.

2. bongo-queuetool's own --host/--port CLI defaults are "localhost"/8670
   -- the *production* queue agent's address, not this instance's
   compiled BONGO_QUEUE_PORT (16870 under BONGO_CTEST_LIVE_INSTANCE). Same
   isolation-breach class as the earlier AuthSqlite_GetUserStore()
   hardcoded-port fix: every _bongo_queue_argv() helper across the 6
   affected scripts now injects --host/--port explicitly from new
   BONGO_TEST_QUEUE_HOST/BONGO_TEST_QUEUE_PORT environment variables, set
   in the shared ctest environment.

collector-quota-retry, queue-quota-partial, smtp-quota-dsn,
smtp-local-delivery, and sendmail-local-submission all now pass.
smtp-client-interoperability still fails on a different, unrelated
question (a curl-submitted message not reaching test2's INBOX) --
investigating separately.

Verified: a from-scratch default build's full 109-case ctest suite still
passes unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-02 09:53:02 +02:00
Mario Fetka 3b444da491 Wire 43 more contrib/testing scripts into ctest; fix isolation bugs found
Extends contrib/testing/CMakeLists.txt with the bulk of the remaining
live-test scripts that need nothing beyond the live_instance fixture --
IMAP, POP3, SMTP, store, and collector coverage.

Running the full wired suite for the first time immediately surfaced real
isolation bugs, all now fixed:

- Several quota-manipulating scripts (imap-overquota, pop3-quota,
  smtp-quota-dsn, collector-quota-retry, store-quota-state/concurrency,
  queue-quota-partial, and others) default BONGO_TEST_ADMIN to the *real*
  production /usr/bin/bongo-admin. Same class of bug as the earlier
  AuthSqlite_GetUserStore() hardcoded-port fix -- now pointed at the
  live-instance's own scratch-prefix binary via the shared ctest
  environment.
- ~18 of those scripts unconditionally wrap bongo-admin calls in
  `sudo -n`, assuming they're targeting the real service user. bongo-admin's
  own IsAdministrativeUser() (src/apps/admin/admin.c) accepts either real
  root or a euid matching the compiled BONGO_USER -- which a
  CTEST_BUILD_NOROOT instance compiles as the invoking user's own name --
  so sudo was never actually required for the live instance. Added a
  BONGO_TEST_ADMIN_UNPRIVILEGED=1 escape hatch, set in the shared ctest
  environment, everywhere this pattern appears.
- smtp-quota-dsn/collector-quota-retry default to their own dedicated
  "stqquota" account when BONGO_TEST_USER is unset, but the shared ctest
  environment was forcibly overriding it to test1 -- now pinned back to
  their own intended account explicitly.
- POP3-touching tests (pop3-quota/retrieval/robustness/transaction) and
  quota-state tests (imap-overquota) share test1's INBOX / account-level
  quota state with everything else if left on the shared account, unlike
  IMAP tests which create/delete their own uniquely-named throwaway
  mailboxes. live-instance-setup now provisions dedicated pop3test1/
  pop3test2/imapquota accounts (see the --user list and the new
  bongo_add_live_test_env() helper for per-test environment overrides).
- BONGO_MAILDROP_DIR (production relies on systemd-tmpfiles to create it,
  which this fixture has no equivalent of) is now created explicitly, same
  as the runtime dir fix from the previous commit.
- Added BONGO_ALLOW_LIVE_SMTP_TEST=1 to the shared environment (several
  SMTP scripts gate on it separately from BONGO_ALLOW_LIVE_USER_TEST).

Two scripts (smtp-command-safety, smtp-forward-header) were caught
red-handed: both load smtp-forward-metadata-check.py as a shared helper
whose restart_bongo() calls `systemctl restart bongo.service` directly
against the *real* production service. Confirmed empirically that this
attempt was rejected ("Access denied", no sudoers grant for this
unprivileged ctest run) and production was never actually restarted
(ActiveEnterTimestamp unchanged) -- but neither belongs in the isolated
suite without rework, so both are removed from the wiring for now.

Also removed store-backup-compare.py (a two-argument comparison utility,
not a standalone test) and store-diagnostics.py (a manual diagnostic dump
with no pass/fail assertions and no host/port configurability) -- both
were misclassified as wireable live tests.

Net result: the wired suite went from 82/99 to 42/50 passing after
narrowing to just the newly-touched tests (full suite numbers differ
since two tests were removed and dedicated accounts added). The 8
remaining failures are real, further findings warranting their own
investigation: a reproducible imap-idle protocol bug (fragmented DONE
continuation during concurrent EXPUNGE notification closes the
connection instead of completing IDLE), an imap-store $AppendKey
keyword-replacement bug, a POP3 TOP inspection error reproducible even
against a fresh dedicated account (not contamination), and
smtp-client-interoperability/collector-quota-retry issues not yet root
caused.

Verified: a from-scratch default build (no BONGO_CTEST_LIVE_INSTANCE)
still passes all 109 existing tests unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-02 09:19:06 +02:00
Mario Fetka b145f52dc9 Distinguish collected mail quota retries
Debian Trixie package bundle / packages (push) Failing after 14m42s
2026-07-23 18:37:51 +02:00