docs: update AI and redesign integration notes
All checks were successful
Source release / source-package (push) Successful in 1m2s
All checks were successful
Source release / source-package (push) Successful in 1m2s
This commit is contained in:
102
AI.md
102
AI.md
@@ -414,6 +414,108 @@ Desired future server log format:
|
|||||||
- Do not invent a parallel logger casually. Reuse existing mars_nwe logging
|
- Do not invent a parallel logger casually. Reuse existing mars_nwe logging
|
||||||
functions/macros and normalize message format gradually.
|
functions/macros and normalize message format gradually.
|
||||||
|
|
||||||
|
|
||||||
|
## 2026-06-05 - current superbuild / storage / TLS handoff
|
||||||
|
|
||||||
|
This chat switched from documentation-only endpoint audit work back to build and
|
||||||
|
integration work around the mars-nwe superbuild. The old patch-number guidance
|
||||||
|
below remains useful historical context for the endpoint-audit series, but the
|
||||||
|
current accepted work in this chat is a separate functional/build patch stack.
|
||||||
|
Do not assume the next patch after this point is an endpoint-audit `0269` patch
|
||||||
|
unless the user explicitly returns to that series.
|
||||||
|
|
||||||
|
Current root/superbuild direction implemented or in progress:
|
||||||
|
|
||||||
|
- `update-submodules.sh` is now the normal helper for keeping mars-nwe's
|
||||||
|
submodules reproducible. Private mars-owned submodules are updated to their
|
||||||
|
configured latest branch, while external upstream snapshots are pinned by a
|
||||||
|
single editable `EXTERNAL_TAG_PINS` block near the top of the script.
|
||||||
|
- Current external pins are `third_party/yyjson=0.12.0`,
|
||||||
|
`third_party/zlog=1.2.18`, and
|
||||||
|
`third_party/libsodium/libsodium=1.0.20-FINAL`.
|
||||||
|
- The helper must not run a root-level `git submodule update --init --recursive`
|
||||||
|
after updating top-level private submodules, because that can reset updated
|
||||||
|
gitlinks such as `third_party/matrixssl` back to the parent commit. Nested
|
||||||
|
submodules are initialized inside the owning top-level submodule instead.
|
||||||
|
- The helper prints recursive submodule status on success and failure and can
|
||||||
|
auto-commit root gitlink/script changes. Root gitlinks must only reference
|
||||||
|
commits that exist in the respective submodule remotes; a parent gitlink to an
|
||||||
|
unpushed submodule commit will make later `git submodule update` fail with
|
||||||
|
`upload-pack: not our ref`.
|
||||||
|
- `yyjson` is being folded into the mars-nwe core library path. Consumers should
|
||||||
|
include it through the `nwcore` include namespace and link `mars_nwe::core`,
|
||||||
|
not link an independent public yyjson target directly.
|
||||||
|
- FLAIM is currently only needed when the directory service is enabled. The root
|
||||||
|
CMake should therefore add/build `third_party/flaim` only when
|
||||||
|
`ENABLE_DIRECTORY=ON`.
|
||||||
|
|
||||||
|
Current `libnwssl` / MatrixSSL / OpenSSL-compat direction:
|
||||||
|
|
||||||
|
- MatrixSSL owns only the renamed low-level backend library (`libnwmatrixssl`).
|
||||||
|
Temporary OpenSSL-compat test headers/sources that were placed in the
|
||||||
|
MatrixSSL fork must move out of MatrixSSL.
|
||||||
|
- `libnwssl` owns the mars-nwe crypto/TLS facade plus the narrow compatibility
|
||||||
|
surfaces needed by FLAIM: OpenSSL-style FTK/network headers and NICI/CCS-style
|
||||||
|
private headers.
|
||||||
|
- OpenSSL-compat headers should live under the `nwssl` include subtree, e.g.
|
||||||
|
`include/nwssl/openssl/*.h`, not as a root-level `include/openssl` directory
|
||||||
|
that could conflict with system OpenSSL headers.
|
||||||
|
- Private FLAIM/NICI compatibility headers should also live under the `nwssl`
|
||||||
|
subtree, e.g. `include/nwssl/private/nici/...`. They are private build
|
||||||
|
compatibility headers, not the public TLS API for normal mars-nwe code.
|
||||||
|
- `smart`, `directory`, and FLAIM should link against `libnwssl` / the
|
||||||
|
`mars_nwe::ssl` target when they need TLS/crypto compatibility. They should
|
||||||
|
not include MatrixSSL headers directly and should not link OpenSSL directly.
|
||||||
|
|
||||||
|
Current FLAIM import/build direction:
|
||||||
|
|
||||||
|
- The imported FLAIM tree is kept under `third_party/flaim` and should remain as
|
||||||
|
close to upstream source as practical. Prefer CMake/build glue, include paths,
|
||||||
|
and tiny compile fixes over broad C/C++ rewrites.
|
||||||
|
- The mars-nwe build currently wants `libnwflaimtk`, `libnwflaim`,
|
||||||
|
`libnwflaimsql`, and `libnwxflaim`. XFLAIM is now included even though it is
|
||||||
|
not immediately required, so it stays build-covered.
|
||||||
|
- FLAIM library versions should come from the public headers when those disagree
|
||||||
|
with `configure.ac`, because the headers are the ABI-facing version source in
|
||||||
|
this import. Current expected shared-object versions are:
|
||||||
|
`libnwflaimtk.so.1.2`, `libnwflaim.so.4.62`,
|
||||||
|
`libnwflaimsql.so.6.00`, and `libnwxflaim.so.5.12`.
|
||||||
|
- All installed FLAIM public headers should go below one mars-nwe namespace
|
||||||
|
directory: `include/nwflaim/`. Do not make `xflaim.h` a special
|
||||||
|
`include/nwxflaim/` exception.
|
||||||
|
- FLAIM command-line utilities should be built and installed with `nw`-prefixed
|
||||||
|
binary names, for example `nwflmcheckdb`, `nwflmrebuild`, `nwflmview`,
|
||||||
|
`nwflmdbshell`, `nwflmgigatest`, and the matching `nwxflm...` utilities.
|
||||||
|
- FLAIM tools require curses/ncurses. CMake should report clearly whether
|
||||||
|
curses/ncurses was found and whether the curses-backed tools will be built.
|
||||||
|
- The uploaded `ncurses-stable.tar.gz` is a valid current upstream ncurses/stable
|
||||||
|
source snapshot for local build testing even if the top-level extracted name
|
||||||
|
does not look like a conventional ncurses release tarball.
|
||||||
|
- The CMake conversion should keep translating the old `Makefile.am` source
|
||||||
|
inventories. Recent build fixes added missing result-set sources and fixed
|
||||||
|
XFLAIM tool include ordering so `xflaim/util` code includes the XFLAIM
|
||||||
|
`flaimsys.h` rather than the classic FLAIM one.
|
||||||
|
- Minimal FLAIM C++ compile fixes are acceptable when required by modern
|
||||||
|
compilers, for example replacing pointer assignments/comparisons using
|
||||||
|
character `\0` with `NULL`/null-pointer checks. Keep those patches small and
|
||||||
|
separate from build-system changes where possible.
|
||||||
|
|
||||||
|
Current local dependency policy for build checks:
|
||||||
|
|
||||||
|
- Build GDBM locally from the uploaded `gdbm-1.26.tar.gz` when testing in an
|
||||||
|
isolated prefix.
|
||||||
|
- Build ncurses locally from the uploaded `ncurses-stable.tar.gz` when testing
|
||||||
|
FLAIM tools in an isolated prefix.
|
||||||
|
- Use the uploaded `Linux-PAM-1.7.2.tar.xz` for PAM headers if needed, but link
|
||||||
|
against the system PAM library. Do not vendor PAM as a mars-nwe library.
|
||||||
|
- These local dependency builds are for verification; they are not new vendored
|
||||||
|
submodules unless a later explicit import decision says otherwise.
|
||||||
|
|
||||||
|
When continuing this work, expect more incremental compile/build patches rather
|
||||||
|
than one large redesign patch. Build after each FLAIM/CMake change, record the
|
||||||
|
next concrete compiler/linker error, and keep generated patches per repository
|
||||||
|
or submodule so the user can apply them with `git am` at the correct path.
|
||||||
|
|
||||||
## Build and test notes
|
## Build and test notes
|
||||||
|
|
||||||
Dependencies used during local checks in this conversation:
|
Dependencies used during local checks in this conversation:
|
||||||
|
|||||||
57
REDESIGN.md
57
REDESIGN.md
@@ -1557,6 +1557,63 @@ mars-tinyldap/
|
|||||||
later wired to libdirectory/libflaim instead of tinyldap's original flat files
|
later wired to libdirectory/libflaim instead of tinyldap's original flat files
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Current superbuild integration status
|
||||||
|
|
||||||
|
The current implementation work has moved several items from planning into an
|
||||||
|
initial mars-nwe superbuild shape. Keep this status separate from the older
|
||||||
|
endpoint-audit patch-number notes: these are functional/build integration facts,
|
||||||
|
not a promise that every compatibility layer is complete.
|
||||||
|
|
||||||
|
Current source layout decisions:
|
||||||
|
|
||||||
|
- `third_party/yyjson` and `third_party/zlog` remain external upstream snapshots
|
||||||
|
pinned by release tag in `update-submodules.sh`.
|
||||||
|
- `third_party/yyjson` is being compiled into `libnwcore`; consumers should use
|
||||||
|
the `nwcore` include namespace and link the core target instead of exposing a
|
||||||
|
standalone yyjson API as a mars-nwe public dependency.
|
||||||
|
- `third_party/libsodium/libsodium` remains a nested external upstream snapshot
|
||||||
|
pinned to `1.0.20-FINAL` inside the mars-libsodium wrapper submodule.
|
||||||
|
- `third_party/matrixssl` is now the mars-maintained MatrixSSL fork producing
|
||||||
|
the renamed backend library `libnwmatrixssl`. It should not contain the
|
||||||
|
temporary OpenSSL-compat shim.
|
||||||
|
- `libnwssl` in the mars-nwe root owns the SSL/crypto facade plus FLAIM
|
||||||
|
compatibility headers. Its compatibility header layout should stay under the
|
||||||
|
`nwssl` include subtree, for example `include/nwssl/openssl/*.h` and
|
||||||
|
`include/nwssl/private/nici/...`.
|
||||||
|
- `third_party/flaim` is the current FLAIM import path used by the working tree.
|
||||||
|
It provides renamed mars-nwe libraries and tools; future prose may still refer
|
||||||
|
to `libflaim` as the logical storage engine, but the concrete submodule path
|
||||||
|
is `third_party/flaim` unless the user explicitly renames it.
|
||||||
|
- `third_party/flaim` is currently gated by `ENABLE_DIRECTORY`. The normal build
|
||||||
|
should not configure or build FLAIM when the directory service is disabled.
|
||||||
|
|
||||||
|
Current FLAIM CMake import decisions:
|
||||||
|
|
||||||
|
- Build `libnwflaimtk`, `libnwflaim`, `libnwflaimsql`, and `libnwxflaim` with
|
||||||
|
mars-nwe names so they do not collide with any system FLAIM installation.
|
||||||
|
- Build and install the FLAIM/XFLAIM utilities with `nw`-prefixed executable
|
||||||
|
names such as `nwflmcheckdb` and `nwxflmcheckdb` when tools are enabled.
|
||||||
|
- Use the ABI-facing version values from the public headers when they disagree
|
||||||
|
with `configure.ac`: `libnwflaimtk.so.1.2`, `libnwflaim.so.4.62`,
|
||||||
|
`libnwflaimsql.so.6.00`, and `libnwxflaim.so.5.12`.
|
||||||
|
- Install all FLAIM public headers under one namespace directory,
|
||||||
|
`include/nwflaim/`, including `xflaim.h`. Do not install a separate
|
||||||
|
`include/nwxflaim/` tree.
|
||||||
|
- Keep CMake messages explicit about curses/ncurses detection so it is obvious
|
||||||
|
whether curses-backed FLAIM tools will be built.
|
||||||
|
- Continue to prefer build glue and include-path fixes over invasive FLAIM source
|
||||||
|
edits. Small modern-compiler fixes are acceptable when necessary to compile,
|
||||||
|
but keep them as small, reviewable patches.
|
||||||
|
|
||||||
|
Current local dependency-test policy:
|
||||||
|
|
||||||
|
- GDBM and ncurses may be built locally from the uploaded release tarballs for
|
||||||
|
verification and passed to CMake through an isolated prefix.
|
||||||
|
- PAM may use headers from the uploaded Linux-PAM tarball for compilation checks,
|
||||||
|
but mars-nwe should link to the system PAM library rather than vendoring PAM.
|
||||||
|
- These local builds are test dependencies, not new third-party submodules.
|
||||||
|
|
||||||
### libowfat dependency rule
|
### libowfat dependency rule
|
||||||
|
|
||||||
`libowfat` should be a hard bundled dependency, initially for the
|
`libowfat` should be a hard bundled dependency, initially for the
|
||||||
|
|||||||
Reference in New Issue
Block a user