diff --git a/AI.md b/AI.md index 3728fb1..939a810 100644 --- a/AI.md +++ b/AI.md @@ -45,75 +45,45 @@ unfinished work out of `TODO.md` merely because its architecture is documented. ## Current accepted patch line -Latest patch marker expected in an up-to-date bundle: +Latest commit in an up-to-date tree: -- `0554 nwnss: import COMN connection structural headers` +- `nwnss: audit COMN main and SBS imports` -When a later chat receives a new `mars-nwe-master` bundle, compare `git log -1` -with this marker. If the uploaded bundle already contains this commit subject, -the last patch was applied. If the bundle is older and the old chat download no -longer works, recreate the missing patch instead of assuming it was applied. - -The active line is expected to include: - -- Quota work through the corrected Linuxquota/NWQUOTA split and DOS/live smokes. -- NSS low-level imports through the Unicode/codepage, UTC, GUID/ID, xCtype and - xString helper line. -- `0434 build: record offline dependency bootstrap layout`. -- `0435 docs: record transport config and nwfs watch plan`. -- `0436 docs: keep unfinished NSS and salvage work in TODO`. +Compare `git log -1 --format="%s"` with this subject. If the tree is older, +check which COMN audit commits are missing and replay them before starting new +work. ## Current patch handoff block -Last generated patch: +Last completed work block: **COMN subsystem audit chain** -- `0554 nwnss: import COMN connection structural headers` +The full COMN import/audit chain is done through Beast/SBS. Audited subsystems +in order: root headers → authsys → beast scheduling → admin volume and beast +storage → WIO userspace boundary → CSA/file dispatch → IO and lookup → rename +task and volume → xaction and helpers → file handle and fsmsg → management and +name → residual COMN common → compression → namespace → main and SBS. -Purpose of that patch: +All COMN headers and source files are now imported into `include/nwnss/` and +`src/nwnss/comn/` with provenance-preserving paths. Userspace boundary guards +(`MARS_NWE_NWNSS_USERSPACE`) are in place for kernel/scheduler/block-IO paths. +The Beast/AdminVolume/SBS connection structure dependencies that `nameSpace.c` +needs are available. -- Reject the previous shortcut direction for namespace registration: do not - disable the AdminVolume/Beast path just to make `nameSpace.c` build. -- Import the next real COMN connection structure headers used by the - AdminVolume/Beast path from `shared/sdk/comnSA` and `shared/sdk/include`. -- Keep this as a header-only bottom-up step before importing - `public_core/comn/common/beastClass.c`, `pssConnection.c`, `adminVolume.c` or - `public_core/comn/namespace/nameSpace.c`. -- Add only a narrow Linux-version macro compatibility fix around the imported - `fileHandle.h` connection-key conditional; no NSS semantics are replaced. +Next step: begin actual namespace implementation in `libnwnss` — wire up +`public_core/comn/namespace/nameSpace.c` against the imported COMN layer. -## 0490 core: import NSS stdio formatter runtime - -Imported the real NSS bounded formatter before parse/FSM/latch code that uses -`LB_snprintf` and friends: - -- `public_core/library/stdio/snprintf.c` -> `src/core/snprintf.c` -- `public_core/library/stdio/snprintf.h` -> `include/core/snprintf.h` - -The Linux userspace port keeps the NSS formatter body and only adapts platform -includes, Unicode declarations, and the formatter-local stack allocation path. -Do not replace this with libc `snprintf`; later stdio/wio imports should build -on this real NSS formatter. - - - -Directory/NDS work order before any FLAIM storage conversion: +## Directory/NDS work order - Do not combine LDAPv2/LDAPv3 compatibility work with the FLAIM storage-engine - swap. First keep TinyLDAP/nwdirectory on its existing flatfile/mmap/journal - backend and add behaviour tests. -- Prefer existing TinyLDAP test/client code for LDAP CTests. LDAP tests should - start `nwdirectory` on `127.0.0.1:` with a temporary flatfile - workdir and test LDAPv2/LDAPv3 bind/search/unbind against test LDIF/schema - data. + swap. Keep TinyLDAP/nwdirectory on its existing flatfile/mmap/journal + backend and add behaviour tests first. +- LDAP CTests should start `nwdirectory` on `127.0.0.1:` with a + temporary flatfile workdir and test LDAPv2/LDAPv3 bind/search/unbind. - Add `libnwds` and console-only `nwsetup` together after LDAPv2/schema flatfile - behavior is testable. Do not make the first setup path depend on the TUI or - on acting as an LDAP client. -- Later, when the flatfile LDAP/setup path is stable, add FLAIM storage as - parallel backend source files selected by CMake. For example, - `flaimstorage_add_bin.c` may provide the same exported `mstorage_add_bin()` - function as the flatfile file when the FLAIM backend is selected. Do not break - upstreamable flatfile TinyLDAP code while adding the mars-nwe FLAIM backend. - + behaviour is testable. Do not make the first setup path depend on the TUI. +- Later, add FLAIM storage as CMake-selected parallel backend source files that + export the same function names as the flatfile files. Do not break + upstreamable flatfile TinyLDAP code. Rejected or superseded patches that must not be reused as-is: @@ -123,12 +93,14 @@ Rejected or superseded patches that must not be reused as-is: ## Current immediate direction -Before starting namespace implementation, keep the documentation boundaries -clean: +The COMN import/audit chain is complete. Documentation boundaries are already +clean. The next concrete step is namespace implementation: -- `TODO.md`: active work only, dashboard retained. -- `REDESIGN.md`: architecture and design separation clarified. -- `AI.md`: concise current state and rules only. +- Wire up `public_core/comn/namespace/nameSpace.c` against the imported COMN + layer in `libnwnss`. +- Keep `src/namspace.c` as NCP glue; do not expand old `namedos`/`nameos2`. +- Resolve missing `nameSpace.c` dependencies bottom-up from the NSS sources + before adding wrappers. Before namespace code changes, keep the legacy logging audit in mind: old MARS accepts numeric debug thresholds from `0` to `99`, but the actual source mostly @@ -353,48 +325,13 @@ work. - PAM and ncurses headers staged while linking to system libraries - Do not vendor random system headers directly into endpoint/provider code. -## New reference inputs available for the next namespace work +## Reference archives -The user supplied fresh reference archives for the next phase: +The following archives were used for the NSS import/audit work and are available +for further namespace implementation and NCP structure verification: - `nss-common.tar.bz2` - `nss.tar.bz2` - `ncp__enu.pdf` - `websdk.tar.gz` - `include.tar.gz` - -Use these for source audits and exact NCP structure/selector verification before -implementing namespace changes. - - -0488 core: import NSS mailbox runtime -0489 core: import NSS production debug header base -- Imported shared/sdk/include/mailbox.h as include/core/mailbox.h. -- Imported public_core/library/os/mailbox.c as src/core/mailbox.c. -- Added nwcore.mailbox CTest. - -- 0491 core: imported NSS stdio sprintf/vsprintf from public_core/library/stdio as the next formatter prerequisite before parse/wio. - -## 0492 core: import NSS Linux WIO output base - -Imported the NSS Linux WIO output path as the next parse prerequisite: - -- `shared/sdk/include/wio.h` -> `include/core/wio.h` -- `public_core/library/wio/wio.c` -> `src/core/wio.c` -- `public_core/library/wio/nssUI.c` -> `src/core/nssUI.c` -- `public_core/library/stdio/{printf,vprintf,aprintf,vaprintf}.c` -> `src/core/` -- `public_core/library/wio/{wprintf,wvprintf,waprintf}.c` -> `src/core/` - -The port keeps the NSS Linux console path instead of adding local wrappers. -Kernel/procfs pieces in `nssUI.c` are reduced to a Linux-userspace stdout -console buffer while `LB__wioOutput()` and `NSS_UI_vprintf()` remain the real -NSS call path. `wio.h` uses `wStdout == NULL` for the Linux userspace build, -matching the NSS Linux restriction that only the console output path is active -at this layer. -## 0493 core: route NSS production debug output through WIO - -After WIO landed in 0492, removed the remaining production-debug header -placeholder by exposing the real NSS stdio/inlines declarations to `pssDebug.h` -unconditionally. The heavy scheduler include remains gated until the real -scheduler/FSM layer is imported. The `nwcore.pssdebug` test now exercises -`PRINT()` through the imported WIO/stdio stack. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..658a642 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,160 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Key context before starting + +Read `AI.md` first — it is the handoff/rules document and contains the current patch marker, rejected patches, and immediate direction. Then read `TODO.md` (active backlog) and `REDESIGN.md` (architecture record) before touching namespace, salvage, transport, or NSS code. For focused topic detail, check the relevant `doc/*.md` file (see list in `AI.md`). + +Default language with the user is **German**; repository documentation stays English unless the existing file is already German. + +## Building + +Out-of-tree build (preferred): +```sh +cmake -S . -B build +cmake --build build +sudo cmake --install build +``` + +In-tree build (legacy style): +```sh +cmake . +make +sudo make install +``` + +### Useful CMake options + +| Option | Default | Purpose | +|---|---|---| +| `ENABLE_DEBUG_BUILD` | OFF | Add `-g3 -O0` debug compiler flags | +| `ENABLE_DEBUG` | OFF | Compile in `XDPRINTF`/`DO_DEBUG` logging code | +| `MAINTAINER_BUILD` | OFF | Enable maintainer-only test helpers; also forces new DOS utils | +| `MARS_NWE_BUILD_TESTS` | OFF | Build integration tests under `tests/` | +| `MARS_NWE_BUILD_NWFS_TESTS` | OFF | Build `libnwfs` unit tests | +| `ENABLE_DIRECTORY` | OFF | Build optional `nwdirectory`/TinyLDAP subsystem | +| `ENABLE_BURSTMODE` | ON | Experimental packet burst support | + +### Required dependencies + +`libcrypt`, `gdbm`, `libpam`, `libdl`. Optional: `libacl` (POSIX ACL trustee projection), `quota`, `libattr` (xattr). + +For an **offline/CI bootstrap** from uploaded tarballs, run `prepare-local-deps.sh` from the repo root before CMake. It stages `yyjson`, `libsodium`, a local `gdbm` build under `.local-deps/prefix`, and PAM/ncurses fake headers. + +## Running tests + +```sh +# Build with tests enabled +cmake -S . -B build -DMARS_NWE_BUILD_TESTS=ON -DMARS_NWE_BUILD_NWFS_TESTS=ON +cmake --build build + +# Run all CTests from the build directory +ctest --test-dir build + +# Run a single named test +ctest --test-dir build -R nwcore.ini + +# Run a specific built test binary directly (verbose) +build/tests/nwfs/nwfs_dirquota_test -v +``` + +CTest names use dotted prefixes: `nwcore.*`, `nwtui.*`, `nwfs.*`, `nwflaim.*`, `nwnss.*`. Keep new tests in that shape. + +Live NCPFS/NCP integration smokes (require a running server and mounted NCPFS volume) live under `tests/nwfs/` and `tests/salvage/`. Entry points are the `*.sh` scripts — see `tests/README.md` for details. + +## Architecture overview + +### Server processes + +MARS_NWE runs as a set of cooperating daemons managed through `nwserv`: + +- **`nwserv`** — master supervisor; spawns and monitors child processes. +- **`nwconn`** — NCP request dispatcher; the main protocol handler. Forks per connection. +- **`nwbind`** — bindery service (user/group/object database via GDBM). +- **`nwrouted`** — RIP/SAP IPX router daemon (only when `ENABLE_INTERNAL_RIP_SAP=ON`). +- **`ncpserv`** / **`nwclient`** — auxiliary NCP helpers. +- **`dbmtool`** / **`ftrustee`** — admin utilities. + +### Library layers (`src/` subdirectories) + +| Library | CMake target | Purpose | +|---|---|---| +| `src/nwcore` | `mars_nwe::core` | Shared Mars helpers: INI reader/writer (`nw_ini_*`), logging facade (`nwlog_*`), imported NSS low-level helpers (Unicode, codepage, UTC, GUID, xCtype, xString, WIO, stdio formatter) | +| `src/nwfs` | `mars_nwe::nwfs` | NWFS bridge: namespace glue, metadata, streams, EA, salvage, compression, quota, xattr persistence | +| `src/nwtui` | `mars_nwe::tui` | Terminal UI using bundled `termbox2`; `nwtoolbox` multi-call applet model | +| `src/nwssl` | `mars_nwe::ssl` | TLS/SSL via bundled MatrixSSL; backs `nwwebui` HTTPS daemon | +| `src/nwnss` | `mars_nwe::nwnss` | Imported NSS userspace library — namespace, trustee/effective-rights, XATTR/EA, AdminVol semantics | + +### `include/` layout mirrors `src/`: + +- `include/nwcore/` — headers for `libnwcore` +- `include/nwnss/` — NSS SDK headers, mapped from original `shared/sdk/{include,internal,public,comnSA}/` +- `include/nwfs/`, `include/nwtransport/`, `include/nwtui/`, `include/nwssl/` + +### NCP dispatch path + +``` +IPX/TCP → nwtransport → nwconn (dispatcher) → NCP providers +``` + +`nwconn` handles `0x2222` NCP packets via a top-level `switch` on the function code, then calls into `nwfile.c`, `namspace.c`, `nwbind.c`, `nwsalvage.c`, `nwxattr.c`, etc. Magic `return(-1)` / `return(-2)` values carry handoff state between `nwconn` and `nwbind`; see `doc/HANDOFF_AUDIT.md` and `include/ncp_endpoint.h` for the ongoing annotation work. + +**NCP numbering rule:** always record both decimal and wire/code hex — e.g. decimal `22/35` == wire/code `0x16/0x23`. `ENDPOINTS.md` is the complete audit table. + +### Key design boundaries + +- **`libnwcore`** must not link against `libnwnss`. +- **`libnwnss`** is for imported NSS semantics only — no disk/VFS I/O; `libnwfs` persists to host xattrs/sidecar state. +- **Compression** belongs in `libnwfs`, not `libnwcore`. +- **Logging:** new code uses `nwlog_*` levels `1=error … 5=trace`; never add `zlog` (Apache-2.0, incompatible with GPL-2.0-only policy). `MAINTAINER_BUILD` enables `nwlog_detail()` only — no INI switch may enable it in production builds. +- **TUI:** do not add new direct `curses.h` users; route future interactive tools through `nwtui`/`nwi18n` and the `nwtoolbox` applet model. +- **Transport:** TCP/IP is a future split under `nwtransport`, not a new daemon. NCP providers stay transport-neutral. + +### NSS import layout + +Imported NSS sources follow provenance-preserving paths: + +| Original NSS path | Mars path | +|---|---| +| `public_core/library/…` | `src/nwnss/library/…` | +| `public_core/nss/…` | `src/nwnss/nss/…` | +| `public_core/comn/…` | `src/nwnss/comn/…` | +| `shared/sdk/include/…` | `include/nwnss/include/…` | +| `shared/sdk/internal/…` | `include/nwnss/internal/…` | +| `shared/sdk/public/…` | `include/nwnss/public/…` | +| `shared/sdk/comnSA/…` | `include/nwnss/comnSA/…` | + +Allowed changes in imported NSS files: compile fixes, modern compiler callback casts, include path fixes, and explicit `MARS_NWE_NWNSS_USERSPACE` guards. Do not replace missing NSS symbols with local semantic wrappers — find and import the real bottom-up dependency instead. + +### Salvage and metadata backend + +- `.recycle/` — Samba-compatible deleted-payload store (rename-based; xattrs stay on inode). +- `netware.metadata` xattr — authoritative for deleted-file state; replaces the old `.salvage` JSON sidecars (legacy/migration only). +- Private stream/compression data lives under `.nwfs_streams//`. +- Dot-directories (`.recycle`, `.salvage`) are not exposed through normal NCP path resolution. + +### Third-party submodules (`third_party/`) + +`libowfat`, `iniparser`, `libsodium`, `matrixssl`, `termbox2`, `yyjson`, `unicodeTables`, `flaim` (optional, for `nwdirectory`). Use audited libowfat API names from the bundled source (e.g. `socket_tcp4()`, `io_*` readiness helpers). + +### License + +Project code: `GPL-2.0-only`. Library components: `LGPL-2.1-only`. New/updated files carry the matching SPDX identifier. Do not import Apache-2.0 or GPL-3.0-or-later code. + +## Documentation map + +| File | Purpose | +|---|---| +| `AI.md` | Handoff rules, current patch marker, rejected patches, immediate direction | +| `TODO.md` | Active backlog with implementation dashboard | +| `REDESIGN.md` | Durable architecture and design record | +| `ENDPOINTS.md` | NCP decimal/hex audit table | +| `nwnss-audit.md` | libnwnss import audit checklist | +| `doc/NSS_NAMESPACE_AUDIT.md` | Namespace-specific NSS audit | +| `doc/NSS_PUBLIC_CORE_AUDIT.md` | NSS public-core audit | +| `doc/HANDOFF_AUDIT.md` | nwconn→nwbind magic-return annotation work | +| `doc/NWFS_SALVAGE_COMPRESSION_TOOLS.md` | Salvage/compression/tool planning | +| `doc/TUI_TOOLBOX_PLAN.md` | Terminal UI and nwtoolbox roadmap | +| `doc/NWCORE_INI_PLAN.md` | Shared INI reader/writer plan | +| `doc/LOG_LEVEL_AUDIT.md` | Logging level audit |