Files
mars-nwe/AI.md
OpenAI bd859420b7
Some checks failed
Source release / source-package (push) Failing after 1m15s
docs: add GPL and LGPL only license files
2026-06-12 23:36:48 +02:00

159 KiB
Raw Blame History

AI working notes for mars-nwe

Current handoff status after NSS low-level imports 0404-0423

Current accepted MARS-NWE server line in this work session includes the NSS low-level libnwcore imports through 0423:

  • 0404 imports NSS bitmap.c directly as src/core/bitmap.c.

  • 0405 moves the imported bitmap/NSS base headers directly into include/core/.

  • 0406 removes trailing whitespace from the imported NSS headers.

  • 0407 imports NSS crc.c/crc.h; Unicode folding still uses the local ASCII-compatible fallback until xUnicode/NSSUniToLower is imported.

  • 0408 imports NSS que.h directly as include/core/que.h.

  • 0409 imports NSS bit.h, hash.h, and hash.c into libnwcore. The supplied NSS archives expose LB_CountBits, LB_findHighBit, LB_findLowBit, LB_RotateLeft, and LB_RotateRight in libNSS.imp, but do not include a bit.c; MARS-NWE therefore provides src/core/bit.c as a small compatibility implementation that preserves the original NSS public API names so imported hash.c links cleanly.

  • 0410 imports NSS xUnicode.h, unitolower.c, and utf_tolower.c directly into libnwcore and removes the private crc.c lowercase fallback. The exported NSSUniToLower[] table is now present in libnwcore with the same ASCII-compatible initialization as the old fallback.

  • 0411 extends the direct NSS Unicode helper import with unitoupper.c, unilwr.c, uniupr.c, uniicmp.c, uniicmpmac.c, and uninicmp.c in libnwcore; unicodeInit.c now exports both NSSUniToLower[] and NSSUniToUpper[] with the same ASCII-compatible bootstrap.

  • 0412 imports the NSS Unicode string helper block into libnwcore: componentUnicpy.c, componentUnilen.c, unicat.c, unicmp.c, unicpy.c, unilen.c, and unimcpy.c, with their matching sharedsrc implementation headers kept local to src/core/. This keeps the original NSS LB_* API names available before replacing older MARS Unicode/string helpers. The full NSS Unicode converter/table startup from public_core/library/unicode/unicodeInit.c is still a separate follow-up because it pulls in the NetWare/libc converter runtime.

  • 0413 switches the NSSUniToLower[]/NSSUniToUpper[] definitions from the temporary ASCII-compatible unicodeInit.c bootstrap to generated BMP tables from the external third_party/unicodeTables submodule. That submodule is tracked on master in the project-owned mars-unicode-tables repository and generates TAB/unicodeTables.c from Unicode Character Database 17.0.0, not from Novell NSS shared/sdk/unitables/*.tab files. unicodeInit.c now keeps only the NSS startup wrapper entry points; no MARS-private Unicode table is authoritative.

  • 0414 imports NSS UTF-8 single-character decode helpers into libnwcore: utf8ToUniChar.c, utf8LenToUniChar.c, plus the matching unicodeInit.h, xError.h, and zError.h headers. No MARS callsites are switched yet; this just makes the NSS UTF-8 conversion API available for later replacement work.

  • 0415 imports the NSS whole-string UTF-8 conversion helpers into libnwcore: uni2utf.c and utf2uni.c, derived from the GPL-2 NSS public_core/sharedsrc/uni2utf.c.h and utf2uni.c.h implementation sources. This keeps the original NSS uni2utf() / utf2uni() APIs available without switching MARS callsites yet.

  • 0416 imports NSS Unicode wildcard/raw override parser helpers into libnwcore via unicodeParse.c and exposes LB_GetNssUnicodeVersion() from getNssUnicodeVersion.c. unicodeInit.c now defines NSSUnicodeFF and NSSUnicodeMacFF sentinels so the imported parser helpers link without the full NSS codepage converter runtime. No new Unicode/codepage tables are introduced; any future table data still belongs in the external mars-unicode-tables submodule.

  • 0417 imports NSS getMacCodePageName.c and fills out the remaining lightweight Unicode init symbols (LB_UnicodeStartup(), LB_UnicodeShutdown(), MacintoshCodePageName) needed by imported NSS converter entry points. The Macintosh codepage name stays NULL until the real codepage table/runtime layer is imported, so no new Unicode/codepage tables are introduced in this patch.

  • 0418 imports the NSS byte/Unicode and Mac byte/Unicode conversion entry points (ByteToUnicode.c, LenByteToUnicode.c, MacByteToUnicode.c, LenMacByteToUnicode.c, UnicodeToByte.c, UnicodeToMacByte.c, UnicodeToUntermByte.c, UnicodeToUntermMacByte.c) into libnwcore. The converter state globals are exported but intentionally empty until DOS/Mac codepage tables/runtime are added from mars-unicode-tables.

  • 0419 imports NSS stdlib allocation compatibility (xStdlib.h, zalloc.c, zrealloc.c) into libnwcore. The original NSS public-core allocator files depend on NSS OS memory tracking (nssOSAPIs.h, intmem.h, MKL_*), so the active libnwcore import preserves the NSS API names (LB_zalloc, zalloc, LB_zrealloc, zrealloc) while mapping them to libc userland allocation. The old private bitmap.c zalloc macro fallback is removed.

  • 0420 imports the NSS UTC/DOS/MS time conversion helper library from public_core/library/utc into libnwcore (utc2dos.c, dos2utc.c, utc2sec.c, sec2utc.c, utc2msTime.c, msTime2utc.c, string formatting and parsing helpers, utcdata.c). Supporting headers utc.h, utcData.h, and enable.h are imported directly under include/core/; procdefs.h is provided as a libnwcore userland compatibility wrapper because the original NSS header is a NetWare/kernel lock assertion shim. utcUserland.c backs the NSS UTC globals and GetUTCTime() with libc time(3) while preserving the original NSS API names.

  • 0421 cleans trailing whitespace from the imported NSS UTC files and fills the remaining UTC userland glue symbols (IgnoreTimeZone, ResetTimeCache, BEASTHASH_InvalidateDOSTimesPtr) when zLINUX is not provided by the imported NSS compatibility headers. This fixes downstream links against libnwcore after 0420 without changing UTC conversion semantics.

  • 0422 imports the NSS Unicode converter registration entry points (RegisterUnicodeConverter.c, UnRegisterUnicodeConverter.c) and adds the initial unicodeTableBuild.c runtime builder. At that stage it still used identity single-byte tables plus NSS wildcard/raw overrides until real codepage data was available.

  • 0423 wires generated Unicode.org codepage descriptors from the third_party/unicodeTables submodule into libnwcore. The submodule now generates TAB/codepageTables.c/.h from MAPPINGS/ and excludes WindowsBestFit/ plus historical DatedVersions/ from direct byte-to-Unicode output. unicodeTableBuild.c builds NSS single-byte, double-byte, reverse Unicode-to-byte, wildcard, and mappability tables from the configured default DOS codepage (VENDORS/MICSFT/PC/CP850) and Mac codepage (VENDORS/APPLE/ROMAN). The data is compiled into libnwcore; no runtime .tab/.txt files are loaded.

Keep future NSS low-level imports directly under src/core/<original>.c and include/core/<original>.h. Do not add a new nwcore/nss/ or src/core/nss/ path for the active libnwcore imports.

Current handoff status after quota completion

This file may keep patch chronology because it is the ChatGPT handoff document. TODO.md and REDESIGN.md should stay topic-sorted and should not carry patch stack listings.

Current accepted MARS-NWE server line in this work session is expected to include quota patches through 0384:

  • 0381 keeps NSS-shaped userquota metadata while computing live usage without a private usage xattr.
  • 0382 and 0383 make the all-quota smoke collect uploadable logs and continue past ctest.
  • 0384 avoids misleading NWQUOTA fallback on Linuxquota set failures.
  • 0380 was rejected and must not be used.

Current DOSUTILS quota smoke line is expected to include patches through 0395:

  • DOS writes prove quota deny-before-data on both QUOTA and SYS.
  • DLYSTRT is used for the DOS handoff/relogin flow.
  • 0395 adds test/quota/dqt_linux_handoff.sh, which reads inuse4k and sets limit4k=inuse4k+12 for both volumes before the DOS helper continues.

Green validation seen before this documentation update:

  • MARS-NWE all-quota smoke: ctest, QUOTA dirquota, QUOTA Linuxquota userquota, SYS metadata, SYS NWQUOTA userquota all passed.
  • DOS quota smoke: QUOTA and SYS both wrote 12 4K files and denied the next 4K write.

Next functional line: move on from quota to DOS namespace compatibility unless the user reports a regression.

Patch 0366 status: corrected Linux project-quota directory quota ownership. For Linux quota-capable volumes, Linux project quota is now the authoritative live directory-quota backend; netware.metadata is only a backup/restore mirror. Decimal 22/35 = wire/code 0x23 first reads Linux project quota. If Linux has no active project quota yet but netware.metadata still contains an active nwm_quota_limit/zMOD_DIR_QUOTA mirror from backup/restore, that metadata value is used once to seed Linux project quota; after that Linux is authoritative again. If neither Linux nor metadata has a limit, decimal 22/35 returns entries=0. Decimal 22/36 = wire/code 0x24 sets/clears Linux project quota first and mirrors the result to netware.metadata. For metadata-only or NWQUOTA volumes, netware.metadata remains authoritative.

Patch 0357 status: live NetWare 3.x directory-quota set/get/clear is audited. Mario retested tests/nwfs/nwfs_ncpfs_dirquota_smoke.sh after the 0357 validator fix: decimal 22/36 = wire/code 0x24 set a finite limit, decimal 22/35 = wire/code 0x23 read one entry, decimal 22/36 = wire/code 0x24 with limit 0 cleared it, and the follow-up decimal 22/35 = wire/code 0x23 read returned entries=0. The host dump after clear correctly showed modify_mask=0x0000000000000000 and dirQuotaLimit=9223372036854775807 inactive. This confirms the earlier failure was smoke-side parsing of inactive, not server-side clear semantics. The separate dual userquota live smoke also remained green on QUOTA/Linuxquota and SYS/NWQUOTA.

Patch 0356 status: fixed 3.x directory-quota clear semantics. Decimal 22/36 = wire/code 0x24 with limit 0 now clears zMOD_DIR_QUOTA in netware.metadata instead of leaving an active unlimited dirQuotaLimit, and nwfs_metadata_get_quota_limit() treats an inactive directory-quota bit as zDIR_NO_QUOTA. Local full CMake build completed with a locally built GDBM and local test-only PAM shim; CTest passed (nwfs_xattr_roundtrip_test, nwfs_dirquota_test, nwfs_metadata_xattr_file_test).

Patch 0353 status: added live NCPFS directory-quota smoke for 3.x endpoints. nwfs_ncpfs_dirquota drives decimal 22/36 = wire/code 0x24 and decimal 22/35 = wire/code 0x23 directly through libncp NCPC_SFN, with readback/expect modes. nwfs_ncpfs_dirquota_smoke.sh sets a limit, reads it back over NCP, verifies netware.metadata, clears it, and verifies that decimal 22/35 = wire/code 0x23 reports no entries.

Patch 0351 status: started closing the MARS-NWE 3.x directory-quota block before namespace work. Added libnwfs dirquota.c/h, CTest nwfs_dirquota_test, active NCP decimal 22/35 = wire/code 0x23 get and decimal 22/36 = wire/code 0x24 set backed by netware.metadata.nwm_quota_limit, and fixed decimal 22/40 = wire/code 0x28 Sequence parsing to Lo-Hi. Code comments name both decimal NCP numbers and wire/code hex bytes. Remaining directory-quota work is enforcement/adjustment on file growth/create/delete/rename and fuller decimal 22/40 = wire/code 0x28 scan-reply validation; later 87/39 stays behind the 4.x line.

AI working notes for mars-nwe

This file is for future ChatGPT sessions. It records general working rules and local build/test notes only. It should not be used as the current project status log; the current patch stack and task context should be pasted into a new chat separately.

Start of a new chat

When the user says this is a new chat or asks to continue mars-nwe work, first read this file before proposing patches or making assumptions. Then ask for, or use, the current project status that the user pasted into the chat.

Current handoff update after quota completion and salvage audit

Quota has been moved out of the active TODO path. The functional state to preserve is: Linuxquota volumes are authoritative/enforcing through kernel quota state, NWQUOTA volumes are authoritative/enforcing through NetWare metadata, and NSS-shaped mirrors (netware.metadata, netware.userquota.0, netware.quota) remain for backup/restore/offline tools. The all-quota smoke and DOS board-tool quota smoke both passed after the live retests.

Do not use the rejected private usage-xattr idea (netware.userquota.mars_usage.*). Live userquota enforcement must compute effective usage from the selected backend and current host state.

NSS salvage audit notes for the next filesystem/metadata line:

  • MARS-NWE already has a Samba-friendly .recycle payload repository and .salvage JSON sidecar backend. Keep that layout; do not replace it with an NSS purge tree.
  • The NSS code worth adapting is the metadata model around deleted objects, not the full ZLSS purge-tree/purge-log implementation. Important reference files inspected: shared/sdk/public/zParams.h, shared/sdk/public/zXattr.h, shared/sdk/include/comnBeasts.h, public_core/zlss/purgeTree*.c, public_core/zlss/purgeLog.*, public_core/zlss/zfsVol.c, and public_core/comn/common/beastDelete.c.
  • NSS exposes deleted info through zGET_DELETED_INFO / zMOD_DELETED_INFO: deleted time and deleted-by user ID. NSS stores deleted name type metadata as zNTYPE_DELETED_FILE plus DeletedPersistentParentEntry_s { time, ID } next to the parent/name identity.
  • NSS volume salvage reporting uses purgeable bytes, non-purgeable bytes, deleted file count, oldest deleted time, min/max keep seconds, and low/high watermarks. These are useful future reporting fields even if MARS computes them by scanning .salvage sidecars.
  • Next MARS step should be a shared libnwcore/libnwfs snapshot/builder layer for salvage metadata: collect source path, recycle path, sidecar path, original parent entry ID, original name, deleted time, deleted-by ID/name, attributes, timestamps, trustees, IRM, AFP hints and size once, then use that structure to write JSON sidecars, NSS-shaped netware.metadata, and NCP salvage replies.
  • External backup tools that read NetWare xattrs should see NSS-shaped metadata on salvaged content as well as on live content. Add tests that dump xattrs on salvaged payloads/sidecars and verify the netware.metadata fields.

Current handoff status after docs/quota patches 0342-0358

The latest patch produced in this work session is 0358-quota-document-audited-netware-3x-quota-status.patch. Build the next patch on top of the 0358 bundle unless the user says a later patch was applied.

Recent green runtime baseline:

  • 0344 split quota into backend-neutral quota.c/h, NetWare metadata nwquota.c/h, and Linux lnxquota.c/h.
  • 0345 added the Linuxquota -> netware.userquota restore mirror while keeping Linux quotactl() primary whenever a kernel quota entry is available.
  • The dual NCPFS userquota smoke passed after 0345 and remained green after the 0357 directory-quota retest on both tested volumes:
    • QUOTA/Linuxquota denied the next 4K write before data and reported inuse4k=11 after the 11 allowed 4K files.
    • SYS/NWQUOTA denied the next 4K write before data and reported inuse4k=184 after rebasing from baseline 173 and writing 11 allowed 4K files.
  • 0346 is docs-only and reorganizes the doc/ tree into topic directories.
  • 0347..0349 are docs-only and record the NSS namespace path, NSS feature scoping, and the MARS-NWE 3.x compatibility roadmap.
  • 0351..0357 add and live-smoke the classic NetWare 3.x directory-quota set/get/clear path.
  • 0358 is docs/audit only and records the audited quota status after the live 0357 retest.

Clean quota model to preserve:

  • quota.c/h: backend-neutral helpers only (nwfs_quota_*).
  • nwquota.c/h: NetWare metadata/NWQUOTA storage and accounting only (nwfs_nwquota_*).
  • lnxquota.c/h: Linux kernel quotactl() backend only (nwfs_lnxquota_*).
  • Future BSD quota support must get a separate bsdquota.c/h and nwfs_bsdquota_* names.
  • Linuxquota is authoritative while available. netware.userquota is mirrored by Linuxquota only as backup/restore metadata; if restored metadata is used to seed Linuxquota, the kernel quota backend becomes primary again.
  • Metadata/NWQUOTA-backed volumes store restriction and used 4K blocks in the volume-root netware.userquota xattr and access that xattr as effective uid 0.
  • Growth must be denied before data when the projected 4K usage reaches or exceeds the user restriction; the NCP completion remains 0xff.
  • Namespace create of a new regular file performs a one-block precheck so a create/write sequence cannot bypass user restrictions before the file-handle growth path runs.

Do not reintroduce the removed quota experiments unless there is a new failing test that proves they are needed. Removed/obsolete ideas include FH_CREATED_NEW, FH_QUOTA_PRECHARGED, nw_mark_file_quota_precharged(), fchown/chown-only quota accounting fixes, namespace precharge bookkeeping, creator-xattr quota scanning, quota-only file-info stamping, and temporary nwarchive.c linkage into ftrustee solely for quota scan support.

Next NSS work: namespace first, then the rest of the useful NSS pieces

The user wants the next functional line to start with namespace compatibility, not more quota work. Do not build another wrapper layer around untouched NSS sources. Continue the existing approach used for lsaComn.c, zXattr.h, and quota: directly adapt the useful NSS source into normal mars-nwe/libnwfs files, remove NSS runtime/VFS dependencies, and then delete or shrink the old mars-nwe duplicate logic.

Source status inspected after 0346:

  • Current mars-nwe namespace implementation is spread across src/namspace.c, src/namedos.c, src/nameos2.c, src/connect.c, and selected src/nwconn.c paths.
  • src/namspace.c is not only namespace logic; it also owns NCP path parsing, base handles, search sequences, create/open/delete/rename/trustee dispatch, salvage helpers, and reply formatting. Do not replace it wholesale in one patch.
  • Current DOS 8.3 aliasing in src/namedos.c is simplified and is called directly from src/connect.c, src/namspace.c, and src/nwconn.c.
  • src/nwfs/nameSpaceModel.c and the top-level src/nwfs/*NSpace.c files currently provide only NSS-derived namespace registration metadata, not real lookup/mangling/wildcard behaviour.
  • The complete NSS reference files are already present under src/nwfs/nss/namespace/ and src/nwfs/nss/common/; no further bulk import is needed.

Namespace adaptation order:

  1. DOS namespace first. Adapt from src/nwfs/nss/namespace/dosNSpace.c and dosNSWild.c into the normal src/nwfs/ build area. Preserve the useful Novell names/headers where practical, but expose clearly separated libnwfs entry points for DOS legal-name checks, uppercase/casefold, reserved names, wildcard matching, and unique 8.3 alias generation. The NSS mangleChars table and DOSNS_generateUniqueName() behaviour are the main compatibility target.
  2. Replace mars-nwe DOS alias users with the libnwfs DOS namespace implementation and then remove the duplicated logic from src/namedos.c instead of keeping it as a permanent wrapper.
  3. LONG/OS2 namespace second. Adapt longNSpace.c legal-name, reserved-name, compare, wildcard and unique-name logic, then shrink src/nameos2.c and OS2-specific branches in src/namspace.c.
  4. Only after DOS/LONG are stable, adapt UNIX/NFS, MAC, Extended Attribute and data-stream namespace split points from unixNSpace.c, macNSpace.c, extAttrNSpace.c, dataStreamNSpace.c, and nameSpace.c.
  5. Mine nameLookup.c, nameScan.c, nameCache.c, comnWild.c, and comnUnicode.c only as concrete consumers appear. They are useful for lookup/search/wildcard/casefold semantics, but their NSS Beast/cache model must not be imported wholesale.

Release-target rule after the 0348/0349 discussion: the next real version line should be a MARS-NWE 3.x compatibility target. 0.99plxx is not yet complete NetWare 3.x, so finish the 1.x/2.x/3.x-compatible filesystem/NCP work first. NetWare 4.x work remains planned and may stay documented or behind #if MARS_NWE_4, but it is not the active default target. Do not add NetWare 5.x/OES/MOAB/newer runtime endpoints during the 3.x push.

After the 0357 live retest, the 3.x quota block has enough set/get/clear coverage to move on to DOS namespace work. Keep directory-quota follow-ups narrow: enforcement/adjustment on file growth/create/delete/rename and fuller decimal 22/40 = wire/code 0x28 scan semantics. Then continue with netware.metadata/trustee effective rights, data streams and extended attributes where the NDK/PDF show 3.x relevance, then object IDs/search maps/salvage follow-ups needed by those calls. NetWare-4.x-only pieces such as namespace-aware variants, compression status/control families, and later directory/NDS identity work stay in the 4.x planning bucket and should remain guarded by MARS_NWE_4 if source stubs are useful.

NCP scope note after checking ncp__enu.pdf: directory disk-space restrictions are not only a NetWare 5.x feature. The old file-system-extension calls are in the NetWare 3.x/4.x scope (decimal 22/35 = wire/code 0x23 Get Directory Disk Space Restriction, decimal 22/36 = wire/code 0x24 Set Directory Disk Space Restriction, decimal 22/40 = wire/code 0x28 Scan Directory Disk Space), so directory quotas belong on the MARS-NWE 3.x roadmap. The namespace-aware 87/39 Get Directory Disk Space Restriction is NetWare 4.x/5.x scope and belongs to the later MARS_NWE_4 line. Patches 0351..0357 changed that from planning into an audited 3.x set/get/clear implementation: libnwfs owns portable dirquota.c helpers, decimal 22/35 = wire/code 0x23 and decimal 22/36 = wire/code 0x24 are wired to netware.metadata.nwm_quota_limit, and decimal 22/40 = wire/code 0x28 now reads its documented Lo-Hi sequence value while continuing to use the existing MARS DOS scan reply shape until resource-fork/MAC_RF work can fill the remaining extended disk-space fields.

For NSS pieces that are outside the current NetWare 1.x/2.x/3.x target or that require a real backend, keep the adapted code dormant and covered by compile/link/logic CTests only. For planned 4.x features, source stubs may be placed behind MARS_NWE_4; for 5.x/OES/newer features, keep notes/tests only unless the user explicitly changes the target scope. Do not expose fake NCP data for any feature without a real backend state.

Full NSS/nss-common source audit after 0349

The user supplied the full Novell/OES NSS Linux kernel module source archives nss.tar(2).bz2 and nss-common.tar(2).bz2. These are more authoritative for NSS layout than the reduced mars-nwe copy under src/nwfs/nss/. When changing filesystem metadata, namespace, AFP/Mac, salvage, directory quota, compression, EA or data-stream behavior, inspect the full archives again and keep the Novell/GPL provenance in adapted files.

Important paths seen in the full NSS tree:

  • public_core/comn/namespace/dosNSpace.c, dosNSWild.c, longNSpace.c, macNSpace.c, extAttrNSpace.c, dataStreamNSpace.c, and nameSpace.c are the namespace reference set.
  • public_core/comn/common/dirQuotas.c plus shared/sdk/internal/dirQuotas.h are the directory quota reference set.
  • shared/sdk/public/zParams.h defines zMacInfo_s: finderInfo[32], proDOSInfo[6], filler[2], dirRightsMask.
  • shared/sdk/internal/macNSpace.h defines PackedMacInfo_s with rvdID, rvdLayout, and zMacInfo_s; MAC_METADATA_LAYOUT is 1.
  • shared/sdk/include/comnBeasts.h defines RVD_MAC_META_DATA as the root variable-data ID used for packed Mac metadata.
  • public_core/comn/namespace/macNSpace.c registers that root variable-data type, packs/unpacks PackedMacInfo_s, zeroes the two filler bytes on disk, and treats missing Mac metadata as a zeroed zMacInfo_s with special default finder-info behaviour.
  • public_core/comn/common/comnMacintosh.c uses the data-stream name MAC_RF for the Mac resource fork.
  • public_core/comn/common/comnDataStream.c and public_core/comn/namespace/dataStreamNSpace.c are the data-stream reference points. Do not invent a mars-nwe-only fork format if the NSS stream model can be adapted.
  • public_core/zlss/salvageLog.c and related ZLSS repair/salvage files are the salvage reference points for later metadata preservation work.
  • public_core/comn/compression/ contains the NSS compression implementation; keep it as a later 4.x/planned-library study until the NCP/PDF scope requires active runtime support.

AFP/Mac metadata decision after comparing mars-nwe and full NSS:

  • Current mars-nwe AFP code in src/nwatalk.c stores private xattrs: org.mars-nwe.afp.entry-id, org.mars-nwe.afp.finder-info, org.mars-nwe.afp.prodos-info, and org.mars-nwe.afp.attributes.
  • Because there has been no public server release of this work since the pl27 line, do not add fallback, migration, or mirror code for those private AFP xattrs. Replace them.
  • New AFP/Mac metadata work must follow the NSS model as far as possible: pack FinderInfo/ProDOSInfo/dirRightsMask as the NSS zMacInfo_s/ PackedMacInfo_s root-variable-data layout, not as a new mars-specific side database and not as a separate convenience netware.macmetadata xattr unless later full-source evidence proves NSS stores it separately at the Linux xattr boundary.
  • If that requires improving the existing netware.metadata writer, trustee variable-length handling, or root-variable-data packing, do that rather than keeping a second metadata world.
  • Salvage must preserve and restore the NSS-style Mac metadata and, later, the MAC_RF resource-fork data stream. Do not expose .recycle or .salvage paths through normal AFP/NCP opens.
  • Existing AFP NCP handlers 35/01..35/19 remain useful, but their backend state should be moved from private mars-nwe xattrs to the NSS-style metadata and data-stream provider.

Directory quota scope after the 0349 discussion:

  • Directory quotas are part of the future MARS-NWE 3.x compatibility target, not only NetWare 5.x work. The 3.x calls are decimal 22/35, 22/36, and 22/40; code comments should also show the wire hex selectors 0x23, 0x24, and 0x28.
  • It is acceptable to adapt dirQuotas.c into libnwfs before the NCP endpoints are fully wired, but only with CTests that link the library and check the quota math/data model. Runtime NCP integration should not be claimed done until those 3.x endpoints behave against real filesystem state.

Version targeting:

  • The active release push is MARS-NWE 3.x compatibility. Finish documented NetWare 1.x/2.x/3.x filesystem and NCP behaviour before enabling new default NetWare 4.x runtime endpoints.
  • Keep NetWare 4.x work behind MARS_NWE_4 or in documentation/tests until the 3.x line is complete. 5.x/OES/newer source can be studied and dormant code can be compile/link/logic-tested, but should not become live endpoints without a deliberate target change.

Current handoff status after patch 0222

The current accepted patch line in this chat is expected to include:

  • endpoint-audit/documentation patches through 0176-docs-audit-direct-lifecycle-buffer-endpoints.patch;
  • redesign documentation patches 0177 through 0198;
  • endpoint-audit/documentation patches 0199 through 0219;
  • redesign clarification patch 0220-docs-record-print-queue-redesign-link.patch;
  • endpoint-audit patch 0221-docs-audit-ncp-extension-stubs.patch;
  • endpoint-audit patch 0222-docs-audit-direct-file-metadata-stubs.patch;
  • latest expected patch name: 0222-docs-audit-direct-file-metadata-stubs.patch.

When continuing in a new chat, first ask the user which patch was actually last applied. If they confirm 0222, build the next patch as 0223-... against a tree that already contains 0222. If they only applied through 0221, apply or rebuild 0222 before continuing endpoint work. If any patch failed or was skipped, rebuild against the last confirmed applied patch instead of assuming the file in /mnt/data was accepted.

Known numbering/patch-history notes from this chat:

  • 0190-docs-clarify-imported-nwlog-backend-layout.patch was superseded because patch number 0189 was accidentally skipped and the old 0190 failed after 0188. Do not reuse that old file.
  • Use 0189-docs-clarify-imported-nwlog-backend-layout.patch instead.
  • Then use 0190-docs-clarify-simple-syslog-nwlog-backends.patch, followed by 0191 ... 0203.

The user prefers patch verification snippets to contain only:

git am patchname.patch

Do not include git diff --check HEAD^..HEAD in the final summary unless the user asks for it.

Current redesign decisions to preserve

REDESIGN.md is now the place for broad architecture notes. Do not keep growing TODO.md with long-term redesign material. TODO.md should remain for concrete endpoint/test/fix follow-ups.

High-level NCP architecture direction:

  • Add a small internal NCP dispatch/handoff layer over time; avoid a large message-bus rewrite.
  • Provider boundary is not the same as process boundary.
  • nwbind remains legacy bindery provider/service.
  • Queue is a strong candidate for a future nwqueue provider/process, but first split it logically from bindery.

NCP Extension note from patch 0221:

  • SDK 0x2222/36 / wire 0x24 NCP Extension information and SDK 0x2222/37 / wire 0x25 Execute NCP Extension are source-stub-audited as planned NetWare 4.x extension-registration work.
  • src/nwconn.c now contains disabled MARS_NWE_4 stubs for 36/00 through 36/06 and direct 37. They intentionally return 0xfb if ever enabled without a real extension registry/provider.
  • Future owner is an extension registry/provider. Do not route extension payloads through nwserv as a data-plane broker; nwserv remains only control-plane/supervision/registry.
  • Patch 0222 audits the remaining old direct file-metadata/open-create compatibility slots SDK 0x2222/79, 0x2222/84, and 0x2222/85 / wire 0x4f, 0x54, and 0x55. No active top-level handlers existed; the patch records disabled #if 0 stubs in src/nwconn.c. Future owner is the filesystem/namespace provider, not NDS.
  • The next patch number should be 0223 if 0222 was applied.

Print/queue redesign note from patch 0220:

  • Do not describe printing as entirely absent. Queue-backed printing already exists in the project through the queue/job printing paths.
  • The old direct 0x2222/17 Print/Spool NCP family is a separate compatibility surface and currently remains documented as disabled stubs.
  • Future direct 17/xx implementation should bridge to existing queue printing mechanics rather than creating a separate print subsystem.
  • Logical owner: queue/print-spool provider area, possible future nwqueue; not nwnds, not nwdirectory.
  • Filesystem/volume/namespace should become a provider/module boundary first; a separate process would be risky and later only.
  • Semaphore, server-management, and most small call families should remain modules/providers, not separate processes.
  • nwserv is the control plane/supervisor/provider registry, not a data-plane payload router. Normal requests should flow client -> nwconn -> provider -> nwconn -> client, not through nwserv as broker.
  • Provider processes must always return one formal internal handoff reply. NO_REPLY is an explicit reply kind, not silence. nwconn owns the final client NCP reply envelope and send.

Transport direction:

  • TCP/IP support is a transport split below nwconn/nwserv, not a new daemon.
  • Planned code layout: src/nwtransport.c, src/nwipx.c, src/nwtcp.c.
  • nwtransport is a code/library boundary, not a process.
  • Higher providers must not depend on raw ipxAddr_t long-term.
  • IPX SAP/RIP/watchdog/broadcast behavior remains isolated as IPX-specific.

Secure IPC/TLS direction:

  • Client-facing NetWare 4.x/NCP/NDS compatibility must not require TLS by default. Keep historical clients compatible.
  • LDAP/LDAPS/StartTLS for nwdirectory should use the nwtls facade with the selected GPL-2.0-compatible backend; MatrixSSL is the current preferred candidate.
  • Internal provider IPC over TCP, if added later, must always use backend-backed TLS with mutual authentication through nwtls. MatrixSSL is the current preferred candidate; no plaintext fallback for TCP provider IPC.
  • Local IPC may remain Unix-domain sockets, pipes, socketpairs, or inherited FDs with strict permissions; still avoid logging decoded secrets.
  • Add nwtls as the internal TLS facade if/when TLS is wired into runtime: include/nwtls.h, src/nwtls.c, src/nwtls_matrixssl.c.

Directory/NetWare 4.x direction:

  • libdirectory is the shared internal C API/library used by nwbind, future nwnds, nwdirectory, and nwsetup. These components should not talk LDAP internally just to reach the directory store.
  • libflaim is the planned persistent store under libdirectory. FLAIM is C++; keep its C++ API behind libdirectory so old mars-nwe C code does not include FLAIM C++ headers directly.
  • nwdirectory is the mars-nwe integration name for the tinyldap-derived LDAP/LDAPS service. Standalone/upstream identity remains tinyldap; inside mars-nwe it builds the nwdirectory service.
  • Future nwnds is the NetWare 4.x/NDS compatibility layer and should use libdirectory, not LDAP protocol calls, as its internal backend path.
  • nwbind should eventually become a legacy bindery adapter over libdirectory/libflaim, not maintain a second persistent truth.
  • Do not mention or design Kerberos for the current NetWare 4.x target.

Configuration and setup direction:

  • Move toward a real typed, documented INI format. Do not use JSON as the admin config format.
  • The generated INI is also user documentation. Writers must preserve comments where possible or regenerate from a full documented template; never rewrite it into an undocumented minimal key/value dump.
  • nwsetup is the provisioning/setup tool. It should initialize the libdirectory/libflaim store, create initial schema/tree/admin/server objects, migrate bindery data later, and edit config atomically.
  • No reusable Admin/Supervisor/NDS/LDAP plaintext passwords in the new typed INI. Initial passwords and recovery resets belong to explicit nwsetup commands and only hashes/verifiers go into the store.
  • Legacy bindery config-password reset may remain only as deprecated compatibility behavior; Directory/NDS mode uses nwsetup recovery commands.

Logging direction:

  • Add a small internal nwlog facade instead of direct zlog/log.c calls in handlers/providers. Project layout: include/nwlog.h, src/nwlog.c.
  • Category wrappers should exist for normal code: nwlog_ncp(), nwlog_handoff(), nwlog_bindery(), nwlog_queue(), nwlog_directory(), nwlog_nds(), nwlog_ldap(), nwlog_auth(), nwlog_acl(), nwlog_recovery(), nwlog_security(). They populate an internal nwlog_event and call nwlog_emit().
  • rxi/log.c may be vendored/adapted as nwlog_simple, not exposed directly: include/nwlog_simple.h, src/nwlog_simple.c. It is a simple stderr/stdout/file/callback basis and is a good default for systemd/journald.
  • nwlog_syslog may later be derived/cloned from the simple backend for classic syslog(3) explicitly: src/nwlog_syslog.c.
  • zlog is the preferred optional advanced routing backend behind the facade: src/nwlog_zlog.c. It may live as a third_party/zlog submodule.
  • Never route raw decoded NCP/handoff/auth payloads to remote loggers. Only redacted structured events should leave the host.

Third-party/fork policy:

  • Fixed third-party libraries live under third_party/, such as existing yyjson, planned matrixssl, planned libflaim, and optional zlog.
  • MatrixSSL is the preferred GPL-2.0-compatible crypto/TLS candidate and should be imported as a maintained fork with a native CMake build if selected. Do not design a first-pass OpenSSL/LibreSSL backend matrix.
  • libflaim should live under third_party/libflaim as a mars-nwe-maintained import/fork/mirror. Source may come from SourceForge/SVN and/or a distro source package such as openSUSE libflaim-4.9.1046. Document exact import, revision/version, license files, distro patches, and local patches in third_party/libflaim/README.mars-nwe.md.
  • FLAIM r1112 has Autotools (configure.ac, Makefile.am, libtool, config.h, subprojects ftk, flaim, sql, xflaim). Do not wrap Autotools from CMake; replace it with a real CMake build. First required targets are FLAIM::ftk and FLAIM::flaim; SQL/XFLAIM/tools/tests/docs can come later.
  • FLAIM source license observed by the user: library sources LGPL-2.1; helper files like svn2cl.xsl may have separate licenses such as BSD-3-Clause. Keep these separated in import docs.
  • Forked/integrated mars components that become project services live in the repository root, matching existing style such as mail, admin, and dosutils. mars-tinyldap belongs in the root, not third_party, because it will be heavily adapted into nwdirectory.
  • tinyldap currently has a hand-written Makefile and flat-file/mmap storage. It needs a real CMake build, not a Makefile wrapper. Standalone remains tinyldap; mars-nwe integration builds nwdirectory.
  • For tinyldap/nwdirectory, first CMake split can expose internal targets such as tinyldap::asn1, tinyldap::ldap, tinyldap::ldif, tinyldap::auth, tinyldap::storage, and tinyldap::server. Replace flat-file storage with libdirectory -> libflaim later.
  • Old tinyldap TLS code can remain reference/legacy/standalone-only; mars-nwe nwdirectory TLS should go through nwtls/MatrixSSL once the backend is selected.

Schema/import direction:

  • Do not invent NetWare 4.11 schema by hand if a real source can be obtained. The user expects the complete schema to be hidden in NetWare 4.11 installation material such as install.dat; a real 4.11 install may be needed to extract it.
  • .SCH files such as uploaded NLS.SCH are useful format examples/fragments. They contain readable ASN.1-like ATTRIBUTE and OBJECT-CLASS blocks, but are not the full schema truth.
  • nwsetup should eventually support native NetWare 4.11 schema import, .SCH fragment import, and LDIF import/export. LDIF remains human-readable, diffable, and testable, but the canonical runtime representation is libdirectory schema objects stored in libflaim.
  • tinyldap has useful ASN.1 BER/DER and LDIF code (scan_asn1*, fmt_asn1*, asn1dump, ldif_parse.c), but it does not appear to be an NDS .SCH or NetWare schema importer. Reuse ideas/code carefully through the nwdirectory fork, but plan a dedicated schema import layer.
  • Samba source4/dsdb/schema and setup schema conversion code are useful references for OID/prefixMap/schema-loading ideas, but Samba is GPL-family; do not blindly copy code into mars-nwe. Use as a reference and implement a mars-nwe-native importer/OID module.

Latest endpoint audit note:

  • Patch 0221 audits SDK 0x2222/36 / wire 0x24 NCP Extension information and SDK 0x2222/37 / wire 0x25 Execute NCP Extension as planned NetWare 4.x extension-registration work.
  • src/nwconn.c contains disabled MARS_NWE_4 stubs for 36/00 through 36/06 and direct 37; they are documentation/source markers only and do not change the default runtime.
  • Future implementation needs an extension registry/provider. nwserv may supervise/register providers but must not become the data-plane broker for extension payloads.
  • Patch 0222 audits SDK 0x2222/79, 0x2222/84, and 0x2222/85 / wire 0x4f, 0x54, and 0x55 as old direct file-metadata/open-create and sparse-data compatibility gaps. It adds disabled #if 0 stubs next to the old direct file-I/O switch in src/nwconn.c; future owner is the filesystem/namespace provider.
  • Previous print note still applies: direct 17/xx spool NCPs are only the old direct-spool compatibility surface; queue-backed printing already exists.

The next patch number should be 0223 if 0222 was applied. Likely next blocks are deeper 0x2222/23 bindery/property/admin subfunction coverage, SDK 0x2222/90 scope, or another user-selected endpoint family.

Patch workflow

  • Produce patches that apply with exactly:

    git am patchname.patch
    
  • Assume the user has already applied and committed accepted earlier patches. Build every new patch against the current tree the user provides.

  • Do not ask the user to apply a long patch chain unless they explicitly say earlier patches were not committed.

  • Keep follow-up patches small and reviewable. Do not mix functional changes, cleanup, and logging refactors unless the user asks for that.

  • If a patch is only documentation or test cleanup, keep it that way.

Current protocol audit scope

  • The current endpoint documentation/audit pass is scoped to compatibility NCPs through NetWare 3.x by default, including NetWare 1.x/2.x legacy calls where they are documented. Bucket endpoints by the oldest NetWare generation that documents them: put 1.x/2.x legacy calls in their own sections, keep the remaining through-3.x compatibility calls in the 3.x/default section, and put endpoints introduced in NetWare 4.x in a separate planning/stub section. Do not create stub work merely for NetWare 5.x/OES/MOAB/newer endpoints.
  • NetWare 4.x-only endpoints are not part of the default implementation target yet, but they are the current forward-planning target. Already implemented compatibility code must not be removed or wrapped just because it is 4.x-era; only new, not-yet-implemented 4.x stubs should be placed behind #if MARS_NWE_4. Do not add disabled stubs for 5.x/OES/MOAB/newer calls unless the user explicitly changes the target scope later. MARS_NWE_4 is currently hard-disabled in include/config.h.cmake and should stay 0 unless the user explicitly asks to start that work.
  • When a 0x2222 group or subfunction is forwarded out of nwconn.c, follow the handoff before declaring the endpoint documented. nwconn.c should document the handoff and the exact header/payload bytes that are preserved or rewritten before forwarding; the destination file (for example nwbind.c) must document the concrete subfunction request/reply layout at the real handler. Do not stop at a comment such as nwbind must do prehandling, nwbind must do the rest, or handled by nwbind.
  • For forwarded paths, document any nwconn-side payload mutation as part of the audit. Examples in the current tree include queue create path expansion, queue job file-handle insertion, quota bindery prehandling, and semaphore/message group forwarding. If a forwarded subfunction is not audited yet, record it as a target-file follow-up rather than only documenting the nwconn dispatcher.
  • For documentation-only endpoint patches, do not change parser offsets, byte order, reply layout, or completion behavior. Always compare the code parser/reply layout against the applicable SDK/WebSDK/PDF request format and, when available, the uploaded SDK include prototypes. If the code differs from the SDK layout, document the concrete difference inline and mirror it in TODO.md for later testing. If it matches, say so in the patch summary so the audit trail is clear.
  • When an SDK/WebSDK/PDF endpoint number is written in decimal notation, convert it carefully to the wire case value before adding inline documentation. Example: Directory Services 0x2222/22/12 in the PDF means SubFunctionCode decimal 12, i.e. wire case 0x0c; it is not the existing case 0x12 / decimal 18 Allocate Permanent Directory Handle. Place disabled stubs directly at the correct numeric slot inside the dispatcher, never appended at the end of the function. For implemented endpoints, keep the detailed documentation inside the relevant case block, immediately after the case label/opening brace, matching the local style; do not leave a large endpoint block before the case label.
  • If a PDF/WebSDK page title and an internal table row disagree, prefer the endpoint title plus include/WebSDK cross-checks and record the mismatch instead of inventing a new wire case. Example: 0x2222/23 Verify Serialization is titled SDK decimal 23/12 / wire 0x0c, even though one PDF table row prints SubFunctionCode (212); do not add a wire 0xd4 case without a packet trace or include-level confirmation.
  • In TODO.md and endpoint summaries, avoid ambiguous mixed notation for grouped subfunctions. Write SDK/PDF numbers as decimal and include the wire byte explicitly when it differs or could be confused, for example SDK 22/18 / wire 0x12 or SDK 22/12 / wire 0x0c. Do not write 22/12 for a wire case 0x12 unless the SDK number is actually decimal 12.
  • Do not assume every 0x2222 endpoint key is only request_type/function/subfunction. Some SDK/PDF/WebSDK families have deeper selectors inside the subfunction payload, such as NDS 0x2222/104/02 with a 32-bit NDS Verb, statistical 0x2222/123/34 with InfoLevelNumber, NCP extension 0x2222/36/37 with dynamic extension numbers, or reply layouts selected by an information type. When auditing such a family, document the selector path explicitly, for example 0x2222/104/02 verb=<n> or 0x2222/123/34 level=<n>, and distinguish true wire dispatch bytes from payload fields that merely select a structure or backend operation.
  • Keep TODO.md endpoint audit notes grouped by endpoint family and NetWare generation instead of as one long flat list.
  • Before starting the next detailed endpoint block, maintain a coverage index for SDK/WebSDK-listed 0x2222 groups that are not yet audited. Classify each group as present in code but not audited, missing a top-level handler, or likely later-generation/unclear. This index is only a planning aid: do not add active TODO work or source stubs until the specific block has been checked for handoffs and bucketed by oldest documented NetWare generation.
  • Before every new endpoint-family patch, first do a missing-endpoint pass for that family: enumerate the SDK/PDF/WebSDK/include endpoint list, compare it against actual case labels and forwarded destination handlers, then document implemented, disabled-stub, and absent slots separately. Do this retroactively for already documented families when touching them again.
  • Always document both the request handoff/parser and the reply builder. For forwarded calls, the nwconn.c comment should explain exactly why return(-1) or return(-2) is used; the destination handler should explain the concrete request bytes and response payload. Do not treat return(-1) inside disabled #if 0 snippets in nwbind.c as a forwarding mechanism.
  • For SDK-listed groups that appear missing from nwconn.c, also search destination files such as nwbind.c, queue helpers, salvage helpers, AFP/name-space dispatchers, and any prehandler path before declaring the endpoint absent.
  • The rejected 0152-docs-note-message-control-subfunction.patch must not be applied: it documented 0x2222/21/0x0c Connection Message Control, which is outside the default NetWare 1.x/2.x/3.x MARS-NWE target scope.

mars-nwe coding style rules

  • Prefer existing mars_nwe / NetWare functions over new helper code.
  • Before adding a helper, search the tree for an existing equivalent.
  • Do not introduce parallel mechanisms for paths, trustees, xattrs, AFP metadata, copy/write/restore, u16/u32 packing, or logging.
  • Use existing integer and wire-format macros such as GET_16, GET_32, U16_TO_16, U32_TO_32, and related mars_nwe helpers instead of open-coded byte parsing/serialization.
  • Use existing namespace/path conversion and basehandle logic instead of parsing NetWare paths by hand.
  • For file restore/copy/write behavior, prefer the existing Novell/mars_nwe file functions over direct POSIX operations. Use POSIX only where there is no suitable internal mechanism, and keep it clearly isolated.
  • Do not add a new trustee or xattr database. Salvage JSON is a snapshot; real restore should feed existing mars_nwe trustee/xattr/AFP mechanisms.

NCP path and hidden repository notes

  • Normal NCP path resolution intentionally treats Unix dot path components as hidden/special. In the classic path resolver (build_dir_name() in connect.c), a component beginning with . is accepted only for ./.. semantics; a component such as .recycle or .salvage returns invalid path (0x899c).
  • nwattrib.c also marks Unix dot files/directories hidden by default when no explicit NetWare attributes are stored.
  • Therefore .recycle and .salvage are backend repositories, not user-visible NCP paths. Tests must not expect SYS:.recycle/... or SYS:.salvage/... to open through ordinary NCP file calls.
  • Use the official salvage endpoints (87/16 scan, 87/17 recover, 87/18 purge, and old 22/27-22/29) to observe or operate on salvage entries. Verify recovered payload content by reading the restored live file through NCP, not by opening backend repository paths through NCP.

Salvage endpoint rules

  • NCP 0x2222 / 87 / 16 is decimal 87/16, implemented as function 0x57, subfunction 0x10.
  • NCP 0x2222 / 87 / 17 is decimal 87/17, function 0x57, subfunction 0x11.
  • NCP 0x2222 / 87 / 18 is decimal 87/18, function 0x57, subfunction 0x12.
  • Legacy salvage endpoints are old function 22 decimal / 0x16: 22/27 scan, 22/28 recover, and 22/29 purge. They should remain thin adapters over the same shared salvage backend, not a second implementation.
  • Keep 0x57 subfunction dispatch in handle_func_0x57() / namespace code, not as a second subfunction switch in nwconn.c.
  • Old 0x16 calls need a minimal bridge in namespace code because short directory handles must be resolved through existing build_base() / dir_base[] internals before reaching the shared backend.
  • Versioned backend payload names follow Samba vfs_recycle literally: Copy #1 of NAME, Copy #2 of NAME, ... . Do not localize this string and do not run it through gettext; the NCP scan reply still reports the original deleted filename for every version.
  • Versioned salvage entries may have different .recycle/.salvage names but 87/16 returns the original deleted filename for every version. Do not match recover/purge by display name alone.
  • Scan must treat .salvage JSON as a sidecar for the matching .recycle payload. If an external tool such as Samba or an administrator removes the payload, 87/16 must not return the stale sidecar and should remove the JSON. The server log should contain a greppable line like WARN SALVAGE 87/16 STALE ... for this cleanup.
  • Scan, recover, and purge should share the same scan/sequence/basehandle view so that a sequence returned by scan identifies the exact sidecar used later.
  • The combined salvage smoke suite now covers NCP write/read payloads, 87/18 purge pre-clean, hidden backend repository behavior, stale sidecar cleanup with a manual payload-removal pause, three version captures, and recovering the oldest version via sequence 0.
  • Append salvage endpoint tests to tests/salvage/salvage_smoke_suite.sh rather than creating unrelated top-level scripts, unless a helper binary is needed and then started by the suite.

AFP 0x13 deleted-file info notes

  • AFP 0x13 Get Macintosh Info On Deleted File is NCP 0x2222 / 35 / 19 (wire subfunction byte 0x13). The Micro Focus / Novell WebSDK request is VolumeNumber plus DOSDirectoryNumber; the reply is FinderInfo[32], ProDOSInfo[6], ResourceForkSize, FileNameLen, FileName.
  • Implement it only as an adapter over the shared mars_nwe salvage/deleted-entry record. Do not expose or normally open .recycle or .salvage through AFP code; those remain hidden backend repositories.
  • The implementation returns FinderInfo[32], ProDOSInfo[6], resource fork size, and deleted original name from the Salvage JSON snapshot. FinderInfo and ProDOSInfo are captured through the existing nwatalk xattr-backed AFP metadata store, not through a parallel AFP metadata database.
  • The AFP smoke suite has a dedicated afp_deleted_info_smoke helper. It pre-cleans salvage entries in the tested directory through NCP purge, creates a temporary AFP file, writes FinderInfo and ProDOSInfo, deletes it, verifies AFP 0x13, and purges the tested deleted entry afterwards.
  • Verified AFP smoke status: the full suite completed with failures=0 after AFP 35/19 and ProDOSInfo work. It verifies live FinderInfo and ProDOSInfo xattrs on SYS:PUBLIC/pmdflts.ini, verifies AFP 35/19 returns prodos=010203040506 from the deleted-file Salvage snapshot, and leaves normal AFP-only attributes absent when Hidden/System/Archive map through the NetWare attribute path.
  • Reuse existing AFP/nwatalk metadata mechanisms for FinderInfo, AFP attributes, entry ids, resource fork state, and related restore/lookup behavior. Do not add a parallel AFP metadata database.

Logging rules

Desired future server log format:

<LVL4> <AREA> <DEC-CODE> <EVENT> key=value ...
  • LVL4 is exactly four characters: INFO, DBUG, WARN, ERRR.

  • AREA examples: NCP, SALVAGE, AFP, MAP, BIND, TRUST, AUTH, CONN, FILE, QUEUE.

  • The front code should be human/protocol decimal where applicable, for example 87/16, 87/17, 87/18.

  • Exact wire values should still be logged later as key/value hex fields, for example fn=0x57 sub=0x10 seq=0x00000000 base=0x00000004 result=0x89ff.

  • Unknown or unimplemented endpoints should be easy to grep, for example:

    INFO NCP 87/18 UNKNOWN fn=0x57 sub=0x12 msg="not implemented"
    INFO NCP 87/255 UNKNOWN fn=0x57 sub=0xff msg="unknown subfunction"
    INFO NCP 136 UNKNOWN fn=0x88 msg="unknown function"
    
  • Do not invent a parallel logger casually. Reuse existing mars_nwe logging functions/macros and normalize message format gradually.

2026-06-06 - handoff for next chat: redesign plus FLAIM/directory foundation

This is the latest working handoff from the long FLAIM/MatrixSSL/nwssl session. If a new chat starts, ask the user for the current root bundle and the current bundles or pushed refs for all submodules before constructing a build tree. Do not assume the bundles named in the old chat are still current.

There are now two active work streams. Keep them separate in patches and in commit messages:

  1. mars-nwe redesign / future-proofing of existing code. This is the REDESIGN.md track. The next intended first task is small: add a typed enum/type layer for internal operations/request kinds/schema-facing IDs so old magic strings and magic numbers can be translated at one boundary before moving larger logic. Read REDESIGN.md, TODO.md, and this file before proposing the first enum patch. Do not start with a large rewrite.
  2. vendored storage/crypto/directory infrastructure. This is the track that introduced libnwowfat, libnwsodium, libnwmatrixssl, libnwssl, libnwflaim*, and libnwdirectory/tinyldap integration. The purpose is to later replace old local helper code and, more importantly, to move tinyldap/nwdirectory away from flat files toward a FLAIM-backed store.

Current green test state reached in the previous chat:

ctest -L flaim --output-on-failure
# nwflaim.database.create-and-check ......... Passed
# mars_nwe.flaim.api-create-query-encrypt ... Passed
# mars_nwe.xflaim.api-alloc ................. Passed

ctest -L nwflaim --output-on-failure
# nwflaim.database.create-and-check ... Passed

Meaning of the green tests:

  • FLAIM tools can create and check a database through CTest.
  • mars-nwe root tests can create a classic FLAIM database, add a dictionary EncDef through the dictionary API, write/read/query records, close/reopen the database, and verify the test secret is not visible as plaintext on disk.
  • libnwssl now has a functional NICI/CCS compatibility layer sufficient for classic FLAIM at-rest encryption tests.
  • XFLAIM stays build-covered but does not use the classic FLAIM NICI compat path; NICI support is scoped to classic FLAIM targets only.
  • FlaimSQL is experimental and must be default OFF; do not let default mars-nwe builds link hard against libnwflaimsql.

Important FLAIM/nwssl fixes that were part of the green state:

  • libnwssl owns include/nwssl/private/nici/* and the NICI/CCS compatibility implementation used by FLAIM.
  • CCS_Init() / CCS_Shutdown() exist for FLAIM startup/cleanup.
  • NICI handle types must match FLAIM FLMUINT width on 64-bit builds.
  • MatrixSSL must export the same AES-related compiler options/defines to consumers that it used for libnwmatrixssl, otherwise psCryptoOpen() fails with a crypto config mismatch.
  • FLAIM dictionary code had EOF-as-not-found / EOF-as-end-of-base64 decode cases that needed to be treated as success in the relevant paths.
  • The encrypted root test must create EncDef through the dictionary API; do not put encdef as a child of a field record.

Known remaining FLAIM follow-up:

  • Manual nwflmgigatest -b with default 100000 records still hit a Gleitkomma-Ausnahme / SIGFPE even though the CTest DB smoke passes. The divisions in gigaUpdateLoadTimes() were already guarded. The likely bug is shutdown ordering in flaim/util/gigatest.cpp: the code stops the screen thread, then later calls gigaUpdateLoadTimes() and stops the screen thread again. The next small mars-flaim patch should move final stats before the first gigaStopScreenThread() and stop the screen thread only once. Verify with:

    cd <build>/third_party/flaim
    ./nwflmgigatest -b
    echo $?
    ctest -L nwflaim --output-on-failure
    ctest -L flaim --output-on-failure
    

At-rest encryption/key policy notes:

  • FLAIM encryption uses a database wrapping key plus EncDef keys. The EncDef key is stored in FLAIM metadata in wrapped/encrypted form; records use the EncDef key for encrypted field/blob storage.
  • The current nwssl NICI/CCS layer is a functional compatibility layer for tests and initial directory storage work. Before storing real production directory secrets, define a key policy: where the server/tree master key lives, how it is created, how backups/restores work, and how rotation will later be handled.
  • Candidate future key locations are root-only files such as /etc/mars_nwe/nwssl.key or /var/lib/mars_nwe/keys/... with mode 0600. Do not hard-code a production master key into the database or source.

TinyLDAP / directory follow-up test ideas before replacing flat-file storage:

  • Create/open/reopen a directory database.
  • Add user object, group object, and user-to-group membership.
  • Lookup by DN/name and by indexed common attributes.
  • Authenticate user/password; password material must not appear plaintext in the FLAIM database files.
  • Duplicate object/name conflict, delete, rename/move, restart/reopen recovery.
  • Once the user extracts real NetWare 4.11 schema data, add schema tests for object classes, mandatory/optional attributes, attribute syntax, single/multi-value rules, naming attributes, inheritance, and default indexes.

NetWare 4.11 LDAP compatibility baseline:

  • Treat stock NetWare 4.11 LDAP as the primary compatibility target for the directory service. The stock LDAP.NLM generation is LDAPv2-only, not LDAPv3.

  • LDAPv2 simple bind, search, and unbind must be first-class, intentional, and covered by tests. LDAPv3 support may remain or be added, but it is additive and must not become the default assumption for the NetWare 4.11 baseline.

  • Baseline references for the stock target are RFC 1777 (LDAPv2 core protocol) and RFC 1778 (LDAPv2 string representation of standard attribute syntaxes).

  • Do not require LDAPv3-only features for the stock 4.11 target: no referrals, no SASL, no controls, no extended operations, and no mandatory LDAPv3 UTF-8 DN behavior.

  • NetWare 4.11 with NDS 8 / eDirectory and newer LDAP.NLM versions is a later optional compatibility profile. That profile may cover LDAPv3-era RFCs such as 2251, 2252, 2253, 2255, and SASL RFC 2222, but it should not drive the initial directory design.

  • The user has started importing real NetWare schema files under opt/schema/ after lower-casing filenames and converting CRLF line endings. Treat these as input for the directory/schema enum/type-layer work.

  • Historical LDAPv2 server reference material inspected in this chat: openldap-1.0.3.tgz is useful because it is still close to the UMich LDAP lineage and contains LDAPv2-era slapd, liblber, RFC 1777/1778 text, and simple backend code. Use it as protocol-behavior reference only; do not import its large server architecture into tinyldap.

  • Samba 2.2.12 and Samba 3.0.37 were inspected for a small embedded LDAPv2 server. They do not contain one. Their LDAP code is primarily client/backend code (pdb_ldap, smbldap, idmap_ldap, ADS/CLDAP client pieces). Do not use Samba 2/3 as the LDAPv2 server model.

  • For LDAPv2 protocol behavior, prefer: RFC 1777/RFC 1778, TinyLDAP's current small C implementation, and UMich/OpenLDAP 1.x as historical C reference. Add tests first, then adjust tinyldap behavior in small patches.

NetWare/NSS xattr and trustee metadata baseline:

  • The corrected compatibility target for NetWare file metadata is the Novell/OES NSS netware.* Linux xattr interface. Use OES/NSS as the primary reference for names, binary layouts, trustee rights constants and inherited-rights behavior.

  • The relevant GPL-2.0 source references are zXattr.h, zParams.h, lsaXattr.c, lsaSuper.c, zasAuthModel.c, zasAuthSpace.c, and sharedsrc/manage.c.h. mars-nwe is GPL-2.0-only, so exact structures/code may be adapted directly when the original copyright/license notices are preserved.

  • Active NSS xattr names include netware.ncpstat, netware.quota, netware.volumeinfo, netware.metadata, and netware.userquota. The first mars-nwe xattr ABI target is still ncpstat, metadata, quota, and userquota; keep volumeinfo as a later volume/tooling target.

  • netware.trustee has helper functions in lsaXattr.c, but its registration entry is disabled/commented in the NSS source; trustees should be represented through netware.metadata first.

  • netware.metadata contains the high-value fields for backup/migration and compatibility: file attributes, timestamps, owner/archiver/modifier/metadata modifier GUIDs, directory quota, inherited rights mask, and trustee array.

  • NSS trustee rights are positive NetWare/NSS rights: R/W/C/E/A/F/M/S plus NSS salvage/secure bits. The baseline is not the Linux trustees-3.0 allow/deny/clear model. The current mars-nwe netware.metadata trustee rights encoding uses the NSS/NCP bit assignments, so the rights masks are layout-compatible with Novell/OES readers.

  • Trustee identity compatibility mode: for the NetWare 3.x/4.x target, keep the classic 32-bit bindery/NDS object ID as the authoritative trustee identity and embed it deterministically in the NSS GUID_t timeLow field with the rest of the GUID zeroed. This preserves the classic NCP/SYSCON/FILER/GRANT/REVOKE object-ID model while using the NSS netware.metadata binary layout. Do not treat these values as real OES/eDirectory authorizer GUIDs.

  • Open directory-identity follow-up: real OES/NSS trustee entries carry GUID-style NSS/eDirectory authorizer identifiers. A real OES/NSS server might not resolve mars-nwe's embedded bindery object ID to the same object. Keep real eDirectory/NDS/AuthID GUID mapping as a later Directory task rather than changing the NetWare 3.x/4.x trustee rights patch.

  • Inheritance model to preserve: only entries marked inherit-down propagate; child inherited rights are filtered by inheritedRightsMask and zVALID_TRUSTEE_RIGHTS; supervisor is preserved in the mask and expands to all valid trustee rights when effective.

  • netware.metadata should be hidden from normal listxattr() by default, like NSS does, unless an explicit admin/backup mode is enabled.

  • NSS maps the same internal metadata into normal Linux attributes too, not only into xattrs. Mode/chmod, owner/chown, timestamps and logical size must stay connected to the same state that serializes as netware.ncpstat and netware.metadata.

  • Current source-tree status: selected NSS/OES GPL-2.0 sources have already been imported into the mars-nwe tree under include/nwfs/nss/, src/nwfs/nss/, and src/core/nss/. trustees-3.0 has been imported under src/nwfs/trustees3/, and nwfs1201/FENRIS has been imported under src/nwfs/nwfs1201/. These are source material for direct adaptation, not build targets yet.

  • First implementation target after the docs: create libnwfs.so from adapted NSS/OES source files, not a wrapper around untouched NSS code. Start by moving/copying the relevant imported files out of src/nwfs/nss/ into the normal src/nwfs/ and include/nwfs/ build area, preserving original Novell headers and function/structure names where they remain useful.

  • The initial libnwfs.so cut should adapt zXattr.h, lsaXattr.c, lsaComn.c, and lsaPrivate.h into mars-nwe-ready sources, removing NSS kernel/VFS/runtime dependencies that mars-nwe does not need while keeping the NSS netware.* names, metadata layout, trustee array, inherited-rights-mask, byteorder/version checks, and modify-mask semantics.

  • Add tests that link directly against libnwfs.so for netware.ncpstat, netware.metadata with trustees, inherited-rights filtering, supervisor expansion, quota/userquota validation, and metadata list visibility. The library tests come before wiring the code into live NCP operations.

  • Move existing trustee storage toward netware.metadata so trustee arrays and inherited-rights-mask are not duplicated in a disconnected .trustees-only path. The existing trustee.c API can stay while its backend migrates to libnwfs.

  • Additional NSS references to keep for later, not the first xattr ABI patch: lsaComn.c for inherited-rights/metadata/quota assembly, public_core/comn/namespace/* for DOS/LONG/UNIX/MAC/EA/data-stream namespace split points, public_core/comn/compression/* for compressed-file policy, and eDir/GUID/ID helper sources for owner/trustee/modifier mapping.

  • Follow-on NSS/OES NCP reference note: the same source drop also contains NCP-adjacent structure definitions in nss/shared/support/lnxmbINC/encp.h, the NSS-to-NCP IPC envelope in nss/shared/sdk/public/ipc2ncp.h, and identity mapping prototypes in nss/shared/sdk/include/ncpIDAPI.h. These are not a complete ncpserv implementation, but they are useful GPL-2.0-compatible structural references for mars-nwe's existing file, namespace, trustee, effective-rights, salvage, and quota NCP handlers. Use them after the xattr layout work so NCP obtain/modify/trustee/effective-rights paths read and write the same metadata that backs netware.ncpstat and netware.metadata.

  • Quota model decision: libnwfs/netware.* is the compatibility metadata model, but quota enforcement is selectable per volume. The planned config key is NWFS_QUOTA_BACKEND with values LINUXQUOTA, METADATAONLY, and NSS. Default must be LINUXQUOTA so normal ext4/xfs/btrfs-style Linux volumes keep the existing host quota enforcement path.

  • LINUXQUOTA: store/serve NSS-compatible netware.quota, netware.userquota, and metadata.nwm_quota_limit, while using Linux quotactl() as the user quota enforcement backend where available. Directory quota still needs mars-nwe-side checks because Linux user quotas do not represent NSS directory quotas 1:1.

  • METADATAONLY: store and roundtrip NSS-compatible quota metadata, but perform no hard host filesystem enforcement. Use this for tests, migration/import, debug volumes, and setups where an external layer handles enforcement.

  • NSS: for real NSS/OES-style volumes, treat NSS itself as the enforcement backend. mars-nwe should consume/export the same netware.* metadata view and avoid trying to mirror NSS enforcement through Linux quotactl().

  • Existing nwvolume.c quota APIs and NCP quota endpoints should stay as stable callers initially. Their backend should migrate from direct Linux quotactl() as the data model toward libnwfs quota metadata plus the selected enforcement backend.

  • Re-check result: the inspected OES/NSS source tree does not provide a reusable non-NSS quota enforcement backend for ordinary Linux filesystems. Its quota enforcement is tied to the NSS internal File_s/Volume/DirectoryQuota/UserSpace model. Keep the existing mars-nwe Linux quotactl() path for the LINUXQUOTA backend, and implement directory-quota checks in mars-nwe/libnwfs for non-NSS volumes.

  • Source-tree status after the expanded imports: the additional NSS authsys, common, main command, compression support, and SDK/header support files are now present under src/nwfs/nss/ and include/nwfs/nss/ as raw GPL-2.0 source material. They are intentionally not build targets yet.

  • Additional NSS areas now available for later direct adaptation include the directory quota engine (src/nwfs/nss/common/dirQuotas.c), file/create/IO paths that call quota checks (comnFile.c, comnIO.c, comnRename.c), name lookup/scan/wildcard helpers, data-stream and extended-attribute beasts, authsys/effective-rights sources, and src/nwfs/nss/main/comnCmdline.c quota/namespace commands. Adapt them only after a concrete libnwfs, tool, or NCP handler consumer exists; do not use them as a replacement for the host Linux quota backend.

  • No more NSS bulk-import work is planned before coding starts. The next step is still 0270: build the first small libnwfs.so from the adapted metadata sources and add unit tests.

nwsetup direction:

  • ncurses is already needed for FLAIM tools and should also support a future nwsetup tool. The user wants a NetWare-like phase-2 setup flow for provisioning the Directory tree. Later nwsetup should initialize the FLAIM-backed directory store, create tree/server/org/user/admin objects, set the admin password, create default SYS/volume/config state, and avoid writing reusable plaintext secrets to config files.

2026-06-05 - current superbuild / storage / TLS handoff

This chat switched from documentation-only endpoint audit work back to build and integration work around the mars-nwe superbuild. The old patch-number guidance below remains useful historical context for the endpoint-audit series, but the current accepted work in this chat is a separate functional/build patch stack. Do not assume the next patch after this point is an endpoint-audit 0269 patch unless the user explicitly returns to that series.

Current root/superbuild direction implemented or in progress:

  • update-submodules.sh is now the normal helper for keeping mars-nwe's submodules reproducible. Private mars-owned submodules are updated to their configured latest branch, while external upstream snapshots are pinned by a single editable EXTERNAL_TAG_PINS block near the top of the script.
  • Current external pins are third_party/yyjson=0.12.0, third_party/zlog=1.2.18, and third_party/libsodium/libsodium=1.0.20-FINAL.
  • The helper must not run a root-level git submodule update --init --recursive after updating top-level private submodules, because that can reset updated gitlinks such as third_party/matrixssl back to the parent commit. Nested submodules are initialized inside the owning top-level submodule instead.
  • The helper prints recursive submodule status on success and failure and can auto-commit root gitlink/script changes. Root gitlinks must only reference commits that exist in the respective submodule remotes; a parent gitlink to an unpushed submodule commit will make later git submodule update fail with upload-pack: not our ref.
  • yyjson is being folded into the mars-nwe core library path. Consumers should include it through the nwcore include namespace and link mars_nwe::core, not link an independent public yyjson target directly.
  • FLAIM is currently only needed when the directory service is enabled. The root CMake should therefore add/build third_party/flaim only when ENABLE_DIRECTORY=ON.
  • Namespace rule: keep historical mars-nwe binaries with their established names (nwserv, ncpserv, nwclient, dbmtool, ftrustee, etc.), but give vendored libraries, vendored headers, CMake packages, and imported helper tools an nw namespace when they could collide with system packages or upstream tool names.
  • Current library namespace examples: libnwowfat, libnwsodium, libnwmatrixssl, libnwflaimtk, libnwflaim, libnwflaimsql, libnwxflaim, libnwcore, libnwssl, and libnwdirectory.
  • Current header namespace examples: include/nwlibowfat/, include/nwsodium/, include/nwmatrixssl/, include/nwflaim/, include/nwssl/, include/nwcore/, and include/nwdirectory/.
  • TinyLDAP/directory tools and FLAIM/XFLAIM tools should install with nw prefixes (nwt2, nwparse, nwldapclient, nwx, nwflmcheckdb, nwxflmdbshell, etc.) because their upstream names are generic or collision-prone.

Current libnwssl / MatrixSSL / OpenSSL-compat direction:

  • MatrixSSL owns only the renamed low-level backend library (libnwmatrixssl). Temporary OpenSSL-compat test headers/sources that were placed in the MatrixSSL fork must move out of MatrixSSL.
  • libnwssl owns the mars-nwe crypto/TLS facade plus the narrow compatibility surfaces needed by FLAIM: OpenSSL-style FTK/network headers and NICI/CCS-style private headers.
  • OpenSSL-compat headers should live under the nwssl include subtree, e.g. include/nwssl/openssl/*.h, not as a root-level include/openssl directory that could conflict with system OpenSSL headers.
  • Private FLAIM/NICI compatibility headers should also live under the nwssl subtree, e.g. include/nwssl/private/nici/.... They are private build compatibility headers, not the public TLS API for normal mars-nwe code.
  • smart, directory, and FLAIM should link against libnwssl / the mars_nwe::ssl target when they need TLS/crypto compatibility. They should not include MatrixSSL headers directly and should not link OpenSSL directly.

Current FLAIM import/build direction:

  • The imported FLAIM tree is kept under third_party/flaim and should remain as close to upstream source as practical. Prefer CMake/build glue, include paths, and tiny compile fixes over broad C/C++ rewrites.
  • The mars-nwe build currently wants libnwflaimtk, libnwflaim, libnwflaimsql, and libnwxflaim. XFLAIM is now included even though it is not immediately required, so it stays build-covered.
  • FLAIM library versions should come from the public headers when those disagree with configure.ac, because the headers are the ABI-facing version source in this import. Current expected shared-object versions are: libnwflaimtk.so.1.2, libnwflaim.so.4.62, libnwflaimsql.so.6.00, and libnwxflaim.so.5.12.
  • All installed FLAIM public headers should go below one mars-nwe namespace directory: include/nwflaim/. Do not make xflaim.h a special include/nwxflaim/ exception.
  • FLAIM command-line utilities should be built and installed with nw-prefixed binary names, for example nwflmcheckdb, nwflmrebuild, nwflmview, nwflmdbshell, nwflmgigatest, and the matching nwxflm... utilities.
  • FLAIM tools require curses/ncurses. CMake should report clearly whether curses/ncurses was found and whether the curses-backed tools will be built.
  • The uploaded ncurses-stable.tar.gz is a valid current upstream ncurses/stable source snapshot for local build testing even if the top-level extracted name does not look like a conventional ncurses release tarball.
  • The CMake conversion should keep translating the old Makefile.am source inventories. Recent build fixes added missing result-set sources and fixed XFLAIM tool include ordering so xflaim/util code includes the XFLAIM flaimsys.h rather than the classic FLAIM one.
  • Minimal FLAIM C++ compile fixes are acceptable when required by modern compilers, for example replacing pointer assignments/comparisons using character \0 with NULL/null-pointer checks. Keep those patches small and separate from build-system changes where possible.

Current local dependency policy for build checks:

  • Build GDBM locally from the uploaded gdbm-1.26.tar.gz when testing in an isolated prefix.
  • Build ncurses locally from the uploaded ncurses-stable.tar.gz when testing FLAIM tools in an isolated prefix.
  • Use the uploaded Linux-PAM-1.7.2.tar.xz for PAM headers if needed, but link against the system PAM library. Do not vendor PAM as a mars-nwe library.
  • These local dependency builds are for verification; they are not new vendored submodules unless a later explicit import decision says otherwise.

When continuing this work, expect more incremental compile/build patches rather than one large redesign patch. Build after each FLAIM/CMake change, record the next concrete compiler/linker error, and keep generated patches per repository or submodule so the user can apply them with git am at the correct path.

Build and test notes

Dependencies used during local checks in this conversation:

  • gdbm-1.26.tar.gz
  • Linux-PAM-1.7.2.tar.xz for PAM headers; link against system PAM if present
  • ncpfs-master.zip for the salvage smoke helper client build
  • yyjson under third_party/yyjson

If CMake finds GDBM but a target still cannot see gdbm.h, pass include paths explicitly for local verification, for example:

CFLAGS="-I/path/to/gdbm/include -I/path/to/Linux-PAM-1.7.2/libpam/include" \
cmake -S . -B build
cmake --build build --target nwconn ncp_salvage_scan_smoke ncp_salvage_recover_smoke

Useful quick checks:

bash -n tests/salvage/salvage_smoke_suite.sh
cc -DLINUX -fsyntax-only -Iinclude -Isrc -Ithird_party/yyjson/src src/nwsalvage.c src/namspace.c

When server-side code or smoke helper clients change, rebuild both the server and the helper targets so the runtime test is not using stale binaries:

cmake --build build --target nwserv ncpserv
cmake --build build --target \
  ncp_delete_smoke \
  ncp_read_smoke \
  ncp_salvage_scan_smoke \
  ncp_salvage_recover_smoke \
  ncp_salvage_purge_smoke \
  afp_entry_id_smoke \
  afp_file_info_smoke \
  afp_scan_info_smoke \
  afp_set_file_info_smoke \
  afp_deleted_info_smoke

Runtime smoke suites:

tests/salvage/salvage_smoke_suite.sh --out /tmp/mars-salvage-report.txt
tests/afp/afp_smoke_suite.sh --out /tmp/mars-afp-smoke.txt

The suite streams the report to --out while running, so a failure before the end should still leave useful output. It has a manual stale-payload pause: the script prints a sudo rm -f .../.recycle/... command; remove that payload in a second shell and press Enter. The next scan should remove the stale sidecar and grep /var/log/mars_nwe/nw.log for WARN SALVAGE 87/16 STALE.

Normal NCP reads of .recycle or .salvage are expected to fail with invalid path. Verify payload data through the visible live file after NCP write or recover, using ncp_read_smoke. Treat the final summary (failures=0, ncp_warnings=0) as the important signal.

AFP ProDOSInfo storage

ProDOSInfo is AFP/NCP per-entry metadata. Store it in the existing nwatalk AFP metadata layer, not in nwarchive/nwxattr directly and not in a parallel DB. The xattr key is user.org.mars-nwe.afp.prodos-info via the mars_nwe xattr wrapper name org.mars-nwe.afp.prodos-info; it is a raw 6-byte value, analogous to FinderInfo's 32-byte org.mars-nwe.afp.finder-info.

Salvage captures this as prodos_info_hex (12 hex characters) beside finder_info_hex. AFP 35/19 Get Macintosh Info On Deleted File returns FinderInfo[32] followed by ProDOSInfo[6] from the Salvage snapshot. The verified smoke value is 010203040506 and the Linux xattr dump should show:

user.org.mars-nwe.afp.prodos-info=0x010203040506

Latest endpoint audit checkpoint

As of patch 0212-docs-audit-namespace-lock-salvage-stubs.patch, the latest audited endpoint block is the Name Space lock/quota/search/salvage-rights subset of NCP 0x2222/87 / wire 0x57 in src/namspace.c. nwconn.c still forwards requestdata starting at the Name Space SubFunction byte to handle_func_0x57(), and the handler return convention remains unchanged: non-negative values are reply payload lengths, negative values are Completion codes.

The previous 87/16..87/29 block contains active source cases for:

  • 87/16 Scan Salvageable Files;
  • 87/17 Recover Salvageable File;
  • 87/18 Purge Salvageable File;
  • 87/20 Search for File or Subdirectory Set;
  • 87/21 Get Path String from Short Directory Handle;
  • 87/22 Generate Directory Base and Volume Number;
  • 87/24 Get Name Spaces Loaded List from Volume Number;
  • 87/26 Get Huge NS Information;
  • 87/28 Get Full Path String;
  • 87/29 Get Effective Directory Rights.

Disabled source stubs exist for eligible 3.x/4.x metadata gaps from that range:

  • 87/19 Get NS Information;
  • 87/23 Query NS Information Format;
  • 87/25 Set NS Information;
  • 87/27 Get Name Space Directory Entry.

Patch 0212 added the next set of disabled source stubs for eligible 1.x/2.x/3.x and planned-4.x namespace/file gaps that were missing from the active switch range:

  • 87/36 Log File;
  • 87/37 Release File;
  • 87/38 Clear File;
  • 87/39 Get Directory Disk Space Restriction;
  • 87/40 Search for File or Subdirectory Set (Extended Errors);
  • 87/41 Scan Salvageable File List;
  • 87/42 Purge Salvageable File List;
  • 87/43 Revoke File Handle Rights.

These stubs are under #if 0, document selector path/request/reply/provider intent, and do not change runtime behavior. 87/44 Update File Handle Rights is NetWare 5.x in the NDK material and was not stubbed under the current scope. The existing 87/26 source slot is still effectively unimplemented and returns the default 0xfb completion.

The next endpoint block can continue with 87/64..87/69, the matching 89 long-name-space family, or another unaudited top-level family such as AFP 0x2222/35, packet burst 0x2222/97/101, or deeper 0x2222/23 bindery/property/admin subfunction coverage, unless the user requests a specific family first.

The next patch number should follow the latest applied patch; after patch 0223, use 0224.

Retro source-stub checkpoint from patch 0207:

  • Already documented eligible gaps in Directory Services 22/12, 22/35, and 22/36 have disabled source stubs at the correct src/nwconn.c dispatch slots. Do not rewrite those stubs unless implementing the endpoint.
  • Already documented File Server Environment 23 queue/server-management gaps have disabled source stubs at the appropriate src/nwbind.c switch slots. Some pre-existing disabled stubs still contain legacy placeholder control flow; leave existing stubs alone unless implementing or explicitly cleaning that exact block.
  • Message 21/04..21/08 were not SDK/PDF server endpoints in the default audit set, so no stubs are required. Message 21/12 is later-generation only and remains prose-only/out-of-scope under the current rules.
  • Physical-record 26..31 plus 110, TTS 34/00..34/10, and direct file 59, 61..77 do not have additional eligible missing slots in the audited ranges; no new source stubs were needed.
  • Future retro-audits must distinguish three cases: add a disabled source stub for an eligible missing endpoint, leave an already-present stub unchanged and only document that it exists, or keep non-endpoints / 5.x+ endpoints out of source.

Remember: for every new endpoint-audit patch, also update this AI handoff file with the latest audited block and expected next patch number. Put detailed Coverage/Request/Reply/Known-difference notes inline at each endpoint case rather than as one large audit block before the switch range.

Missing-endpoint rule: when an audited SDK/PDF/WebSDK/Header endpoint is not implemented but belongs to the compatibility scope, document it at the appropriate dispatch location as a disabled #if 0 stub instead of only mentioning it in prose. The compatibility scope for stubs is NetWare 1.x/2.x legacy calls, NetWare 3.x/default compatibility calls, and explicitly planned NetWare 4.x/NDS work. Do not add stubs merely for NetWare 5.x/OES/MOAB/newer endpoints: those are outside the current target unless the user explicitly asks for that later generation. A 3.x-compatible server should remain compatible with documented 1.x/2.x calls, and the current forward plan is only through 4.x. Disabled stubs should include selector path, name, request/reply sketch, provider/out-of-scope reason, and no active behavior change. Disabled stubs must not use misleading control flow such as return(-1) where that return value has no local handoff meaning.

Latest endpoint audit checkpoint from patch 0223:

  • Direct NCP 0x2222/111 / wire 0x6f Semaphore is now source-stub-audited in src/nwconn.c. There is no active top-level handler for this newer NetWare 3.x/4.x semaphore family.
  • Patch 0223 records disabled #if 0 selector slots for 111/00 Open/Create a Semaphore, 111/01 Examine Semaphore, 111/02 Wait On (P) Semaphore, 111/03 Signal (V) Semaphore, and 111/04 Close Semaphore.
  • The old 32/xx semaphore implementation in src/sema.c remains the active compatibility path. Future work should bridge both families through one semaphore provider/state table and verify the documented Lo-Hi handle order against existing MARS-NWE big-endian handle helpers before changing behavior.
  • This block is local synchronization, not nwnds/directory work.

Next patch number should be 0224.

Latest endpoint audit checkpoint from patch 0224:

  • SDK 0x2222/90 / wire 0x5a Data Migration / parse-tree / compression metadata is now source-stub-audited as planned NetWare-4.x filesystem and namespace work. There was no active top-level handler in src/nwconn.c.
  • Patch 0224 records the selector map behind MARS_NWE_4: 90/00 Parse Tree, 90/10 Get Reference Count from Dir Entry Number, 90/11 Get Reference Count from Dir Handle, 90/12 Set Compressed File Size, 90/128 Move File Data To DM, 90/129 DM File Information, 90/130 Volume DM Status, 90/131 Migrator Status Info, 90/132 DM Support Module Information, 90/133 Move File Data From DM, 90/134 Get/Set Default Read-Write Support Module ID, 90/135 DM Support Module Capacity Request, 90/136 RTDM Request, and 90/150 File Migration Request.
  • The future owner is the filesystem/namespace provider, not nwnds. There is no active Data Migration support module, parse-tree engine, compressed file-size backend, or RTDM provider yet. Keep this as unsupported 0xfb unless the filesystem provider grows real backing state.

Next patch number should be 0225.

Latest endpoint audit checkpoint from patch 0225:

  • SDK 0x2222/92 / wire 0x5c SecretStore is now scope-audited as later-generation and out of the current source-stub target. The NDK PDF marks SecretStore Services as NetWare Server 5.x and eDirectory 8.5 or later, with subverbs 0 Query Server through 9 Get Service Information.
  • No active top-level case 0x5c exists in src/nwconn.c, and no indirect handler/provider path was found during this audit. Do not add a disabled source stub for SecretStore while the target remains 1.x/2.x/3.x plus planned 4.x only.
  • SecretStore is not the same as the planned 4.x libdirectory/nwnds work. If a future post-4.x/eDirectory target is ever added, it should be designed as a separate secure secret-storage provider with strict no-secret logging rules.

Next patch number should be 0226.

Latest endpoint audit checkpoint from patch 0227:

  • After Accounting, the next relevant block checked was the already-present bindery property/password/set/access portion of SDK 0x2222/23 / wire 0x17, especially SDK 23/57..23/76 in src/nwbind.c.
  • The source already contains per-endpoint comments for the property calls (23/57..23/62) and the set/password/access calls (23/63..23/76), including the disabled 23/63 Verify Bindery Object Password stub and the 23/71 note that trustee-path scanning is handled in nwconn.c.
  • Patch 0227 only updates the coverage index: this is no longer a missing-stub/source-change target. Further work in the 23 family should be targeted behavior/security/provider review rather than another broad endpoint-presence pass.

Latest endpoint audit checkpoint from patch 0228:

  • SDK 0x2222/131 / wire 0x83 RPC / server-control is now source-stub-audited as a NetWare-4.x server-control planning family. No active top-level handler exists in src/nwconn.c.
  • Patch 0228 records disabled MARS_NWE_4 selector slots for 131/01 RPC Load an NLM, 131/02 RPC Unload an NLM, 131/03 RPC Mount Volume, 131/04 RPC Dismount Volume, 131/05 RPC Add Name Space To Volume, 131/06 RPC Set Set Command Value, and 131/07 RPC Execute NCF File.
  • Future ownership belongs to the servermgmt/RPC provider boundary. Do not wire these calls to fake success: the real operations can change server state and the documented reply carries an RPCccode. nwserv may supervise or register providers, but must not become a data-plane broker for RPC payloads.

Latest endpoint audit checkpoint from patch 0231:

  • SDK 0x2222/123 / wire 0x7b server information/statistics is now partially source-stub-audited as planned NetWare-4.x server-management work. There is no active top-level handler in src/nwconn.c.
  • Patch 0231 records the first core selector block behind MARS_NWE_4: 123/01 Get Cache Information, 123/02 Get File Server Information, 123/03 NetWare File Systems Information, 123/04 User Information, 123/05 Packet Burst Information, 123/06 IPX SPX Information, 123/07 Garbage Collection Information, 123/08 CPU Information, 123/09 Volume Switch Information, 123/10 Get NLM Loaded List, 123/11 NLM Information, 123/12 Get Directory Cache Information, 123/13 Get Operating System Version Information, 123/14 Get Active Connection List by Type, 123/15 Get NLM Resource Tag List, 123/16 Enumerate Connection Information from Connection List, and 123/17 Enumerate NCP Service Network Addresses.
  • The common SDK 123 request wrapper is SubFuncStrucLen Hi-Lo, SubFuncCode, VersionNumber, RevisionNumber, plus selector-specific payload. Replies commonly start with CurrentServerTime Lo-Hi, VConsoleVersion, VConsoleRevision, and reserved fields, then selector-specific structures.
  • Future ownership belongs to the servermgmt/information provider boundary. It should adapt existing mars-nwe/host/transport/filesystem state and must not grow a second management database or return fake success for detailed information that is not modeled.

Latest redesign checkpoint from patch 0232:

  • REDESIGN.md now records nwservermgmt as a possible future process for the servermgmt provider once multiple NetWare-4.x management families become real: 123/xx server information/statistics, 114/xx TimeSync adapter, selected 23/200+ console/server-management calls, and guarded 131/xx RPC server-control requests.
  • This does not make nwserv a data-plane router. nwserv remains supervisor, provider registry, and control plane. If nwservermgmt exists later, nwconn sends normalized handoff requests directly to it; nwservermgmt may query nwserv only for control-plane state such as provider status/capabilities.
  • Small static/status calls may stay in-process until a dedicated process is justified. RPC/server-control calls must not fake success and require a real privilege model before activation.

Next patch number should be 0233.

Latest endpoint audit checkpoint from patch 0233:

  • SDK 0x2222/123 / wire 0x7b server information/statistics now has a second source-stub-audited selector block behind MARS_NWE_4 in src/nwconn.c: 123/20 Active LAN Board List, 123/21 LAN Configuration Information, 123/22 LAN Common Counters Information, 123/23 LAN Custom Counters Information, 123/25 LSL Information, and 123/26 LSL Logical Board Statistics.
  • This audit was checked against the local NDK/Core Protocols PDF plus the uploaded WebSDK/include nwfse material. Patch 0234 corrects the adjacent LAN/LSL coverage by adding the PDF-listed 123/24 LAN Name Information and 123/27 MLID Board Information selector slots.
  • Future ownership remains the servermgmt/information provider boundary, adapting real transport/IPX/adapter/LSL state. Do not synthesize fake LAN boards, expose raw Linux-interface details directly, or route this data plane through nwserv; nwserv remains supervisor/provider registry/control plane.
  • Remaining 123/30 and higher Media Manager, volume, protocol-stack, router/SAP, server/set-command, and compression information selectors still need later follow-up auditing.

Latest endpoint audit checkpoint from patch 0234:

  • SDK 0x2222/123 / wire 0x7b LAN/LSL source-stub coverage now also includes the adjacent local-PDF selectors 123/24 LAN Name Information and 123/27 MLID Board Information behind MARS_NWE_4 in src/nwconn.c.
  • 123/27 is documented with a NetWare 4.x versus NetWare 5.x reply-layout difference: the current guarded mars-nwe planning scope should model only the 4.x semantics if this slot is later activated.
  • The earlier note that 123/24 was not listed is superseded by this correction; it was present in the local NDK/Core Protocols PDF even though the uploaded include/WebSDK material surfaced the surrounding nwfse structs more clearly.

Next patch number should be 0235.

Latest endpoint audit checkpoint from patch 0235:

  • SDK 0x2222/123 / wire 0x7b server information/statistics now records the next Media Manager / volume selector run behind MARS_NWE_4 in src/nwconn.c: 123/30 Get Media Manager Object Information, 123/31 Get Media Manager Objects List, 123/32 Get Media Manager Object Children's List, 123/33 Get Volume Segment List, and 123/34 Get Volume Information by Level.
  • This block was checked against the local NDK/Core Protocols PDF plus the uploaded WebSDK/include nwfse material. 123/34 has an internal InfoLevelNumber payload selector for the returned volume-information structure; do not treat those levels as separate wire subfunctions.
  • Future ownership remains the servermgmt/information provider boundary, with filesystem/namespace/volume-provider input for real volume/media state. Do not invent a separate NetWare Media Manager database and do not route these data-plane requests through nwserv.
  • Remaining 123/40 and higher protocol-stack, router/SAP, server/set-command, and compression information selectors still need later follow-up auditing.

Latest endpoint audit checkpoint from patch 0237:

  • SDK 0x2222/123 / wire 0x7b server information/statistics now records the router/SAP/server/source selector run behind MARS_NWE_4 in src/nwconn.c: 123/50 Get General Router and SAP Information, 123/51 Get Network Router Information, 123/52 Get Network Routers Information, 123/53 Get Known Networks Information, 123/54 Get Server Information, 123/55 Get Server Sources Information, and 123/56 Get Known Servers Information.
  • This block was checked against the local NDK/Core Protocols PDF plus the uploaded WebSDK/include nwfse material. PDF/WebSDK/includes jump from 123/56 to the server set-command group at 123/60; do not invent 123/57..123/59 selector slots unless a new primary source is found.
  • Future ownership remains the servermgmt/information provider boundary with IPX/SAP/RIP transport/provider input for real routing and advertised-server state. Do not synthesize fake routers, known networks, SAP server lists, or server-source records, and do not route these read-only data-plane requests through nwserv.
  • Remaining 123/60 and higher server set-command and compression information selectors still need later follow-up auditing.

Latest endpoint audit checkpoint from patch 0238:

  • SDK 0x2222/123 / wire 0x7b server information/statistics now records the NetWare-4.x server SET query selector run behind MARS_NWE_4 in src/nwconn.c: 123/60 Get Server Set Commands Information and 123/61 Get Server Set Categories.
  • This block was checked against the local NDK/Core Protocols PDF plus the uploaded WebSDK/include nwfse material. The PDF also documents 123/62 Get Server Set Commands Information By Name, but marks it as NetWare 5.x; do not add a source stub for 123/62 under the current through-4.x planning scope unless the target scope changes.
  • Future ownership remains the servermgmt/configuration provider boundary. The SET command/category views should expose real mars-nwe configuration and runtime state once modeled; do not synthesize a fake NetWare SET database or route these read-only data-plane requests through nwserv.
  • Patch 0239 records the final in-scope compression/decompression selector run: 123/70 Get Current Compressing File, 123/71 Get Current DeCompressing File Info List, and 123/72 Get Compression and Decompression Time and Counts.

Latest endpoint audit checkpoint from patch 0239:

  • SDK 0x2222/123 / wire 0x7b server information/statistics now records the NetWare-4.x compression/decompression information selector run behind MARS_NWE_4 in src/nwconn.c: 123/70 Get Current Compressing File, 123/71 Get Current DeCompressing File Info List, and 123/72 Get Compression and Decompression Time and Counts.
  • This block was checked against the local NDK/Core Protocols PDF plus the uploaded WebSDK/include nwfse material. The PDF/WebSDK/include material jumps from the NetWare-5.x-only 123/62 SET-by-name selector to the compression group at 123/70; do not invent 123/63..123/69 selector slots unless a new primary source is found.
  • Future ownership remains the servermgmt/information provider boundary with filesystem/volume compression state as the source of truth. Do not synthesize fake current-compression files, decompression file lists, or compression byte/tick counters, and do not route these read-only data-plane requests through nwserv.
  • This completes the currently identified in-scope 123/xx NetWare-4.x server-information/statistics source-stub audit. The next endpoint-audit block should be selected from a different SDK family after rechecking active dispatch and handoff paths.

Latest endpoint audit checkpoint from patch 0240:

  • After completing the 123/xx SDK server-information audit, the next adjacent source block checked was the disabled _MAR_TESTS_XX wire 0x5f UNIX-client probe in src/nwconn.c. It is documented as local/test-only, not an SDK 0x2222/95 endpoint family.
  • The local NDK/Core Protocols NCP-by-number table does not list a documented 0x2222/95 group, and the original source comment records that a NetWare 4.1 server also did not know this call. Do not promote it to a normal default handler or add compatibility stubs without a real client trace and explicit provider ownership.
  • The observed disabled branch shape is FunctionCode 0x5f followed by four unknown bytes, historically seen as 0x10 00 00 00; its old success reply is not exposed outside _MAR_TESTS_XX.

Next patch number should be 0241.

Latest endpoint audit checkpoint from patch 0241:

  • Direction corrected back to NDK-first endpoint selection: choose the next documented NetWare 1.x/2.x/3.x endpoint gap or planned NetWare 4.x endpoint from the local NDK/Core Protocols PDF first, then compare mars-nwe dispatch. Local/test-only source probes such as the disabled wire 0x5f branch are not endpoint-audit candidates unless an in-scope NDK entry exists.
  • SDK 0x2222/22 / wire 0x16 directory-services source-stub coverage now records two NDK-first gaps in src/nwconn.c: 22/49 Open Data Stream and 22/52 Get Mount Volume List.
  • 22/49 is a NetWare 3.x/4.x data-stream open endpoint. It remains disabled until the filesystem/namespace/datastream provider can open real alternate data streams and return real file handles; do not synthesize fake stream handles.
  • 22/52 is a NetWare 4.x mounted-volume-list endpoint. It remains disabled behind MARS_NWE_4 until mounted-volume and namespace-specific volume-list state is modeled by the servermgmt/information provider with volume-provider input. Do not route this read-only data plane through nwserv.

Latest endpoint audit checkpoint from patch 0242:

  • Continuing the corrected NDK-first pass, SDK 0x2222/23 / wire 0x17 File Server Environment now records the NetWare-2.x server statistics block 23/212 through 23/217 as disabled source stubs in src/nwbind.c: Get File System Statistics, Get Transaction Tracking Statistics, Read Disk Cache Statistics, Get Drive Mapping Table, Read Physical Disk Statistics, and Get Disk Channel Statistics.
  • These selectors are reached through the normal nwconn.c File Server Environment forward path into nwbind.c. There was no active implementation for the wire 0xd4..0xd9 subfunction slots before this documentation marker.
  • Do not synthesize fake NetWare FAT/cache, TTS, SFT mirror, physical-disk, or disk-channel counter blocks. Future ownership belongs to a servermgmt/ statistics provider backed by real filesystem, volume, and backend state.
  • The next NDK-first 23/xx statistics/monitoring audit block should continue with the nearby open-file/lock/semaphore/usage selectors such as 23/219 through 23/242, keeping small reviewable sub-blocks.

Latest endpoint audit checkpoint from patch 0243:

  • Continuing the corrected NDK-first pass, SDK 0x2222/23 / wire 0x17 File Server Environment now records the NetWare-2.x/3.x legacy monitor scan block 23/219 through 23/226 as disabled source stubs in src/nwbind.c: Get Connection's Open Files (old), Get Connection Using A File (old), Get Physical Record Locks By Connection And File (old), Get Physical Record Locks By File (old), Get Logical Records By Connection (old), Get Logical Record Information (old), Get Connection's Semaphores (old), and Get Semaphore Information (old).
  • These selectors are reached through the normal nwconn.c File Server Environment forward path into nwbind.c. There was no active implementation for the wire 0xdb..0xe2 subfunction slots before this documentation marker.
  • Do not synthesize fake open-file, file-user, physical-lock, logical-lock, or semaphore lists from partial state. Future ownership belongs to servermgmt/ statistics with real file-handle, byte-range-lock, logical-lock, and semaphore provider input.
  • The next NDK-first 23/xx statistics/monitoring audit block should continue with the nearby LAN-driver, connection-usage, disk-space, LAN-I/O, misc/volume, and newer open-file/lock/semaphore selectors such as 23/227, 23/229..23/242, keeping the patch scope small.

Next patch number should be 0244.

Endpoint audit formatting requirement from patch 0244:

  • Disabled source stubs for audited NDK endpoints must not group multiple selectors under one shared explanatory block. Each switch case must be documented as its own case.
  • Active fall-through handlers do not need control-flow rewrites only for documentation. It is acceptable to keep adjacent case labels sharing one handler, as long as each label has its own adjacent concise Request: and Response: summary before the shared implementation block.
  • Each audited case comment must include a concise Request: summary and a concise Response: summary from NDK/PDF/WebSDK/include sources.
  • Keep provider ownership and "do not fake" constraints in the same case block when relevant, so future implementers do not need to infer requirements from a surrounding grouped comment.
  • Endpoint selection remains NDK-first: NetWare 1.x/2.x/3.x plus planned 4.x; ignore 5.x+ only endpoints unless the project scope is explicitly changed.
  • Final verification snippets should show only the new patch, using git am patchname.patch with no path prefix.

Latest endpoint audit checkpoint from patch 0244:

  • Corrects the previously applied 23/212 through 23/217 and 23/219 through 23/226 source stubs in src/nwbind.c from grouped cases into one documented disabled case per selector.
  • The request/response details are now adjacent to each wire case: 0xd4 through 0xd9 for statistics and 0xdb through 0xe2 for old monitor scan calls.
  • The next NDK-first 23/xx audit block can continue with the nearby LAN-driver, connection-usage, disk-space, LAN-I/O, misc/volume, and newer open-file/lock/ semaphore selectors such as 23/227, 23/229..23/242.

Next patch number should be 0245.

Latest endpoint audit checkpoint from patch 0245:

  • Continued the NDK-first File Server Environment pass with 23/227 and 23/229 through 23/236 at wire 0xe3 and 0xe5 through 0xec. 23/228 is not listed in the NDK table and no wire 0xe4 marker was added.
  • Added/split individual disabled src/nwbind.c cases for LAN driver configuration, connection usage statistics, object remaining disk space, LAN I/O statistics, file-server misc information, volume information, connection task information, connection open files, and connection-using-a-file scans.
  • The old grouped 23/232/23/235 placeholder is gone. Each audited selector now has its own case-local Request: and Response: summary.
  • Do not implement these from approximations. They need real LAN binding, NCP byte/request counter, bindery quota, volume/free-space, routing/LAN I/O, server memory/utilization, volume-table, task/lock-wait, open-file/share/deny, and file-user provider state.

Next patch number should be 0246.

Latest endpoint audit checkpoint from patch 0246:

  • Continued the NDK-first File Server Environment pass with the newer NetWare 3.x/4.x monitor selectors 23/237 through 23/242 at wire 0xed through 0xf2: physical record locks by connection/file, physical record locks by file, logical records by connection, logical record information, connection semaphores, and semaphore information.
  • Each selector is documented as its own disabled #if 0 src/nwbind.c case with a case-local Request: and Response: summary, matching the post-0244 endpoint-audit formatting rule.
  • These endpoints must not be implemented from approximations. They require real byte-range lock state, namespace/data-stream mapping, logical-record lock/log state, and semaphore open/value/wait provider state.
  • The next NDK-first 23/xx pass should skip unlisted gaps and continue with the next documented NetWare 1.x/2.x/3.x selector or planned 4.x selector after 23/242.

Next patch number should be 0247.

Latest endpoint audit checkpoint from patch 0247:

  • Continued the NDK-first File Server Environment pass after 23/242 by skipping the already implemented 23/243 Map Directory Number to Path and 23/244 Convert Path to Dir Entry helpers, then tightening the existing disabled 23/253 and 23/254 tail cases in src/nwbind.c.
  • 23/253 / wire 0xfd Send Console Broadcast now has a case-local Request summary for NumberOfStations, StationList long[] and BroadcastMessage, plus a Response summary covering status-only success/error returns.
  • 23/254 / wire 0xfe Clear Connection Number now has a case-local Request summary for the long ConnectionNumber variant and a Response summary covering status-only success/error returns.
  • These endpoints must not be implemented from approximations. They require real console/supervisor privilege checks, broadcast disabled/delivery state, valid target connection handling, and full logout/resource-release semantics.
  • The next NDK-first pass should continue outside this 23/xx tail with the next documented NetWare 1.x/2.x/3.x endpoint or planned 4.x endpoint, while skipping unlisted gaps and ignoring 5.x-only endpoints.

Next patch number should be 0248.

Latest endpoint audit checkpoint from patch 0248:

  • Continued NDK-first outside the 23/xx tail with the in-scope Transaction Tracking System family 34/00 through 34/10 in src/nwconn.c.
  • Replaced the old grouped TTS documentation block with an explicit switch (ufunc) so every NDK selector has its own case and adjacent Request:/Response: summary: 34/00 availability, 34/01 begin, 34/02 end, 34/03 abort, 34/04 status, 34/05/34/06 application thresholds, 34/07/34/08 workstation thresholds, and 34/09/34/10 transaction control bits.
  • Runtime behavior is preserved: 34/00 reports TTS unavailable and all other TTS selectors return 0xfb. Do not return synthetic success without real transaction files, transaction status tracking, lock integration, threshold state, control-bit state, and rollback/backout logic.
  • The next NDK-first pass should continue with the next documented NetWare 1.x/2.x/3.x endpoint or planned 4.x endpoint after the TTS family, skipping unlisted gaps and ignoring 5.x-only endpoints.

Next patch number should be 0249.

Latest endpoint audit checkpoint from patch 0249:

  • Continued NDK-first after the TTS family with the in-scope AFP/Mac namespace family 35/01 through 35/19 in src/nwconn.c. The NDK lists these AFP calls for NetWare 2.x/3.x/4.x, so they remain relevant even though the same pages also mention 5.x.
  • Converted the AFP dispatcher from grouped if/else pairs into an explicit switch (ufunc) with one case per NDK selector: create directory/file, delete, entry-ID lookup by name/handle/path, rename, open file fork, get/set/scan file information, AFP 2.0 create/get/set/scan variants, DOS-name lookup, and deleted-file Macintosh-info lookup.
  • Runtime behavior is unchanged. Some selectors still share the same helper, but each selector now has its own case-local Request: and Response: summary matching the endpoint-audit rule. Do not regroup these cases in a later cleanup.
  • Future AFP work must keep stable AFP entry IDs/CNIDs, FinderInfo/ProDOSInfo, data/resource fork identity, directory enumeration state, and Salvage metadata grounded in real provider state. Do not fake AFP replies from plain Unix path names when the NDK requires namespace identity or metadata persistence.
  • The next NDK-first pass should continue with the next documented NetWare 1.x/2.x/3.x endpoint or planned 4.x endpoint after the AFP family, skipping unlisted gaps and ignoring 5.x-only endpoints.

2026-06-02 - Patch 0250 NCP Extension selector notes 36/00..06 and 37

  • Continued NDK-first after AFP with the NetWare-4.x planned-scope NCP Extension family: 0x2222/36 selectors 36/00 through 36/06, plus 0x2222/37 Execute NCP Extension.
  • The source already carried an explicit switch for the NCP Extension selectors. Patch 0250 tightens each selector-local comment so every case has concrete NDK Request/Response fields rather than a generic extension summary:
    • 36/00 Get NCP Extension Information old
    • 36/01 Get NCP Extension Maximum Data Size
    • 36/02 Get NCP Extension Information by Name
    • 36/03 Get Number of Registered NCP Extensions
    • 36/04 Get NCP Extension Registered Verbs List
    • 36/05 Return NCP Extension Information
    • 36/06 Return NCP Extension Maximum Data Size
    • 37 Execute NCP Extension
  • Runtime behavior is unchanged: all extension-registration and execution requests still return 0xfb until a real extension registry/provider exists.
  • Future NCP Extension work must model a real registered-extension table, version/name/custom-data records, maximum data-size policy, registered verb enumeration, and provider-owned execute payload dispatch. Do not route extension payloads through nwserv as a generic data-plane broker.
  • Continue NDK-first with the next documented NetWare 1.x/2.x/3.x endpoint or planned 4.x endpoint after the NCP Extension family, skipping 5.x-only NDS unless explicitly brought into scope.

2026-06-02 - Patch 0251 old direct create-file selector split 67/77

  • Continued NDK-first after the NCP Extension checkpoint by returning to the in-scope old direct file-I/O calls in src/nwconn.c. The NDK lists both 0x2222/67 Create File and 0x2222/77 Create New File for NetWare 2.x/3.x/4.x, so they remain relevant even though the same pages also mention 5.x.
  • Split the old grouped case 0x43 / case 0x4d implementation into two explicit case bodies. Each case now has its own adjacent Request: and Response: notes:
    • 67 / wire 0x43 Create File: replace-if-existing semantics when the caller has sufficient create/delete rights.
    • 77 / wire 0x4d Create New File: no-replace semantics; fail if the target already exists.
  • Runtime behavior is intentionally unchanged. Both cases still use the existing nw_creat_open_file() path and return the old six-byte file-handle plus NW_FILE_INFO reply layout, but they no longer rely on a grouped case label or a function == 0x43 mode selector inside a shared block.
  • Do not regroup these two direct create-file cases in a later cleanup; keep the per-case Request/Response audit rule intact.

2026-06-02 - Patch 0252 SDK 90 tree/reference/compression selector notes

  • Continued NDK-first after the direct create-file split. The NDK Enhanced NCP 89/xx chapter is not taken as the next implementation target here because the 89/01 page is marked NetWare Servers: 6.5, SP2 and later; keep those 5.x+/6.x-only enhanced namespace endpoints out of this 1.x-4.x audit unless project scope changes.
  • Audited the first NetWare-4.x-relevant SDK 90/xx block in src/nwconn.c:
    • 90/00 Parse Tree
    • 90/10 Get Reference Count from Dir Entry Number
    • 90/11 Get Reference Count from Dir Handle
    • 90/12 Set Compressed File Size
  • Each selector already had its own switch case; patch 0252 tightens those case-local comments with exact NDK SubFuncStrucLen, request fields and response fields.
  • Runtime behavior remains unchanged: all four selectors still return 0xfb until a real namespace tree/reference-count/compression metadata provider exists.
  • Do not synthesize reference counts or compressed-size replies from partial Unix stat(2) state. These replies require NetWare directory-base identity, namespace handles, reference tracking and compression metadata.

Next patch number should be 0253.

2026-06-02 - Patch 0253 SDK 90 data-migration selector notes

  • Continued NDK-first after the first SDK 90/xx tree/reference/compression selector pass with the planned NetWare-4.x Data Migration selector block in src/nwconn.c:
    • 90/128 Move File Data To DM
    • 90/129 DM File Information
    • 90/130 Volume DM Status
    • 90/131 Migrator Status Info
    • 90/132 DM Support Module Information
    • 90/133 Move File Data From DM
    • 90/134 Get/Set Default Read-Write Support Module ID
    • 90/135 DM Support Module Capacity Request
    • 90/136 RTDM Request
    • 90/150 File Migration Request
  • Each selector already had its own switch case. Patch 0253 tightens the case-local comments with exact NDK SubFuncStrucLen, request fields, and response fields so the Data Migration block follows the current endpoint audit rule.
  • Runtime behavior remains unchanged: all Data Migration selectors still return 0xfb until a real filesystem/data-migration provider exists.
  • Do not fake Data Migration replies from ordinary Unix file metadata. These endpoints require NetWare volume/directory-entry identity, namespace-aware migrated-file state, support-module registration, capacity accounting, migrator status, RTDM verbs, and file-migration attribute persistence.

2026-06-02 - Patch 0254 TimeSync selector split 114/01..12

  • Continued NDK-first after SDK 90 Data Migration with the planned NetWare-4.x Time Synchronization family in src/nwconn.c:
    • 114/01 Timesync Get Time
    • 114/02 Timesync Exchange Time
    • 114/05 Timesync Get Server List
    • 114/06 Timesync Set Server List
    • 114/12 Timesync Get Version
  • The source already had selector coverage behind MARS_NWE_4, but the cases were grouped with fall-through to one shared unsupported return. Patch 0254 splits them so each selector has its own case, own Request: summary, own Response: summary, and own 0xfb return.
  • Runtime behavior remains unchanged: TimeSync remains unsupported until a real server-management/time provider exists.
  • Do not emulate the NDK note that 114/06 returns success in all cases until there is an explicit TimeSync compatibility policy and real server-list state.

2026-06-02 - Patch 0255 RPC selector split 131/01..07

  • Continued NDK-first after Time Synchronization by revisiting the planned NetWare-4.x RPC/server-control family in src/nwconn.c:
    • 131/01 RPC Load an NLM
    • 131/02 RPC Unload an NLM
    • 131/03 RPC Mount Volume
    • 131/04 RPC Dismount Volume
    • 131/05 RPC Add Name Space To Volume
    • 131/06 RPC Set Set Command Value
    • 131/07 RPC Execute NCF File
  • The source already had selector coverage behind MARS_NWE_4, but the cases were grouped through fall-through to one shared unsupported return. Patch 0255 splits them so each selector has its own case, own Request: summary, own Reply: summary, and own 0xfb return.
  • Runtime behavior remains unchanged. RPC/server-control remains unsupported until a real server-management/RPC provider exists.
  • Do not fake RPC success. These calls can load/unload NLMs, mount/dismount volumes, add name spaces, change SET commands, and execute NCF files; they require supervisor-equivalent authentication, real provider state, and documented RPCccode mapping.
  • Keep nwserv as control-plane supervisor/registry only; do not route these RPC payloads through nwserv as a generic data-plane broker.

2026-06-02 - Patch 0256 Physical-record selector notes without control-flow split

  • Continued NDK-first after RPC/server-control with the implemented physical record lock family in src/nwconn.c:
    • 26 Log Physical Record (old)
    • 27 Lock Physical Record Set (old)
    • 28 Release Physical Record
    • 29 Release Physical Record Set
    • 30 Clear Physical Record
    • 31 Clear Physical Record Set
    • 110 Lock Physical Record Set
  • Patch 0256 is documentation-only and intentionally keeps the existing fall-through handler structure. Adjacent cases may remain adjacent when they share parser/handler logic; the requirement is that each wire case has its own nearby Request: and Response: notes, not that active control flow be split for documentation.
  • Keep the known parser audit notes unchanged: old 27 is documented as Lo-Hi timeout while the shared handler reads Hi-Lo, and the current LockFlag mapping differs from the NDK/Core-Protocols text until real requester traces justify a behavior change.

2026-06-02 - Patch 0257 Clear Lock Wait Node stub 112

  • Continued NDK-first after the implemented physical-record family with the direct NetWare-3.x/4.x asynchronous-lock cleanup endpoint:
    • 112 Clear Lock Wait Node
  • Patch 0257 adds a disabled source stub in src/nwconn.c for wire 0x70. The stub records the NDK request as FunctionCode=112 plus a WaitNode structure and the response as completion-only: success, ERR_LOCK_WAITING, or lock error.
  • The endpoint must not fake success from the current synchronous lock tables. It requires real asynchronous wait-node state allocated by Log/Lock File, Logical Record, and Physical Record async requests (105 through 110).
  • This is synchronization/lock-provider state, not server-management or directory state.

2026-06-02 - Patch 0258 async synchronization direct stubs 105/107/108/109

  • Continued NDK-first after Clear Lock Wait Node with the missing direct asynchronous synchronization endpoints in src/nwconn.c:
    • 105 / wire 0x69 Log File (old)
    • 107 / wire 0x6b Log Logical Record
    • 108 / wire 0x6c Lock Logical Record Set
    • 109 / wire 0x6d Log Physical Record
  • These are NetWare-3.x/4.x-relevant endpoints. The NDK also lists 5.x, but this audit keeps only the 3.x/4.x contract.
  • Patch 0258 adds disabled source stubs only. Runtime behavior is unchanged.
  • Do not map these direct async-capable calls onto the existing old synchronous handlers (03, 09, 10, 26) or namespace handlers (87/36, 87/67) without a real provider design. The request encodings and async wait-node semantics differ.
  • These calls share state requirements with 112 Clear Lock Wait Node: pending lock allocation, async wait-node lifetime, and correct completion-code mapping.

2026-06-02 - Patch 0259 fall-through synchronization selector notes

  • Continued NDK-first after the direct async synchronization stubs by tightening the implemented old file/logical synchronization fall-through handlers in src/nwconn.c:
    • 04 Lock File Set (old) and 106 Lock File Set
    • 05 Release File (old) and 07 Clear File (old)
    • 06 Release File Set and 08 Clear File Set
    • 11 Clear Logical Record and 12 Release Logical Record
    • 14 Clear Logical Record Set and 13 Release Logical Record Set
  • This is documentation-only. The active fall-through control flow remains unchanged: shared parser/handler branches stay shared, but each wire case label now has its own nearby Request: and Response: notes.
  • Keep the parser audit items unchanged: old 04 documents Lo-Hi timeout while the shared handler uses GET_BE16(), and the set release/clear handlers for file and logical records still ignore the documented LockFlag byte until direct requester traces justify changing behavior.

2026-06-02 - Patch 0260 final NDK/WebSDK/include endpoint-audit closure

  • Re-ran the endpoint-audit inventory NDK-first against the local Core Protocols PDF text, uploaded WebSDK HTML, and uploaded SDK includes after patches 0255 through 0259.
  • No additional in-scope NetWare 1.x/2.x/3.x endpoint family or planned NetWare-4.x endpoint family was found that still needs a new source-dispatch stub in this documentation pass.
  • Current coverage state is considered documentation-complete for this audit: each in-scope family is either actively handled, forwarded to the file that parses it, represented by a disabled source stub with request/reply notes, or explicitly documented as out of scope because it is NetWare 5.x/OES/MOAB/newer or requires a future provider.
  • This does not mean all endpoints are behaviorally complete. It only closes the NDK/WebSDK/include source-stub inventory. Remaining work should be driven by tests, requester traces, and provider implementation tasks, not by adding more placeholder endpoint stubs.
  • Keep using the corrected documentation rule from patches 0256 and 0259: do not split active fall-through control flow merely for documentation; instead keep shared parser/handler branches shared and place the appropriate Request:/Response: notes next to the relevant case labels.

2026-06-02 - Patch 0261 source/header subtree layout rules

  • Documented the planned large-source-file split as a module-subtree layout, not a semantic provider change. Source files should move under src/<module>/ while headers mirror the same hierarchy under include/<module>/.
  • Keep ownership names explicit: nwconn code stays under src/nwconn/, nwbind code under src/nwbind/, nwqueue code under src/nwqueue/, nwnds code under src/nwnds/, and directory code under src/nwdirectory/.
  • Flat headers remain umbrella headers. For example, include/nwbind.h should include public headers from include/nwbind/*.h; do the same later for nwconn.h, nwqueue.h, nwnds.h, and nwdirectory.h.
  • Private implementation headers should be named include/<module>/internal.h and may only be included by files in the matching src/<module>/ subtree.
  • Mechanical move/split patches must not change runtime behavior and must not be combined with endpoint semantics, provider IPC changes, or switch cleanup.
  • Start with build-system support for src/<module>/*.c plus include/<module>/*.h, then move smaller modules such as nwdirectory before splitting very large files such as nwconn.c and nwbind.c.

2026-06-02 - Patch 0262 libowfat hard dependency policy

  • Reviewed the uploaded libowfat-0.34.tar.xz archive as the dependency used by tinyldap-style code. Its README states that libowfat provides general purpose APIs extracted from Dan Bernstein's software and reimplemented under GNU GPL version 2 only, with no later-version grant.
  • Treat libowfat as a planned hard third-party dependency for the tinyldap-derived mars-tinyldap/nwdirectory work, similar to how yyjson is a required bundled dependency for salvage metadata.
  • The planned import path is third_party/libowfat. Do not keep it merely as a reference archive and do not place it at the repository root.
  • The pinned initial source should be libowfat 0.34 unless a later explicit dependency bump patch chooses a different version. The import must carry a mars-nwe note documenting source/version, GPL-2.0-only license, local CMake changes, and how mars-tinyldap/nwdirectory links it.
  • The first integration should expose a normal CMake target, for example OWFAT::owfat, and should be usable from both standalone mars-tinyldap and the mars-nwe superbuild. Do not wrap libowfat's original Makefile as the long-term build path.
  • Scope the first direct libowfat dependency to the tinyldap-derived directory-service build so the initial import remains reviewable.
  • It is also acceptable for mars-nwe core code to use libowfat later when a concrete call site benefits from it, for example byte, buffer, fmt, scan, stralloc, uint, socket, or io helpers. Do that deliberately in follow-up patches with an explicit consumer and without scattering convenience includes through unrelated dispatch code.
  • Prefer a small mars-nwe facade when multiple modules need the same libowfat helper pattern, but do not invent wrappers merely to hide a useful hard dependency.
  • Keep GPLv3 code out of this dependency path.

2026-06-02 - Patch 0263 libowfat consumer scope clarification

  • Relaxed the 0262 wording that made libowfat sound restricted to only tinyldap/nwdirectory compatibility code.
  • libowfat remains a hard bundled GPL-2.0-only dependency under third_party/libowfat, exposed as OWFAT::owfat.
  • The first consumer should still be the tinyldap-derived mars-tinyldap / nwdirectory work, but mars-nwe core code may use libowfat later when the helper is technically useful and the patch names the concrete consumer.
  • Keep direct includes deliberate: do not add libowfat to unrelated switch dispatch code only because it is available, and prefer a small mars-nwe facade if several modules need the same helper pattern.

2026-06-02 - Patch 0264 GPL-2.0-only source/header normalization

  • Normalize mars-nwe-owned .c and .h files to GPL-2.0-only. Existing GPL-2-or-later text should be rewritten to version 2 only, and files without a license header should gain a project header.
  • C/header files should carry SPDX-License-Identifier: GPL-2.0-only at the top.
  • Preserve/restore copyright attribution for Martin Stover and add current maintenance attribution for Mario Fetka where missing.
  • Root COPYING.md should state the project-level GPL-2.0-only policy before the full GNU GPL version 2 text, and COPYING.LGPL-2.1.md should carry the LGPL-2.1-only library license notice and full text.
  • README and README.md should describe the GPL-2.0-only/LGPL-2.1-only split and point to COPYING.md plus COPYING.LGPL-2.1.md.
  • Do not introduce GPLv3-or-later wording in mars-nwe-owned source/header files or dependency documentation unless a separate explicit relicensing decision is made.

2026-06-02 - Patch 0265 MatrixSSL fork/CMake policy

  • MatrixSSL is now the preferred GPL-2.0-compatible crypto/TLS candidate to evaluate for the FLAIM CCS/NICI compatibility layer and possible future TLS needs, instead of pulling OpenSSL into the tree.
  • Treat MatrixSSL as a mars-nwe-maintained fork/import under third_party/matrixssl, not as an opaque system probe and not as a source tree built by its original Makefiles.
  • The import must document the exact upstream/fork URL, revision, license terms, any local patches, and the reason it is GPL-2.0-compatible with mars-nwe.
  • Add a native CMake build in the MatrixSSL fork. It must work standalone and as a mars-nwe subdirectory, and should expose normal targets such as MATRIXSSL::crypto and, if TLS is built, MATRIXSSL::tls.
  • FLAIM must not call MatrixSSL directly. FLAIM still talks to the future CCS/NICI compatibility layer; that layer may use MatrixSSL crypto primitives underneath.
  • Keep old OpenSSL-facing FLAIM/FTK code disabled by default. Do not introduce OpenSSL as a default mars-nwe dependency.
  • Do not mix MatrixSSL import/CMake work with FLAIM CCS/NICI implementation or source tree layout changes in the same patch.

2026-06-02 - Patch 0266 shared library layering for MatrixSSL, FLAIM, and directory services

  • The third-party storage/crypto plan now uses mars-nwe-named shared library layers, not raw upstream target names as public project interfaces.

  • libnwmatrixssl is the patched MatrixSSL fork/library. It exists only to make the GPL-2.0-compatible MatrixSSL source build cleanly under mars-nwe CMake and to avoid collisions with any system MatrixSSL package. Keep local changes limited to portability, naming, CMake, and build hygiene unless a later security/compatibility patch explicitly says otherwise.

  • libnwssl is the mars-nwe SSL/crypto facade. It owns the MatrixSSL-facing wrappers, future TLS abstraction used by apps/services, and the CCS/NICI compatibility layer that FLAIM needs. Application code should include/use libnwssl, not raw MatrixSSL headers.

  • libnwflaimtk and libnwflaim are the renamed mars-nwe FLAIM libraries built from the imported FLAIM sources. The rename is intentional so mars-nwe never accidentally links against, or conflicts with, a system libflaim/FTK install.

  • libnwdirectory is the directory abstraction library above the FLAIM store. It is the API used by nwdirectory, future nwnds, setup/import tooling, and later Bindery/NDS integration; those consumers should not call libnwflaim directly.

  • Keep the layering explicit:

    apps/services/setup/nwnds -> libnwdirectory -> libnwflaim -> libnwflaimtk
                              -> libnwssl CCS/NICI -> libnwmatrixssl
    apps/services TLS         -> libnwssl TLS      -> libnwmatrixssl
    
  • FLAIM source import, MatrixSSL fork/CMake work, libnwssl facade work, CCS/NICI implementation, and libnwdirectory API work should remain separate patches.

2026-06-02 - Patch 0267 libnwssl FLAIM OpenSSL-compat boundary

  • The inspected flaim-code-r1112-trunk uses OpenSSL-style APIs only in the FTK network/TLS helper code, not as the primary FLAIM database-at-rest encryption API. FLAIM storage encryption still goes through NICI/CCS-style CCS_* calls.
  • libnwssl should therefore own two separate compatibility surfaces:
    • a CCS/NICI compatibility API for FLAIM encrypted storage, backed by libnwmatrixssl crypto primitives; and
    • a narrow OpenSSL-compatibility facade for the legacy FLAIM/FTK network code, backed by libnwmatrixssl TLS/crypto when that code cannot be disabled.
  • Do not expose OpenSSL-compatible types outside the FLAIM/FTK import boundary. If wrappers are needed, they should be private to libnwssl plus the libnwflaimtk build glue.
  • The FTK OpenSSL-compat surface should be limited to the symbols actually seen in r1112 (SSL_*, SSL_CTX_*, BIO_*, X509_*, EVP_PKEY_*, and error initialization helpers). Do not implement a general OpenSSL replacement API unless a later source audit proves it is required.
  • Keep the FLAIM network/TLS compatibility path separate from mars-nwe LDAP, provider IPC, and application TLS policy. Those consumers use the native libnwssl TLS facade, not the OpenSSL-compat shim.

2026-06-02 - Patch 0268 third-party compat-header boundary

  • Keep imported third-party code as third-party code. FLAIM, MatrixSSL, and libowfat imports may receive build fixes, CMake integration, portability fixes, local library naming, and documented minimal compatibility fixes, but they must not grow mars-nwe-specific public APIs or be edited merely to call mars-nwe wrappers directly.
  • Prefer include-path compatibility over source edits for old FLAIM dependencies: libnwssl should provide private compatibility headers under include/nwssl/private/, for example:
    • include/nwssl/private/nici.h for NICI/CCS declarations expected by FLAIM; and
    • include/nwssl/private/openssl/*.h for the narrow OpenSSL-style headers required by the FTK network/TLS code.
  • The libnwflaimtk/libnwflaim CMake targets may add include/nwssl/private to their private include path so legacy includes such as <openssl/ssl.h> resolve to the libnwssl compatibility headers without patching FLAIM call sites.
  • The compatibility headers are private to the FLAIM build. mars-nwe modules, LDAP, provider IPC, nwconn, nwbind, and libnwdirectory consumers must use the normal libnwssl and libnwdirectory public headers instead.
  • MatrixSSL remains a renamed third-party backend producing libnwmatrixssl and must not expose mars-nwe-specific APIs. libnwssl owns the public TLS/crypto facade and bridges to MatrixSSL underneath.
  • libowfat remains a renamed/bundled hard dependency with its own mars-nwe build target/library, including the planned libnwowfat shared library name. Keep libowfat changes limited to import/build/CMake/portability work in the same spirit as MatrixSSL and FLAIM.

2026-06-05 - Current handoff for next chat: redesign + FLAIM/directory storage track

This section is the current working handoff for the next mars-nwe chat. It is newer than the old patch-number notes above. If the user starts a fresh chat, ask for the current bundles/commits of all submodules first and rebuild the tree from those, because the live work has moved beyond the old 0269 documentation line.

Current repositories used in this work:

  • mars-nwe: https://gitea.disconnected-by-peer.at/mars_nwe/mars-nwe
  • mars-flaim: https://gitea.disconnected-by-peer.at/mars_nwe/mars-flaim

There are now two separate workstreams. Keep patches and tests separated by workstream.

Workstream A: mars-nwe redesign of existing code

Goal: make the existing mars-nwe codebase more future-proof without doing a big rewrite.

Near-term first redesign task:

  • introduce small enum/type layers before moving logic;
  • reduce magic strings / magic numbers at provider and directory boundaries;
  • centralize string-to-enum mapping at the edges;
  • add smoke/CTest coverage for known mappings;
  • do not move large chunks of logic in the first enum patch.

Likely first targets:

  • operation/request enums for internal handoff/provider dispatch;
  • directory/NDS object and attribute identifiers once the real NetWare 4.11 schema is available;
  • typed config/setup enums for future nwsetup.

Keep the redesign direction from REDESIGN.md: nwserv is control plane and provider registry, not a data-plane payload broker; nwconn owns client reply envelopes; providers/modules get clearer boundaries over time.

Workstream B: vendored infrastructure for future directory/storage rewrite

Todays integrated infrastructure work is not the redesign itself. It prepares the replacement of old ad-hoc/self-built functions and flat-file directory storage with maintained, namespaced libraries:

  • libnwowfat
  • libnwsodium
  • libnwmatrixssl
  • libnwssl
  • libnwflaimtk, libnwflaim, libnwxflaim
  • libnwdirectory / nwdirectory from the tinyldap-derived tree

Namespace policy:

  • historical mars-nwe programs keep their established names: nwserv, ncpserv, nwclient, dbmtool, ftrustee;
  • vendored/system-colliding libraries, headers, CMake packages, and imported helper tools use the nw namespace/prefix;
  • TinyLDAP/directory tools and FLAIM/XFLAIM tools install with nw prefixes;
  • vendored headers stay below namespaced include directories such as nwcore/, nwssl/, nwflaim/, nwmatrixssl/, nwsodium/, nwlibowfat/, and nwdirectory/.

FLAIM/MatrixSSL/NWSSL status at this handoff:

  • MatrixSSL is built as libnwmatrixssl and must export CPU/compiler feature options such as AES flags to consumers so PSCRYPTO_CONFIG matches between library and consumer.
  • yyjson is built into libnwcore; public header is installed under include/nwcore/yyjson.h.
  • nwssl owns OpenSSL-compat headers under include/nwssl/openssl/*.h and private NICI/CCS headers under include/nwssl/private/nici/*.h.
  • The temporary OpenSSL-compat code was removed from MatrixSSL; OpenSSL-style compatibility belongs in nwssl, not MatrixSSL.
  • nwssl now has a functional NICI/CCS compatibility layer used by classic FLAIM at-rest encryption.
  • NICI support must be scoped to classic FLAIM targets only. Do not define FLM_USE_NICI globally for XFLAIM; XFLAIM has a different NICI path and will not compile against the classic FLAIM compat API.
  • FLAIMSQL remains experimental and default-OFF. Do not make the default build depend on libnwflaimsql; keep SQLFLAIM work separate.
  • FLAIM is currently required only when directory support is enabled.

Current green tests at the end of the 2026-06-05 session:

ctest -L nwflaim --output-on-failure
# nwflaim.database.create-and-check ... Passed

ctest -L flaim --output-on-failure
# nwflaim.database.create-and-check ......... Passed
# mars_nwe.flaim.api-create-query-encrypt ... Passed
# mars_nwe.xflaim.api-alloc ................. Passed

These tests confirm:

  • FLAIM tools can create and check a database;
  • mars-nwe can use classic FLAIM API from the root test tree;
  • classic FLAIM at-rest encryption works through nwssl NICI/CCS compat;
  • XFLAIM still builds/runs its allocation smoke test without the incompatible classic NICI path.

Known important FLAIM fixes from this session:

  • GigaTest final rate calculation had a divide-by-zero/SIGFPE when elapsed time was zero; this was fixed in mars-flaim source, not hidden in CTest.
  • FLAIM dictionary add path needed to treat missing dictionary-name lookup EOF as no-conflict.
  • FLAIM base64 key decode needed to treat EOF after successful decode as successful completion.
  • FLAIM EncDef test dictionary syntax: encdef is valid on index definitions; encrypted data fields in the root API test are created by adding an EncDef dictionary record and using the API with that EncDef ID, not by putting encdef below a field record.

Planned directory/storage direction:

  • tinyldap/nwdirectory currently has flat-file/mmap storage. Long term it should write through libdirectory -> libnwflaim, not directly to flat files.
  • At-rest encryption was tested hard now because the future directory store will contain keys, password verifiers, and other sensitive objects.
  • When the user extracts the real NetWare 4.11 Directory/NDS schema, use it as truth rather than inventing schema by hand. Important data to capture: object classes, numeric IDs if available, attributes, syntax types, single/multi-value flags, mandatory/optional attributes, naming attributes, inheritance/superclasses, and index expectations.

Useful future TinyLDAP/Directory tests:

  • initialize an empty directory database;
  • add organization/container/user/group objects;
  • authenticate a user/password;
  • ensure password/secret material is not plaintext in the FLAIM files;
  • close/reopen and verify all objects still exist;
  • duplicate-name/object conflict handling;
  • rename/move/delete object behavior;
  • group membership add/remove lookup;
  • indexed lookup for common attributes;
  • schema validation once the real NetWare 4.11 schema is available.

Future nwsetup direction:

  • Add a curses/ncurses nwsetup tool after the directory foundation exists.
  • It should mimic the NetWare setup flow from installation phase 2 enough to provision a mars-nwe directory: tree/server/org/O/user/admin password, defaults, config files, and initial storage.
  • nwsetup should write only hashes/verifiers/sealed keys to the store; no reusable plaintext passwords in config.
  • The user referenced the NetWare 6.5 Proxmox installation article as UI/setup inspiration, not as an implementation dependency.

When tomorrows chat starts, ask the user for the current bundles of mars-nwe and all submodules, then rebuild a clean tree before producing new patches.

Next patch number should be 0271.

0343 quota file/name split handoff note

0343 keeps the quota backends deliberately distinguishable. The generic quota frontend helpers live in include/nwfs/quota.h and src/nwfs/quota/quota.c with nwfs_quota_* names only. The NetWare metadata backend lives in include/nwfs/nwquota.h and src/nwfs/quota/nwquota.c with nwfs_nwquota_* public names and nwfs_nwquota_* private helpers.

Do not merge Linux quota and NWQUOTA back into one source file. Future Linux quotactl() relocation should get a separate backend implementation while keeping the generic quota.c file backend-neutral.

0342 quota relocation handoff note

0342 starts the planned quota move into libnwfs. It moves the metadata/NWQUOTA backend helpers from src/nwvolume.c into src/nwfs/quota/quota.c and src/nwfs/quota/nwquota.c with public declarations in include/nwfs/quota.h and include/nwfs/nwquota.h. src/nwvolume.c remains the mars-nwe volume/NCP entry point and still handles Linux quotactl() probing, but now calls libnwfs for NWQUOTA restriction, usage, and adjust operations.

Do not reintroduce the removed precharge/fchown/creator-xattr experiments while working on this area. If 0342 regresses, inspect the libnwfs xattr name mapping and temporary euid-0 access first, because those preserve the 0339/0340 green behavior.

0344 quota backend naming

Quota code is split by backend so future BSD quota support does not get mixed with Linux-specific quotactl code:

  • include/nwfs/quota.h, src/nwfs/quota/quota.c: backend-neutral helpers only (nwfs_quota_*).
  • include/nwfs/nwquota.h, src/nwfs/quota/nwquota.c: NetWare metadata/NWQUOTA backend only (nwfs_nwquota_*).
  • include/nwfs/lnxquota.h, src/nwfs/quota/lnxquota.c: Linux kernel quotactl backend only (nwfs_lnxquota_*).

Do not merge these back together; a later BSD backend should use its own bsdquota.c/h and nwfs_bsdquota_* names.

0345 Linuxquota restore mirror handoff note

0345 keeps Linuxquota authoritative while adding a backup/restore mirror in netware.userquota:

  • nw_set_vol_restrictions() writes Linux quotactl() first when Linuxquota is available, then mirrors the same restriction to the NWQUOTA metadata backend.
  • nw_get_vol_restrictions() reads Linuxquota first. If Linux reports that the per-user quota entry is missing but netware.userquota has a restriction, the value is treated as restored metadata: mars-nwe tries to write it back to Linuxquota and then reads Linuxquota again.
  • If the restore write succeeds, Linuxquota is again the primary source. If it cannot be written, the metadata value is returned as fallback so NetWare clients still see the restored restriction.
  • nwquota.c and lnxquota.c remain separate backend files. The restore mirror is coordinated by the volume/NCP layer and documented in doc/quota/README.md.

0381 quota state for future AI/debug sessions

  • Do not resurrect 0380's netware.userquota.mars_usage.0; it was rejected as a private persistent parallel accounting store.
  • Linuxquota volumes: Linux quota remains authoritative for limits and kernel enforcement. The NetWare/NSS userquota xattr is a mirror only.
  • NWQUOTA/metadata volumes: netware.userquota.0 is authoritative for the restriction and the runtime enforcement usage is computed from the host tree; nwur_reserved_2 must stay zero for NSS-shaped backup/restore compatibility.
  • The combined live evidence collector is nwfs_ncpfs_all_quota_smoke.sh; use it before asking for scattered logs. It captures QUOTA.log, SYS.log, optional CTest output, an nw.log slice from test start, and a compressed bundle.

2026-06-12 all-smoke log access note

The all-in-one quota smoke keeps its temporary evidence directory world-readable and traversable immediately after creation, matching the dual userquota smoke. This is intentional because the live smokes are commonly run as root but their logs/archives are usually copied or uploaded later by a normal desktop user.

0383 all-smoke finalization fix

  • The all-quota wrapper must never exit from inside run_logged; doing so stops the script after the first subtest (typically CTest) and prevents the later live smokes, nw.log slice, tar.gz and zip from being emitted.
  • Keep the archive outside the output directory (/tmp/<timestamp>-quota-all-smoke.*) so tar/zip do not recursively include their own output file.

0384 Linuxquota clear/log cleanup note

  • Linuxquota set/clear must not echo a broad dqb_valid mask from Q_GETQUOTA back into Q_SETQUOTA. Set only block-limit fields for NetWare user-volume restrictions; usage and inode fields remain kernel-owned.
  • AUTO fallback to NWQUOTA is for genuinely unavailable Linux quota devices (no-device, unsupported, probe-failed). A real Linuxquota set-failed on a QUOTA-style volume is an error to fix, not a reason to create a parallel metadata-authoritative state.

NSS low-level library import policy

  • Keep original NSS file names and public API names when importing small GPL-2 NSS library helpers.

  • Do not hide imported helpers under an artificial nss/ public API/source path in libnwcore; imported core helpers live directly in src/core/ and expose the original header/API names so their NSS origin remains recognizable.

  • First imported compiled helper: NSS public_core/nss/lib/bitmap.c -> MARS-NWE src/core/bitmap.c, linked into libnwcore with original bitmap.h API (BitMap_s, newBitMap, findBits, etc.).

  • The existing NSS SDK include layout under include/nwfs/nss/sdk/... remains available for compatibility and provenance; the compiled library source is what moves into src/core/.

  • NSS runtime-only dependencies may be reduced only as needed to compile outside NSS; for bitmap this only maps NSS zalloc() to libc calloc().

  • Future candidates to import with original names: CRC/hash helpers after Unicode dependency review, queue macros, bit helpers, and media/type helpers already represented by the SDK headers.

  • 0411 extends the direct NSS Unicode helper import with unitoupper.c, unilwr.c, uniupr.c, uniicmp.c, uniicmpmac.c, and uninicmp.c in libnwcore; unicodeInit.c now exports both NSSUniToLower[] and NSSUniToUpper[] with the same ASCII-compatible bootstrap until the full NSS converter/table startup is imported.

  • 0412 imports the next safe NSS Unicode library block into libnwcore: the component/string helpers componentUnicpy.c, componentUnilen.c, unicat.c, unicmp.c, unicpy.c, unilen.c, and unimcpy.c. Their sharedsrc implementation headers (*.c.h) are kept local under src/core/ and are not installed as public API. No MARS callsites are switched yet; this is a prerequisite for replacing the older hand-written MARS Unicode/string code piece by piece with NSS-compatible primitives.

  • 0413 switches the NSS case-map globals to the external third_party/unicodeTables submodule (TAB/unicodeTables.c). That submodule is project-managed and currently tracks master; it generates NSSUniToLower[] and NSSUniToUpper[] from Unicode UCD data instead of copying Novell shared/sdk/unitables/*.tab files. Any future Unicode/codepage tables belong there first, then MARS-NWE consumes the generated output.

  • 0414 and 0415 import NSS UTF-8 conversion helpers that need no tables: single-character UTF-8 decode (utf8ToUniChar.c, utf8LenToUniChar.c) and whole-string UTF-8/Unicode conversion (uni2utf.c, utf2uni.c).

  • 0416 imports NSS Unicode parser/override helpers (unicodeParse.c) and getNssUnicodeVersion.c. It also exports NSSUnicodeFF and NSSUnicodeMacFF as temporary 0xff sentinel values. Keep those on the same table/runtime watchlist as the case maps; the final values should come from DOS/Mac codepage tables or derived converter startup, not from private MARS conversion state.

  • 0417 imports getMacCodePageName.c and exports NSS Unicode startup/shutdown entry points plus MacintoshCodePageName. The name remains NULL until real Mac/DOS codepage table/runtime support is imported via the Unicode tables submodule.

  • 0418 imports the NSS byte/Unicode and Mac byte/Unicode conversion entry points (ByteToUnicode.c, LenByteToUnicode.c, MacByteToUnicode.c, LenMacByteToUnicode.c, UnicodeToByte.c, UnicodeToMacByte.c, UnicodeToUntermByte.c, UnicodeToUntermMacByte.c) directly into libnwcore. The NSS converter state globals are present but intentionally empty, so these APIs return zERR_UNICODE_INVALID_CONVERSION_TYPE until the real codepage tables/runtime are added from mars-unicode-tables.

  • 0419 imports NSS stdlib allocation compatibility (xStdlib.h, zalloc.c, zrealloc.c) and removes the private bitmap.c zalloc macro fallback. Original NSS public-core allocator sources exist under public_core/library/stdlib, but they depend on NSS OS memory tracking; libnwcore therefore preserves the NSS API names with libc userland allocation until the full NSS memory runtime is imported.

  • 0420 imports NSS UTC/DOS/MS timestamp helpers from public_core/library/utc into libnwcore. The active import keeps original NSS function names and public headers, with only small userland glue for current time/timezone globals where the NSS public source expects NetWare or Linux-kernel state.

  • 0421 follows up 0420 by whitespace-cleaning the imported UTC block and defining the missing cache-control globals in utcUserland.c for userland builds where zLINUX is unset.

  • 0422 imports the NSS Unicode converter registration entry points (RegisterUnicodeConverter.c, UnRegisterUnicodeConverter.c) and adds a small unicodeTableBuild.c userland builder for the NSS byte/unicode table shapes. The builder intentionally only provides identity single-byte tables plus the already imported NSS wildcard overrides; full DOS/Mac codepage tables still belong in the external mars-unicode-tables submodule before MARS-NWE enables NetWare-codepage-accurate conversions.

  • 0423 consumes the external mars-unicode-tables codepage output and builds NSS-shaped byte/unicode converter tables from compiled-in Unicode.org mapping descriptors. Runtime does not load .tab/.txt files; the submodule is a source/build dependency only.

  • 0424 imports NSS GUID/ID helpers into libnwcore (guid.h, guid.c, local guid.c.h, id.h, id.c). The import keeps NSS GUID/ID API names but deliberately excludes eDir/DDC/NDS runtime pieces; userland GUID generation is adapted with libc time and /dev/urandom fallback glue. Namespace DOS/LONG/MAC replacement remains the next larger planned block after parse/xCtype/string review.

0425 NSS xCtype/xString import

  • Imported NSS xCtype/xString API headers into include/core/:
    • xCtype.h
    • xString.h
  • Added libnwcore userland implementations for exported NSS LB_* symbols in:
    • src/core/xCtype.c
    • src/core/xString.c
  • Source NSS tree only exposes these routines through headers/libNSS.imp, so the implementations use libc-compatible byte/string operations while preserving the NSS API names.
  • Did not import parse/pcmdline.c yet: it depends on setparms, errPrintf, message/category plumbing, and is not a clean lowlevel block.
  • No MARS callsites were changed. Namespace DOS/LONG/MAC remains the next planned larger audit/import track after this lowlevel helper pass.

0426 NSS namespace audit / replacement plan

  • Audited NSS namespace sources under public_core/comn/namespace and recorded the migration plan in doc/NSS_NAMESPACE_AUDIT.md.
  • NSS provides the target replacement set for old MARS namespace code:
    • DOS namespace
    • LONG namespace
    • MAC namespace
    • UNIX namespace
    • Data Stream namespace
    • Extended Attribute namespace
  • Do not expand old MARS namedos/nameos2; they are now replacement targets.
  • Namespace is not a simple helper import like crc/unicode/utc: the NSS sources depend on common-layer beasts, AdminVolume registration, variable-data hooks, message plumbing, and name-cache structures.
  • Follow-up 0427 corrects this plan: do not build a permanent wrapper layer. The namespace path is direct NSS source import/adaptation followed by removal of old MARS namedos/nameos2 logic.

0427 namespace plan correction and libnwfs follow-ups

  • Corrected doc/NSS_NAMESPACE_AUDIT.md to remove the earlier wrapper-oriented wording. The target is direct import/adaptation of NSS namespace sources, then retirement of old MARS namedos/nameos2 code after DOS behavior is covered by tests.
  • Recorded _ADMIN as future libnwfs work, not libnwcore and not pure libnwnds. Preserve the NSS/OES volume-ID convention in later code:
    • SYS => volume ID 0
    • _ADMIN => reserved virtual admin volume ID 1
  • _ADMIN must remain hidden/admin-only and should not be enabled by default on the NetWare 3.x path. Its eDirectory-backed views can later call into libnwnds, but the virtual filesystem/runtime belongs to libnwfs.
  • Added NSS compression to the libnwfs follow-up list. The lowlevel algorithm sources live in public_core/comn/compression/ (cdcomp.c, cduncomp.c, nwAlgo.c, copyAlgo.c, etc.); the larger compression manager/runtime should wait until namespace/data-stream/volume metadata is present.
  • Compression-related NCP endpoints remain guarded/stubbed until real libnwfs state exists:
    • decimal 90/12 == wire/code 0x5a/0x0c
    • decimal 123/70 == wire/code 0x7b/0x46
    • decimal 123/71 == wire/code 0x7b/0x47
    • decimal 123/72 == wire/code 0x7b/0x48
    • decimal 22/51 == wire/code 0x16/0x33 compression counters

0428 NSS public_core audit and authsys boundary notes

  • Added doc/NSS_PUBLIC_CORE_AUDIT.md to classify the top-level NSS public_core tree before more large imports.
  • Kept the namespace policy strict: no permanent wrapper over old MARS namedos/nameos2; import/adapt NSS namespace directly and retire the old files after tests cover DOS/LONG behavior.
  • Clarified comn/authsys: do not wrap old MARS auth logic as the target. Import/adapt useful NSS authsys logic directly, but adapt only at real platform/backend boundaries:
    • AES/crypto/RNG => existing MatrixSSL/libsodium/libc layers
    • Bindery identity/storage => future libnwbind
    • NDS/eDir identity => future libnwnds
    • filesystem hooks/Beasts => future libnwfs
  • Classified useful public_core areas:
    • library/, nss/lib, sharedsrc => selective libnwcore helpers
    • comn/namespace, comn/common, lsa, zlss, compression => future libnwfs path
    • manage/_ADMIN => later management/libnwfs-admin path
    • ndpmod, library/eDir => future libnwnds
    • comn/authsys identity bridge pieces => future libnwbind/libnwfs split
    • admindrv, nebdrv, nsslnxlib, library/os => platform glue only, import narrow userland-compatible pieces when concrete consumers need them.

0429 salvage/compression/tool roadmap documentation

  • Added doc/NWFS_SALVAGE_COMPRESSION_TOOLS.md as the current storage-backend roadmap for salvage, .nwfs_streams, compression and host tools.
  • Salvage direction changed from permanent .salvage JSON sidecars to authoritative netware.metadata on the recycled payload:
    • .recycle remains the payload backend for Samba compatibility.
    • netware.metadata carries NSS-shaped deleted-file metadata.
    • .salvage JSON is legacy transition/debug data only and should stop being written after the metadata path is implemented.
  • Samba 4.23.6 vfs_recycle was inspected: the normal recycle path uses rename into the repository, so existing Linux xattrs remain attached to the recycled inode. Do not add automatic synthetic metadata fallback for files manually copied into .recycle; those are not valid NetWare salvage objects unless an explicit admin repair command marks them.
  • Long-term yyjson target: remove third_party/yyjson after new deletes no longer write .salvage JSON, old sidecars are migrated/retired, and no other required code uses yyjson.
  • .nwfs_streams scope clarified:
    • use it for internal streams, EA, compression blobs and related future state;
    • do not move primary Samba-compatible salvage payloads there;
    • key entries by a stable MARS/NWFS/NSS-shaped file ID from netware.metadata, not by Linux inode and not by visible filename.
  • Compression direction documented: Linux ext3/XFS do not provide a portable NSS-compatible compression model. Compression belongs to future libnwfs stream/metadata work and must feed real state to decimal 90/12 == wire/code 0x5a/0x0c, decimal 123/70..72 == wire/code 0x7b/0x46..0x48, and decimal 22/51 == wire/code 0x16/0x33.
  • Tool roadmap recorded:
    • nwsalvage for list/info/restore/finaldelete/purge/verify/explicit repair;
    • nwmetadata for metadata dump/verify/set-deleted/clear-deleted;
    • nwcompress for compression info/compress/uncompress/verify/list;
    • nwstreams for stream list/dump/extract/remove;
    • nwea for EA list/dump/set/remove.
  • No runtime code changed in 0429.

0430 compression stream layout and recycled payload clarification

  • Clarified the future .nwfs_streams layout in doc/NWFS_SALVAGE_COMPRESSION_TOOLS.md:
    • key internal stream/compression backends by a stable MARS/NWFS/NSS-shaped file ID from netware.metadata;
    • do not use Linux inode numbers;
    • do not use visible DOS/LONG/MAC/UNIX filenames;
    • do not encode compression state in a compressed_ filename prefix.
  • Future compressed stream backend shape:
    • /export/SYS/.nwfs_streams/<stable-file-id>/compression/primary
    • compression state/algorithm/logical size/compressed size live in netware.metadata or stream descriptors.
  • If a compressed file is recycled, the .recycle payload must be a normal uncompressed Linux file so Samba and host-side tools can read it. Store the previous compression descriptor in netware.metadata; NCP recover may later recreate/recompress according to volume policy.
  • .recycle remains the Samba-compatible deleted-payload backend. .nwfs_streams remains private NWFS state for live/future streams, EA and compression.
  • No runtime code changed in 0430.

2026-06-12 - Patch 0431 license files for GPL/LGPL-only split

  • Root license files are Markdown now:
    • COPYING.md for MARS-NWE GPL-2.0-only programs/tools/non-library code.
    • COPYING.LGPL-2.1.md for explicitly LGPL-covered MARS-NWE libraries.
  • Both policies are "only", not "or later": use GPL-2.0-only and LGPL-2.1-only SPDX identifiers.
  • Adding LGPL-2.1-only license text does not relicense the whole tree; individual files/libraries must opt in explicitly.