mars-nwe/TODO.md

TODO

This file is the active project backlog, not a changelog and not the broad architecture document. Keep patch chronology and handoff details in AI.md. Keep durable architecture prose in REDESIGN.md and focused audits in doc/*.md. The dashboard at the top is intentionally retained as the short active-work index. Remove items from this file only when the work is finished or explicitly superseded; unfinished long-running tracks such as salvage and low-level NSS imports stay visible here. This file should answer: what remains to be done, why it matters, and where the work belongs.

Implementation dashboard

Area	Priority	Status	Notes
DOS namespace compatibility	P0	Active next	Legal names, wildcard semantics, case folding, reserved names and stable 8.3 aliases.
Logging level cleanup	P0	Pre-namespace audit done / cleanup pending	Keep new code at levels 1..5; review touched legacy call sites while namespace code moves.
Salvage metadata / backup-tool compatibility	P0	Planned next / unfinished	Keep .recycle payloads; make netware.metadata authoritative for deleted entries; deprecate .salvage JSON.
NetWare 3.x NCP endpoint completion	P1	Active audit + implementation	ENDPOINTS.md is the detailed decimal/hex audit table; keep this TODO as the short implementation queue.
NCP provider handoff cleanup	P0	Pre-namespace audit started	Use doc/HANDOFF_AUDIT.md and include/ncp_endpoint.h; annotate real nwconn magic handoffs before changing control flow.
NetWare 4.x endpoint compatibility	P2	Partial / guarded	ENDPOINTS.md tracks active compatibility vs guarded 4.x selectors.
Admin, queues, printing and TTS	P2	Ongoing	Console-operator model, queue path case handling, print/TTS compatibility cleanup.
Test infrastructure	P1	Ongoing	CTest where offline, live all-smokes where server/client state is required, DOS tool smokes.
NSS low-level import and helper extraction	P1	Ongoing / cleanup started	Low-level core helper staging is removed once imported into normal src/core/include/core; continue selective NSS imports for namespace, salvage, streams, metadata or compression.
Shared library layering	P2	Planned cleanup	Move reusable metadata/salvage/stream/compression helpers into libnwcore/libnwfs without importing whole NSS runtime subsystems.
Transport split and IPX/TCP config	P2	Documented plan	Keep TCP/IP and IPX under nwtransport; make IPX kernel setup internal later, with ipx-utils as fallback/debug tooling.
External file reconciliation	P1	Planned next	Add libnwfs watcher/scanner support so Samba, rsync and host-created files receive netware.metadata and namespace records.

NCP provider handoff cleanup

Before namespace implementation grows new provider paths, finish the first handoff audit pass:

• keep include/ncp_endpoint.h declarative until endpoint rows and magic-return sites are annotated;
• do not bulk-rewrite every return(-1) in the tree, because most are ordinary local error returns;
• start with the active nwconn.c dispatcher clusters listed in doc/HANDOFF_AUDIT.md;
• add provider/result flags to endpoint-table work before replacing existing return(-1) / return(-2) handoff behaviour;
• new namespace/NWFS code may use provider vocabulary, but must not create new process-specific magic return conventions.

NetWare 3.x endpoint implementation queue

Default runtime work should finish documented NetWare 1.x/2.x/3.x behaviour first. The rows below are grouped by feature area, not by patch sequence. Status meanings: done = implemented and tested enough to leave the active queue; partial = useful code exists but layout/edge cases/tests remain; open = known implementation work remains; later = document or guard until a client/test requires it.

Family / endpoint area	Generation	Status	Next work
Bindery/property relations, including NCP 17/4C List Relations of an Object	2.x/3.x	open	Add direct tests for group membership, GROUPS_I'M_IN, and relation-listing edge cases.
Message/bindery forwarded groups from nwconn to nwbind	2.x/3.x	partial	Keep dispatcher comments paired with parser layouts; verify forward-vs-incomplete paths.
Old direct file/logical/physical synchronization and lock calls 0x2222/01 through 0x2222/0e	1.x/2.x legacy	partial	Preserve old-client behaviour where DOS clients still exercise these calls; add minimal direct probes.
Broadcast/message compatibility around 0x2222/12 through 0x2222/15	1.x/2.x legacy	partial	Recheck observable DOS utility behaviour before changing parser logic.
Directory Services group 0x2222/22 namespace/directory selectors through the 3.x range	3.x	partial	Complete request/reply comments and tests for namespace, search-map and directory-handle edge cases.
Directory quota calls decimal 22/35, 22/36 (wire/code 0x23, 0x24)	3.x	done	Stable; keep regression tests only.
Directory quota scan decimal 22/40 (wire/code 0x28)	3.x	partial	Sequence byte order is handled; enrich scan reply semantics after related resource-fork/namespace fields are understood.
Salvage scan/recover/purge bridges, including old 22/27, 22/28, 22/29 style views	3.x	partial	Align NCP replies, JSON sidecars and NSS-shaped deleted metadata from one snapshot builder.
File Server Environment group 0x2222/23 usage/volume/LAN/server-status selectors	3.x	partial	Add selector-specific tests for monitor tools and ensure dummy/no-op replies are explicitly documented.
File Server Environment open-file, lock/semaphore monitor and console-control selectors	3.x	open	Decide exact compatibility replies for old admin tools; add direct tests where DOS tools do not trigger paths.
Semaphore groups, including forwarded selector families and direct old-style calls	2.x/3.x	partial	Separate implemented forwards from incomplete direct calls; add lock/semaphore probes.
Extended Attribute group 0x2222/86	3.x	partial	Keep work tied to backup/metadata compatibility and existing xattr roundtrip tests.
Name Space groups 0x2222/87 and 0x2222/89, DOS namespace selectors	3.x	open	Make DOS namespace rules match old clients before broad LONG/OS2 expansion.
Name Space groups 0x2222/87 and 0x2222/89, LONG/OS2 metadata/search selectors	3.x	partial	Implement only the pieces required by existing clients and 3.x filesystem semantics.
AFP/Mac selector families where they touch existing metadata storage	3.x adjacent	partial	Keep behind the AFP backend boundary; do not block DOS namespace or salvage metadata work.
Queue, print and TTS compatibility selectors	3.x	partial	Fix queue path case handling and keep TTS limited to documented client needs.

NetWare 4.x endpoint status queue

NetWare 4.x work is not a separate future-only bucket: some compatibility calls are already present. The rule is that new broad 4.x runtime behaviour must be explicitly chosen or guarded, while already useful compatibility implementations may remain active.

Family / endpoint area	Status	Policy / next work
0x2222/22/50 Get Object Effective Rights for Directory Entry	partial / implemented compatibility	Keep active compatibility code; verify layout and rights semantics against clients and direct tests.
0x2222/22/51 Get Extended Volume Information	partial / implemented compatibility	Keep active compatibility code; verify volume fields and ensure 3.x clients are not regressed.
Other Directory Services 4.x high selectors	later / guarded	Document exact request/reply layout before enabling default runtime behaviour.
Additional File Server Environment 4.x monitor/admin selectors	later / guarded	Implement only when a concrete admin tool or compatibility test requires it.
NDS/NCP Fragger and related 4.x infrastructure	later / guarded	Keep out of default runtime until transport/client scope is explicit.
TimeSync and NCP Extension families	later / guarded	Reference/stub only unless a real 3.x/4.x client path requires them.
Migration, compression, data migration and later OES/MOAB-style selectors	reference-only	Study reusable data models only; do not add default live endpoints during the 3.x push.

Active backlog by work area

1. Filesystem and namespace compatibility

Goal: finish NetWare 1.x/2.x/3.x filesystem behaviour before broad new runtime work. The next implementation line should start here after the documentation boundaries are clean.

• Adapt DOS namespace behaviour from the NSS/reference model:
  • legal-name checks;
  • case folding;
  • wildcard matching;
  • reserved names;
  • stable 8.3 alias generation;
  • host-path conflict handling on case-sensitive filesystems.
• After DOS namespace behaviour is stable, adapt LONG/OS2 namespace behaviour required by 3.x clients and existing MARS-NWE paths.
• Keep trustee/effective-rights, data-stream, extended-attribute, object-id, search-map and salvage work scoped to what supports the 1.x/2.x/3.x target.
• Store stable namespace identity in netware.metadata: file ID, parent file ID, DOS, LONG/OS2, MAC, UNIX/backend names, casefold/hash helpers and namespace flags.
• Treat MAC namespace as a libnwfs namespace/stream problem, not as a transport problem; resource forks and Finder info belong in later stream/metadata work.
• Add watcher/scanner support for files created outside MARS/NCP:
  • inotify/fanotify for CREATE, MOVED_TO, DELETE, RENAME and ATTRIB;
  • startup/full reconcile scan for missed offline changes;
  • missing netware.metadata generation for normal host-created files;
  • namecache/search invalidation after external changes;
  • orphaned .nwfs_streams and invalid .recycle metadata reporting.
• Avoid importing whole NSS/OES subsystems. Adapt data models and helper logic into MARS-NWE/libnwfs where useful. Remove NSS staging duplicates after the adapted/stubbed MARS path exists; keep only still-open reference areas.

2. Salvage metadata and backup-tool compatibility

Current base:

• MARS-NWE already has a Samba-friendly salvage/recycle model: live deletes move payloads into .recycle. Keep .recycle as the payload backend.
• New salvage state should be authoritative in netware.metadata on the recycled payload. .salvage JSON sidecars are a legacy transition/cache and should be phased out instead of becoming a second metadata authority.
• Samba 4.23.6 vfs_recycle was checked: the normal recycle path uses rename, so Linux xattrs stay attached to the recycled inode. Files manually copied into .recycle without netware.metadata are not valid NetWare salvage objects unless an explicit admin repair tool marks them.

Follow-up:

• Add shared libnwcore/libnwfs helpers for deleted metadata, xattr validation, NCP salvage reply input and future host tools.
• Preserve .recycle payload compatibility. Do not replace the host layout with an NSS purge tree and do not move primary salvage payloads into .nwfs_streams.
• If a compressed file is recycled, materialize the .recycle payload as a normal uncompressed Linux file so Samba/host tools can see it; keep the former compression state in netware.metadata for NCP recover/recompress policy.
• For each salvaged file, set netware.metadata with the same NSS-shaped fields used for live files plus deleted-object information:
  • DOS attributes;
  • create/archive/modify/access times where available;
  • owner/archiver/modifier IDs;
  • inherited-rights mask;
  • trustees;
  • deleted time and deleted-by identity;
  • original parent/name identity.
• Keep volume salvage reporting vocabulary aligned with NSS concepts such as purgeable bytes, non-purgeable bytes, deleted-file count, oldest deleted time, keep seconds and watermarks, even if MARS computes them by scanning .recycle payloads with netware.metadata.
• Add host-side tests and tools that verify salvaged payloads expose NSS-shaped netware.metadata.
• Remove the yyjson dependency only after new deletes no longer write .salvage JSON, old sidecars are migrated/retired, and no other required consumer remains.
• Keep the offline dependency bootstrap aligned with the root/submodule layout: mars-nwe-master is the root, uploaded mars-* bundles populate their configured submodule paths, yyjson/zlog/nested libsodium release tarballs go into the existing third-party snapshot locations, gdbm is locally built, and PAM/ncurses provide headers only while linking to system libraries.

3. Administration, queues, printing and operational behaviour

• Add a real console-operator privilege model instead of mapping console rights directly to supervisor equivalence.
• Decide where console-operator state should live:
  • bindery property;
  • server configuration;
  • explicit internal list similar to queue operator handling.
• Fix queue spool path case handling so DOS/bindery names do not create duplicate Unix directories that differ only by case.
• Resolve queue job file paths case-insensitively or use the queue object's already-resolved Unix spool directory instead of rebuilding it from a DOS path.
• Continue Q_UNIX_PRINT cleanup where it improves observable NetWare client behaviour.
• Keep TTS work focused on documented 2.x/3.x client compatibility; do not add a broad transaction subsystem unless a real client path needs it.

4. Extended attributes, AFP/Mac and metadata families

• Continue extended-attribute work only where it supports the 3.x filesystem target, existing clients or backup-tool compatibility.
• Keep AFP/Mac metadata work behind the existing backend boundary; do not let AFP cleanup block DOS namespace or salvage metadata work.
• Align xattr naming and validation helpers so live files, salvaged files and test dump tools use the same NSS-shaped metadata builders.

5. Test infrastructure

• Keep live smokes easy to collect: one bundle per functional area, with one log per volume and an nw.log slice from the start of the test.
• Keep DOS tool smokes paired with Linux helper scripts when setup requires host state, for example quota limits or filesystem images.
• Extend CTest only with checks that can run without a live IPX/NCP server.
• Plan an isolated live-test server tree that can be driven from the CMake build once the transport no longer depends on host IPX setup.
• The future test environment should generate its own SYS tree and quota image, then start a local MARS-NWE instance from the test build.

6. Low-level NSS import follow-ups

This track is not complete. Keep it in the active TODO file until the remaining NSS-derived helpers needed by the NetWare 3.x filesystem line have been adapted, built and tested.

• Continue the direct-import pattern for small GPL-2.0 NSS helpers whose APIs are useful outside the original NSS runtime.
• Keep imported libnwcore helpers in normal src/core/ and include/core/ locations, with original NSS names preserved where that improves source compatibility.
• Do not import entire NSS/OES runtime subsystems just to satisfy helper dependencies; replace kernel/VFS/platform hooks with small userland glue.
• Treat remaining namespace, metadata, stream, salvage and compression helper needs as active follow-up work tied to libnwfs/libnwcore boundaries.
• Keep Unicode/codepage table generation based on the external third_party/unicodeTables submodule and Unicode.org data, not Novell unitables/*.tab files.

7. Third-party and shared-library layering

• Keep mars-nwe-owned .c and .h files under GPL-compatible project control.
• Maintain clear boundaries for third-party code:
  • libowfat-style helpers where they simplify core code;
  • MatrixSSL/libnwssl only where needed for existing SSL/FLAIM/NICI work;
  • FLAIM wrappers without invasive upstream source rewrites.
• Keep shared helper naming consistent:
  • generic filesystem/quota helpers under nwfs_*;
  • Linux quota helpers under nwfs_lnxquota_*;
  • NetWare metadata quota helpers under nwfs_nwquota_*;
  • salvage/metadata snapshot helpers in a shared libnwcore/libnwfs layer once they exist.

8. Transport split and IPX/TCP configuration

• Keep TCP/IP support as a code/library split under nwtransport, not a new daemon.
• In the implementation patch for src/nwtcp.c, check against socket.h and io.h from the bundled mars-libowfat submodule before coding call names.
• Add src/nwtransport.c as the common transport boundary, src/nwipx.c for IPX-specific behaviour and later src/nwtcp.c for TCP/IP.
• Keep NCP providers transport-neutral: IPX and TCP clients both flow through nwtransport -> nwconn -> NCP dispatcher -> providers.
• For TCP/IP, use the audited libowfat API names: socket_tcp4(), socket_tcp6(), socket_bind4_reuse(), socket_bind6_reuse(), socket_listen(), nonblocking accept helpers and io_* readiness helpers; support default and per-interface listener configuration.
• Move IPX kernel configuration into mars-nwe later so ipx-utils is no longer a runtime requirement. Keep ipx-utils as fallback/debug/admin tooling.
• Keep the existing AF_IPX backend, but document and later add an optional backend = l2 path that sends/receives Ethernet IPX frames directly when a host has no usable kernel IPX stack.
• Do not add a bundled capability library for the L2 backend. Try the packet socket, report missing CAP_NET_RAW/CAP_NET_ADMIN clearly, and leave root, setcap, or systemd capability assignment to deployment policy.
• Treat SPX as a separate later userland layer above the common IPX boundary; do not depend on kernel SPX and do not block namespace work on SPX.
• Use the imported Linux 2.4.37.9 SPX files under src/kernel/ and include/kernel/ only as reference when designing nwspx.c; they are not build inputs.
• Use section-encoded IPX frame tokens instead of dotted frame names: 8022, 8023, etherii, snap, tr8022, auto.
• Reserve [transport.ipx.local] for the internal network and [transport.ipx.route.<target>] for explicit route entries.
• Emit a clear error when IPX is requested but socket(AF_IPX, ...) fails because AF_IPX is unavailable.

Logging level cleanup

Legacy MARS accepts debug thresholds from 0 through 99 in INI entries 100 through 106. The audit in doc/LOG_LEVEL_AUDIT.md shows that the old numbers are thresholds, not strict severities, and that the old source is mostly centered on 1..5 but unevenly distributed. New code should use the semantic 1..5 facade vocabulary from include/nwlog.h: error, warn, info, debug, trace, where debug is local diagnostic detail and trace follows packet/message/handoff flow across process or provider boundaries. Useful legacy 6..99 deep traces should migrate to nwlog_detail(), not to new numeric high levels. nwlog_detail() is active only in MAINTAINER_BUILD, is not controlled by an INI option, and in maintainer builds may emit independently from the normal configured 1..5 threshold.

Namespace work should opportunistically clean logging in files it touches, especially src/namspace.c, src/connect.c and the future libnwfs namespace engine. Do not start a tree-wide logging rewrite before the namespace split; keep this as a local cleanup rule for touched code.

Concrete follow-up for the future facade:

• add the normal nwlog_error() / nwlog_warn() / nwlog_info() / nwlog_debug() / nwlog_trace() facade wrappers around the semantic 1..5 vocabulary;
• define the config parser so 0, 1, 12, 123, 1234, 12345 and off/error/warn/info/debug/trace are synonyms, with one global [logging] level plus optional [logging.process.<name>] level overrides while legacy 100..106 entries remain accepted;
• add nwlog_detail() as the only maintainer-build-only target for useful legacy 6..99 call sites;
• make normal builds return 0/no-op from nwlog_detail();
• make maintainer builds allow nwlog_detail() independently from the normal configured 1..5 threshold;
• do not expose maintainer-detail logging through nw.ini;
• remove obsolete high-level traces instead of preserving all of them.