mars-nwe/doc/DIRECTORY_STACK_PLAN.md

Directory stack and storage-backend plan

This note records the current nwdirectory/TinyLDAP staging rule before the NetWare 4.x/NDS work starts. It exists to keep LDAP compatibility, schema import, setup tooling and the later FLAIM storage conversion from being mixed into one unreviewable change.

Current code shape

The current TinyLDAP-derived component already builds a library and server under mars-nwe names:

libnwdirectory
  TinyLDAP-derived library output
  contains LDAP/ASN.1/LDIF/auth/helper code
  contains the current flatfile/mmap/journal storage helpers

nwdirectory
  LDAP server binary/service
  uses libnwdirectory

The current storage files are the TinyLDAP flatfile path, including:

mstorage_add.c
mstorage_add_bin.c
mstorage_init.c
mstorage_init_persistent.c
mstorage_unmap.c
mduptab_*.c
strstorage/strduptab helpers

FLAIM is built in the mars-nwe directory-enabled superbuild today, but it should not be wired into the TinyLDAP storage path until the flatfile LDAP behaviour is covered by tests.

Target library chain

The later directory-service chain should be:

libnwds
  -> libnwdirectory
       -> libnwflaim
            -> FLAIM

The nwdirectory name is the server/binary. libnwdirectory is the library used by that server. libnwds is the future NetWare Directory Services API layer above it, used by setup/import tools, future nwnds, Bindery integration and any NetWare-facing directory policy code.

Implementation order

Do not combine LDAP protocol compatibility, NetWare schema import, setup tooling and the storage-engine swap. The first directory implementation phase is flatfile-only.

1. Keep libnwdirectory on the current flatfile/mmap/journal backend.
2. Add LDAPv2 support to the TinyLDAP-derived server first. TinyLDAP is currently treated as the LDAPv3-era baseline; the uploaded OpenLDAP 1.0.3 tree is the historical LDAPv2 server reference for bind/search/result/BER behaviour, not a source tree to import wholesale.
3. Add LDAPv2 CTests over TCP against nwdirectory with isolated temporary workdirs and free loopback ports.
4. Keep/add LDAPv3 CTests for behaviour that is already present or intentionally supported, but do not make LDAPv3 the NetWare 4.11 compatibility baseline.
5. Make NetWare/NDS schema handling work on the flatfile backend. Prefer native .sch import if practical; otherwise add a reviewed .sch to LDIF conversion path.
6. Add libnwds and the first nwsetup path together. The first nwsetup implementation should be console/CLI-only so CTest can exercise initial setup without the future nwtui frontend. It should call libnwds, and libnwds should call libnwdirectory; it should not bootstrap by acting as an LDAP client to nwdirectory.
7. Only after LDAPv2, LDAPv3 smoke coverage, schema import, libnwds and the console nwsetup path are stable on flatfile storage, add a FLAIM storage backend.
8. After the FLAIM backend is working, add the interactive nwtui/nwtoolbox setup frontend and the certificate/TLS directory-storage work as separate follow-up changes.

This keeps LDAPv2 improvements generally useful to the TinyLDAP-derived code, keeps nwsetup testable before the UI exists, and keeps the later FLAIM backend as a clearly mars-nwe-specific storage change.

Future storage-backend swap

The later FLAIM backend should be a parallel source set selected by CMake, not a large rewrite of every LDAP call site.

Example layout:

flatfile backend, default:
  mstorage_add.c
  mstorage_add_bin.c
  mstorage_init.c
  mstorage_init_persistent.c
  mstorage_unmap.c
  mduptab_*.c

flaim backend, later:
  flaimstorage_add.c
  flaimstorage_add_bin.c
  flaimstorage_init.c
  flaimstorage_init_persistent.c
  flaimstorage_unmap.c
  flaimduptab_*.c

The FLAIM files may keep the same exported function names as the flatfile files. For example, flaimstorage_add_bin.c can provide mstorage_add_bin() when the FLAIM backend is selected. Only one backend source set may be compiled into libnwdirectory at a time.

That gives a reviewable storage replacement:

LDAP/server code -> existing mstorage/mduptab API -> selected backend

and avoids changing LDAP protocol logic while changing persistence.

CTest direction

LDAP server tests should not depend on system ldapsearch or OpenLDAP tools by default. Prefer existing TinyLDAP client/test helpers where suitable, or add a small in-tree client test.

A standard test should:

1. allocate a unique temporary workdir;
2. prepare test LDIF/schema data for the flatfile backend;
3. start nwdirectory on 127.0.0.1:<free-port>;
4. wait until the TCP port is ready;
5. run LDAPv2 or LDAPv3 bind/search/unbind checks;
6. stop the server;
7. remove the workdir.

Suggested future test names:

nwdirectory.ldap.v2.bind-search-flatfile
nwdirectory.ldap.v3.bind-search-flatfile
nwdirectory.schema.sch-import-flatfile
nwds.setup.console-flatfile
nwdirectory.storage.flatfile.smoke
nwdirectory.storage.flaim.smoke   # later, after the backend swap