mars-nwe/doc/DIRECTORY_AND_ADMIN_PLAN.md

# Directory stack and admin client plan

This file consolidates the directory stack/storage-backend plan and the nwAdmin/nwConsole roadmap.

---

## Directory stack and storage backend

This section records the current `nwdirectory`/TinyLDAP staging rule before the
NetWare 4.x/NDS work starts.  It exists to keep LDAP compatibility, schema
import, setup tooling and the later FLAIM storage conversion from being mixed
into one unreviewable change.

### Current code shape

The current TinyLDAP-derived component already builds a library and server under
mars-nwe names:

```text
libnwdirectory
  TinyLDAP-derived library output
  contains LDAP/ASN.1/LDIF/auth/helper code
  contains the current flatfile/mmap/journal storage helpers

nwdirectory
  LDAP server binary/service
  uses libnwdirectory
```

The current storage files are the TinyLDAP flatfile path, including:

```text
mstorage_add.c
mstorage_add_bin.c
mstorage_init.c
mstorage_init_persistent.c
mstorage_unmap.c
mduptab_*.c
strstorage/strduptab helpers
```

FLAIM is built in the mars-nwe directory-enabled superbuild today, but it should
not be wired into the TinyLDAP storage path until the flatfile LDAP behaviour is
covered by tests.

### Target library chain

The later directory-service chain should be:

```text
libnwds
  -> libnwdirectory
       -> libnwflaim
            -> FLAIM
```

The `nwdirectory` name is the server/binary.  `libnwdirectory` is the library
used by that server.  `libnwds` is the future NetWare Directory Services API
layer above it, used by setup/import tools, future `nwnds`, Bindery integration
and any NetWare-facing directory policy code.

### Implementation order

Do not combine LDAP protocol compatibility, NetWare schema import, setup tooling
and the storage-engine swap.  The first directory implementation phase is
flatfile-only.

1. Keep `libnwdirectory` on the current flatfile/mmap/journal backend.
2. Add LDAPv2 support to the TinyLDAP-derived server first.  TinyLDAP is
   currently treated as the LDAPv3-era baseline; the uploaded OpenLDAP 1.0.3
   tree is the historical LDAPv2 server reference for bind/search/result/BER
   behaviour, not a source tree to import wholesale.
3. Add LDAPv2 CTests over TCP against `nwdirectory` with isolated temporary
   workdirs and free loopback ports.
4. Keep/add LDAPv3 CTests for behaviour that is already present or intentionally
   supported, but do not make LDAPv3 the NetWare 4.11 compatibility baseline.
5. Make NetWare/NDS schema handling work on the flatfile backend.  Prefer native
   `.sch` import if practical; otherwise add a reviewed `.sch` to LDIF
   conversion path.
6. Add `libnwds` and the first `nwsetup` path together.  The first `nwsetup`
   implementation should be console/CLI-only so CTest can exercise initial
   setup without the future `nwtui` frontend.  It should call `libnwds`, and
   `libnwds` should call `libnwdirectory`; it should not bootstrap by acting as
   an LDAP client to `nwdirectory`.
7. Only after LDAPv2, LDAPv3 smoke coverage, schema import, `libnwds` and the
   console `nwsetup` path are stable on flatfile storage, add a FLAIM storage
   backend.
8. After the FLAIM backend is working, add the interactive `nwtui`/`nwtoolbox`
   setup frontend and the certificate/TLS directory-storage work as separate
   follow-up changes.

This keeps LDAPv2 improvements generally useful to the TinyLDAP-derived code,
keeps `nwsetup` testable before the UI exists, and keeps the later FLAIM backend
as a clearly mars-nwe-specific storage change.

### Future storage-backend swap

The later FLAIM backend should be a parallel source set selected by CMake, not a
large rewrite of every LDAP call site.

Example layout:

```text
flatfile backend, default:
  mstorage_add.c
  mstorage_add_bin.c
  mstorage_init.c
  mstorage_init_persistent.c
  mstorage_unmap.c
  mduptab_*.c

flaim backend, later:
  flaimstorage_add.c
  flaimstorage_add_bin.c
  flaimstorage_init.c
  flaimstorage_init_persistent.c
  flaimstorage_unmap.c
  flaimduptab_*.c
```

The FLAIM files may keep the same exported function names as the flatfile files.
For example, `flaimstorage_add_bin.c` can provide `mstorage_add_bin()` when the
FLAIM backend is selected.  Only one backend source set may be compiled into
`libnwdirectory` at a time.

That gives a reviewable storage replacement:

```text
LDAP/server code -> existing mstorage/mduptab API -> selected backend
```

and avoids changing LDAP protocol logic while changing persistence.

### CTest direction

LDAP server tests should not depend on system `ldapsearch` or OpenLDAP tools by
default.  Prefer existing TinyLDAP client/test helpers where suitable, or add a
small in-tree client test.

A standard test should:

1. allocate a unique temporary workdir;
2. prepare test LDIF/schema data for the flatfile backend;
3. start `nwdirectory` on `127.0.0.1:<free-port>`;
4. wait until the TCP port is ready;
5. run LDAPv2 or LDAPv3 bind/search/unbind checks;
6. stop the server;
7. remove the workdir.

Suggested future test names:

```text
nwdirectory.ldap.v2.bind-search-flatfile
nwdirectory.ldap.v3.bind-search-flatfile
nwdirectory.schema.sch-import-flatfile
nwds.setup.console-flatfile
nwdirectory.storage.flatfile.smoke
nwdirectory.storage.flaim.smoke   # later, after the backend swap
```

---

## Admin client and nwConsole

`mars-nweadmin` is useful as a historical workflow reference, but the old
Pascal program must not become the architecture for new administrator clients.
The current source is a Delphi/VCL Windows 3.x style program which depends on a
running NetWare client.  Its bundled NwTP Pascal units call old DOS/Windows
NetWare client mechanisms such as VLM/NETX/DPMI helpers.  That makes the code a
reference for screens and operations, not a portable Linux/macOS/Windows client
library.

The first new admin path is the NetWare 3.x path.  Directory/NDS management is a
later extension, but the design should keep a ConsoleOne-like direction in mind:
one administrator application can grow from bindery/server administration into
directory administration when the directory stack is ready.

### Product names

Use distinct user-facing names for the two frontend families:

- `nwAdmin` is the remote administrator client family.  It may start as a
  console/CTest client and later gain a desktop GUI.
- `nwConsole` is the web administration surface.  Its small web listener process
  may still be named `nwwebui` internally, but the UI, page titles and user
  documentation should call the browser interface `nwConsole`.

The names intentionally separate the remote client from the server-local web
console even when both share the same icon catalog, navigation vocabulary and
admin operation model.

### Client/server boundary

Administrator tools are remote clients.  They must talk to a mars-nwe server
through NCP or later directory protocols instead of linking directly to server
implementation libraries.

The boundary is therefore:

```text
admin GUI or console command
  -> portable admin client library
       -> platform NCP transport adapter
            -> mars-nwe server over NCP/IPX/TCP-backed client stack
```

They must not call local server libraries such as `libnwdirectory`,
`libnwflaim`, or server-private `libnwds` internals to modify a running server's
state.  Those libraries belong on the server/setup side.  The administrator
client library may expose similar high-level operations, but it should perform
those operations by sending NCP requests to the selected server.

### Platform transport plan

The transport layer should be behind an adapter interface so the GUI and common
admin logic do not know which platform backend is used.

Initial direction:

- Linux: prefer the existing ncpfs client library where it can provide the 3.x
  NCP operations needed by the admin client.
- Windows: expect the Novell Client to be the practical first backend.  A
  Windows administrator workstation normally needs that client stack anyway for
  authentic NetWare access, so `nwAdmin` should wrap a Windows Novell Client
  backend through the same transport adapter instead of reviving the old Pascal
  VLM/NWTP layer.  A project-owned NCP backend can remain a later fallback only
  if the native client path is unavailable or too limited.
- Classic/PPC macOS: treat the historical Novell NetWare Client for Macintosh as
  a compatibility reference for old systems where it is available, not as a
  modern portable dependency.
- Modern macOS: plan for either an ncpfs client-library port or a small
  project-owned NCP transport behind the adapter, because the old Macintosh
  client does not solve current macOS targets.

The GUI must call only the portable admin client API.  Platform-specific NCP
attachment, login, request framing, timeouts, reconnect behaviour, and native
client integration belong below that API.

### GUI direction

A cross-platform GUI is allowed, but it must remain a frontend over the portable
admin client library.

Possible frontends:

- Lazarus LCL / FreePascal, if the Pascal route is still wanted.
- Another cross-platform GUI toolkit, if it gives a cleaner long-term result.
- A console/CLI frontend for tests and recovery should exist regardless of the
  GUI toolkit.

If Lazarus LCL is used, treat the old Delphi forms as a workflow/screen reference
only.  Do not try to preserve the old VCL/NwTP implementation as the new runtime
transport layer.

### GUI and web frontend dependency policy

Keep the first client simple.  The portable admin client API and the console/CLI
frontend are more important than a rich desktop framework.  A graphical frontend
can be Lazarus LCL/FreePascal or another cross-platform toolkit, but the selected
toolkit must not pull a large runtime stack into the server or common libraries.

Evaluate heavier Pascal GUI/web stacks only after the 3.x NCP admin API is
proven.  Custom-drawn Lazarus/Delphi libraries such as Cross.Codebot can be
looked at for a later modern desktop UI, and browser-oriented Pascal stacks such
as TMS WEB Core or uniGUI can be compared for a later web UI, but none of them is
a first-pass dependency.  The first-pass rule is: common admin logic stays in the
portable client library, and every frontend remains replaceable.

### Server-local web administration / nwConsole

A server-local web admin path is allowed and may eventually replace the old
SMArT role.  Treat it as a separate frontend, not as the architecture for the
common admin client library.

Preferred shape:

```text
service manager / nwserv supervisor role
  -> starts/supervises nwserv service family
       -> nwserv / ncpserv / nwconn / nwdirectory / nwftp / ...
  -> starts/supervises nwwebui independently when configured
       -> owns HTTP/HTTPS, TLS offload, sessions and static assets only
       -> forwards authenticated web-admin requests to nwadmin
            -> owns web/config plugin dispatch and admin operation routing
            -> calls the portable admin API
                 -> localhost/server NCP transport or explicit server-control IPC
                      -> mars-nwe service being managed
```

In this shape `nwwebui` is deliberately small.  It is the stable web listener
and SSL offloader that can remain alive while server services restart.
`nwadmin` is the management process behind it.  Web modules, configuration
modules and later service-specific extensions belong to `nwadmin`, not to the
TLS/HTTP frontend.  The connection between both processes should be an explicit
local IPC socket/protocol, not the old SMArT-style subprocess stdout/console
scraping model.

`nwwebui` may reuse the existing SMArT-derived web service direction as a
feature reference, but new implementation work should replace local ad-hoc socket,
HTTP and TLS helpers with the project's normal bundled dependencies.  Socket and
buffer/event-loop code should prefer `libowfat` helpers where they fit the
project style.  TLS/HTTPS must go through the project TLS facade backed by
MatrixSSL/`libnwssl`, not through another SSL library or a private web-server TLS
stack.

Login must not remain primarily PAM-based.  For the 3.x path, the first
authentication choice is the mars-nwe server's bindery login state over localhost
NCP or an explicit server-control authentication path.  PAM is only a fallback
for emergency/local host policy when bindery authentication is not available or
is explicitly disabled.  For the later 4.x/directory path, user authentication
should switch to the directory service path through `libnwds` or the equivalent
server-side directory authentication API, with PAM still only a fallback.

Management operations should move behind the same portable admin API used by
console and GUI frontends.  `nwwebui` should forward to the `nwadmin` process,
and `nwadmin` should perform the plugin dispatch and admin API calls.  Neither
process may call local storage libraries such as `libnwdirectory`, `libnwflaim`,
or private storage internals directly to manage a running server.

For the first server-local web UI, prefer simple generated HTML, small static
assets and normal forms.  Avoid making a large single-page-application framework
or a Pascal-to-JavaScript compiler a required dependency for a server admin tool.
A richer browser UI can be added later if the small web UI and admin API prove
insufficient.

`nwwebui` and `nwserv` have separate lifecycle rules depending on who requested
the restart.  A host-level restart of the MARS-NWE service through systemd, an
init script or a package/service manager may restart the whole service family,
including `nwwebui`, because that is an explicit server-side service restart.
A restart requested from the web UI must not restart the web listener itself as a
side effect.  In that case `nwwebui` should keep its HTTP/HTTPS listener alive,
report the selected backend as restarting or temporarily unavailable, and
reconnect or refresh its backend IPC/server session after the backend comes back.
Restarting `nwwebui` should be an explicit service-manager action, not an
implicit side effect of restarting `nwserv`, `nwadmin`, `nwconn`, `nwbind`,
`ncpserv`, `nwdirectory`, `nwftp` or another managed backend from nwConsole.

Because `nwwebui` remains up during web-requested backend restarts, nwConsole
needs embedded static/status pages that do not depend on a live `nwadmin`
backend.  Those pages should use the same nwConsole visual language and be able
to show messages such as `Restarting nwadmin`, `Restarting ncpserv`,
`Restarting nwconn`, `Restarting nwbind`, `Restarting nwdirectory`, `Service
unavailable`, and `Reconnect pending`.

### Shared visual language and plugins

The nwConsole web UI and the nwAdmin desktop/client frontends should share the
same visual vocabulary.  Use an open-source icon set or project-owned icons, but
keep a common icon ID catalog so the desktop UI and web UI show the same symbols
for users, groups, volumes, trustees, server status, restart actions and later
directory objects.  Do not copy proprietary Novell artwork; use an
iManager/ConsoleOne-inspired layout and project-owned assets.

`nwadmin`, not `nwwebui`, owns the plugin system.  A first plugin interface can
be simple and static: each plugin declares an ID, title, icon ID, navigation
section, required server capability, and command handlers for read-only views or
configuration changes.  Later plugins can cover optional services such as NFS,
FTP, WebDAV, directory/schema tools, certificate management or other server
modules.  The same plugin metadata should drive both the web navigation tree and
a future desktop/client navigation tree.

The split keeps dependencies small:

```text
nwwebui
  -> HTTP/HTTPS/TLS/session/static frontend
  -> local IPC socket client/server boundary to nwadmin

nwadmin
  -> plugin registry
  -> web/config module dispatch
  -> portable admin API / server-control client
```

When `nwserv` gains service-management awareness, it may support targeted
restarts such as restarting only `nwadmin`, only `nwconn`, only `ncpserv`, only
`nwbind`, only the directory server, only `nwftp`, or another optional service.
For a host-level systemd/init restart of the MARS-NWE service, restarting the
whole service family including `nwwebui` is allowed.  For a restart requested
through nwConsole, the frontend must keep running unless the operator explicitly
chooses to restart the web frontend itself.  If `nwadmin` or another backend is
restarted, `nwwebui` should keep the listener and sessions alive, reconnect to
the IPC socket when the backend comes back, and show an embedded nwConsole
`restarting` or `backend unavailable` page meanwhile.

### Work order

1. Audit `mars-nweadmin` workflows for the 3.x feature set: users, passwords,
   bindery object operations, server information, connections, messages and
   other admin calls that the old tool exposes.
2. Define a portable admin client API over NCP for the 3.x path.
3. Implement a Linux backend using ncpfs where practical, with the backend hidden
   behind the transport adapter.
4. Add a Windows transport adapter that wraps the installed Novell Client where
   practical, because that is the expected first Windows backend.
5. Audit the old Macintosh NetWare client as a Classic/PPC reference only, then
   decide whether modern macOS needs an ncpfs port or a project-owned NCP
   transport.
6. Add a console/CLI test frontend so CTest can exercise the client API without a
   GUI.
7. Add a simple server-local nwConsole (`nwwebui`) plus `nwadmin` pair only after
   the API is stable enough: `nwwebui` handles HTTP/HTTPS/TLS/session/static
   frontend work, while `nwadmin` owns plugin dispatch and management operations.
8. Add a desktop GUI frontend only after the admin client API and transport
   behaviour are stable and the toolkit choice does not force large dependencies
   into the common/server side.
9. Later, when LDAPv2/LDAPv3, schema import, `libnwds`, console `nwsetup`, and
   the FLAIM-backed directory stack are ready, extend the admin client toward
   directory administration in a ConsoleOne-like direction.

### Non-goals for the first pass

- Do not port the old Delphi program line-for-line to FreePascal as the first
  step.
- Do not make the admin GUI link to local server libraries to manage a remote
  server.
- Do not combine the 3.x admin-client transport work with the later Directory/NDS
  management work.
- Do not make modern macOS depend on the old Classic/PPC Novell Client.  Use it
  only as a compatibility/reference path; modern macOS needs either an ncpfs port
  or a project-owned NCP transport adapter.
- Do not revive the old Pascal VLM/NWTP runtime layer for Windows.  Prefer the
  installed Novell Client backend first, hidden behind the portable transport
  adapter.
- Do not make Cross.Codebot, TMS WEB Core, uniGUI, or any other large GUI/web
  framework a required first-pass dependency.
- Do not merge HTTP/HTTPS/session handling into `nwserv`; keep `nwwebui` or an
  equivalent frontend process responsible for the web surface.
- Do not add new local socket, HTTP or TLS helper stacks to `nwwebui` when the
  project-owned path can use bundled `libowfat` helpers and MatrixSSL/`libnwssl`.
  The first code step may move the existing SMArT-derived listener to libowfat
  socket helpers and `nwlog` while leaving the old Perl subprocess bridge in
  place; the explicit `nwwebui` <-> `nwadmin` IPC socket comes later.
- Do not keep PAM as the primary `nwwebui` login path.  Bindery/server
  authentication is primary for 3.x; later directory authentication through
  `libnwds` or the matching server-side directory API is primary for 4.x.
  `nwwebui` may keep a minimal PAM-only recovery login for the break-glass case
  where `nwadmin` is down and the web frontend must ask `nwserv` to start or
  restart it, but that recovery path must not grow into normal administration.
- Do not make web-requested backend restarts restart `nwwebui` as a side
  effect.  A host-level systemd/init restart of the MARS-NWE service may restart
  the whole service family, including `nwwebui`; nwConsole-requested backend
  restarts must keep the web listener alive unless the operator explicitly
  restarts the web frontend itself.  `nwserv` or a service manager may restart
  backend services such as `nwadmin`, `nwconn`, `ncpserv`, `nwbind`, directory,
  FTP or other optional daemons individually.
- Do not put the web/config plugin system into `nwwebui`.  Keep `nwwebui` as
  the HTTP/HTTPS/TLS/session/static frontend and put plugin dispatch into the
  `nwadmin` backend process.
- Replace the old SMArT-style subprocess/console-output coupling with an
  explicit local IPC socket between `nwwebui` and `nwadmin`, with reconnect and
  backend-unavailable handling in the frontend.