uce/WASM-PROPOSAL.md

# WASM-PROPOSAL: WebAssembly Unit Runtime for UCE

- **Status:** proposal / design draft (2026-06-11)
- **Scope:** replace the native unit pipeline (generated C++ → clang → `.so` →
  `dlopen`) with per-unit WebAssembly modules executed in a per-request,
  runtime-linked workspace, exposing the same API surface to page code.
- **Origin:** design discussion 2026-06-11; incorporates the post-mortem of the
  earlier per-invocation arena attempt (preserved in
  `src/lib/_scratchpad.cpp`).

---

## 1. Motivation

Two long-standing structural problems and one strategic opportunity share a
single root cause: **request code shares an address space and an allocator
with the runtime.**

1. **The arena attempt failed for a structural reason.** The
   `GLOBAL_ARENA_ALLOCATOR` design in `_scratchpad.cpp` swapped the global
   `operator new`/`delete` against a `current_memory_arena`. A single global
   allocator cannot distinguish request-lifetime allocations from
   process-lifetime ones: during a request, server-lifetime structures
   (compile registry, sessions, config trees, unit statics) also allocate.
   Under the arena those dangle on reset; with `delete` as a no-op,
   system-allocated objects released mid-request leak. The lifetime
   distinction lives in the type system and call graph, not at the allocator
   boundary — fixing it in-process means threading PMR allocators through
   `String`, `DTree`, and every container.

2. **Fault recovery is best-effort, not sound.** The current
   SIGSEGV → `sigsetjmp`/`siglongjmp` recovery performs no unwinding, skips
   destructors, can resume over corrupted worker state, and (see
   RECOMMENDATIONS.md 1.5) cannot reliably produce a useful backtrace.
   A faulting unit *can* have scribbled on runtime memory before the signal.

3. **UCE can only ever host fully-trusted code.** A `.uce` page is arbitrary
   native code with full process privileges. Multi-tenant or user-supplied
   page hosting is structurally impossible in the native model.

A WASM execution model deletes the shared-fate fact itself rather than
patching around it:

- **Arena by construction:** each request runs in its own linear memory,
  dropped wholesale at request end. Request-lifetime memory physically cannot
  outlive the request; server-lifetime state physically cannot live inside
  it. The `_scratchpad.cpp` design becomes correct because the boundary is
  structural, not typological.
- **Sound recovery:** a trap (null deref, OOB, stack exhaustion) is a defined
  host-side error that unwinds cleanly, with a precise guest stack trace, and
  the unit cannot have touched host memory. Render error page, drop
  workspace, keep serving — actually correct, not hopeful.
- **Capability security:** page code gets exactly the imported API surface
  and nothing else. Multi-tenant hosting becomes possible.
- **Secondary wins:** architecture-independent unit artifacts (no clang
  required on prod), safe module unload/replace (vs. never-safe `dlclose`),
  first-class limits (linear-memory cap = RAM limit, epoch/fuel = CPU
  timeout), per-request memory stats for free.

**Accepted costs:** ~1.2–2× compute slowdown vs. native (still far ahead of
interpreted runtimes); a real ABI/membrane design; ownership of a custom
loader; toolchain rough edges (§10).

---

## 2. Rejected alternatives (recorded so they stay rejected)

- **In-process PMR arena.** Requires re-typedefing `String`/`DTree` and
  threading allocators through the entire codebase; the failed global-swap
  shortcut is the only cheap version and it is unsound (§1.1).
- **One instance per component.** UCE components are function calls, not
  RPCs: callees receive the context **by reference**, mutate `context.call`,
  share the `ob_*` capture stack and `ONCE` dedup state. Per-component
  instances force serialize/copy/deserialize of the context on every
  `component()` call (a dozen+ per page in the starter), require
  host-mediation of the ob stack, and silently change reference semantics to
  copy semantics. Rejected.
- **One linked module per app ("the blob").** Introduces an "app" concept
  UCE does not have, makes every edit a global relink, turns the artifact
  cache into a build graph — webpack's worst traits without its benefits.
  Rejected. The file stays the unit.
- **Eager pre-loading of known units at worker warm-up.** Rejected; loading
  is strictly lazy, on first explicit call, preserving current semantics.

---

## 3. Architecture overview

### 3.1 Execution model

```
host process (linux_fastcgi, per worker)
│
├─ vendored wasm runtime (§10.1)
├─ unit artifact cache: one PIC .wasm per unit (replaces per-unit .so)
├─ loader (§6): dylink parsing, base allocation, GOT resolution,
│   symbol registry, ABI stamp check
├─ core snapshot: "core module, initialized" memory+table image
│
└─ per request: WORKSPACE
   ├─ linear memory (CoW-born from core snapshot)
   ├─ shared funcref table
   ├─ core module instance  (uce_lib + libc compiled to wasm)
   ├─ unit module instances (loaded lazily, on first call, incl. mid-request)
   └─ host handle table     (sqlite/mysql/file/socket handles + closers)
```

- **Workspace = request.** Born from the core snapshot, dropped at request
  end. Memory drop is the arena; handle-table drop is resource cleanup
  (this generalizes and replaces the per-connector
  `cleanup_*_connections()` pattern — RECOMMENDATIONS.md 1.7 / 5.1 become
  structurally unrepresentable).
- **One unit = one PIC wasm module.** Compile, cache, and invalidation
  granularity stay per-file. The `.uce → C++` translation pipeline is
  unchanged; only the compile target changes
  (`clang --target=wasm32-wasi -fPIC` + `wasm-ld -shared`).
- **Strictly lazy loading.** A unit's module is instantiated into a
  workspace the first time that workspace calls it — including mid-request.
  This is the wasm equivalent of today's compile-and-`dlopen`-on-first-hit
  and requires no restart, no fallback path. Placement memoization
  (deterministic bases so repeat instantiation is cheaper) is a permitted
  optimization; it must not change the loading policy.
- **In-flight isolation.** Module versions are immutable; a recompiled unit
  becomes a new module. Running workspaces keep what they loaded; new
  workspaces get the new version. Old modules are dropped when unreferenced
  (safe unload — impossible with `dlclose`).

### 3.2 Memory model

- **One heap, one allocator, one DTree implementation** — all owned by the
  core module. Unit modules *import* `malloc`/`free`/runtime symbols via GOT;
  the loader **rejects any unit module that defines rather than imports
  them** (two allocators on one heap is the one fatal misconfiguration).
- **Bump allocator option.** Because the workspace heap is dropped wholesale,
  the core's allocator may be a bump allocator with no-op free — the
  `_scratchpad.cpp` design, now correct by construction. Per-deployment
  flag; fallback is wasi-libc dlmalloc. Memory stats = heap pointer − base
  (replaces the tracking `operator new` in `types.h`).
- **Unit statics reset per request** (workspace is born from the core
  snapshot, which does not include unit data; unit data segments initialize
  at unit load within the workspace). This is *more* shared-nothing than
  today, where `.so` statics persist across requests within a worker.
  Cross-request state must use explicit host facilities (sessions, caches).
  **Breaking change — must be called out in docs and checked against the
  site/ tree during Phase 5.**

### 3.3 The host membrane

Exactly three currencies cross between host and workspace:

1. **scalars** (i32/i64/f64),
2. **byte buffers** (`ptr+len` into linear memory; inbound buffers are
   placed via the core's exported allocator),
3. **handles** (opaque `u32` indices into the per-workspace host handle
   table; each entry carries a closer callback).

Everything pointer-shaped stays on its own side. Hostcall surface budget:
30–60 functions (§5.1). Host errors return as error values; traps are
reserved for unit faults. Nothing throws across the membrane.

**DTrees cross the membrane only as the versioned wire encoding** (§5.3),
at exactly three sites: request context in (once), response out (once),
bulk I/O results in (per query, optional vs. cursor-style hostcalls).

### 3.4 DTree inside the workspace: no serialization, ever

Within a workspace, all modules share one address space, one toolchain, one
set of headers — the C/C++ ABI is intact across module boundaries. Therefore:

- A `DTree` is a pointer (an i32 offset into linear memory).
- `component(path, context)` resolves path → table index (host registry or
  guest-resident map) and `call_indirect`s, passing the context pointer.
  Reference semantics, mutation visibility, shared ob stack, working `ONCE`
  — identical to today.
- Function pointers are shared-table indices, valid across modules: virtual
  calls, `std::function` callbacks (`dtree_map` lambdas) work across units.
- Cost: one `call_indirect` (single-digit ns) + GOT loads for cross-module
  symbols — the same shape of overhead native PIC pays through the PLT/GOT,
  i.e. what dlopened `.so` units pay today.

Encode/decode is **not** part of internal component calls. It exists only at
the membrane (§3.3) and on the cross-instance plane (§4).

### 3.5 The DTree C ABI (load-bearing, build it first)

The stable contract of the workspace is a **C ABI**, not the C++ class:
the core exports `extern "C"` accessors over an opaque `uce_dtree*` (§5.2),
plus the string/ob/print helpers. C++ units may bypass it and use the class
directly (same headers, zero cost — a private fast path). Every other
workspace language uses the C surface.

This ABI is versioned: every unit artifact carries a custom section
(`uce.abi`: core ABI version + toolchain fingerprint); the loader refuses
stale units and triggers lazy recompilation (units are lazily compiled
anyway, so this costs nothing structurally).

**Phase 1 of the implementation plan is to introduce this C ABI in the
current native runtime** — it is useful immediately (plugin surface,
testability) and de-risks the rest.

---

## 4. Two component-call planes / multi-language support

The design stratifies languages by one question: *can the toolchain produce a
PIC linear-memory module that adopts a foreign allocator?*

**Plane A — workspace peers** (C++, C, Rust, Zig, …):
- Join the workspace as PIC modules importing core symbols.
- Must adopt the core allocator (Rust: `#[global_allocator]`; Zig: allocator
  parameter) and must not unwind across boundaries (`panic=abort` /
  catch-at-edge).
- Access DTrees through the C ABI: pointer semantics, no copies, ns-scale
  calls. Idiomatic wrappers per language (e.g. Rust `DTree<'request>` —
  the borrow checker enforces the arena invariant).

**Plane B — runtime-carrying languages** (JS, Python, Go, C#, …):
- Their GC/runtime owns its memory; they run as **separate instances**
  within the request and communicate through the host using the wire
  encoding and handles. Bindings choose per-access hostcalls or bulk
  subtree hydration into native dicts/objects — a tuning decision, not an
  architectural fork.

**The semantic rule (enforced by the loader, not by convention):**
cross-plane components do **not** receive the mutable context. They get an
explicit interface — props in (copied by definition), rendered output and
declared results back. Plane A keeps the full "here's the world, mutate it"
contract. A `component()` call must never silently change mutation semantics
based on the callee's implementation language.

The cross-instance call mechanism is shared by: Plane B units, and future
cross-trust-boundary components (multi-tenant). Serialization boundaries and
isolation boundaries are the same lines, by design.

---

## 5. ABI sketches (to be finalized in Phase 0/1)

### 5.1 Hostcall surface (grouped; target ≤ 60 functions)

```
request:   uce_host_ctx_read(buf) → len          // wire-encoded context, once
response:  uce_host_respond(status, hdrs_buf, body_buf)
           uce_host_stream_write(buf)            // chunked/streaming path
log:       uce_host_log(level, buf)
sqlite:    uce_host_sqlite_connect(path_buf) → handle | err
           uce_host_sqlite_query(handle, sql_buf, params_buf) → result_buf | err
           uce_host_sqlite_cursor_*(...)         // optional row-cursor variant
           uce_host_sqlite_insert_id/affected/error/disconnect(handle)
mysql:     (same shape; existing connector APIs are already handle-shaped)
files:     uce_host_file_read/write/stat/list(path_buf, ...)   // policy-gated
session:   uce_host_session_get/set(key_buf, val_buf)
http:      uce_host_http_request(req_buf) → handle/result_buf  // outbound
misc:      uce_host_time(), uce_host_random(buf), uce_host_env(key_buf)
loader:    uce_host_component_resolve(path_buf) → table_index  // may load (§6)
ws:        uce_host_ws_send(buf), event delivery via render entry re-invocation
```

Conventions: all errors as result codes + `uce_host_last_error(buf)`;
inbound buffers placed via the core's exported `uce_alloc`; no hostcall
traps on bad input (clamp/error instead).

### 5.2 DTree C ABI (core exports; sketch)

```c
typedef struct uce_dtree uce_dtree;                 // opaque; workspace-owned

uce_dtree*  uce_dtree_root(void);                   // request context
uce_dtree*  uce_dtree_get(uce_dtree*, const char* key, size_t klen); // create-on-write
uce_dtree*  uce_dtree_find(uce_dtree*, const char* key, size_t klen); // NULL if absent
const char* uce_dtree_value(uce_dtree*, size_t* len);
void        uce_dtree_set_value(uce_dtree*, const char* v, size_t vlen);
size_t      uce_dtree_count(uce_dtree*);
int         uce_dtree_is_list(uce_dtree*);
/* iteration */
uce_dtree_iter uce_dtree_iter_begin(uce_dtree*);
int         uce_dtree_iter_next(uce_dtree*, uce_dtree_iter*,
                                const char** key, size_t* klen, uce_dtree** child);
/* encode/decode at the membrane */
size_t      uce_dtree_encode(uce_dtree*, char* buf, size_t cap);   // → UCEB1
uce_dtree*  uce_dtree_decode(const char* buf, size_t len);
/* ob / print / helpers: uce_print, uce_ob_start, uce_ob_get_close,
   uce_html_escape, uce_json_encode, ... (mirror uce_lib surface) */
```

No unwinding across this surface; C++ exceptions are caught at the edge and
surfaced as error returns where fallible.

### 5.3 Wire encoding "UCEB1" (membrane + cross-instance only)

Length-prefixed binary tree; **not** JSON. Sketch (finalize against DTree's
actual fields — value + ordered children):

```
node  := value children
value := varint len, bytes (utf-8)
children := varint count, count × ( key: varint len + bytes, node )
flags := one leading byte per node reserved (bit0: is_list hint)
header := "UCEB" u8 version
```

This encoding is a **versioned protocol** from day one (header byte). It is
the Plane B contract and the membrane format; internal calls never see it.

### 5.4 Unit module contract

```
custom sections:  dylink.0 (standard), uce.abi { abi_version, toolchain_id }
imports:          env.memory, env.__indirect_function_table,
                  env.__memory_base, env.__table_base,
                  GOT.mem.* / GOT.func.* (resolved by loader),
                  core symbols (malloc, uce_dtree_*, uce_print, ...)
exports:          uce_unit_setup, uce_unit_render,
                  uce_unit_component, uce_unit_websocket
                  (same roles as today's UCE_SETUP/RENDER/COMPONENT/WEBSOCKET
                   dlsym symbols in compiler.cpp)
forbidden:        defining malloc/free/operator new, own memory, start fn
                  with side effects beyond data init
```

---

## 6. The loader (host-side, custom, load-bearing)

Owned code, ~1–2k lines, vendored-runtime-adjacent. Reference logic:
Emscripten's dylink loader (the ABI is the stable, battle-tested part; the
server-side loader is what doesn't exist off the shelf).

Per `load(unit)` into a workspace:

1. Fetch compiled module from artifact cache (compile on miss — today's
   lazy-compile path, retargeted).
2. Verify `uce.abi` stamp against the core; on mismatch, recompile unit.
3. Verify import discipline (no allocator/runtime definitions; §3.2).
4. Parse `dylink.0`: data size/alignment, table slots needed.
5. Allocate `__memory_base` (bump within workspace data region) and
   `__table_base` (append to shared table).
6. Instantiate with bases; resolve `GOT.*` imports against the workspace
   symbol registry (core symbols + previously loaded units); register the
   unit's exports.
7. Register entry points in the path → table-index dispatch map.

`uce_host_component_resolve(path)` consults the dispatch map and calls
`load()` on miss — this is how lazy, programmatic, mid-request loading works
with no special cases.

Placement memoization (optional, later): record each unit's first-assigned
bases; reuse across workspaces so instantiation is cheaper and snapshot
growth (below) stays consistent. Does not change the lazy policy.

**Core snapshot:** the only pre-built state is "core module, initialized" —
memory bytes + table state captured once per core build. Workspaces are born
from it via CoW (`mmap(MAP_PRIVATE)` of the snapshot image; the host owns
the Memory object, so OS-level CoW is available). No units are pre-fed.

---

## 7. Request lifecycle (replaces the native flow in linux_fastcgi.cpp)

```
1. accept request
2. workspace = birth_from_core_snapshot()          (CoW, ~µs)
3. write wire-encoded context into workspace; core decodes → context DTree
4. resolve entry unit (load on first call); call uce_unit_render(ctx_ptr)
5. component(path) inside guest → resolve hostcall → (lazy load) →
   call_indirect — reference semantics throughout
6. I/O via hostcalls; resources land in the workspace handle table
7. on return: encode response/headers out; write FastCGI response
   on trap: defined error → render error page with guest stack trace;
   workspace state is irrelevant because…
8. drop workspace: linear memory gone (arena), handle table closed
   (generalized resource cleanup), instances released
```

CPU limit: epoch/fuel interruption → same path as trap.
Memory limit: linear memory max → allocation failure / trap → same path.

---

## 8. What carries over unchanged

- The `.uce → C++` translation, parser, and page semantics.
- The lazy compile-on-first-request model and per-unit artifact caching
  (different artifact format).
- The host-side connectors (sqlite/mysql) — already handle-shaped APIs; they
  move behind hostcalls with the same `.uce`-visible signatures.
- The site tree, docs, demo, and the network test suite
  (`tests/run_network_tests.py`) — which becomes the parity harness (§9).
- nginx/FastCGI front-end integration, worker model, websocket event flow
  (events re-enter via the websocket entry point; note: a workspace-per-event
  or workspace-per-connection decision is an open question, §11).

---

## 9. Implementation plan

Phases are sequential; each has an exit criterion. No phase except 0 and 2's
scaffolding produces throwaway work.

**Phase 0 — toolchain & runtime spike (timeboxed).**
Validate: wasi-sdk `-fPIC` + `wasm-ld -shared` on a representative generated
unit; cross-module C++ calls with shared memory/table; exceptions decision
(wasm EH vs. error-code discipline at unit boundaries — pick one, record it);
vendored runtime selection. Candidates: **WAMR** (C, small, designed for
embedding, easiest to vendor and patch — fits the project's vendoring
practice) vs. **Wasmtime** (fastest, best AOT/CoW machinery, Rust — heavier
to vendor/patch). Selection criteria: imported-memory + shared-table support,
AOT artifact quality, patchability. Exit: a two-module (core stub + unit
stub) hello-world linked at runtime by a minimal loader, in the chosen
vendored runtime.

**Phase 1 — DTree C ABI in the native runtime.**
Introduce `uce_dtree_*` (§5.2) and the UCEB1 codec in `src/lib/`, used
natively. Zero wasm dependency; immediately testable; freezes the contract
everything else builds on. Exit: codec round-trip + accessor tests in the
existing suite; ABI doc checked in.

**Phase 2 — core module + membrane.**
Compile `uce_lib` (+ wasi-libc) to wasm as the core module; implement the
hostcall surface (§5.1) in the host; temporary scaffolding allowed: one
statically-linked unit + core to validate codegen and membrane without the
loader. Exit: one real `.uce` page (e.g. `site/tests/core.uce`) renders
correctly through the membrane. Scaffolding is marked throwaway.

**Phase 3 — the loader + workspace.**
Implement §6 in full: dylink parsing, base allocation, GOT resolution,
ABI/import verification, lazy mid-request loading, path dispatch. Per-request
workspace birth/drop (plain memcpy birth is fine here; CoW is Phase 4).
Exit: the uce-starter renders end-to-end with components loading lazily;
`tests/run_network_tests.py --match starter` passes against the wasm worker.

**Phase 4 — production mechanics.**
Core snapshot + CoW birth; bump-allocator flag; epoch/memory limits; trap →
error-page path with guest stack traces (this supersedes the
signal/longjmp machinery and closes RECOMMENDATIONS.md 1.5 structurally);
handle-table cleanup (closes 1.7/5.1); artifact/ABI versioning end-to-end.
Exit: kill-tests (deliberate null-deref page, infinite-loop page, OOM page)
produce clean error pages and an unharmed worker.

**Phase 5 — parity & performance.**
Full network suite green on the wasm worker; differential native-vs-wasm runs
on the site tree; audit `site/` for cross-request-static reliance (§3.2
breaking change); benchmark suite (template-heavy page, sqlite page,
component-heavy starter page) with budgets: ≤2× native page latency,
workspace birth ≤100µs, internal component call overhead within 10× native
call cost. Exit: numbers published in this document; go/no-go for default
backend.

**Phase 6 — second plane (deferred until wanted).**
Cross-instance call mechanism (props-in/output-out, UCEB1), first Plane B
language binding, loader enforcement of the cross-plane context rule (§4).
Plane A second language (Rust) as the cheaper first polyglot proof.

The native `.so` backend remains in-tree and selectable until Phase 5's
go/no-go; both backends share the Phase 1 C ABI.

---

## 10. Risks & mitigations

| Risk | Mitigation |
|---|---|
| wasi-sdk PIC / shared-library maturity (least-trodden toolchain path) | Phase 0 spike before any commitment; pin toolchain versions; statically link libc into the core and export from there (avoid shared wasi-libc entirely) |
| C++ exceptions × PIC × wasm EH | Phase 0 decision point; fallback is error-code discipline at unit entry points (units already have a uniform entry shape) |
| Custom loader correctness (GOT, bases, relocation) | Small, contained (~1–2k lines); crib logic from Emscripten's reference loader; fuzz with adversarial modules; loader rejects > loader guesses |
| Vendored runtime patches drift from upstream | Same practice as vendored SQLite: provenance + patch files under `docs/patches/`; pin upstream tag; tests gate upgrades |
| Performance regression beyond budget | Phase 5 gates; bump allocator and placement memoization in reserve; native backend retained |
| Multi-module DWARF / debugging story | Trap stack traces cover the production case (better than today); accept weaker interactive debugging; keep native backend for local deep-debugging |
| Unit-statics semantic change breaks existing pages | Phase 5 audit of `site/`; documented migration note; host-side cache facility if a real need surfaces |

## 11. Open questions

1. **Exceptions:** wasm EH or error codes at unit boundaries? (Phase 0.)
2. **WebSocket granularity:** workspace per event (pure arena, statics reset
   per event) or per connection (state across events, bounded lifetime)?
   Leaning per-event for consistency; needs a look at current WS page usage.
3. **UCEB1 final layout** vs. DTree's actual field set (value/children/list
   flag) — finalize in Phase 1.
4. **Cursor vs. bulk** as the default for `sqlite_query` results at the
   membrane (offer both; pick the default after Phase 5 benchmarks).
5. **Streaming output:** today's ob model buffers; does the membrane expose
   `uce_host_stream_write` from day one or post-MVP?
6. **Core snapshot rebuild cadence** once placement memoization lands
   (dead-base reclamation policy).

## 12. Summary

The module is the **unit** (file-grained, lazily compiled, lazily loaded —
including mid-request). The instance set is the **workspace** (one per
request; shared memory + table; born from a core-only CoW snapshot; dropped
wholesale — the arena, done right). The contract is the **DTree C ABI**
inside the workspace (pointer semantics, no serialization) and the **UCEB1
wire encoding + handles** at every true address-space boundary (host
membrane, Plane B languages, future trust boundaries). Serialization
boundaries and isolation boundaries are the same lines; component calls stay
function calls; and the file stays the unit.