# UCE WASM Runtime Architecture

Status: current as of the W7e native-pipeline removal (June 2026). This document
describes the **runtime architecture as built** — the process topology, the
wasm membrane, the unified request dispatch, and the central WebSocket broker.
Native `.so` unit execution/dlopen fallback has been removed; the parser and
preprocessor remain only as the front-end that emits C++ for wasm side-module
compilation.

The guiding principle: **request code never shares an address space or an
allocator with the runtime.** Every unit runs as a WebAssembly module inside a
per-request workspace behind a narrow host membrane. Long-lived connection
state (WebSocket connections, listening HTTP sockets) lives in dedicated native
broker processes that own no unit code — they hold the connection and forward
the actual unit invocation to a clean-engine worker, exactly like a normal
page render.

---

## 1. Process topology

A single native binary (`uce_fastcgi`) forks into a small set of long-lived
roles. None of them is special-cased per request mode; the differences are
purely *which socket a process listens on* and *which function inside a unit
gets invoked*.

```
                          ┌────────────────────────────┐
        nginx  ──FastCGI──►  worker pool (N processes)  │  /run/uce.sock
   (port 80 etc.)          │  uniform unit renderers    │  (FastCGI + CLI)
                          └─────────────▲──────────────┘
                                        │ forward render (FastCGI, uce.sock)
                                        │
   browser ──raw HTTP / WS──►  ┌────────┴─────────┐
        (HTTP_PORT 8080)       │   WS broker      │  owns HTTP_PORT + every
                               │  (1 process)     │  WS connection; renders
                               └────────▲─────────┘  nothing itself
                                        │ ws_* command flush
                                        │ (FastCGI, ws-broker.sock)
   on-demand serve_http   ──►  ┌────────┴─────────┐
        (per bind addr)        │ custom-server    │  owns one serve_http
                               │ dispatcher(s)    │  bind addr; forwards to pool
                               └──────────────────┘

   parent process: spawns/respawns all of the above + the proactive compiler.
```

| Process | Owns | Renders units? | Source |
|---|---|---|---|
| **Parent** | nothing; supervises children | no | `main()`, `init_base_process()` |
| **Worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH` (`/run/uce.sock`) + `CLI_SOCKET_PATH` | **yes** — the only processes that run wasm | `listen_for_connections()` |
| **WS broker** (×1) | `HTTP_PORT` + every live WS connection + `WS_BROKER_SOCKET_PATH` | no — forwards to the pool | `run_ws_broker()` |
| **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` |
| **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` |

**only workers instantiate Wasmtime and run unit code.**
Every connection-owning process (broker, serve_http dispatcher) forwards the
request invocation back to a worker over `/run/uce.sock` using the minimal
FastCGI client in [`src/lib/fcgi_forward.h`](../src/lib/fcgi_forward.h). This is
forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across
`fork()`, and the brokers fork from the parent that already touched the
runtime. So the brokers hold the long-lived connection and units respond to
events the same way they respond to a page request — through a clean worker.

---

## 2. The membrane and the DValue ABI

Units are compiled to WebAssembly and linked against a host import surface (the
"membrane"). Host and guest exchange structured values as **UCEB** (a compact
binary `DValue` encoding — `ucb_encode`/`ucb_decode`); strings cross as
`std::string` on the native side throughout (no raw `char*` ownership across
the boundary).

- `DValue` is the universal value type: scalars plus an ordered child map
  (`_map`, a `std::map<String,DValue>`). `operator[]` is non-const (creates);
  `.key(k)` is the const probe returning `const DValue*`; `.each(fn)` iterates
  children yielding `const DValue&`.
- The request context (`params`/`get`/`post`/`cookies`/`session`, the raw body
  `in`, and—for WS—the connection context) is marshalled into a single `ctx`
  DValue, UCEB-encoded, and handed to the workspace. The response (body,
  headers, status, and any `meta` such as `ws_commands`) comes back the same
  way.

See [`docs/wasm-phase1-dvalue-abi.md`](wasm-phase1-dvalue-abi.md) for the wire
format details.

---

## 3. Units, handlers, and export naming

A `.uce` unit compiles to a wasm module that exports one symbol per **handler**.
There is no per-mode machinery — a "CLI unit", "WebSocket unit", and "page" are
the same compiled artifact invoked at different exports. The dispatcher passes a
**handler string**; the host maps it to an export symbol:

```
__uce_<base>[_<sanitize(suffix)>]
```

`handler_export_symbol()` (`src/wasm/worker.cpp`) splits on the first `:`:

| Handler string | Export symbol |
|---|---|
| `render` | `__uce_render` |
| `cli` | `__uce_cli` |
| `websocket` | `__uce_websocket` |
| `once` | `__uce_once` (optional; absence is not an error) |
| `serve_http` | `__uce_serve_http` |
| `serve_http:named` | `__uce_serve_http_named` |
| `component:CARD` | `__uce_component_CARD` |
| `exists` | probe only — resolves the unit, loads nothing |

`sanitize_symbol_suffix()` keeps `[A-Za-z0-9_]` (mirrors `ascii_safe_name`).
`wasm_resolve_target(unit, handler)` (`src/wasm/core.cpp`) resolves the source
path and looks up the export's funcref slot; `exists` lets callers probe a unit
without instantiating it.

`wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
backend is initialized and the requested artifact/handler is currently
available. If an artifact is cold or stale, dispatch compiles it on demand via
`get_shared_unit()` and rechecks. There is no native unit-execution fallback.

---

## 4. The workspace runtime

Each request gets a fresh **workspace** — a per-request wasm instance tree with
the membrane wired in. `wasm_worker_serve(worker, ctx, entry_unit, handler)`
(`src/wasm/worker.cpp`) is the single entry point for *every* mode:

1. Birth a workspace (CoW-snapshot-based where available).
2. Resolve `entry_unit` + `handler` to an export; components referenced at
   runtime are resolved on demand via the `uce_host_component_resolve` hostcall
   (`component_resolve()` → `__uce_<...>` slot), loading dependency modules
   lazily and recording resolve counts/timings.
3. Invoke the export. Unit code calls host functions (filesystem, sqlite, regex,
   markdown, time, tasks, `ws_*`, …) across the membrane.
4. Collect the response (`WasmResponse`: body, headers, status, `meta`).

Because the workspace owns no process-lifetime state, a unit crash or trap is
contained: it fails the one request and the worker stays healthy. Workers
suspend the native SIGSEGV/SIGILL recovery handler around the wasm call so that
Wasmtime's own trap signals are not escalated into a native fatal signal (see
`serve_via_wasm` in `handle_complete`).

---

## 5. Request dispatch (`handle_complete`)

`handle_complete()` (`src/linux_fastcgi.cpp`) is the worker's single dispatch
point. It selects a handler string from request params and calls the same
`serve_via_wasm(entry_unit, handler)` lambda for all of them:

```
UCE_WS == "1"            → serve_via_wasm(entry_unit, "websocket")
request.resources.is_cli → serve_via_wasm(entry_unit, "cli")
UCE_SERVE_HTTP == "1"    → serve_via_wasm(entry_unit, "serve_http"[:fn])
otherwise (page)         → serve_via_wasm(entry_unit, "render")
```

The `UCE_*` params are set by whichever broker forwarded the request:

- **Page render**: FastCGI nginx → `/run/uce.sock` directly; no `UCE_*` flags → `render`.
- **CLI**: the CLI socket sets `is_cli`.
- **serve_http**: the custom-server dispatcher sets `UCE_SERVE_HTTP=1` plus
  `UCE_SERVE_HTTP_FUNCTION` and rewrites `SCRIPT_FILENAME` to the configured
  unit (`custom_server_http_complete`).
- **WebSocket**: the WS broker sets `UCE_WS=1` and carries the connection
  identity as `UCE_WS_*` params (see §6). `handle_complete` rebuilds
  `request.resources.websocket_*` and `request.connection` from them before
  invoking `__uce_websocket`.

If a unit still cannot be served after the on-demand wasm compile, the worker
returns a clean 500 with the wasm/compile error; it does not execute native unit
code.

---

## 6. The central WebSocket broker

**One process owns the HTTP port
and every WebSocket connection**, so any unit's `ws_*` call can reach any or all
connections, and a unit-code crash (which happens in a worker) never drops live
connections.

### 6.1 Inbound: a WS frame → a worker render (non-blocking)

`ws_broker_ws_message(request, message, opcode)` fires when a complete
(reassembled) WS message arrives on a connection. It does **not** block the
broker loop:

1. Build FastCGI params: `SCRIPT_FILENAME`, `REQUEST_METHOD=GET`, a
   `REQUEST_URI` (required — `handle_request()` rejects requests without one
   *before* `on_complete` runs), `UCE_WS=1`, and the connection context as
   `UCE_WS_CONNECTION_ID / SCOPE / OPCODE / BINARY / CONNECTIONS / STATE`.
2. The message rides as `UCE_WS_MESSAGE` (base64) with an **empty STDIN body** —
   a non-empty STDIN makes the FastCGI transport flush a premature response
   before `on_complete` ever runs.
3. Connect to `/run/uce.sock` (non-blocking) and queue the encoded request in
   `ws_broker_outbound[fd]`.

`ws_broker_drain_outbound()` runs after every `process(50)` tick: it finishes
writing each queued request, then drains and discards the reply (the unit's
output comes back via the command socket, not this reply), closing the fd when
the worker closes its end.

### 6.2 Outbound: `ws_*` commands flushed back to the broker

Any unit code — not just WebSocket handlers — may call `ws_send` / `ws_send_to`
/ `ws_close`. In the workspace these **record dispatch commands** rather than
touching a socket (the workspace owns no connections); `wasm-core`'s `ws_*`
(`src/lib/sys.cpp`) append to `websocket_dispatch_commands`, and
`finish_response_meta` (`src/wasm/core.cpp`) emits them as `ws_commands` (plus
`ws_connection_state` if the handler mutated per-connection state).

`wasm_backend_serve` (`src/wasm/backend.cpp`) flushes that batch at workspace
teardown — in **any** scenario, not just WS handlers — to the broker's command
socket (`WS_BROKER_SOCKET_PATH`, `/run/uce/ws-broker.sock`) via
`fcgi_forward_request` with `UCE_WS_DISPATCH=1`. This is the only path WS data
takes out of a workspace.

`ws_broker_apply_commands()` decodes the batch and applies each command against
the full registry it owns: `broadcast` (by scope), `send_to` (by connection id),
`close`. If the batch carries `connection_state`, it persists that onto the
matching live connection's `websocket_state`.

### 6.3 Un-upgraded HTTP on the WS port

The WS port can also receive ordinary (non-Upgrade) HTTP requests.
`ws_broker_complete()` routes by param: `UCE_WS_DISPATCH=1` → apply commands;
otherwise → `forward_request_to_worker()` — the *same* shared facility the
serve_http dispatcher uses, so there is no duplicated request-forwarding code.

### 6.4 The broker loop

`run_ws_broker()` drops the worker listeners it inherited
(`close_inherited_server_sockets`), installs permissive `on_request`/`on_data`
handlers (it renders nothing, so it accepts every request straight through to
`on_complete`), wires `on_complete=ws_broker_complete` and
`on_websocket_message=ws_broker_ws_message`, listens on `HTTP_PORT` and the
command socket, and loops `process(50)` + `drain_outbound()`. The design is
**non-blocking outbound dispatch + async command-socket flush, all in the
broker's single epoll loop** — the broker never blocks on a worker.

The parent respawns the broker if it dies (`ws_broker_alive` / `ensure_ws_broker`
in `main()`). 

---

## 7. The serve_http facility

`serve_http` units are reachable on their own bind address via a custom-server
dispatcher (`custom_server_http_dispatcher_loop`). The dispatcher owns the bind
socket and, on each request, sets `UCE_SERVE_HTTP=1` + the configured unit/
function and calls `forward_request_to_worker()`. The worker then runs
`serve_via_wasm(entry_unit, "serve_http"[:fn])`. This is the same
hold-connection-forward-render model as the WS broker, sharing
`forward_request_to_worker` and `fcgi_forward.h`.

---

## 8. Build / object layout

The native side is split into separately-compiled objects so editing the wasm
runtime does not recompile the whole TU (`scripts/build_linux.sh`):

| Object | Contents |
|---|---|
| `bin/sqlite3.o` | sqlite amalgamation (cached) |
| `bin/wasm.o` | `src/wasm/wasm_module.cpp` (backend.cpp + worker.cpp + wasmtime) |
| `bin/main.o` | `src/linux_fastcgi.cpp` (includes `lib/uce_lib.cpp`) |

Linked into one `-rdynamic` binary. ODR hazards across the objects are handled
deliberately: `context` and `my_pid`/`parent_pid` are `extern` (guarded for the
wasm core vs. unit builds), `operator new`/`delete` live in `types.cpp`, and
header free-functions are `inline`. The wasm backend exposes only declarations
(`src/wasm/backend.h`) to `main.o`.

---

## 9. Configuration keys

| Key | Default | Meaning |
|---|---|---|
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
| `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). |
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. |
| `HTTP_PORT` | `8080` | Raw HTTP + WebSocket port — owned by the WS broker. |
| `WS_BROKER_SOCKET_PATH` | `/run/uce/ws-broker.sock` | Broker command socket for `ws_*` flushes. |
| `WORKER_COUNT` | `4` | Number of uniform worker processes. |

---

## 10. Testing

- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
  in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages.

- **WebSocket end-to-end**: a headless client performs a raw WS handshake to
  `:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
  `SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
  broker → worker → broker → client chain across process boundaries.


