Compare commits

..

5 Commits

Author SHA1 Message Date
root
bfd6d33829 W7f: sweep dead/legacy/fallback leftovers after native-pipeline removal
Post-deletion cleanup (units run only on wasm):
- types.h/compiler.cpp: drop the native-era SharedUnit fields so_name,
  bin_file_name, and the opt_so_optional cache-mode plumbing (no native
  optional .so path remains). The per-unit compile lock is re-keyed from
  so_name+.lock to wasm_name+.lock (still per-unit).
- unit_info() and to_string(SharedUnit*) no longer expose .so artifact fields.
- backend.h: drop the stale "+ fallback-token gate" comment.
- Docs/comments corrected to wasm-only reality: README, tests/README,
  site/doc C++ preprocessor + error_pages + unit_info pages, site/info intro,
  site/demo/unit-browser artifact card; the Phase-5 native-vs-wasm benchmark
  harness (tests/wasm_benchmark.py) reframed for the wasm-only backend.

Audit confirmed no live references remain to so_handle, load_shared_unit,
compiler_load_shared_unit, compiler_invoke*/_cli/_websocket/_serve_http,
COMPILE_SCRIPT/COMPILE_WASM_UNITS, or the native export-symbol constants;
request_ref_handler/dv_call_handler are kept (live wasm funcref casts).

Swept via the pi agent (delegated to a gpt-5.3-codex-spark sub-model);
independently re-verified on the host: run_cli_tests --include-wasm-kill =>
87 passed, 0 failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:43:16 +00:00
root
cf51336873 W7e: delete the native unit pipeline (.so compile + dlopen execution)
Units now run exclusively on wasm; the native generated-C++ -> clang -> .so ->
dlopen path and its request-time fallback are removed.

- Dispatch (linux_fastcgi.cpp): the 4 handle_complete branches + the CLI-socket
  path route every request through wasm (wasm_ready compiles cold/stale on
  demand); a wasm-unavailable unit now yields a clean error page
  (fail_wasm_unavailable / render_request_failure) instead of native execution.
  compiler_invoke / _cli / _websocket / _serve_http deleted.
- compiler.cpp (-1274): removed the native .so compile (COMPILE_SCRIPT),
  load_shared_unit, dlopen/dlsym/dlclose, compiler_load_shared_unit, and the
  SharedUnit .so function-pointer fields (on_setup/on_render/on_component/
  on_websocket/on_cli/on_once/on_init) in types.h/types.cpp. compile_shared_unit
  now builds only the .wasm side-module; the .uce preprocessor/parser front-end
  is kept (it emits the C++ the wasm compile consumes).
- unit_call()/component()/once/init now resolve across units through the wasm
  host component resolver (uce_host_component_resolve) instead of native dlsym;
  configured runtime error pages render through the wasm backend.
- Dropped the WASM_BACKEND_ENABLED feature flag and dead COMPILE_SCRIPT /
  COMPILE_WASM_UNITS config; unit ABI freshness tied to UCE_UNIT_ABI_VERSION;
  guard against serving a stale .wasm for a deleted source; retired the obsolete
  W5 native-vs-wasm toggle script. Docs updated.

Implemented via the pi agent (with 3 delegated sub-reviews); independently
re-verified on the build host: run_cli_tests --include-wasm-kill => 87 passed,
0 failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:20:50 +00:00
root
fb728d63bc W7e stage B: factor the .wasm artifact into unit compile-freshness
inspect_shared_unit_filesystem() tracked only the .so mtime, so a missing or
stale .wasm with a current .so never triggered a rebuild and the unit fell to
native indefinitely (the in-process cache also never invalidated). Account for
su->wasm_name when wasm unit compilation is enabled: a unit counts as compiled
only as of the OLDER of the two artifacts; a missing .wasm forces a recompile.
Closes the cached-vanished-.wasm gap noted in the stage A review.

Verified independently on the build host: run_cli_tests --include-wasm-kill =>
87 passed, 0 failed; delete-.wasm-then-request rebuilds the artifact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:50:31 +00:00
root
2debd33804 W7e: wasm-preferred dispatch + compile-on-demand; harden unit ABI validator
- Dispatch (linux_fastcgi.cpp): route every request through wasm. On a
  cold/stale artifact, compile the unit on demand (get_shared_unit, forced) and
  serve wasm; native compiler_invoke* remains only as a fallback when wasm
  cannot be made ready (compile failure / backend disabled). Applies to the 4
  handle_complete branches and the CLI socket path.
- backend.cpp: delete the now-vestigial native-only fallback token gate
  (wasm_backend_native_fallback_*), empty since W7d; should_handle now gates on
  config + current artifact + healthy worker only.
- check_unit_wasm.py: skip the defense-in-depth llvm-nm allocator scan when
  llvm-nm SIGSEGVs on a degenerate-but-valid module (e.g. a unit with no
  exported handlers). Fixes site/demo/empty.uce, the last unit that could not
  produce a .wasm; forbidden allocator *exports* are still rejected.

A pi-assisted review caught that the compile-freshness check keys off the .so
mtime only, so force_recompile is required to rebuild a missing/stale .wasm; a
unit already cached in-process whose .wasm later vanished still uses native
fallback (closed in W7e stage B). Native execution is otherwise bypassed for
all real traffic.

Verified: scripts/run_cli_tests.sh --include-wasm-kill -> 87 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:20:12 +00:00
root
8587fbc5aa wasm runtime: central WS broker, unified handlers, W7d holdouts, membrane completeness
- WS: a dedicated broker process owns HTTP_PORT + every connection; it forwards
  renders to the worker pool over uce.sock (non-blocking) and applies ws_*
  command batches flushed back at workspace teardown. Removes the now-dead
  per-worker websocket executor (-509 lines).
- Dispatch: unify CLI / WebSocket / serve_http / page render through one
  serve_via_wasm(entry_unit, handler) path; handler string -> __uce_<handler>
  export symbol.
- W7d: rewrite zip.uce to the membrane return-value error contract (no C++
  try/catch), error-reporting.uce to genuine wasm traps instead of throw, and
  sharedunit.uce to unit_info(); empty the native-only token gate.
- Membrane: wire ls / mkdir / file_mtime through new uce_host_file_list /
  uce_host_file_mkdir / uce_host_file_mtime hostcalls (resolve_guest_file gains
  directory support). Fixes /doc/index.uce listing nothing; adds a regression
  assertion that the index enumerates items.
- Docs: add docs/wasm-runtime-architecture.md; record the W7e staged native-
  deletion plan in WASM-PROPOSAL.md.

Verified: scripts/run_cli_tests.sh --include-wasm-kill -> 87 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 17:51:51 +00:00
37 changed files with 1731 additions and 2348 deletions

View File

@ -8,7 +8,7 @@ This is in the early stages of development. Don't use this for anything importan
UCE is a PHP-inspired server-side runtime that lets you build web pages and handlers in C++ using a small `.uce` preprocessor plus a FastCGI application server.
- `.uce` pages compile to shared objects on demand
- `.uce` pages compile to WebAssembly side modules on demand
- normal HTTP pages expose `RENDER(Request& context)`
- WebSocket pages can additionally expose `WS(Request& context)`
- local CLI/admin/test entrypoints can expose `CLI(Request& context)` and are invoked through the Unix CLI socket
@ -103,7 +103,7 @@ Those are intended for sub-rendering through helpers such as `component("compone
Additional lifecycle hooks are also available on ordinary `.uce` units:
- `INIT(Request& context)` runs once when a worker loads that unit's shared object into memory
- `INIT(Request& context)` runs once when a worker instantiates that unit's wasm module
- `ONCE(Request& context)` runs once per request before the first `RENDER()`, `CLI()`, or `COMPONENT...` entrypoint from that file
CLI units can be invoked locally with the convenience wrapper or directly over HTTP-over-Unix:
@ -330,7 +330,7 @@ Proactive compilation settings:
- `SITE_DIRECTORY=site` tells the runtime which tree to scan on startup for `.uce` files when `PRECOMPILE_FILES_IN` is left empty.
- `PRECOMPILE_FILES_IN=` can override that startup scan root with a different absolute or runtime-relative directory.
- `PROACTIVE_COMPILE_CHECK_INTERVAL=60` controls how often the low-priority background compiler rechecks known `.uce` files for stale or missing shared objects.
- `PROACTIVE_COMPILE_CHECK_INTERVAL=60` controls how often the low-priority background compiler rechecks known `.uce` files for stale or missing wasm modules.
The runtime keeps a shared known-file registry under `BIN_DIRECTORY` and updates it as request handling discovers new `.uce` files, so proactive recompiles are not limited to the initial startup scan.

View File

@ -1,5 +1,10 @@
# WASM-PROPOSAL: WebAssembly Unit Runtime for UCE
> **Steady-state reference:** for the runtime as it is actually built today
> (process topology, the membrane, unified dispatch, and the central WebSocket
> broker), see [`docs/wasm-runtime-architecture.md`](docs/wasm-runtime-architecture.md).
> This proposal retains the motivation and the phased migration plan.
- **Status:** building the production worker per §9.1 (W-phases). W1 (core
module), W2 (wasm unit compile target), W3 (workspace runtime + membrane,
`src/wasm/worker.cpp`), and W4 (config-selectable FastCGI backend,
@ -970,14 +975,63 @@ and the operational services test) longer per-request timeouts while preserving
strict status/body checks, and the services test drains split custom-server
response packets before deciding that the named handler failed.
> **Status: W7a/W7b PARTIAL (2026-06-13).** CLI/WebSocket/SERVE_HTTP entry kind
> plumbing exists, CLI socket error propagation is fixed, and post-fork ticker
> reset is safer. Directly enabling wasm `SERVE_HTTP` in the in-process custom
> HTTP dispatcher was tested and reverted: the dispatcher can hang/spin when it
> renders wasm in the forked broker. W7c therefore remains a design item: route
> custom-server requests back through a normal worker/workspace boundary rather
> than rendering inside the broker child. Remaining native fallback tokens after
> W6/W7 hardening are still `zip_` and direct `compiler_load_shared_unit(`.
> **Status: W7a/W7b/W7c/W7d DONE (2026-06-14).** Handler invocation is now
> unified: CLI / WebSocket / serve_http / page render all route through one
> `serve_via_wasm(entry_unit, handler)` path, the handler string mapping to a
> `__uce_<handler>` export (see
> [`docs/wasm-runtime-architecture.md`](docs/wasm-runtime-architecture.md)).
> W7c was resolved by the central WebSocket broker + serve_http dispatcher both
> forwarding to a clean worker over `uce.sock` (no wasm render inside a forked
> broker child). W7d cleared the last native-only holdouts: `zip.uce` was
> rewritten to use the membrane's return-value error contract (no C++
> try/catch), `error-reporting.uce` now triggers genuine wasm traps instead of
> `throw`, and `sharedunit.uce` uses `unit_info()` instead of
> `compiler_load_shared_unit()`. **The native-only token gate is now empty**
> no unit source forces native anymore.
>
> **W7e plan — deleting the native pipeline (the remaining work).** The native
> pipeline is not just a fallback; it is currently the **wasm compile driver**:
> `compiler.cpp` preprocesses `.uce → C++` (shared front-end, KEEP), then
> compiles the native `.so` (`COMPILE_SCRIPT`), and only *then*, gated on that
> native compile succeeding, compiles the `.wasm` side-module
> (`compile_wasm_unit`). The backend's native fallback on a cold/stale artifact
> exists so the native path can JIT-rebuild **both** `.so` and `.wasm`. So the
> deletion must be staged, not a single excision:
> 1. **Decouple the compile driver.** Make the `.wasm` side-module compile run
> unconditionally from the shared preprocessed C++, independent of the
> native `.so` compile and of the `SharedUnit` struct's native bookkeeping.
> 2. **Make wasm self-healing on cold/stale.** Have the backend trigger (or
> wait on) an on-demand wasm compile instead of falling back to native, so
> `wasm_artifact_exists` staleness no longer needs the native rebuild. (Note:
> until this lands, a cold request — before the proactive compiler warms a
> unit — to a unit rewritten for the membrane error contract, e.g. `zip.uce`,
> can `500` on the native path because native `zip_create` still throws.)
> 3. **Remove native execution + machinery.** Delete `compiler_invoke` /
> `_cli` / `_websocket` / `_serve_http`, `compiler_load_shared_unit`,
> `load_shared_unit`, the `dlopen`/`dlsym`/`dlclose` path and `.so`
> function-pointer fields on `SharedUnit`, the `COMPILE_SCRIPT` native
> compile, and the native-fallback branches in `handle_complete` and
> `wasm_backend_should_handle`.
> 4. **Drop the feature flag.** Remove `WASM_BACKEND_ENABLED` and the now-dead
> `wasm_backend_native_fallback_*` gate (the token scan is already vestigial).
>
> Each stage must rebuild green on `scripts/run_cli_tests.sh --include-wasm-kill`
> before the next. W7f (broad dead/legacy sweep) folds into stages 34.
>
> **Prerequisite — membrane silent-stub completeness (in progress).** Removing
> native fallback (stages 24) turns any silently-stubbed wasm-core function
> into a hard failure with no safety net. A real one slipped through this way:
> `ls()` was a stub returning empty, so `doc/index.uce` (which enumerates pages
> via `ls("pages/")`) rendered but listed nothing — and the suite missed it
> because it only asserted the page returned `<html>`, never that it enumerated
> items. Fixed (2026-06-14) by wiring `ls`, `mkdir`, and `file_mtime` through new
> `uce_host_file_list` / `uce_host_file_mkdir` / `uce_host_file_mtime` hostcalls
> (and teaching `resolve_guest_file` to accept directories), plus a regression
> assertion that the doc index contains `func-item`. Remaining wasm-core stubs
> are intentional: `shell_exec` (no shell in the sandbox), the `*_locked_fd`
> fd-locking variants, and the `backtrace_*` / `signal_name` host-only helpers.
> Before stage 2, audit the rest of the wasm-core block in `src/lib/sys.cpp` for
> any other `(void)arg; return {}` stub a unit can reach.
---

View File

@ -10,7 +10,7 @@ Replace the Python-based network test runner and plugins with UCE unit tests inv
- [x] UCE CLI tests cover every current Python plugin behavior or explicitly document a deliberate replacement.
- [x] Existing W5/WASM gates use the UCE CLI test runner instead of `tests/run_network_tests.py`.
- [x] Python test runner/plugins are deleted after parity validation.
- [x] Full suite passes on `uce-dev` with `WASM_BACKEND_ENABLED=1`.
- [x] Full suite passes on `uce-dev` with wasm-only unit execution.
## Current State

View File

@ -0,0 +1,319 @@
# UCE WASM Runtime Architecture
Status: current as of the W7e native-pipeline removal (June 2026). This document
describes the **runtime architecture as built** — the process topology, the
wasm membrane, the unified request dispatch, and the central WebSocket broker.
Native `.so` unit execution/dlopen fallback has been removed; the parser and
preprocessor remain only as the front-end that emits C++ for wasm side-module
compilation. For the historical motivation and the phased migration plan, see
[`WASM-PROPOSAL.md`](../WASM-PROPOSAL.md); this file is the steady-state
reference that proposal points at.
The guiding principle: **request code never shares an address space or an
allocator with the runtime.** Every unit runs as a WebAssembly module inside a
per-request workspace behind a narrow host membrane. Long-lived connection
state (WebSocket connections, listening HTTP sockets) lives in dedicated native
broker processes that own no unit code — they hold the connection and forward
the actual unit invocation to a clean-engine worker, exactly like a normal
page render.
---
## 1. Process topology
A single native binary (`uce_fastcgi`) forks into a small set of long-lived
roles. None of them is special-cased per request mode; the differences are
purely *which socket a process listens on* and *which function inside a unit
gets invoked*.
```
┌────────────────────────────┐
nginx ──FastCGI──► worker pool (N processes) │ /run/uce.sock
(port 80 etc.) │ uniform unit renderers │ (FastCGI + CLI)
└─────────────▲──────────────┘
│ forward render (FastCGI, uce.sock)
browser ──raw HTTP / WS──► ┌────────┴─────────┐
(HTTP_PORT 8080) │ WS broker │ owns HTTP_PORT + every
│ (1 process) │ WS connection; renders
└────────▲─────────┘ nothing itself
│ ws_* command flush
│ (FastCGI, ws-broker.sock)
on-demand serve_http ──► ┌────────┴─────────┐
(per bind addr) │ custom-server │ owns one serve_http
│ dispatcher(s) │ bind addr; forwards to pool
└──────────────────┘
parent process: spawns/respawns all of the above + the proactive compiler.
```
| Process | Owns | Renders units? | Source |
|---|---|---|---|
| **Parent** | nothing; supervises children | no | `main()`, `init_base_process()` |
| **Worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH` (`/run/uce.sock`) + `CLI_SOCKET_PATH` | **yes** — the only processes that run wasm | `listen_for_connections()` |
| **WS broker** (×1) | `HTTP_PORT` + every live WS connection + `WS_BROKER_SOCKET_PATH` | no — forwards to the pool | `run_ws_broker()` |
| **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` |
| **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` |
The key invariant: **only workers instantiate Wasmtime and run unit code.**
Every connection-owning process (broker, serve_http dispatcher) forwards the
real invocation back to a worker over `/run/uce.sock` using the minimal
FastCGI client in [`src/lib/fcgi_forward.h`](../src/lib/fcgi_forward.h). This is
forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across
`fork()`, and the brokers fork from the parent that already touched the
runtime. So the brokers hold the long-lived connection and units respond to
events the same way they respond to a page request — through a clean worker.
---
## 2. The membrane and the DValue ABI
Units are compiled to WebAssembly and linked against a host import surface (the
"membrane"). Host and guest exchange structured values as **UCEB** (a compact
binary `DValue` encoding — `ucb_encode`/`ucb_decode`); strings cross as
`std::string` on the native side throughout (no raw `char*` ownership across
the boundary).
- `DValue` is the universal value type: scalars plus an ordered child map
(`_map`, a `std::map<String,DValue>`). `operator[]` is non-const (creates);
`.key(k)` is the const probe returning `const DValue*`; `.each(fn)` iterates
children yielding `const DValue&`.
- The request context (`params`/`get`/`post`/`cookies`/`session`, the raw body
`in`, and—for WS—the connection context) is marshalled into a single `ctx`
DValue, UCEB-encoded, and handed to the workspace. The response (body,
headers, status, and any `meta` such as `ws_commands`) comes back the same
way.
See [`docs/wasm-phase1-dvalue-abi.md`](wasm-phase1-dvalue-abi.md) for the wire
format details.
---
## 3. Units, handlers, and export naming
A `.uce` unit compiles to a wasm module that exports one symbol per **handler**.
There is no per-mode machinery — a "CLI unit", "WebSocket unit", and "page" are
the same compiled artifact invoked at different exports. The dispatcher passes a
**handler string**; the host maps it to an export symbol:
```
__uce_<base>[_<sanitize(suffix)>]
```
`handler_export_symbol()` (`src/wasm/worker.cpp`) splits on the first `:`:
| Handler string | Export symbol |
|---|---|
| `render` | `__uce_render` |
| `cli` | `__uce_cli` |
| `websocket` | `__uce_websocket` |
| `once` | `__uce_once` (optional; absence is not an error) |
| `serve_http` | `__uce_serve_http` |
| `serve_http:named` | `__uce_serve_http_named` |
| `component:CARD` | `__uce_component_CARD` |
| `exists` | probe only — resolves the unit, loads nothing |
`sanitize_symbol_suffix()` keeps `[A-Za-z0-9_]` (mirrors `ascii_safe_name`).
`wasm_resolve_target(unit, handler)` (`src/wasm/core.cpp`) resolves the source
path and looks up the export's funcref slot; `exists` lets callers probe a unit
without instantiating it.
`wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
backend is initialized and the requested artifact/handler is currently
available. If an artifact is cold or stale, dispatch compiles it on demand via
`get_shared_unit()` and rechecks. There is no native unit-execution fallback.
---
## 4. The workspace runtime
Each request gets a fresh **workspace** — a per-request wasm instance tree with
the membrane wired in. `wasm_worker_serve(worker, ctx, entry_unit, handler)`
(`src/wasm/worker.cpp`) is the single entry point for *every* mode:
1. Birth a workspace (CoW-snapshot-based where available).
2. Resolve `entry_unit` + `handler` to an export; components referenced at
runtime are resolved on demand via the `uce_host_component_resolve` hostcall
(`component_resolve()``__uce_<...>` slot), loading dependency modules
lazily and recording resolve counts/timings.
3. Invoke the export. Unit code calls host functions (filesystem, sqlite, regex,
markdown, time, tasks, `ws_*`, …) across the membrane.
4. Collect the response (`WasmResponse`: body, headers, status, `meta`).
Because the workspace owns no process-lifetime state, a unit crash or trap is
contained: it fails the one request and the worker stays healthy. Workers
suspend the native SIGSEGV/SIGILL recovery handler around the wasm call so that
Wasmtime's own trap signals are not escalated into a native fatal signal (see
`serve_via_wasm` in `handle_complete`).
---
## 5. Request dispatch (`handle_complete`)
`handle_complete()` (`src/linux_fastcgi.cpp`) is the worker's single dispatch
point. It selects a handler string from request params and calls the same
`serve_via_wasm(entry_unit, handler)` lambda for all of them:
```
UCE_WS == "1" → serve_via_wasm(entry_unit, "websocket")
request.resources.is_cli → serve_via_wasm(entry_unit, "cli")
UCE_SERVE_HTTP == "1" → serve_via_wasm(entry_unit, "serve_http"[:fn])
otherwise (page) → serve_via_wasm(entry_unit, "render")
```
The `UCE_*` params are set by whichever broker forwarded the request:
- **Page render**: nginx → `/run/uce.sock` directly; no `UCE_*` flags → `render`.
- **CLI**: the CLI socket sets `is_cli`.
- **serve_http**: the custom-server dispatcher sets `UCE_SERVE_HTTP=1` plus
`UCE_SERVE_HTTP_FUNCTION` and rewrites `SCRIPT_FILENAME` to the configured
unit (`custom_server_http_complete`).
- **WebSocket**: the WS broker sets `UCE_WS=1` and carries the connection
identity as `UCE_WS_*` params (see §6). `handle_complete` rebuilds
`request.resources.websocket_*` and `request.connection` from them before
invoking `__uce_websocket`.
If a unit still cannot be served after the on-demand wasm compile, the worker
returns a clean 500 with the wasm/compile error; it does not execute native unit
code.
---
## 6. The central WebSocket broker
The broker is the architectural centerpiece. **One process owns the HTTP port
and every WebSocket connection**, so any unit's `ws_*` call can reach any or all
connections, and a unit-code crash (which happens in a worker) never drops live
connections.
### 6.1 Why central, and why it forwards
Earlier designs gave each worker its connections and captured `ws_*` output to
return inline. That was wrong: connections couldn't outlive a worker, and one
unit could only talk to connections it happened to own. The broker fixes both —
it is the sole connection registry and data broker between connected clients and
the units that handle them. It renders nothing itself; like every other
connection owner it forwards unit invocation to the worker pool.
### 6.2 Inbound: a WS frame → a worker render (non-blocking)
`ws_broker_ws_message(request, message, opcode)` fires when a complete
(reassembled) WS message arrives on a connection. It does **not** block the
broker loop:
1. Build FastCGI params: `SCRIPT_FILENAME`, `REQUEST_METHOD=GET`, a
`REQUEST_URI` (required — `handle_request()` rejects requests without one
*before* `on_complete` runs), `UCE_WS=1`, and the connection context as
`UCE_WS_CONNECTION_ID / SCOPE / OPCODE / BINARY / CONNECTIONS / STATE`.
2. The message rides as `UCE_WS_MESSAGE` (base64) with an **empty STDIN body**
a non-empty STDIN makes the FastCGI transport flush a premature response
before `on_complete` ever runs.
3. Connect to `/run/uce.sock` (non-blocking) and queue the encoded request in
`ws_broker_outbound[fd]`.
`ws_broker_drain_outbound()` runs after every `process(50)` tick: it finishes
writing each queued request, then drains and discards the reply (the unit's
output comes back via the command socket, not this reply), closing the fd when
the worker closes its end.
### 6.3 Outbound: `ws_*` commands flushed back to the broker
Any unit code — not just WebSocket handlers — may call `ws_send` / `ws_send_to`
/ `ws_close`. In the workspace these **record dispatch commands** rather than
touching a socket (the workspace owns no connections); `wasm-core`'s `ws_*`
(`src/lib/sys.cpp`) append to `websocket_dispatch_commands`, and
`finish_response_meta` (`src/wasm/core.cpp`) emits them as `ws_commands` (plus
`ws_connection_state` if the handler mutated per-connection state).
`wasm_backend_serve` (`src/wasm/backend.cpp`) flushes that batch at workspace
teardown — in **any** scenario, not just WS handlers — to the broker's command
socket (`WS_BROKER_SOCKET_PATH`, `/run/uce/ws-broker.sock`) via
`fcgi_forward_request` with `UCE_WS_DISPATCH=1`. This is the only path WS data
takes out of a workspace.
`ws_broker_apply_commands()` decodes the batch and applies each command against
the full registry it owns: `broadcast` (by scope), `send_to` (by connection id),
`close`. If the batch carries `connection_state`, it persists that onto the
matching live connection's `websocket_state`.
### 6.4 Un-upgraded HTTP on the WS port
The WS port can also receive ordinary (non-Upgrade) HTTP requests.
`ws_broker_complete()` routes by param: `UCE_WS_DISPATCH=1` → apply commands;
otherwise → `forward_request_to_worker()` — the *same* shared facility the
serve_http dispatcher uses, so there is no duplicated request-forwarding code.
### 6.5 The broker loop
`run_ws_broker()` drops the worker listeners it inherited
(`close_inherited_server_sockets`), installs permissive `on_request`/`on_data`
handlers (it renders nothing, so it accepts every request straight through to
`on_complete`), wires `on_complete=ws_broker_complete` and
`on_websocket_message=ws_broker_ws_message`, listens on `HTTP_PORT` and the
command socket, and loops `process(50)` + `drain_outbound()`. The design is
**non-blocking outbound dispatch + async command-socket flush, all in the
broker's single epoll loop** — the broker never blocks on a worker.
The parent respawns the broker if it dies (`ws_broker_alive` / `ensure_ws_broker`
in `main()`). A broker restart loses live connections (acceptable: crashes
happen in workers, so the broker stays up in practice), but no unit state is at
risk because the broker holds none.
---
## 7. The serve_http facility
`serve_http` units are reachable on their own bind address via a custom-server
dispatcher (`custom_server_http_dispatcher_loop`). The dispatcher owns the bind
socket and, on each request, sets `UCE_SERVE_HTTP=1` + the configured unit/
function and calls `forward_request_to_worker()`. The worker then runs
`serve_via_wasm(entry_unit, "serve_http"[:fn])`. This is the same
hold-connection-forward-render model as the WS broker, sharing
`forward_request_to_worker` and `fcgi_forward.h`.
---
## 8. Build / object layout
The native side is split into separately-compiled objects so editing the wasm
runtime does not recompile the whole TU (`scripts/build_linux.sh`):
| Object | Contents |
|---|---|
| `bin/sqlite3.o` | sqlite amalgamation (cached) |
| `bin/wasm.o` | `src/wasm/wasm_module.cpp` (backend.cpp + worker.cpp + wasmtime) |
| `bin/main.o` | `src/linux_fastcgi.cpp` (includes `lib/uce_lib.cpp`) |
Linked into one `-rdynamic` binary. ODR hazards across the objects are handled
deliberately: `context` and `my_pid`/`parent_pid` are `extern` (guarded for the
wasm core vs. unit builds), `operator new`/`delete` live in `types.cpp`, and
header free-functions are `inline`. The wasm backend exposes only declarations
(`src/wasm/backend.h`) to `main.o`.
---
## 9. Configuration keys
| Key | Default | Meaning |
|---|---|---|
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
| `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). |
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. |
| `HTTP_PORT` | `8080` | Raw HTTP + WebSocket port — owned by the WS broker. |
| `WS_BROKER_SOCKET_PATH` | `/run/uce/ws-broker.sock` | Broker command socket for `ws_*` flushes. |
| `WORKER_COUNT` | `4` | Number of uniform worker processes. |
---
## 10. Testing
- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages.
Current baseline: **87 passed, 0 failed**.
- **WebSocket end-to-end**: a headless client performs a raw WS handshake to
`:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
`SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
broker → worker → broker → client chain across process boundaries.
All builds, runs, and installs happen on the dev host (`k-uce` / uce-dev) over
SSH; the local checkout is edit-only.

View File

@ -21,13 +21,10 @@ SITE_DIRECTORY=site
# ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT
JIT_COMPILE_ON_REQUEST=1
# OPTIONAL W2 WEBASSEMBLY SIDE-MODULE COMPILATION BESIDE NATIVE .so UNITS
COMPILE_WASM_UNITS=0
# WASM SIDE-MODULE COMPILER USED FOR .uce UNITS
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
# DEFAULT PAGE RENDER BACKEND. W5 defaults to the WASM backend with explicit
# native fallbacks for host-owned surfaces that are not membrane APIs yet.
WASM_BACKEND_ENABLED=1
# WASM RUNTIME SETTINGS. Unit execution is always routed through wasm.
WASM_BACKEND_VERBOSE=0
WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm
WASM_MEMORY_LIMIT_BYTES=536870912

View File

@ -9,11 +9,11 @@ GF="uce_fastcgi"
mkdir bin > /dev/null 2>&1
mkdir bin/tmp > /dev/null 2>&1
mkdir bin/assets > /dev/null 2>&1
mkdir bin/wasm > /dev/null 2>&1
mkdir work > /dev/null 2>&1
COMPILER="clang++"
# -rdynamic is a link-time flag (exports the binary's symbols so dlopen'd .uce
# units resolve the runtime against it); the -c compiles below do not need it.
# -rdynamic is a link-time flag; the -c compiles below do not need it.
FLAGS="-g -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math"
# Wasmtime C++ API — needed only by the wasm backend object (src/wasm).
@ -30,8 +30,8 @@ SRCFLAGS="-D EXEC_NAME=\"$GF\" -D PLATFORM_NAME=\"linux\""
# bin/sqlite3.o vendored SQLite amalgamation (depends only on its own source)
# bin/wasm.o wasm backend + worker + wasmtime.hh (src/wasm)
# bin/main.o linux_fastcgi.cpp + the uce_lib core amalgamation
# All link into the single -rdynamic binary, so the dlopen unit model is
# unchanged. Delete bin/*.o to force a clean rebuild.
# All link into the single -rdynamic binary. Delete bin/*.o to force a clean
# rebuild.
# Rebuild $1 if it is missing or anything under the remaining find-args is newer.
needs_rebuild() {
@ -41,6 +41,14 @@ needs_rebuild() {
return 1
}
# core.wasm: guest runtime loaded by the native wasm backend.
if needs_rebuild bin/wasm/core.wasm src/wasm/core.cpp src/lib src/wasm/core_hostcalls.syms src/wasm/core_libc_exports.syms scripts/build_core_wasm.sh; then
echo "Compiling wasm core..."
bash scripts/build_core_wasm.sh || exit 1
else
echo "Reusing bin/wasm/core.wasm"
fi
# SQLite: vendored C, depends only on its own source (not our headers).
if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then
echo "Compiling SQLite..."

View File

@ -1,47 +0,0 @@
#!/bin/bash
cd "$(dirname "$0")"
cd ..
SRC_DIR="$1"
DEST_DIR="$2"
SRC_FN="$3"
PP_FN="$4"
SO_FN="$5"
LINK_OBJECTS="$6"
#echo "Source Dir: $SRC_DIR"
#echo "Dest Dir: $DEST_DIR"
#echo "Source File: $SRC_FN"
#echo "Preprocessed File: $PP_FN"
#echo "Dest File: $SO_FN"
mkdir -p "$DEST_DIR" > /dev/null 2>&1
export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH:+${CPLUS_INCLUDE_PATH}:}$SRC_DIR"
BUILDMODE="debug"
OPT_FLAG="O0"
COMPILER="clang++"
#COMPILER="g++"
FLAGS="-shared -g -rdynamic -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math -fPIC -Isrc/lib"
LIBS="-ldl -lm -lpthread"
SRCFLAGS="-D PLATFORM_NAME=\"linux\""
# echo "Compliling executable..."
$COMPILER "$DEST_DIR/$PP_FN" $SRCFLAGS $FLAGS $LIBS -o "$DEST_DIR/$SO_FN"
# separate .o file
#$COMPILER -c "$DEST_DIR/$PP_FN" $SRCFLAGS $FLAGS $LIBS -o "$DEST_DIR/$PP_FN.o"
#$COMPILER "$DEST_DIR/$PP_FN.o" "$LINK_OBJECTS" $SRCFLAGS $FLAGS $LIBS -o "$DEST_DIR/$SO_FN"
if [ $? -eq 0 ]
then
# ls -lh "$DEST_DIR"
exit 0
else
exit 1
fi

View File

@ -21,9 +21,9 @@ COMMON_FLAGS=(
--target=wasm32-wasip1
-fPIC -fvisibility=default -fvisibility-inlines-hidden
-O1 -g -std=c++20
# -w as in scripts/compile: warnings are not failures. The server captures
# this script's output and treats any non-empty result as a compile failure
# (then drops the .wasm), so a successful build must be silent.
# The server captures this script's output and treats any non-empty result as
# a compile failure (then drops the .wasm), so a successful build must be
# silent; warnings are intentionally suppressed.
-w
# must match the core build ABI: units with RTTI/EH enabled import
# typeinfo/unwind symbols the -fno-rtti/-fno-exceptions core cannot provide
@ -50,7 +50,7 @@ build_pch_if_needed() {
return 0
fi
mkdir -p "$PCH_DIR"
if [ -s "$PCH_FN" ]; then
if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then
return 0
fi
"$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \

View File

@ -20,21 +20,12 @@ fi
count=0
checked=0
skipped=0
# Native-only units that cannot be wasm side modules (yet).
# - error-reporting.uce deliberately throws to exercise the native exception
# path; the wasm backend replaces that machinery with traps (§11.1).
# - tests/zip.uce uses try/catch around the zip library, which is carved out
# of the wasm core until it moves behind a hostcall (W4+ membrane work).
SKIP_PATTERN=${UCE_W2_SKIP:-(error-reporting|tests/zip)\.uce$}
for unit in "${UNITS[@]}"; do
case "$unit" in
*.uce|*.ws.uce) ;;
*) continue ;;
esac
if [[ "$unit" =~ $SKIP_PATTERN ]]; then
continue
fi
src_dir=$(dirname "$unit")
base=$(basename "$unit")
dest_dir="$BIN_DIR$src_dir"

View File

@ -140,6 +140,14 @@ def dylink_has_valid_mem_info(payload: bytes) -> bool:
def defined_symbols(path: Path, llvm_nm: str) -> list[str]:
proc = subprocess.run([llvm_nm, "--defined-only", str(path)], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if proc.returncode != 0:
# llvm-nm (wasi-sdk) can SIGSEGV on degenerate-but-valid modules — e.g. a
# unit with no exported handlers (`empty.uce`). A toolchain crash is not
# evidence of a forbidden allocator, so skip this defense-in-depth scan
# rather than fail the unit; forbidden allocator *exports* are still
# rejected by the export-section check above.
crashed = proc.returncode < 0 or "Stack dump:" in proc.stderr or "PLEASE submit a bug report" in proc.stderr
if crashed:
return []
raise RuntimeError(proc.stderr.strip() or "llvm-nm failed")
symbols = []
for line in proc.stdout.splitlines():

View File

@ -1,94 +1,19 @@
#!/bin/bash
# W5 parity/performance gate for the config-selectable WASM backend.
# Runs on k-uce. Requires root because the live service reads /etc/uce/settings.cfg.
# Retired W5 parity/performance gate.
#
# W7e removed the config-selectable native backend and all native unit
# execution. There is no longer a valid WASM_BACKEND_ENABLED=0 baseline to
# switch to here; doing so would only create a misleading service state.
set -euo pipefail
cd "$(dirname "$0")/../.."
cat >&2 <<'EOF'
run_w5.sh is retired: UCE unit execution is wasm-only after W7e.
Use the supported regression gate instead:
if [ "${EUID:-$(id -u)}" -ne 0 ]; then
echo "run_w5.sh must run as root so it can switch /etc/uce/settings.cfg and restart uce.service" >&2
exit 1
fi
bash scripts/build_linux.sh && systemctl restart uce.service && sleep 3 \
&& bash scripts/run_cli_tests.sh --include-wasm-kill
OUT=${UCE_W5_OUT:-/tmp/uce/wasm-w5}
CONFIG=${UCE_CONFIG:-/etc/uce/settings.cfg}
mkdir -p "$OUT"
BACKUP="$OUT/settings.cfg.before-w5"
cp "$CONFIG" "$BACKUP"
restore_on_error() {
if [ "${UCE_W5_KEEP_BACKEND:-0}" != "1" ]; then
cp "$BACKUP" "$CONFIG"
systemctl restart uce.service >/dev/null 2>&1 || true
fi
}
trap restore_on_error EXIT
set_backend() {
local enabled="$1"
local tmp
tmp=$(mktemp)
awk -F= -v enabled="$enabled" '
BEGIN { found = 0 }
/^WASM_BACKEND_ENABLED=/ { print "WASM_BACKEND_ENABLED=" enabled; found = 1; next }
{ print }
END { if(!found) print "WASM_BACKEND_ENABLED=" enabled }
' "$CONFIG" > "$tmp"
for kv in \
'WASM_BACKEND_VERBOSE=0' \
'WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm' \
'WASM_MEMORY_LIMIT_BYTES=536870912' \
'WASM_EPOCH_DEADLINE_TICKS=200' \
'WASM_EPOCH_PERIOD_MS=50'
do
key=${kv%%=*}
if ! grep -q "^${key}=" "$tmp"; then
printf '%s\n' "$kv" >> "$tmp"
fi
done
cat "$tmp" > "$CONFIG"
rm -f "$tmp"
systemctl restart uce.service >/dev/null
sleep 1
}
summary_total() {
awk '/^Summary:/ { print $2; found=1 } END { if(!found) print 0 }' "$1"
}
# Native reference baseline.
set_backend 0
scripts/run_cli_tests.sh > "$OUT/native-warmup.txt" || true
scripts/run_cli_tests.sh > "$OUT/native-network.txt"
python3 tests/wasm_benchmark.py --out-dir "$OUT/native-benchmark" --samples "${UCE_W5_BENCH_SAMPLES:-20}" --timeout 30 >/dev/null
# WASM default backend with W5 native fallbacks for host-owned surfaces.
set_backend 1
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-warmup.txt" || true
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-network.txt"
python3 tests/wasm_benchmark.py \
--out-dir "$OUT/benchmark" \
--backend-label wasm \
--compare-native-json "$OUT/native-benchmark/benchmark.json" \
--samples "${UCE_W5_BENCH_SAMPLES:-20}" \
--timeout 30
python3 tests/wasm_site_audit.py --out-dir "$OUT" >/dev/null
network_cases=$(summary_total "$OUT/wasm-network.txt")
if [ "$network_cases" -lt 80 ]; then
echo "W5 HARNESS: FAIL"
echo "wasm network ran $network_cases cases, expected at least 80"
exit 1
fi
echo 'W5 HARNESS: PASS'
echo "network_cases=$network_cases benchmark_rows=see benchmark.json"
echo "reports=$OUT"
if [ "${UCE_W5_KEEP_BACKEND:-0}" = "1" ]; then
trap - EXIT
printf 'WASM backend left enabled in %s\n' "$CONFIG"
else
cp "$BACKUP" "$CONFIG"
systemctl restart uce.service >/dev/null
trap - EXIT
fi
For performance comparisons, use tests/wasm_benchmark.py against the current
wasm backend; native-baseline artifacts under docs/wasm-baselines are historical
only.
EOF
exit 2

View File

@ -1,5 +1,13 @@
#include "demo_guard.h"
// Fault-injection demo. Under the wasm runtime every unit fault is a guest
// trap that the workspace turns into a clean 500 without harming the worker;
// these modes exercise the three distinct trap causes (mirrors tests/wasm-kill).
u64 error_reporting_recurse(volatile u64 depth)
{
volatile u64 next = depth + 1;
return(next + error_reporting_recurse(next));
}
RENDER(Request& context)
{
@ -10,12 +18,19 @@ RENDER(Request& context)
}
String mode = context.get["mode"];
if(mode == "exception")
throw std::runtime_error("Intentional test exception from /test/error-reporting.uce");
if(mode == "abort")
raise(SIGABRT);
if(mode == "segfault")
raise(SIGSEGV);
if(mode == "trap")
__builtin_trap();
if(mode == "recurse")
{
volatile u64 sink = error_reporting_recurse(0);
(void)sink;
}
if(mode == "loop")
{
volatile u64 i = 0;
while(i >= 0)
i++;
}
<>
<link rel="stylesheet" href='style.css?v=<?= time() ?>'></link>
@ -23,11 +38,11 @@ RENDER(Request& context)
<a href="index.uce">UCE Test</a>:
Error reporting
</h1>
<p>These actions intentionally trigger failures so you can verify that UCE returns a usable `500` response instead of dropping the upstream connection.</p>
<p>These actions intentionally trigger guest traps so you can verify that UCE returns a usable `500` response from a healthy worker instead of dropping the upstream connection.</p>
<ul>
<li><a href="?mode=exception">Trigger uncaught exception</a></li>
<li><a href="?mode=abort">Trigger SIGABRT</a></li>
<li><a href="?mode=segfault">Trigger SIGSEGV</a></li>
<li><a href="?mode=trap">Trigger an explicit trap (`__builtin_trap`)</a></li>
<li><a href="?mode=recurse">Trigger stack exhaustion (unbounded recursion)</a></li>
<li><a href="?mode=loop">Trigger a runaway loop (epoch interrupt)</a></li>
</ul>
</>
}

View File

@ -1,7 +1,9 @@
RENDER(Request& context)
{
auto p = compiler_load_shared_unit(&context, "post.uce");
if(p)
print(to_string(p));
// The native SharedUnit loader has been retired. Unit metadata is now
// exposed through the wasm membrane via unit_info() (uce_host_units), which
// resolves and introspects another unit without dlopen-ing a native .so.
DValue info = unit_info("post.uce");
print(json_encode(info));
}

View File

@ -157,16 +157,16 @@ RENDER(Request& context)
</div>
<div class="detail-card">
<strong>Runtime Flags</strong>
<span>loaded <?= unit_flag_label(selected_info["loaded"]) ?>, stale <?= unit_flag_label(selected_info["stale"]) ?>, current <?= unit_flag_label(selected_info["current_unit"]) ?></span>
<span>wasm <?= unit_flag_label(selected_info["wasm_available"]) ?>, stale <?= unit_flag_label(selected_info["stale"]) ?>, current <?= unit_flag_label(selected_info["current_unit"]) ?></span>
</div>
<div class="detail-card">
<strong>Artifacts</strong>
<code><?= selected_info["so_name"].to_string() ?></code>
<code><?= selected_info["wasm_name"].to_string() ?></code>
</div>
<div class="detail-card">
<strong>Timestamps</strong>
<span>Source file: <?= time_format_relative(selected_info["source_mtime"].to_u64()) ?><br>
Binary: <?= time_format_relative(selected_info["compiled_mtime"].to_u64()) ?><br>
Wasm: <?= time_format_relative(selected_info["compiled_mtime"].to_u64()) ?><br>
Meta info: <?= time_format_relative(selected_info["metadata_mtime"].to_u64()) ?></span>
</div>
</div>

View File

@ -15,7 +15,7 @@ INIT(Request& context)
:content
Defines a worker-load hook for the current `.uce` unit.
When a worker loads the unit's compiled shared object into memory, the runtime checks whether the unit exposes `INIT(Request& context)`. If it does, the hook runs once for that load before the unit begins serving later requests from that in-memory copy.
When a worker instantiates the unit's compiled wasm module, the runtime checks whether the unit exposes `INIT(Request& context)`. If it does, the hook runs once for that instance before the unit begins serving requests from that worker-local copy.
Because UCE usually loads units on demand during a request, `INIT()` still receives a valid `Request& context`. Use it for worker-local initialization, not for request-local state that should reset each request.

View File

@ -14,7 +14,7 @@ unit_call
:content
UCE runs a small custom source-to-source preprocessor before Clang sees a `.uce` or `.ws.uce` file.
The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, with orchestration in `src/lib/compiler.cpp`. It does not try to parse all of C++. Instead, it performs a narrow character-wise rewrite that understands literal output, inline code islands, `#load`, and `EXPORT` harvesting, then writes a generated `.cpp` file and compiles that file into a shared object.
The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, with orchestration in `src/lib/compiler.cpp`. It does not try to parse all of C++. Instead, it performs a narrow character-wise rewrite that understands literal output, inline code islands, `#load`, and `EXPORT` harvesting, then writes a generated `.cpp` file and compiles that file into a WebAssembly side module.
## Syntax
@ -32,7 +32,7 @@ The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, wi
## Pipeline
- The generated file starts by including the logical runtime header `uce_lib.h`; native and WASM compile scripts provide the include path.
- The generated file starts by including the logical runtime header `uce_lib.h`; the wasm unit compile script provides the include path.
- It then inlines the configured setup template from `SETUP_TEMPLATE` (by default `scripts/setup.h.template`), which defines the internal hook `__uce_set_current_request(Request*)`.
- It inserts `#line 1` before page code so compiler diagnostics point back to the original `.uce` file.
- Each literal region is rewritten into one or more `print(R"...( ... )...");` calls using a safe raw-string delimiter selected for that literal content.
@ -47,8 +47,8 @@ The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, wi
- Lines beginning with `RENDER:NAME(...)` are rewritten into exported `__uce_render_NAME(...)` functions.
- Lines beginning with `COMPONENT:NAME(...)` are rewritten into exported `__uce_component_NAME(...)` functions for the component helpers.
- The final generated source is written to `BIN_DIRECTORY + src_path + "/" + source_file + ".cpp"`.
- `scripts/compile` then compiles that generated `.cpp` into `source_file + ".so"` with `clang++ -shared -std=c++20 ...`.
- When a worker loads the compiled unit into memory, the runtime checks for `INIT(Request& context)` and calls it once for that worker-side load.
- `scripts/compile_wasm_unit` then compiles that generated `.cpp` into `source_file + ".wasm"` as a PIC WebAssembly side module.
- When a worker instantiates the compiled unit, the runtime checks for `INIT(Request& context)` and calls it once for that worker-side instance.
- On each request, the first time a given unit is entered through `RENDER()`, `CLI()`, or any `COMPONENT...` handler, the runtime checks for `ONCE(Request& context)` and calls it before the selected handler.
## Generated Files
@ -56,7 +56,7 @@ The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, wi
For a source file like `/some/path/page.uce`, the preprocessor produces:
- generated C++: `BIN_DIRECTORY/some/path/page.uce.cpp`
- shared object: `BIN_DIRECTORY/some/path/page.uce.so`
- wasm side module: `BIN_DIRECTORY/some/path/page.uce.wasm`
- export list: `BIN_DIRECTORY/some/path/page.uce.exports.txt`
## Examples
@ -153,7 +153,7 @@ The page template can then render `context.call["fragments"]["head"]` inside `<h
- `EXPORT` harvesting only triggers when the current line starts with `EXPORT` at column 1 and is followed by whitespace.
- Relative `#load` paths are expanded against the including unit's source directory.
- `unit_render()` and `unit_call()` are runtime APIs. `#load` is a compile-time composition feature.
- `INIT()` runs when the shared object is loaded into a worker during a request-triggered load, so it still receives a valid `Request& context`.
- `INIT()` runs when the wasm unit is instantiated by a worker during a request-triggered load, so it still receives a valid `Request& context`.
- `ONCE()` is tracked per request and per resolved unit file. A file entered multiple times in one request only runs `ONCE()` once.
## Limitations

View File

@ -42,7 +42,7 @@ RENDER(Request& context)
}
```
The compiling page is only used while `PROACTIVE_COMPILE_ENABLED` is on (the default) — otherwise nothing would finish the build, and the runtime falls back to the blocking compile.
The compiling page is only used while `PROACTIVE_COMPILE_ENABLED` is on (the default) — otherwise nothing would finish the build asynchronously, and the runtime waits for the on-request wasm compile.
## page_compiler_error

View File

@ -23,7 +23,7 @@ The returned tree includes:
- request and invocation counters
- best, worst, last, and average render time
- compile counters plus best, worst, last, and average compile time
- file mtimes, stale status, ABI compatibility status, load status, and exported API declarations
- file mtimes, stale status, ABI compatibility status, wasm availability, and exported API declarations
If `path` is relative, it is resolved relative to the current executing unit.

View File

@ -166,7 +166,7 @@ RENDER(Request& context)
<h1>Make dynamic websites like it's 2006.</h1>
<p class="hero-lead">
UCE is a deliberately direct web runtime: files are pages, templates live next to control flow,
nginx fronts the whole thing, and a request can compile into a shared object on first hit.
nginx fronts the whole thing, and a request can compile into a wasm module on first hit.
It is built in the image of PHP's non-architecture, but with an explicit request object,
component rendering, and a built-in WebSocket broker.
</p>
@ -177,7 +177,7 @@ RENDER(Request& context)
</div>
<div class="signal-grid">
<? render_signal("Files are routes", "A .uce page is the thing you edit and the thing nginx serves."); ?>
<? render_signal("Compile on demand", "First request builds a shared object, later requests reuse it."); ?>
<? render_signal("Compile on demand", "First request builds a wasm module, later requests reuse it."); ?>
<? render_signal("WebSockets included", "Same runtime, same page path, broker-managed connection state."); ?>
</div>
</div>
@ -314,7 +314,7 @@ RENDER(Request& context)
</details>
<details>
<summary>What makes it different from old PHP?</summary>
<p>The request object is explicit, components and named handlers are native, markdown support is built in, and WebSockets use a broker-owned connection model inside the runtime.</p>
<p>The request object is explicit, components and named handlers are first-class, markdown support is built in, and WebSockets use a broker-owned connection model inside the runtime.</p>
</details>
</section>
</main>

View File

@ -155,6 +155,10 @@ void cli_run_demo_smoke()
void cli_run_http_smoke()
{
cli_expect_http("uce_http_smoke:doc index", "/doc/index.uce", 200, "<html>");
// Regression guard: the index enumerates pages via ls("pages/"). When that
// returned empty (wasm ls() was a stub), the page still rendered <html> but
// listed nothing — so assert the function grid is actually populated.
cli_expect_http("uce_http_smoke:doc index lists functions", "/doc/index.uce", 200, "func-item");
cli_expect_http("uce_http_smoke:doc singlepage", "/doc/singlepage.uce", 200, "<html>");
cli_expect_http("uce_http_smoke:doc component page", "/doc/index.uce?p=component", 200, "component()");
cli_expect_http("uce_http_smoke:doc regex page", "/doc/index.uce?p=regex_search", 200, "regex_search");

View File

@ -49,33 +49,19 @@ RENDER(Request& context)
check("zip_read()", hello == "Hello ZIP", hello);
check("zip_extract()", extracted && nested == "Nested file" && value == "42", nested + " / " + value);
bool unsafe_rejected = false;
try
{
DValue unsafe_entries;
unsafe_entries["/absolute.txt"] = "bad";
zip_create(path_join(base, "unsafe.zip"), unsafe_entries);
}
catch(std::exception& e)
{
unsafe_rejected = contains(e.what(), "unsafe");
}
// Unsafe member names are rejected host-side; the membrane reports the
// failure as a false return (no thrown exception crosses into the unit).
DValue unsafe_entries;
unsafe_entries["/absolute.txt"] = "bad";
bool unsafe_rejected = !zip_create(path_join(base, "unsafe.zip"), unsafe_entries);
check("zip_create() rejects unsafe names", unsafe_rejected, "absolute member name rejected");
bool nul_name_rejected = false;
try
{
DValue unsafe_entries;
String nul_name = "prefix";
nul_name.push_back((char)0x00);
nul_name += "suffix.txt";
unsafe_entries[nul_name] = "bad";
zip_create(path_join(base, "nul-name.zip"), unsafe_entries);
}
catch(std::exception& e)
{
nul_name_rejected = contains(e.what(), "unsafe");
}
DValue nul_entries;
String nul_name = "prefix";
nul_name.push_back((char)0x00);
nul_name += "suffix.txt";
nul_entries[nul_name] = "bad";
bool nul_name_rejected = !zip_create(path_join(base, "nul-name.zip"), nul_entries);
check("zip_create() rejects NUL entry names", nul_name_rejected, "embedded NUL member name rejected");
String binary_source("UCE", 3);
@ -98,15 +84,9 @@ RENDER(Request& context)
check("gz_compress()", gz_body.size() > gz_source.size() && (u8)gz_body[0] == 0x1f && (u8)gz_body[1] == 0x8b, "bytes=" + std::to_string((u64)gz_body.size()));
check("gz_uncompress()", gz_roundtrip == gz_source && gz_binary_roundtrip == binary_source && gz_binary_roundtrip.size() == binary_source.size(), "text=" + gz_roundtrip + ", binary bytes=" + std::to_string((u64)gz_binary_roundtrip.size()));
bool bad_gz_rejected = false;
try
{
gz_uncompress("not gzip data");
}
catch(std::exception& e)
{
bad_gz_rejected = contains(e.what(), "gz_uncompress");
}
// Invalid gzip input fails host-side; the membrane returns an empty result
// rather than throwing into the unit.
bool bad_gz_rejected = gz_uncompress("not gzip data") == "";
check("gz_uncompress() rejects invalid data", bad_gz_rejected, "invalid stream rejected");
site_tests_summary(passed, failed, skipped, "ZIP tests write only under /tmp/uce-site-tests-zip.");

View File

@ -564,7 +564,10 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
parsed_content.resize(parsed_content.length() - current_line.length());
nibble(current_line, "\"");
String unit_name = nibble(current_line, "\"");
SharedUnit* sub_su = compiler_load_shared_unit(context, unit_name, su->src_path, true);
String resolved_unit = unit_name;
if(resolved_unit != "" && resolved_unit[0] != '/')
resolved_unit = expand_path(resolved_unit, su->src_path);
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit(context, resolved_unit));
if(sub_su)
parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n");
}

File diff suppressed because it is too large Load Diff

View File

@ -13,16 +13,9 @@ String preprocess_shared_unit(Request* context, SharedUnit* su);
String compiler_generated_cpp_path(Request* context, String source_file);
String compiler_generated_cpp_path(SharedUnit* su);
void setup_unit_paths(Request* context, SharedUnit* su, String file_name);
void load_shared_unit(Request* context, SharedUnit* su);
void compile_shared_unit(Request* context, SharedUnit* su);
SharedUnit* get_shared_unit(Request* context, String file_name, bool opt_so_optional = false);
void compiler_invoke(Request* context, String file_name);
void compiler_invoke_websocket(Request* context, String file_name);
void compiler_invoke_cli(Request* context, String file_name);
void compiler_invoke_serve_http(Request* context, String file_name, String handler_name = "");
SharedUnit* compiler_load_shared_unit(Request* context, String file_name, String current_path = "", bool opt_so_optional = false);
SharedUnit* get_shared_unit(Request* context, String file_name);
String compiler_error_page_unit(Request* context, String config_key);
bool compiler_render_error_page(Request* context, String config_key, s32 status_code, String status_reason, DValue error_info);
bool compiler_unit_compile_pending(Request* context, String file_name);
String compiler_site_directory(Request* context);
StringList compiler_scan_site_units(Request* context);

172
src/lib/fcgi_forward.h Normal file
View File

@ -0,0 +1,172 @@
#pragma once
// Minimal FastCGI client used by the connection brokers (the custom HTTP server
// dispatcher and the websocket exec child) to render a request through a normal
// worker on /run/uce.sock instead of rendering wasm in the broker's own forked
// process. Wasmtime cannot be safely re-created across fork, so the broker owns
// the connection but forwards the actual unit invocation to a clean-engine
// worker — the "broker holds connections, units respond like RENDER()" model.
//
// Native-only; compiled into the main object. Needs String/StringMap (uce_lib.h)
// plus the socket headers below.
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
#include <cstring>
#include <algorithm>
struct FcgiForwardResult {
bool ok = false;
String error;
int status = 200; // parsed from a CGI "Status:" header if present
StringMap headers; // response headers (excluding Status)
String body; // response body
};
// One FastCGI stream (PARAMS/STDIN): data records (<=64KB each) then the
// terminating empty record the protocol requires to close the stream.
inline void fcgi_forward_stream(String& out, unsigned char type, const String& content)
{
size_t off = 0, len = content.size();
while(off < len)
{
size_t chunk = std::min(len - off, (size_t)0xffff);
unsigned char hdr[8] = { 1, type, 0, 1, (unsigned char)((chunk >> 8) & 0xff), (unsigned char)(chunk & 0xff), 0, 0 };
out.append((const char*)hdr, 8);
out.append(content.data() + off, chunk);
off += chunk;
}
unsigned char term[8] = { 1, type, 0, 1, 0, 0, 0, 0 };
out.append((const char*)term, 8);
}
// FastCGI name/value pair length prefix: 1 byte if < 128, else 4 bytes (high bit set).
inline void fcgi_forward_put_len(String& out, size_t n)
{
if(n < 128)
out.push_back((char)(unsigned char)n);
else
{
out.push_back((char)(unsigned char)(((n >> 24) & 0xff) | 0x80));
out.push_back((char)(unsigned char)((n >> 16) & 0xff));
out.push_back((char)(unsigned char)((n >> 8) & 0xff));
out.push_back((char)(unsigned char)(n & 0xff));
}
}
// Build the full FastCGI request bytes (BEGIN_REQUEST + PARAMS + STDIN) for a
// RESPONDER request id 1. The broker reuses this to fire renders non-blocking.
inline String fcgi_build_request(const StringMap& params, const String& stdin_body)
{
String request;
unsigned char begin[16] = { 1, 1, 0, 1, 0, 8, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0 };
request.append((const char*)begin, 16);
String params_encoded;
for(auto& kv : params)
{
fcgi_forward_put_len(params_encoded, kv.first.size());
fcgi_forward_put_len(params_encoded, kv.second.size());
params_encoded.append(kv.first);
params_encoded.append(kv.second);
}
fcgi_forward_stream(request, 4 /*FCGI_PARAMS*/, params_encoded);
fcgi_forward_stream(request, 5 /*FCGI_STDIN*/, stdin_body);
return(request);
}
// Forward `params` + `stdin_body` to the FastCGI responder at unix `socket_path`
// and return the parsed CGI response. Times out (recv) at `timeout_seconds`.
inline FcgiForwardResult fcgi_forward_request(const String& socket_path,
const StringMap& params, const String& stdin_body, u32 timeout_seconds = 30)
{
FcgiForwardResult result;
int fd = ::socket(AF_UNIX, SOCK_STREAM, 0);
if(fd < 0)
{
result.error = "fcgi_forward: socket() failed";
return(result);
}
struct sockaddr_un addr;
memset(&addr, 0, sizeof(addr));
addr.sun_family = AF_UNIX;
strncpy(addr.sun_path, socket_path.c_str(), sizeof(addr.sun_path) - 1);
if(::connect(fd, (struct sockaddr*)&addr, sizeof(addr)) < 0)
{
::close(fd);
result.error = "fcgi_forward: connect(" + socket_path + ") failed";
return(result);
}
struct timeval tv;
tv.tv_sec = timeout_seconds;
tv.tv_usec = 0;
setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof(tv));
String request = fcgi_build_request(params, stdin_body);
if(::send(fd, request.data(), request.size(), MSG_NOSIGNAL) != (ssize_t)request.size())
{
::close(fd);
result.error = "fcgi_forward: short write to responder";
return(result);
}
// Read records; collect FCGI_STDOUT content until FCGI_END_REQUEST or EOF.
String inbuf, stdout_data;
char buf[65536];
bool ended = false;
while(!ended)
{
ssize_t n = ::recv(fd, buf, sizeof(buf), 0);
if(n <= 0)
break;
inbuf.append(buf, n);
while(inbuf.size() >= 8)
{
const unsigned char* h = (const unsigned char*)inbuf.data();
size_t content = ((size_t)h[4] << 8) + h[5];
size_t padding = h[6];
size_t record_len = 8 + content + padding;
if(inbuf.size() < record_len)
break;
unsigned char type = h[1];
if(type == 6 /*FCGI_STDOUT*/)
stdout_data.append(inbuf.data() + 8, content);
else if(type == 3 /*FCGI_END_REQUEST*/)
ended = true;
inbuf.erase(0, record_len);
}
}
::close(fd);
// Parse the CGI response: header block, then body. Status: header (if any)
// sets the HTTP status; everything else is a response header.
size_t sep = stdout_data.find("\r\n\r\n");
size_t sep_len = 4;
if(sep == String::npos)
{
sep = stdout_data.find("\n\n");
sep_len = 2;
}
String header_block = sep == String::npos ? String() : stdout_data.substr(0, sep);
result.body = sep == String::npos ? stdout_data : stdout_data.substr(sep + sep_len);
for(String line : split(header_block, "\n"))
{
line = trim(line);
if(line == "")
continue;
size_t colon = line.find(":");
if(colon == String::npos)
continue;
String name = trim(line.substr(0, colon));
String value = trim(line.substr(colon + 1));
if(to_lower(name) == "status")
result.status = (int)int_val(value);
else
result.headers[name] = value;
}
result.ok = true;
return(result);
}

View File

@ -48,7 +48,7 @@ inline String to_string(SharedUnit* u) {
result += String("SharedUnit( \n")+
"Source:"+(u->file_name)+"\n"+
"SharedObject:"+(u->so_name)+"\n"+
"Wasm:"+(u->wasm_name)+"\n"+
"API:"+(u->api_file_name)+"\n"+
to_string(u->api_declarations);

View File

@ -2,6 +2,7 @@
#include <cmath>
#include "types.h"
#include "functionlib.h"
#include "uri.h"
#include "sys.h"
extern "C" {
@ -17,6 +18,9 @@ int uce_host_file_exists(const char* path, size_t path_len, const char* current,
size_t uce_host_file_read(const char* path, size_t path_len, const char* current, size_t current_len, char* buf, size_t cap);
int uce_host_file_write(const char* path, size_t path_len, const char* current, size_t current_len, const char* content, size_t content_len, int append);
void uce_host_file_unlink(const char* path, size_t path_len, const char* current, size_t current_len);
size_t uce_host_file_list(const char* path, size_t path_len, const char* current, size_t current_len, char* buf, size_t cap);
int uce_host_file_mkdir(const char* path, size_t path_len, const char* current, size_t current_len);
int64_t uce_host_file_mtime(const char* path, size_t path_len, const char* current, size_t current_len);
int uce_host_task_spawn(const char* key, size_t key_len, uint64_t callback_id, double interval, uint64_t timeout, int repeat);
int uce_host_task_pid(const char* key, size_t key_len);
int uce_host_task_kill(int pid, int sig);
@ -43,7 +47,11 @@ String dirname(String fn) { auto pos = fn.find_last_of('/'); return(pos == Strin
String path_join(String base, String child) { if(base == "") return(child); if(child == "") return(base); if(child[0] == '/') return(child); return(base + (base.back() == '/' ? "" : "/") + child); }
String path_real(String path) { return(path); }
bool path_is_within(String path, String root) { return(str_starts_with(path, root)); }
bool mkdir(String path) { (void)path; return(false); }
bool mkdir(String path)
{
String current = wasm_current_unit_file();
return(uce_host_file_mkdir(path.data(), path.size(), current.data(), current.size()) != 0);
}
bool file_exists(String path)
{
String current = wasm_current_unit_file();
@ -78,14 +86,30 @@ bool file_append_contents(String file_name, String content)
String cwd_get() { return("/"); }
void cwd_set(String path) { (void)path; }
String process_start_directory() { return("/"); }
time_t file_mtime(String file_name) { (void)file_name; return(0); }
time_t file_mtime(String file_name)
{
String current = wasm_current_unit_file();
return((time_t)uce_host_file_mtime(file_name.data(), file_name.size(), current.data(), current.size()));
}
void file_unlink(String file_name)
{
String current = wasm_current_unit_file();
uce_host_file_unlink(file_name.data(), file_name.size(), current.data(), current.size());
}
String expand_path(String path, String relative_to_path) { return(path_join(relative_to_path, path)); }
StringList ls(String dir) { (void)dir; return(StringList()); }
StringList ls(String dir)
{
String current = wasm_current_unit_file();
size_t required = uce_host_file_list(dir.data(), dir.size(), current.data(), current.size(), 0, 0);
if(required == 0)
return(StringList());
String listing(required, 0);
size_t got = uce_host_file_list(dir.data(), dir.size(), current.data(), current.size(), &listing[0], required);
listing.resize(got <= required ? got : 0);
if(listing == "")
return(StringList());
return(split(listing, "\n"));
}
u64 config_map_u64(StringMap& cfg, String key, u64 fallback) { String raw = first(cfg[key], std::to_string(fallback)); char* end = 0; unsigned long long v = strtoull(raw.c_str(), &end, 10); return(end && *end == 0 ? (u64)v : fallback); }
f64 config_map_f64(StringMap& cfg, String key, f64 fallback) { String raw = first(cfg[key], std::to_string(fallback)); char* end = 0; double v = strtod(raw.c_str(), &end); return(end && *end == 0 ? (f64)v : fallback); }
bool config_bool_value(String raw, bool fallback) { if(raw == "") return(fallback); return(raw != "0" && raw != "false" && raw != "no" && raw != "off"); }
@ -169,11 +193,49 @@ String ws_connection_id() { return(context ? context->resources.websocket_connec
String ws_scope() { return(context ? context->resources.websocket_scope : ""); }
u8 ws_opcode() { return(context ? context->resources.websocket_opcode : 0); }
bool ws_is_binary() { return(context && context->resources.websocket_is_binary); }
StringList ws_connections(String scope) { (void)scope; return(StringList()); }
u64 ws_connection_count(String scope) { (void)scope; return(0); }
bool ws_send(String message, bool binary, String scope) { (void)message; (void)binary; (void)scope; return(false); }
bool ws_send_to(String connection_id, String message, bool binary) { (void)connection_id; (void)message; (void)binary; return(false); }
bool ws_close(String connection_id) { (void)connection_id; return(false); }
StringList ws_connections(String scope) { (void)scope; return(context ? context->resources.websocket_scope_connection_ids : StringList()); }
u64 ws_connection_count(String scope) { return(ws_connections(scope).size()); }
// The wasm workspace owns no connections; ws_send/ws_close record dispatch
// commands (same shape as the native websocket_exec capture) that the host
// carries back to the broker, which sends them over the real connections.
bool ws_send(String message, bool binary, String scope)
{
if(!context)
return(false);
DValue command;
command["action"] = "broadcast";
command["binary"].set_bool(binary);
command["message_b64"] = base64_encode(message);
command["scope"] = scope;
context->resources.websocket_dispatch_commands.push(command);
return(true);
}
bool ws_send_to(String connection_id, String message, bool binary)
{
if(!context)
return(false);
DValue command;
command["action"] = "send_to";
command["binary"].set_bool(binary);
command["message_b64"] = base64_encode(message);
command["connection_id"] = connection_id;
context->resources.websocket_dispatch_commands.push(command);
return(true);
}
bool ws_close(String connection_id)
{
if(!context)
return(false);
if(connection_id == "")
connection_id = ws_connection_id();
DValue command;
command["action"] = "close";
command["connection_id"] = connection_id;
command["status_code"] = (f64)1000;
command["reason"] = "";
context->resources.websocket_dispatch_commands.push(command);
return(true);
}
String backtrace_frames_string(void* const* frames, size_t size, u32 skip_frames) { (void)frames; (void)size; (void)skip_frames; return(""); }
String capture_backtrace_string(u32 max_frames, u32 skip_frames) { (void)max_frames; (void)skip_frames; return(""); }
String signal_name(int sig) { (void)sig; return(""); }
@ -1252,10 +1314,7 @@ StringMap make_server_settings()
StringMap cfg;
cfg["BIN_DIRECTORY"] = "/tmp/uce/work";
cfg["COMPILE_SCRIPT"] = "scripts/compile";
cfg["WASM_COMPILE_SCRIPT"] = "scripts/compile_wasm_unit";
cfg["COMPILE_WASM_UNITS"] = "0";
cfg["WASM_BACKEND_ENABLED"] = "1";
cfg["WASM_BACKEND_VERBOSE"] = "0";
cfg["WASM_CORE_PATH"] = "";
cfg["WASM_MEMORY_LIMIT_BYTES"] = std::to_string(512ull * 1024 * 1024);
@ -1266,6 +1325,9 @@ StringMap make_server_settings()
cfg["CONTENT_TYPE"] = "text/html; charset=utf-8";
cfg["FCGI_SOCKET_PATH"] = "/run/uce.sock";
cfg["CLI_SOCKET_PATH"] = "/run/uce/cli.sock";
// Command socket the WS broker listens on; workers flush ws_* dispatch
// command batches here at workspace teardown.
cfg["WS_BROKER_SOCKET_PATH"] = "/run/uce/ws-broker.sock";
cfg["TMP_UPLOAD_PATH"] = "/tmp/uce/uploads";
cfg["SESSION_PATH"] = "/tmp/uce/sessions";
cfg["COMPILER_SYS_PATH"] = ".";

View File

@ -79,12 +79,6 @@ String http_status_reason(s32 code)
SharedUnit::~SharedUnit()
{
#ifndef __UCE_WASM_CORE__
if(so_handle)
{
dlclose(so_handle);
}
#endif
}
String nibble(String div, String& haystack)

View File

@ -57,16 +57,14 @@ typedef std::ostringstream ByteStream;
struct Request;
struct DValue;
typedef void (*request_ref_handler)(Request& request);
typedef DValue* (*dv_call_handler)(DValue* call_param);
typedef void (*request_handler)(Request* request);
typedef void (*WasmRequestHandler)(Request& request);
typedef DValue* (*WasmDValueCallHandler)(DValue* call_param);
inline String to_string(s64 v) { return(std::to_string(v)); }
struct SharedUnit {
String file_name;
String so_name;
String wasm_name;
String wasm_check_file_name;
String api_file_name;
@ -74,33 +72,20 @@ struct SharedUnit {
String compile_output_file_name;
String setup_file_name;
StringList api_declarations;
std::map<String, void*> api_functions;
String src_path;
String bin_path;
String pre_path;
String src_file_name;
String bin_file_name;
String wasm_file_name;
String pre_file_name;
void* so_handle = 0;
request_handler on_setup = 0;
request_ref_handler on_render = 0;
request_ref_handler on_component = 0;
request_ref_handler on_websocket = 0;
request_ref_handler on_cli = 0;
request_ref_handler on_once = 0;
request_ref_handler on_init = 0;
String compiler_messages;
String compile_status = "unknown";
String compile_error_status = "";
String runtime_error_status = "";
time_t last_compiled = 0;
time_t observed_compiled_time = 0;
time_t last_loaded = 0;
time_t last_rendered = 0;
time_t last_error = 0;
String observed_metadata_content = "";
@ -123,8 +108,6 @@ struct SharedUnit {
f64 best_render_duration = 0;
f64 worst_render_duration = 0;
bool opt_so_optional = false;
~SharedUnit();
};
@ -153,8 +136,6 @@ String nibble(String div, String& haystack);
#include "dvalue.h"
void compiler_invoke(Request* context, String file_name);
struct Request {
ServerState* server = 0;

File diff suppressed because it is too large Load Diff

View File

@ -2,12 +2,8 @@
//
// Included into the native server TU (src/linux_fastcgi.cpp) after uce_lib.cpp,
// so it shares String/DValue/config and the UCEB1 codec. Provides a
// config-selectable page-render backend: when WASM_BACKEND_ENABLED and a wasm
// artifact exists for the entry unit, the request is served through a
// per-request wasm workspace instead of the native dlopen path.
//
// The seam is narrow on purpose — only the page render branch in
// handle_complete() changes. CLI, serve_http, and websocket stay native.
// Always-on wasm backend: every unit request is served through a per-request
// wasm workspace. The legacy native dlopen execution path has been removed.
#include "../lib/wasm_trace.h"
// The native server TU has the real connectors (sqlite/mysql) compiled in, so
@ -19,6 +15,8 @@
#include <atomic>
#include <sys/stat.h>
#include <thread>
// Worker flushes ws_* dispatch batches to the broker via the FastCGI client.
#include "../lib/fcgi_forward.h"
// per forked worker process: one engine + compiled-core cache, one epoch ticker
static WasmWorker* g_wasm_worker = 0;
@ -29,9 +27,7 @@ static bool g_wasm_init_attempted = false;
bool wasm_backend_configured(Request* context)
{
if(!context || !context->server)
return(false);
return(config_bool("WASM_BACKEND_ENABLED", false));
return(context && context->server);
}
// Lazily bring up the per-process worker on first use inside a forked child
@ -88,78 +84,23 @@ static bool wasm_artifact_exists(Request* context, const String& entry_unit)
struct stat wasm_st;
if(stat(wasm_path.c_str(), &wasm_st) != 0 || !S_ISREG(wasm_st.st_mode))
return(false);
// The wasm path bypasses the native JIT recompile, so a source edit would
// otherwise be served from a stale .wasm. Require the artifact to be newer
// than the source; if it is stale, fall back to native — which JIT-rebuilds
// both .so and .wasm — so the next request gets a fresh artifact.
// Require the artifact to be newer than the source. If it is stale,
// dispatch compiles the unit on demand and then rechecks this predicate.
struct stat src_st;
if(stat(entry_unit.c_str(), &src_st) == 0 && wasm_st.st_mtime < src_st.st_mtime)
if(stat(entry_unit.c_str(), &src_st) != 0 || !S_ISREG(src_st.st_mode))
return(false);
if(wasm_st.st_mtime < src_st.st_mtime)
return(false);
return(true);
}
// Decide, by source inspection, whether a page must stay on the native
// backend for now (W5 surfaces not yet behind the membrane).
//
// Caveats this deliberately accepts:
// - ENTRY-UNIT ONLY: a clean entry page that component()s into a native-only
// unit is still served by wasm. With signals_based_traps off (see
// make_engine) the worst case is a clean wasm error or a stubbed-empty
// result, never a worker crash — so this is a best-effort routing hint,
// not a guarantee. Full coverage is a W5 concern (un-stub the APIs).
// - SUBSTRING match, intentionally conservative: a token like "xml_" also
// matches an identifier or doc string containing it. That only ever
// over-routes to native (always correct), never the reverse.
static bool wasm_backend_native_fallback_uncached(Request* context, const String& entry_unit)
{
String source = file_get_contents(entry_unit);
// Supported by the wasm core, intentionally NOT in this list:
// - regex_* (host PCRE2 hostcall), xml_*/yaml_*/markdown_* (compiled in),
// - unit_render()/component() (host resolver).
// What remains is genuinely host-owned / native-only for now:
// - zip pages that use C++ try/catch around zip_* error cases;
// - direct compiler_load_shared_unit() access to native SharedUnit internals.
// Background tasks, sleep/usleep, sockets/custom servers, memcache, mysql,
// and unit_call/unit_compile/unit_info/units_list are host-owned but now have
// membrane hostcalls, so they intentionally do not fallback here.
StringList native_only_tokens = {
"zip_", "compiler_load_shared_unit("
};
for(auto& token : native_only_tokens)
if(source.find(token) != String::npos)
return(true);
return(false);
}
static bool wasm_backend_native_fallback_needed(Request* context, const String& entry_unit)
{
if(!context || !context->server || entry_unit == "")
return(true);
// Cache the verdict per process, keyed on path + mtime: the source scan is
// otherwise a file read + 20 substring searches on every page request.
struct CachedVerdict { time_t mtime; bool fallback; };
static std::map<String, CachedVerdict> cache;
struct stat st;
time_t mtime = (stat(entry_unit.c_str(), &st) == 0) ? st.st_mtime : 0;
auto it = cache.find(entry_unit);
if(it != cache.end() && it->second.mtime == mtime)
return(it->second.fallback);
bool fallback = wasm_backend_native_fallback_uncached(context, entry_unit);
cache[entry_unit] = { mtime, fallback };
return(fallback);
}
// True if this request should be served by the wasm backend. Falls through to
// native when disabled, when init failed, or when the unit has no wasm artifact
// (e.g. units skip-listed for try/catch — automatic, graceful fallback). Mode-
// agnostic: the caller (page render / cli / serve_http) decides which entry to
// route here; this only gates on config, fallback tokens, artifact, and worker.
// True if this request can be served by the wasm backend now. Every unit runs
// on wasm; cold/stale artifacts return false so dispatch can compile on demand
// and then recheck.
bool wasm_backend_should_handle(Request& request, const String& entry_unit)
{
if(!wasm_backend_configured(&request))
return(false);
if(wasm_backend_native_fallback_needed(&request, entry_unit))
return(false);
if(!wasm_artifact_exists(&request, entry_unit))
return(false);
if(wasm_backend_ensure_started(&request) != "")
@ -168,11 +109,12 @@ bool wasm_backend_should_handle(Request& request, const String& entry_unit)
}
// Serve a request through a wasm workspace using the unit handler selected by
// `kind` (page render or cli). Populates the native Request
// (status/headers/cookies/session/body) so the existing transport writes the
// response unchanged. Returns "" on success, or a collapsed error/trace string
// for the caller to route into the configured error page.
String wasm_backend_serve(Request& request, const String& entry_unit, int32_t kind = WasmWorkspace::RESOLVE_RENDER)
// `kind` (page render / cli / serve_http, with an optional named serve_http
// handler). Populates the native Request (status/headers/cookies/session/body)
// so the existing transport writes the response unchanged. Returns "" on
// success, or a collapsed error/trace string for the caller to route into the
// configured error page.
String wasm_backend_serve(Request& request, const String& entry_unit, const String& handler = "render")
{
DValue ctx;
auto copy_map = [&](const StringMap& source, const char* key) {
@ -186,19 +128,55 @@ String wasm_backend_serve(Request& request, const String& entry_unit, int32_t ki
copy_map(request.session, "session");
ctx["entry_unit"] = entry_unit;
// Raw request body: cli_input() parses a JSON CLI payload from context.in,
// carried into the workspace, not just the form-decoded post map.
// and serve_http handlers read it as req->in; carried into the workspace.
ctx["in"] = request.in;
// WebSocket event context: the workspace owns no connections, so the frame's
// connection identity goes in and the handler's ws_send/ws_close dispatch
// commands come back out (below) for the native broker to apply.
if(handler == "websocket")
{
ctx["ws"]["connection_id"] = request.resources.websocket_connection_id;
ctx["ws"]["scope"] = request.resources.websocket_scope;
ctx["ws"]["opcode"] = (f64)request.resources.websocket_opcode;
ctx["ws"]["binary"].set_bool(request.resources.websocket_is_binary);
for(auto& id : request.resources.websocket_scope_connection_ids)
{
DValue v; v = id;
ctx["ws"]["connections"].push(v);
}
ctx["ws"]["connection_state"] = request.connection;
}
WasmResponse response = wasm_worker_serve(*g_wasm_worker, ctx, entry_unit, kind);
WasmResponse response = wasm_worker_serve(*g_wasm_worker, ctx, entry_unit, handler);
if(!response.ok)
return(response.error == "" ? String("wasm workspace failed") : response.error);
// A cli unit that does not export __uce_cli is a 404, matching native
// compiler_invoke_cli ("CLI Entry Point Not Found"). For page render a
// missing handler is simply an empty body (native parity).
if(!response.handler_present && kind == WasmWorkspace::RESOLVE_CLI)
// Any handler may have called ws_send/ws_close (not just WS handlers). If it
// left dispatch commands, flush the batch to the central WS broker, which owns
// all connections and resolves each command's target (id/scope/broadcast)
// against the full registry. This is the only path WS data takes out — the
// workspace owns no connections.
if(DValue* cmds = response.meta.key("ws_commands"))
{
request.set_status(404, "CLI Entry Point Not Found");
DValue batch;
batch["commands"] = *cmds;
if(DValue* cstate = response.meta.key("ws_connection_state"))
{
batch["connection_id"] = request.resources.websocket_connection_id;
batch["connection_state"] = *cstate;
}
StringMap dispatch_params;
dispatch_params["UCE_WS_DISPATCH"] = "1";
String broker_socket = first(request.server ? request.server->config["WS_BROKER_SOCKET_PATH"] : String(),
"/run/uce/ws-broker.sock");
fcgi_forward_request(broker_socket, dispatch_params, ucb_encode(batch), 5);
}
// A cli/serve_http unit that does not export the requested handler is a 404.
// For page render a missing handler is simply an empty body.
if(!response.handler_present && handler != "render")
{
request.set_status(404, handler == "cli" ? "CLI Entry Point Not Found" : "Handler Not Found");
return("");
}

View File

@ -6,27 +6,15 @@
// backend.cpp — so it does not have to compile worker.cpp + wasmtime.hh on
// every build. Include only after uce_lib.h (needs Request / String).
#include <cstdint>
// Resolve-kind selector for wasm_backend_serve, mirrored from
// WasmWorkspace::ResolveKind in worker.cpp. Only the entry kinds actually wired
// to the wasm backend live here; WebSocket/serve_http re-enter when W7b/W7c land
// (via the broker→worker-pool forwarding model, not in-fork rendering).
namespace wasm_kind {
enum {
RENDER = 1,
CLI = 4,
};
}
// True if this request should be served by the wasm backend (config + artifact
// + fallback-token gate); mode-agnostic, the caller picks the kind.
// True if this request can be served by the wasm backend for the named unit
// artifact; handler-agnostic, the caller names the handler.
bool wasm_backend_should_handle(Request& request, const String& entry_unit);
// Serve a request through a wasm workspace using the unit handler for `kind`,
// populating the native Request. Returns "" on success or a collapsed error.
String wasm_backend_serve(Request& request, const String& entry_unit,
int32_t kind = wasm_kind::RENDER);
// Serve a request through a wasm workspace by invoking a named unit handler —
// "render", "cli", "websocket", "serve_http", "serve_http:named" — and populate
// the native Request. Returns "" on success or a collapsed error. The handler is
// just an export name; there is no per-mode machinery.
String wasm_backend_serve(Request& request, const String& entry_unit, const String& handler = "render");
// Join the per-process epoch ticker before the worker process exits.
void wasm_backend_shutdown();

View File

@ -139,32 +139,6 @@ bool unit_compile(String path)
}
static DValue wasm_unit_call_result;
DValue* unit_call(String file_name, String function_name, DValue* call_param)
{
DValue request;
request["op"] = "call";
request["file"] = file_name;
request["function"] = function_name;
if(call_param)
request["param"] = *call_param;
DValue response = wasm_units_call(request);
String output = response["output"].to_string();
if(output != "")
print(output);
DValue* result = response.key("result");
if(result)
{
wasm_unit_call_result = *result;
return(&wasm_unit_call_result);
}
return(0);
}
SharedUnit* compiler_load_shared_unit(Request* context, String file_name, String current_path, bool opt_so_optional)
{
(void)context; (void)file_name; (void)current_path; (void)opt_so_optional;
return(0);
}
static DValue wasm_mysql_call(DValue request)
{
@ -475,7 +449,8 @@ extern "C" void uce_wasm_link_anchors()
// slots (loading units lazily) and writes the resolved unit path back so
// nested relative component resolution keeps working.
extern "C" int32_t uce_host_component_resolve(
const char* target, size_t target_len, int32_t kind,
const char* target, size_t target_len,
const char* handler, size_t handler_len,
const char* current_unit, size_t current_unit_len,
char* resolved_buf, size_t resolved_cap);
@ -483,9 +458,9 @@ extern "C" int32_t uce_host_component_resolve(
// but a single workspace can render the same component many times)
static std::map<String, s32> wasm_component_slots;
// These mirror small page-runtime pieces of compiler.cpp, which is carved
// out of the wasm core wholesale (it is the native toolchain: parser, clang
// driver, dlopen). Kept byte-identical where possible.
// These mirror small page-runtime pieces that cannot include compiler.cpp in
// the wasm core (compiler.cpp owns parser/clang/cache bookkeeping for the host
// build). Kept byte-identical where possible.
String component_normalize_path(String name)
{
name = trim(name);
@ -534,38 +509,37 @@ struct RequestPropsScope
}
};
// kind values shared with the host loader (src/wasm/worker.cpp)
enum WasmResolveKind {
WASM_RESOLVE_COMPONENT = 0,
WASM_RESOLVE_RENDER = 1,
WASM_RESOLVE_EXISTS = 2,
WASM_RESOLVE_ONCE = 3,
WASM_RESOLVE_CLI = 4,
};
static s32 wasm_resolve_target(String target, s32 kind, String* resolved_out = 0)
// A unit is a bag of exported handlers; invoking any of them is one operation —
// the host resolves __uce_<handler> in the loaded module to a funcref slot. The
// handler is just a string: "render", "component:CARD", "render:VARIANT",
// "once", "cli", "websocket", "serve_http:named" — or "exists" (an existence
// probe that loads nothing). No per-mode kinds.
static s32 wasm_resolve_target(String unit_target, String handler, String* resolved_out = 0)
{
String cache_key = std::to_string(kind) + ":" + target;
String cache_key = handler + "\t" + unit_target;
bool is_exists = (handler == "exists");
auto cached = wasm_component_slots.find(cache_key);
if(cached != wasm_component_slots.end() && kind != WASM_RESOLVE_EXISTS)
if(cached != wasm_component_slots.end() && !is_exists)
return(cached->second);
char resolved[512];
String current = context ? context->resources.current_unit_file : "";
s32 slot = uce_host_component_resolve(
target.data(), target.size(), kind,
unit_target.data(), unit_target.size(), handler.data(), handler.size(),
current.data(), current.size(),
resolved, sizeof(resolved));
if(resolved_out && slot)
*resolved_out = String(resolved, strnlen(resolved, sizeof(resolved)));
if(kind != WASM_RESOLVE_EXISTS)
if(!is_exists)
wasm_component_slots[cache_key] = slot;
return(slot);
}
String component_resolve(String name)
{
String file_name, render_name;
component_parse_target(trim(name), file_name, render_name);
String resolved;
if(wasm_resolve_target(trim(name), WASM_RESOLVE_EXISTS, &resolved))
if(wasm_resolve_target(file_name, "exists", &resolved))
return(resolved);
return("");
}
@ -585,20 +559,79 @@ static void wasm_run_once(const String& resolved, Request& request)
if(request.once_units.find(resolved) != request.once_units.end())
return;
request.once_units.insert(resolved);
s32 once_slot = wasm_resolve_target(resolved, WASM_RESOLVE_ONCE);
s32 once_slot = wasm_resolve_target(resolved, "once");
if(once_slot == 0)
return;
String previous_unit = request.resources.current_unit_file;
request.resources.current_unit_file = resolved;
request_ref_handler once_handler = (request_ref_handler)(uintptr_t)once_slot;
WasmRequestHandler once_handler = (WasmRequestHandler)(uintptr_t)once_slot;
once_handler(request);
request.resources.current_unit_file = previous_unit;
}
DValue* unit_call(String file_name, String function_name, DValue* call_param)
{
String macro = to_upper(trim(function_name));
String handler = "";
if(macro == "RENDER" || macro.rfind("RENDER:", 0) == 0)
handler = "render" + (macro.length() > 7 ? ":" + trim(function_name.substr(function_name.find(":") + 1)) : String(""));
else if(macro == "COMPONENT" || macro.rfind("COMPONENT:", 0) == 0)
handler = "component" + (macro.length() > 10 ? ":" + trim(function_name.substr(function_name.find(":") + 1)) : String(""));
else if(macro == "ONCE")
handler = "once";
else if(macro == "INIT")
handler = "init";
if(handler != "")
{
String resolved;
s32 slot = wasm_resolve_target(file_name, handler, &resolved);
if(!slot)
{
print("Error: unit_call() function '", function_name, "' not found");
return(0);
}
RequestPropsScope props_scope(context, (call_param ? *call_param : DValue()));
if((handler == "render" || handler.rfind("render:", 0) == 0 || handler == "component" || handler.rfind("component:", 0) == 0) && resolved != "")
wasm_run_once(resolved, *context);
String previous_unit = context->resources.current_unit_file;
if(resolved != "")
context->resources.current_unit_file = resolved;
WasmRequestHandler handler_fn = (WasmRequestHandler)(uintptr_t)slot;
handler_fn(*context);
context->resources.current_unit_file = previous_unit;
return(0);
}
String resolved;
s32 slot = wasm_resolve_target(file_name, "export:" + function_name, &resolved);
if(!slot)
{
print("Error: unit_call() function '", function_name, "' not found");
return(0);
}
String previous_unit = context->resources.current_unit_file;
if(resolved != "")
context->resources.current_unit_file = resolved;
WasmDValueCallHandler handler_fn = (WasmDValueCallHandler)(uintptr_t)slot;
DValue* result = handler_fn(call_param);
context->resources.current_unit_file = previous_unit;
if(result)
{
wasm_unit_call_result = *result;
return(&wasm_unit_call_result);
}
return(0);
}
void component_render(String name, DValue props, Request& request)
{
String file_name, render_name;
component_parse_target(trim(name), file_name, render_name);
String handler = render_name == "" ? String("component") : "component:" + render_name;
String resolved;
s32 slot = wasm_resolve_target(trim(name), WASM_RESOLVE_COMPONENT, &resolved);
s32 slot = wasm_resolve_target(file_name, handler, &resolved);
if(!slot)
{
print(component_error_banner("component not found: " + trim(name)));
@ -611,8 +644,8 @@ void component_render(String name, DValue props, Request& request)
request.resources.current_unit_file = resolved;
// a wasm function pointer is its index in the shared funcref table; the
// host returned the handler's slot, so this is a plain call_indirect
request_ref_handler handler = (request_ref_handler)(uintptr_t)slot;
handler(request);
WasmRequestHandler handler_fn = (WasmRequestHandler)(uintptr_t)slot;
handler_fn(request);
request.resources.current_unit_file = previous_unit;
}
@ -633,8 +666,11 @@ String component(String name, DValue props) { return(component(name, props, *con
void unit_render(String file_name, Request& request)
{
String unit_name, render_name;
component_parse_target(trim(file_name), unit_name, render_name);
String handler = render_name == "" ? String("render") : "render:" + render_name;
String resolved;
s32 slot = wasm_resolve_target(trim(file_name), WASM_RESOLVE_RENDER, &resolved);
s32 slot = wasm_resolve_target(unit_name, handler, &resolved);
if(!slot)
{
print(component_error_banner("unit not found: " + trim(file_name)));
@ -644,8 +680,8 @@ void unit_render(String file_name, Request& request)
String previous_unit = request.resources.current_unit_file;
if(resolved != "")
request.resources.current_unit_file = resolved;
request_ref_handler handler = (request_ref_handler)(uintptr_t)slot;
handler(request);
WasmRequestHandler handler_fn = (WasmRequestHandler)(uintptr_t)slot;
handler_fn(request);
request.resources.current_unit_file = previous_unit;
}
@ -653,25 +689,25 @@ void unit_render(String file_name) { unit_render(file_name, *context); }
extern "C" {
// Host calls this to invoke the request's entry unit through one of its
// directive handlers (render/cli/websocket/serve_http). Routing through the
// resolver gives the entry the same ONCE() + dispatch semantics as components
// (the host pre-loaded it, so resolution is a cache hit). The host pre-checks
// that the handler export exists, so a missing slot here is a silent no-op.
void uce_wasm_invoke_entry(const char* path, size_t len, int32_t kind)
// Host calls this to invoke the request's entry unit through a named handler
// ("render"/"cli"/"websocket"/"serve_http:named"). It is the same resolve +
// ONCE() + dispatch path as components — the entry handler is just another
// export. The host pre-checks the export exists, so a missing slot is a no-op.
void uce_wasm_invoke_entry(const char* path, size_t path_len, const char* handler, size_t handler_len)
{
// the host always runs uce_wasm_core_init + apply_context before this
String file_name(path, len);
String file_name(path, path_len);
String handler_name(handler, handler_len);
String resolved;
s32 slot = wasm_resolve_target(trim(file_name), kind, &resolved);
s32 slot = wasm_resolve_target(trim(file_name), handler_name, &resolved);
if(!slot)
return;
wasm_run_once(resolved, *context);
String previous_unit = context->resources.current_unit_file;
if(resolved != "")
context->resources.current_unit_file = resolved;
request_ref_handler handler = (request_ref_handler)(uintptr_t)slot;
handler(*context);
WasmRequestHandler handler_fn = (WasmRequestHandler)(uintptr_t)slot;
handler_fn(*context);
context->resources.current_unit_file = previous_unit;
}
@ -761,6 +797,27 @@ int uce_wasm_apply_context(const char* buf, size_t len)
wasm_request.resources.current_unit_file = entry->to_string();
DValue* raw_in = decoded.key("in");
wasm_request.in = raw_in ? raw_in->to_string() : "";
// websocket event context: ws_send()/ws_close() capture into the dispatch
// list (the workspace owns no connections), which collect() carries back to
// the broker. Reset per invocation.
wasm_request.resources.websocket_dispatch_commands = DValue();
wasm_request.resources.websocket_dispatch_capture = false;
DValue* ws = decoded.key("ws");
if(ws)
{
wasm_request.resources.websocket_connection_id = (*ws)["connection_id"].to_string();
wasm_request.resources.websocket_scope = (*ws)["scope"].to_string();
wasm_request.resources.websocket_opcode = (u8)(*ws)["opcode"].to_u64();
wasm_request.resources.websocket_is_binary = (*ws)["binary"].to_bool();
wasm_request.resources.websocket_scope_connection_ids.clear();
if(DValue* conns = ws->key("connections"))
conns->each([&](const DValue& v, String) {
wasm_request.resources.websocket_scope_connection_ids.push_back(v.to_string());
});
wasm_request.resources.websocket_dispatch_capture = true;
if(DValue* cstate = ws->key("connection_state"))
wasm_request.connection = *cstate;
}
return(0);
}
@ -787,6 +844,14 @@ void uce_wasm_finish_response_meta()
}
for(auto& entry : wasm_request.session)
meta["session"][entry.first] = entry.second;
// Any unit code (not just WS handlers) may call ws_send/ws_close; whenever the
// dispatch list is non-empty, carry it back so the worker can flush it to the
// broker. ws_connection_state rides along for stateful WS handlers.
if(!wasm_request.resources.websocket_dispatch_commands._map.empty())
{
meta["ws_commands"] = wasm_request.resources.websocket_dispatch_commands;
meta["ws_connection_state"] = wasm_request.connection;
}
wasm_response_meta = ucb_encode(meta);
}

View File

@ -22,5 +22,8 @@ uce_host_file_exists
uce_host_file_read
uce_host_file_write
uce_host_file_unlink
uce_host_file_list
uce_host_file_mkdir
uce_host_file_mtime
uce_host_regex
uce_host_sqlite

View File

@ -42,6 +42,9 @@
#include <unistd.h>
#include <sys/time.h>
#include <vector>
#include <algorithm>
#include <dirent.h>
#include <cerrno>
struct WasmDylinkInfo
{
@ -388,8 +391,6 @@ public:
// resolve-kind values shared with the guest core (src/wasm/core.cpp)
// must match WasmResolveKind in src/wasm/core.cpp
enum ResolveKind { RESOLVE_COMPONENT = 0, RESOLVE_RENDER = 1, RESOLVE_EXISTS = 2, RESOLVE_ONCE = 3,
RESOLVE_CLI = 4 };
String birth()
{
@ -482,38 +483,52 @@ public:
return("");
}
// Invoke the entry unit through one of its directive handlers
// (render/cli/websocket/serve_http, selected by kind). handler_present, when
// provided, reports whether the unit actually exports that handler — the
// caller maps a missing render to an empty body (native parity) and a
// missing cli/serve handler to a 404.
String invoke_entry(const String& entry_source_path, int32_t kind, bool* handler_present = 0)
// Invoke the entry unit through a named handler ("render"/"cli"/"websocket"/
// "serve_http:named"). The handler is just an export name — same resolve +
// ONCE + dispatch path as a component. handler_present, when provided,
// reports whether the unit exports it (caller maps a missing render to an
// empty body, a missing cli/serve handler to a 404).
String invoke_entry(const String& entry_source_path, const String& handler, bool* handler_present = 0)
{
entry_dir = dir_of(entry_source_path);
size_t unit_index = 0;
String error = load_unit(entry_source_path, unit_index);
if(error != "")
return(error);
String symbol = kind == RESOLVE_CLI ? "__uce_cli" : "__uce_render";
bool present = (bool)unit_func(unit_index, symbol);
bool present = (bool)unit_func(unit_index, handler_export_symbol(handler));
if(handler_present)
*handler_present = present;
if(!present)
return("");
// Invoke through the core so the entry gets the same ONCE() + dispatch
// path as components (unit is pre-loaded above; resolution is cached).
// Invoke through the core: it resolves the same handler and runs ONCE +
// dispatch (unit is pre-loaded above; resolution is cached). The core
// entry takes two guest buffers — the unit path and the handler name.
auto entry = core_func("uce_wasm_invoke_entry");
if(!entry)
return("core does not export uce_wasm_invoke_entry");
int32_t guest_ptr = 0;
error = call_core("uce_alloc", { (int32_t)entry_source_path.size() }, &guest_ptr);
if(error != "" || guest_ptr == 0)
int32_t path_ptr = 0, handler_ptr = 0;
error = call_core("uce_alloc", { (int32_t)entry_source_path.size() }, &path_ptr);
if(error != "" || path_ptr == 0)
return(error == "" ? String("guest uce_alloc failed for entry path") : error);
error = guest_write((u32)guest_ptr, entry_source_path);
error = guest_write((u32)path_ptr, entry_source_path);
if(error == "")
{
error = call_core("uce_alloc", { (int32_t)handler.size() }, &handler_ptr);
if(error == "" && handler_ptr == 0)
error = "guest uce_alloc failed for handler";
}
if(error == "")
error = guest_write((u32)handler_ptr, handler);
if(error != "")
{
if(path_ptr) call_core("uce_free", { path_ptr }, 0);
if(handler_ptr) call_core("uce_free", { handler_ptr }, 0);
return(error);
auto result = entry->call(ctx(), { wasmtime::Val(guest_ptr), wasmtime::Val((int32_t)entry_source_path.size()), wasmtime::Val(kind) });
call_core("uce_free", { guest_ptr }, 0);
}
auto result = entry->call(ctx(), { wasmtime::Val(path_ptr), wasmtime::Val((int32_t)entry_source_path.size()),
wasmtime::Val(handler_ptr), wasmtime::Val((int32_t)handler.size()) });
call_core("uce_free", { path_ptr }, 0);
call_core("uce_free", { handler_ptr }, 0);
if(!result)
return(trap_text(result.err()));
return("");
@ -521,7 +536,7 @@ public:
String render_entry(const String& entry_source_path)
{
return(invoke_entry(entry_source_path, RESOLVE_RENDER));
return(invoke_entry(entry_source_path, "render"));
}
String collect(WasmResponse& response)
@ -891,6 +906,12 @@ private:
return(stat(path.c_str(), &st) == 0 && S_ISREG(st.st_mode));
}
static bool dir_exists_host(const String& path)
{
struct stat st;
return(stat(path.c_str(), &st) == 0 && S_ISDIR(st.st_mode));
}
String resolve_source_path(const String& file_name, const String& current_unit)
{
std::vector<String> bases;
@ -926,7 +947,7 @@ private:
// guest file access policy: only inside the site tree, resolved against
// the entry unit's directory first (the native cwd convention), then the
// site root; containment checked on the canonicalized path
String resolve_guest_file(const String& raw, const String& current_unit = "")
String resolve_guest_file(const String& raw, const String& current_unit = "", bool allow_dir = false)
{
if(raw == "" || raw.find('\0') != String::npos)
return("");
@ -970,7 +991,7 @@ private:
}
if(!allowed)
continue;
if(file_exists_host(path))
if(file_exists_host(path) || (allow_dir && dir_exists_host(path)))
return(path);
}
return("");
@ -1022,8 +1043,23 @@ private:
return(result);
}
// hostcall body: uce_host_component_resolve(target, kind, current) → slot
int32_t component_resolve(const String& target, int32_t kind, const String& current_unit, String& resolved_out)
// __uce_<base>[_<suffix>] for a handler spec like "component:CARD" / "render".
static String handler_export_symbol(const String& handler)
{
String export_prefix = "export:";
if(handler.rfind(export_prefix, 0) == 0)
return(handler.substr(export_prefix.size()));
auto colon = handler.find(":");
String symbol = "__uce_" + (colon == String::npos ? handler : handler.substr(0, colon));
if(colon != String::npos)
symbol += "_" + sanitize_symbol_suffix(handler.substr(colon + 1));
return(symbol);
}
// hostcall body: uce_host_component_resolve(unit, handler, current) → slot.
// `handler` names the export ("render", "component:CARD", "cli",
// "serve_http:named", "once") or is "exists" (probe only, loads nothing).
int32_t component_resolve(const String& target, const String& handler, const String& current_unit, String& resolved_out)
{
auto probe_start = std::chrono::steady_clock::now();
auto record_probe = [&]() {
@ -1032,13 +1068,6 @@ private:
std::chrono::steady_clock::now() - probe_start).count();
};
String file_name = target;
String render_name;
auto split = target.find(":");
if(split != String::npos)
{
render_name = target.substr(split + 1);
file_name = target.substr(0, split);
}
if(file_name == "" && current_unit != "")
file_name = current_unit;
if(file_name == "")
@ -1054,12 +1083,15 @@ private:
return(0);
}
resolved_out = resolved;
if(kind == RESOLVE_EXISTS)
if(handler == "exists")
{
record_probe();
return(1);
}
if(!file_exists_host(worker.unit_wasm_path(resolved)) || compiler_unit_needs_recompile(context, resolved, 0))
get_shared_unit(context, resolved);
size_t unit_index = 0;
String error = load_unit(resolved, unit_index);
if(error != "")
@ -1068,12 +1100,7 @@ private:
record_probe();
return(0);
}
String symbol = kind == RESOLVE_RENDER ? "__uce_render"
: kind == RESOLVE_ONCE ? "__uce_once"
: kind == RESOLVE_CLI ? "__uce_cli"
: "__uce_component";
if(render_name != "")
symbol += "_" + sanitize_symbol_suffix(render_name);
String symbol = handler_export_symbol(handler);
String slot_key = resolved + ":" + symbol;
auto cached = handler_slots.find(slot_key);
if(cached != handler_slots.end())
@ -1081,17 +1108,17 @@ private:
record_probe();
return((int32_t)cached->second);
}
auto handler = unit_func(unit_index, symbol);
if(!handler)
auto handler_fn = unit_func(unit_index, symbol);
if(!handler_fn)
{
// ONCE is optional per unit; a missing __uce_once is not an error
if(kind != RESOLVE_ONCE)
if(handler != "once")
fprintf(stderr, "[wasm] %s does not export %s\n", resolved.c_str(), symbol.c_str());
record_probe();
return(0);
}
u32 slot = 0;
error = place_funcref(*handler, slot);
error = place_funcref(*handler_fn, slot);
if(error != "")
{
fprintf(stderr, "[wasm] %s\n", error.c_str());
@ -1184,6 +1211,31 @@ private:
results[0] = Val(resolved != "" ? (int32_t)1 : (int32_t)0);
return(std::monostate());
}));
if(mod == "env" && name == "uce_host_file_mkdir")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String path, current;
self->hostcall_read(args[0].i32(), args[1].i32(), path);
self->hostcall_read(args[2].i32(), args[3].i32(), current);
String resolved = self->resolve_guest_write(path, current);
int ok = 0;
if(resolved != "")
ok = (::mkdir(resolved.c_str(), 0777) == 0 || errno == EEXIST) ? 1 : 0;
results[0] = Val((int32_t)ok);
return(std::monostate());
}));
if(mod == "env" && name == "uce_host_file_mtime")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String path, current;
self->hostcall_read(args[0].i32(), args[1].i32(), path);
self->hostcall_read(args[2].i32(), args[3].i32(), current);
String resolved = self->resolve_guest_file(path, current);
int64_t mtime = 0;
struct stat st;
if(resolved != "" && stat(resolved.c_str(), &st) == 0)
mtime = (int64_t)st.st_mtime;
results[0] = Val((int64_t)mtime);
return(std::monostate());
}));
if(mod == "env" && name == "uce_host_file_read")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String path, current;
@ -1204,6 +1256,38 @@ private:
results[0] = Val((int32_t)bytes.size());
return(std::monostate());
}));
if(mod == "env" && name == "uce_host_file_list")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String path, current;
self->hostcall_read(args[0].i32(), args[1].i32(), path);
self->hostcall_read(args[2].i32(), args[3].i32(), current);
String resolved = self->resolve_guest_file(path, current, true /*allow_dir*/);
String listing;
if(resolved != "")
{
std::vector<String> names;
if(DIR* d = opendir(resolved.c_str()))
{
while(struct dirent* e = readdir(d))
{
String n = e->d_name;
if(n != "." && n != "..")
names.push_back(n);
}
closedir(d);
}
// match the native ls -1 convention: bare names, sorted
std::sort(names.begin(), names.end());
listing = join(names, "\n");
}
u32 cap = (u32)args[5].i32();
int32_t buf = args[4].i32();
// length-query convention: no copy unless the buffer fits
if(buf != 0 && cap >= listing.size())
self->hostcall_write(buf, listing);
results[0] = Val((int32_t)listing.size());
return(std::monostate());
}));
if(mod == "env" && name == "uce_host_file_write")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String path, current, content;
@ -1713,20 +1797,22 @@ private:
return(std::monostate());
}));
if(mod == "env" && name == "uce_host_component_resolve")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String target, current, resolved;
return(add([self](Caller caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String target, handler, current, resolved;
self->hostcall_read(args[0].i32(), args[1].i32(), target);
self->hostcall_read(args[3].i32(), args[4].i32(), current);
int32_t slot = self->component_resolve(target, args[2].i32(), current, resolved);
u32 cap = (u32)args[6].i32();
self->hostcall_read(args[2].i32(), args[3].i32(), handler);
self->hostcall_read(args[4].i32(), args[5].i32(), current);
int32_t slot = self->component_resolve(target, handler, current, resolved);
u32 cap = (u32)args[7].i32();
if(cap > 0)
{
if(resolved.size() >= cap)
resolved = resolved.substr(0, cap - 1);
resolved.push_back('\0');
self->hostcall_write(args[5].i32(), resolved);
self->hostcall_write(args[6].i32(), resolved);
}
results[0] = Val(slot);
caller.context().set_epoch_deadline(self->worker.cfg.epoch_deadline_ticks);
return(std::monostate());
}));
@ -1752,7 +1838,7 @@ private:
// ---- public entry: one request through one workspace -----------------------
inline WasmResponse wasm_worker_serve(WasmWorker& worker, const DValue& context_tree, const String& entry_source_path,
int32_t kind = WasmWorkspace::RESOLVE_RENDER)
const String& handler = "render")
{
WasmResponse response;
WasmWorkspace workspace(worker);
@ -1763,7 +1849,7 @@ inline WasmResponse wasm_worker_serve(WasmWorker& worker, const DValue& context_
if(error == "")
error = workspace.apply_context(context_tree);
if(error == "")
error = workspace.invoke_entry(entry_source_path, kind, &response.handler_present);
error = workspace.invoke_entry(entry_source_path, handler, &response.handler_present);
if(error == "")
error = workspace.collect(response);
response.workspace_birth_us = workspace.workspace_birth_us;

View File

@ -10,7 +10,7 @@ scripts/run_cli_tests.sh --include-wasm-kill
scripts/run_cli_tests.sh --list
```
`--include-wasm-kill` adds the wasm trap/loop/recurse kill cases and should be used when `WASM_BACKEND_ENABLED=1`.
`--include-wasm-kill` adds the wasm trap/loop/recurse kill cases and should be used for the normal wasm-only runtime gate.
The script reads `CLI_SOCKET_PATH` from `/etc/uce/settings.cfg` unless `UCE_CLI_SOCKET` is set.

View File

@ -1,9 +1,10 @@
#!/usr/bin/env python3
"""Phase 5 native/WASM benchmark harness.
"""Current wasm-runtime HTTP benchmark harness.
This runner intentionally has no third-party dependencies. It records a warmed
native HTTP baseline now and can compare against a future wasm worker by passing
--wasm-base-url. Results are written as JSON plus a small Markdown table.
This runner intentionally has no third-party dependencies. It measures one or
more HTTP endpoints served by the wasm-only UCE runtime. Optionally pass a prior
benchmark JSON as a baseline; old native baselines are treated as historical
reference data only, not as a runnable backend mode.
"""
from __future__ import annotations
@ -95,14 +96,14 @@ def measure_backend(backend: str, base_url: str, host_header: str, targets: list
return results
def compare(results: list[Measurement]) -> list[str]:
native = {r.target: r for r in results if r.backend == "native"}
def compare(results: list[Measurement], baseline_label: str) -> list[str]:
baseline = {r.target: r for r in results if r.backend == baseline_label}
lines = ["| target | backend | median ms | mean ms | budget | status |", "|---|---:|---:|---:|---|---|"]
for result in results:
budget = "baseline"
budget = "baseline" if result.backend == baseline_label else "n/a"
status = "PASS" if result.ok else "FAIL"
if result.backend != "native" and result.target in native:
limit = native[result.target].median_ms * 2.0
if result.backend != baseline_label and result.target in baseline:
limit = baseline[result.target].median_ms * 2.0
budget = f"{limit:.1f} ms"
if result.median_ms > limit:
status = "FAIL"
@ -119,33 +120,30 @@ def build_targets() -> list[Target]:
def main() -> int:
parser = argparse.ArgumentParser(description="Phase 5 native/wasm benchmark harness")
parser.add_argument("--native-base-url", default="http://localhost:80")
parser.add_argument("--wasm-base-url", default="", help="optional wasm worker URL; omitted until worker exists")
parser.add_argument("--backend-label", default="native", help="label for --native-base-url measurements when running one backend at a time")
parser.add_argument("--compare-native-json", default="", help="optional prior native benchmark.json for budget comparison")
parser = argparse.ArgumentParser(description="UCE wasm-runtime HTTP benchmark harness")
parser.add_argument("--base-url", "--native-base-url", dest="base_url", default="http://localhost:80", help="runtime base URL (legacy alias --native-base-url is accepted)")
parser.add_argument("--backend-label", default="wasm", help="label for this run's measurements")
parser.add_argument("--compare-baseline-json", "--compare-native-json", dest="compare_baseline_json", default="", help="optional prior benchmark.json for budget comparison")
parser.add_argument("--baseline-label", default="native", help="backend label in the prior JSON to use as the budget baseline")
parser.add_argument("--host-header", default="uce.openfu.com")
parser.add_argument("--warmups", type=int, default=2)
parser.add_argument("--samples", type=int, default=20)
parser.add_argument("--timeout", type=float, default=10.0)
parser.add_argument("--out-dir", default="/tmp/uce/wasm-phase5")
parser.add_argument("--out-dir", default="/tmp/uce/wasm-benchmark")
args = parser.parse_args()
targets = build_targets()
results: list[Measurement] = []
if args.compare_native_json:
for row in json.loads(Path(args.compare_native_json).read_text()):
if args.compare_baseline_json:
for row in json.loads(Path(args.compare_baseline_json).read_text()):
row.setdefault("backend", args.baseline_label)
results.append(Measurement(**row))
results.extend(measure_backend(args.backend_label, args.native_base_url, args.host_header, targets, args.warmups, args.samples, args.timeout))
if args.wasm_base_url:
results.extend(measure_backend("wasm", args.wasm_base_url, args.host_header, targets, args.warmups, args.samples, args.timeout))
results.extend(measure_backend(args.backend_label, args.base_url, args.host_header, targets, args.warmups, args.samples, args.timeout))
out_dir = Path(args.out_dir)
out_dir.mkdir(parents=True, exist_ok=True)
(out_dir / "benchmark.json").write_text(json.dumps([asdict(r) for r in results], indent=2) + "\n")
md_lines = ["# Phase 5 benchmark report", "", *compare(results), ""]
if not args.wasm_base_url and args.backend_label == "native" and not args.compare_native_json:
md_lines.append("WASM worker URL was not provided; this report is the native baseline that future wasm runs compare against.")
md_lines = ["# UCE wasm benchmark report", "", *compare(results, args.baseline_label), ""]
(out_dir / "benchmark.md").write_text("\n".join(md_lines) + "\n")
print("\n".join(md_lines))
print(f"wrote {out_dir / 'benchmark.json'} and {out_dir / 'benchmark.md'}")