Compare commits

..
21 Commits
Author SHA1 Message Date
udo 34a97e2577 chore: use version file for package builds 2026-06-15 16:19:55 +00:00
root 6be92ef93e smolstuf 2026-06-15 15:45:52 +00:00
root 5ee257565c smolstuf 2026-06-15 15:40:24 +00:00
root 5f430155b7 deb update 2026-06-15 13:16:39 +00:00
root 762b586242 post-W7 cleanup 2026-06-15 12:31:33 +00:00
root 1a5c6547b9 post-W7 cleanup 2026-06-15 12:00:46 +00:00
root 75bccb778c W7 done done 2026-06-15 11:14:03 +00:00
root ad2ba7b632 W7 done done 2026-06-15 11:08:44 +00:00
root abcb66717e W7 done done 2026-06-15 11:06:35 +00:00
root 1743c51e46 W7 done done 2026-06-15 11:04:16 +00:00
root 04745f39a8 W7 done done 2026-06-15 10:53:36 +00:00
root f5637bb587 W7 done done 2026-06-15 10:28:23 +00:00
root b1a0df9c93 W7 done done 2026-06-15 10:19:06 +00:00
rootandClaude Opus 4.8 4f84ac544d feat: request_perf() worker-side timing hostcall; restore demo System Info
Units run in the wasm sandbox, so my_pid/parent_pid/context.server->request_count
read as sandbox stubs — the demo System Info counters were broken, and there was
no authoritative server-side request timing available to unit code (client-side
measurement cannot see queue/dispatch latency).

Add a request_perf() unit API backed by a new uce_host_request_perf hostcall.
The native worker answers it live, returning a DValue:
  worker_pid, parent_pid, request_count,
  accept_us  = (time_start - time_init)*1e6   (entry -> dispatch wait),
  running_us = (now - time_start)*1e6         (since dispatch, live),
  total_us   = (now - time_init)*1e6          (since the request entered UCE),
  workspace_birth_us.
time_init is captured at request entry (handle_request, with a handle_complete
fallback); a RequestPerfSnapshot {pids, request_count, time_init, time_start} is
threaded from wasm_backend_serve through wasm_worker_serve onto the workspace,
and the hostcall computes the live deltas at call time. Wired like uce_host_units
(sized DValue hostcall): core_hostcalls.syms + sys.cpp/sys.h request_perf().

site/demo/index.uce System Info now uses request_perf() and shows the real worker
PID, an incrementing per-worker request count, and the timing counters.

Implemented via the pi agent (gpt-5.3-codex-spark); a review of the live numbers
caught accept_us mistakenly computed as (now - time_init) (== total_us), fixed to
the dispatch wait (time_start - time_init). Independently verified on the host:
System Info shows non-zero PIDs, incrementing count, accept_us ~50us << total_us
~2.4ms with accept+running==total; run_cli_tests --include-wasm-kill => 87 passed,
0 failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 08:18:04 +00:00
rootandClaude Opus 4.8 83ab9e10f7 perf: extend the compiled-module disk cache to per-unit modules
Follow-up to 560290c. Per-unit .wasm modules were still Cranelift-compiled per
worker on first use (~40-70ms each), so with 8-call worker recycling fresh
workers re-JIT every unit they touch.

Refactor the core cache logic into a shared WasmWorker helper:
- cached_wasm_path(p): maps <...>.wasm -> <...>.cwasm.
- load_or_compile_cached_module(engine, cached, wasm, bytes, err): deserialize_file
  the .cwasm when it is newer than the .wasm; otherwise Module::compile + serialize,
  written atomically (temp+rename); deserialize failure falls back to compile.
Both the core module load and unit_module() now go through this helper, so unit
artifacts get the same <unit>.uce.cwasm cache the core got.

Independently verified on the host: fresh-worker first-hit unit latency ~40-70ms
-> ~3-21ms; full suite wall-clock 20.5s -> 6.5s (and ~40s before any caching);
run_cli_tests --include-wasm-kill => 87 passed, 0 failed, 0 skipped.

Implemented via the pi agent (gpt-5.3-codex-spark sub-model).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:16:41 +00:00
rootandClaude Opus 4.8 560290ca1d perf: cache the compiled core module so fresh workers deserialize, not recompile
Workers recycle every 8 requests (calls_until_termination=8). Each fresh worker
ran wasmtime::Module::compile() on the 6.8MB bin/wasm/core.wasm — a ~1.3s
Cranelift JIT — on its first request, so every ~8th request spiked to ~1.3s and
dominated suite wall-clock.

Cache the compiled artifact: on worker core-module load, if bin/wasm/core.cwasm
exists and is newer than core.wasm, load it via Module::deserialize_file() (mmap,
~ms); otherwise Module::compile() as before and atomically (temp+rename) write
the serialized artifact for the next worker. Deserialize failure / stale cache
falls back to a normal compile, so it is self-healing; rebuilding core.wasm
(newer mtime) invalidates the cache. Engine config (epoch_interruption,
signals_based_traps(false)) is unchanged, which the serialized format requires.

Implemented via the pi agent (delegated to a gpt-5.3-codex-spark sub-model).
Independently verified on the host: worker-startup request latency drops from
~1.3s to ~23ms (warm 12x /demo/hello.uce: all <=0.023s, no spikes); full suite
wall-clock ~40s -> 20.5s; run_cli_tests --include-wasm-kill => 87 passed, 0
failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 00:02:22 +00:00
rootandClaude Opus 4.8 bfd6d33829 W7f: sweep dead/legacy/fallback leftovers after native-pipeline removal
Post-deletion cleanup (units run only on wasm):
- types.h/compiler.cpp: drop the native-era SharedUnit fields so_name,
  bin_file_name, and the opt_so_optional cache-mode plumbing (no native
  optional .so path remains). The per-unit compile lock is re-keyed from
  so_name+.lock to wasm_name+.lock (still per-unit).
- unit_info() and to_string(SharedUnit*) no longer expose .so artifact fields.
- backend.h: drop the stale "+ fallback-token gate" comment.
- Docs/comments corrected to wasm-only reality: README, tests/README,
  site/doc C++ preprocessor + error_pages + unit_info pages, site/info intro,
  site/demo/unit-browser artifact card; the Phase-5 native-vs-wasm benchmark
  harness (tests/wasm_benchmark.py) reframed for the wasm-only backend.

Audit confirmed no live references remain to so_handle, load_shared_unit,
compiler_load_shared_unit, compiler_invoke*/_cli/_websocket/_serve_http,
COMPILE_SCRIPT/COMPILE_WASM_UNITS, or the native export-symbol constants;
request_ref_handler/dv_call_handler are kept (live wasm funcref casts).

Swept via the pi agent (delegated to a gpt-5.3-codex-spark sub-model);
independently re-verified on the host: run_cli_tests --include-wasm-kill =>
87 passed, 0 failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:43:16 +00:00
rootandClaude Opus 4.8 cf51336873 W7e: delete the native unit pipeline (.so compile + dlopen execution)
Units now run exclusively on wasm; the native generated-C++ -> clang -> .so ->
dlopen path and its request-time fallback are removed.

- Dispatch (linux_fastcgi.cpp): the 4 handle_complete branches + the CLI-socket
  path route every request through wasm (wasm_ready compiles cold/stale on
  demand); a wasm-unavailable unit now yields a clean error page
  (fail_wasm_unavailable / render_request_failure) instead of native execution.
  compiler_invoke / _cli / _websocket / _serve_http deleted.
- compiler.cpp (-1274): removed the native .so compile (COMPILE_SCRIPT),
  load_shared_unit, dlopen/dlsym/dlclose, compiler_load_shared_unit, and the
  SharedUnit .so function-pointer fields (on_setup/on_render/on_component/
  on_websocket/on_cli/on_once/on_init) in types.h/types.cpp. compile_shared_unit
  now builds only the .wasm side-module; the .uce preprocessor/parser front-end
  is kept (it emits the C++ the wasm compile consumes).
- unit_call()/component()/once/init now resolve across units through the wasm
  host component resolver (uce_host_component_resolve) instead of native dlsym;
  configured runtime error pages render through the wasm backend.
- Dropped the WASM_BACKEND_ENABLED feature flag and dead COMPILE_SCRIPT /
  COMPILE_WASM_UNITS config; unit ABI freshness tied to UCE_UNIT_ABI_VERSION;
  guard against serving a stale .wasm for a deleted source; retired the obsolete
  W5 native-vs-wasm toggle script. Docs updated.

Implemented via the pi agent (with 3 delegated sub-reviews); independently
re-verified on the build host: run_cli_tests --include-wasm-kill => 87 passed,
0 failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 23:20:50 +00:00
rootandClaude Opus 4.8 fb728d63bc W7e stage B: factor the .wasm artifact into unit compile-freshness
inspect_shared_unit_filesystem() tracked only the .so mtime, so a missing or
stale .wasm with a current .so never triggered a rebuild and the unit fell to
native indefinitely (the in-process cache also never invalidated). Account for
su->wasm_name when wasm unit compilation is enabled: a unit counts as compiled
only as of the OLDER of the two artifacts; a missing .wasm forces a recompile.
Closes the cached-vanished-.wasm gap noted in the stage A review.

Verified independently on the build host: run_cli_tests --include-wasm-kill =>
87 passed, 0 failed; delete-.wasm-then-request rebuilds the artifact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:50:31 +00:00
rootandClaude Opus 4.8 2debd33804 W7e: wasm-preferred dispatch + compile-on-demand; harden unit ABI validator
- Dispatch (linux_fastcgi.cpp): route every request through wasm. On a
  cold/stale artifact, compile the unit on demand (get_shared_unit, forced) and
  serve wasm; native compiler_invoke* remains only as a fallback when wasm
  cannot be made ready (compile failure / backend disabled). Applies to the 4
  handle_complete branches and the CLI socket path.
- backend.cpp: delete the now-vestigial native-only fallback token gate
  (wasm_backend_native_fallback_*), empty since W7d; should_handle now gates on
  config + current artifact + healthy worker only.
- check_unit_wasm.py: skip the defense-in-depth llvm-nm allocator scan when
  llvm-nm SIGSEGVs on a degenerate-but-valid module (e.g. a unit with no
  exported handlers). Fixes site/demo/empty.uce, the last unit that could not
  produce a .wasm; forbidden allocator *exports* are still rejected.

A pi-assisted review caught that the compile-freshness check keys off the .so
mtime only, so force_recompile is required to rebuild a missing/stale .wasm; a
unit already cached in-process whose .wasm later vanished still uses native
fallback (closed in W7e stage B). Native execution is otherwise bypassed for
all real traffic.

Verified: scripts/run_cli_tests.sh --include-wasm-kill -> 87 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 18:20:12 +00:00
rootandClaude Opus 4.8 8587fbc5aa wasm runtime: central WS broker, unified handlers, W7d holdouts, membrane completeness
- WS: a dedicated broker process owns HTTP_PORT + every connection; it forwards
  renders to the worker pool over uce.sock (non-blocking) and applies ws_*
  command batches flushed back at workspace teardown. Removes the now-dead
  per-worker websocket executor (-509 lines).
- Dispatch: unify CLI / WebSocket / serve_http / page render through one
  serve_via_wasm(entry_unit, handler) path; handler string -> __uce_<handler>
  export symbol.
- W7d: rewrite zip.uce to the membrane return-value error contract (no C++
  try/catch), error-reporting.uce to genuine wasm traps instead of throw, and
  sharedunit.uce to unit_info(); empty the native-only token gate.
- Membrane: wire ls / mkdir / file_mtime through new uce_host_file_list /
  uce_host_file_mkdir / uce_host_file_mtime hostcalls (resolve_guest_file gains
  directory support). Fixes /doc/index.uce listing nothing; adds a regression
  assertion that the index enumerates items.
- Docs: add docs/wasm-runtime-architecture.md; record the W7e staged native-
  deletion plan in WASM-PROPOSAL.md.

Verified: scripts/run_cli_tests.sh --include-wasm-kill -> 87 passed, 0 failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-14 17:51:51 +00:00
188 changed files with 4756 additions and 4883 deletions
+1 -1
View File
@@ -36,7 +36,7 @@
tmp/*
bin/*
pkg/*
dist/*
# Python cache artifacts
__pycache__/
+46 -43
View File
@@ -2,19 +2,19 @@
## Current State
This is in the early stages of development. Don't use this for anything important (or at all)!
This is in the early stages of development. Don't use this for anything important!
## Overview
UCE is a PHP-inspired server-side runtime that lets you build web pages and handlers in C++ using a small `.uce` preprocessor plus a FastCGI application server.
- `.uce` pages compile to shared objects on demand
- `.uce` pages compile to WebAssembly side modules on demand
- normal HTTP pages expose `RENDER(Request& context)`
- WebSocket pages can additionally expose `WS(Request& context)`
- local CLI/admin/test entrypoints can expose `CLI(Request& context)` and are invoked through the Unix CLI socket
- sub-rendering and components pass structured data through `context.props`
- nginx can forward normal `.uce` requests and ordinary `.ws.uce` page loads to the FastCGI socket, while real WebSocket upgrade requests for `.ws.uce` endpoints go to the built-in HTTP/WebSocket listener
- the nginx-published application tree lives under `site/`
- nginx can forward normal `.uce` requests to the FastCGI socket, while WebSocket upgrade requests for `.uce` endpoints go to the built-in HTTP/WebSocket listener
- the example application tree lives under `site/`; deployments should publish app files to a normal web root such as `/var/www/html`
- you can include C++ code as much as you want, but only .uce files called via API functions and entry points will be pre-processed
- the preprocessor has two jobs:
- allow for inline HTML within C++ and the use of templating tags inside of that HTML
@@ -29,7 +29,7 @@ RENDER(Request& context)
}
```
The runtime is still experimental.
*The runtime is still experimental. This is not production-ready. Use at your own risk!*
## Build
@@ -45,8 +45,10 @@ The current build expects:
- `mysql_config`
- PCRE2 development headers and library (`libpcre2-dev` on Debian / Ubuntu)
- standard Linux development headers for `dl`, `pthread`, sockets, and backtrace support
- Wasmtime C API / C++ headers, defaulting to `/opt/wasmtime` or `WASMTIME_HOME`
- WASI SDK tools, defaulting to `/opt/wasi-sdk` or `WASI_SDK`, for `scripts/build_core_wasm.sh` and unit compilation
SQLite is vendored under `src/3rdparty/sqlite/` and compiled by `scripts/build_linux.sh`; no system SQLite package is required.
SQLite and miniz are vendored under `src/3rdparty/`; no system SQLite or zlib package is required for those helpers.
The binary is written to:
@@ -103,7 +105,7 @@ Those are intended for sub-rendering through helpers such as `component("compone
Additional lifecycle hooks are also available on ordinary `.uce` units:
- `INIT(Request& context)` runs once when a worker loads that unit's shared object into memory
- `INIT(Request& context)` runs once when a worker instantiates that unit's wasm module
- `ONCE(Request& context)` runs once per request before the first `RENDER()`, `CLI()`, or `COMPONENT...` entrypoint from that file
CLI units can be invoked locally with the convenience wrapper or directly over HTTP-over-Unix:
@@ -184,7 +186,7 @@ The runtime keeps the socket lifecycle in-process and exposes a low-boilerplate
- `ws_send_to(connection_id, message[, binary])`
- `ws_close([connection_id])`
By default, the WebSocket scope is the current page file, so `ws_send()` queues a message for clients connected to that same `.ws.uce` endpoint.
By default, the WebSocket scope is the current page file, so `ws_send()` queues a message for clients connected to that same `.uce` endpoint.
Each live WebSocket connection owns a broker-side `DValue` exposed to page code as `context.connection`. Mutations to that tree persist for the life of the socket and are visible on later `WS(Request& context)` calls for the same client.
@@ -248,8 +250,8 @@ Representative test pages:
The intended production shape is:
- nginx serves static files directly
- nginx forwards `.uce` requests and ordinary `.ws.uce` page loads to the UCE FastCGI Unix socket
- nginx proxies actual WebSocket upgrade requests for `.ws.uce` endpoints to the runtime's built-in HTTP/WebSocket listener
- nginx forwards ordinary `.uce` page loads to the UCE FastCGI Unix socket
- nginx proxies WebSocket upgrade requests for `.uce` endpoints to the runtime's built-in HTTP/WebSocket listener
- systemd keeps the runtime built, started, and restarted on failure
The repository ships the pieces used for this:
@@ -264,7 +266,7 @@ On a Debian or Ubuntu host, start with the packages needed to build and run UCE
```bash
apt update
apt install -y nginx clang mariadb-client libmariadb-dev libpcre2-dev build-essential
apt install -y nginx clang mariadb-client libmariadb-dev libpcre2-dev build-essential curl rsync ca-certificates
```
The exact package names may vary by distro. The important requirements are:
@@ -274,16 +276,26 @@ The exact package names may vary by distro. The important requirements are:
- `mysql_config`
- PCRE2 development headers and library (`libpcre2-dev` on Debian / Ubuntu)
- normal Linux development headers for threads, sockets, `dl`, and backtrace support
- Wasmtime C API / C++ headers installed at `/opt/wasmtime` or configured with `WASMTIME_HOME`
- WASI SDK installed at `/opt/wasi-sdk` or configured with `WASI_SDK`
### 2. Put the repo on the server
This README assumes the repository lives at:
```bash
/Code/uce.openfu.com/uce
/opt/uce
```
That is what the shipped `scripts/systemd/uce.service` file currently uses as its `WorkingDirectory` and build path. If you deploy somewhere else, update that unit file before enabling the service.
The examples below use that path for the runtime. Publish public application files under the normal web root, for example:
```bash
cd /opt/uce
mkdir -p /var/www/html
rsync -a site/ /var/www/html/
```
If you deploy somewhere else, update the systemd unit's `WorkingDirectory`, build path, and `ExecStart` path before enabling the service.
### 3. Configure `/etc/uce/settings.cfg`
@@ -293,7 +305,7 @@ The runtime reads its server settings from:
/etc/uce/settings.cfg
```
The shipped example contains the important filesystem and FastCGI settings:
The example contains the filesystem and FastCGI settings:
```ini
BIN_DIRECTORY=/var/cache/uce/work
@@ -304,7 +316,7 @@ FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
FCGI_PORT=9993
PRECOMPILE_FILES_IN=
SITE_DIRECTORY=site
SITE_DIRECTORY=/var/www/html
PROACTIVE_COMPILE_CHECK_INTERVAL=60
WORKER_COUNT=4
@@ -328,9 +340,9 @@ HTTP_PORT=8080
Proactive compilation settings:
- `SITE_DIRECTORY=site` tells the runtime which tree to scan on startup for `.uce` files when `PRECOMPILE_FILES_IN` is left empty.
- `SITE_DIRECTORY=/var/www/html` tells the runtime which public web tree to scan on startup for `.uce` files when `PRECOMPILE_FILES_IN` is left empty.
- `PRECOMPILE_FILES_IN=` can override that startup scan root with a different absolute or runtime-relative directory.
- `PROACTIVE_COMPILE_CHECK_INTERVAL=60` controls how often the low-priority background compiler rechecks known `.uce` files for stale or missing shared objects.
- `PROACTIVE_COMPILE_CHECK_INTERVAL=60` controls how often the low-priority background compiler rechecks known `.uce` files for stale or missing wasm modules.
The runtime keeps a shared known-file registry under `BIN_DIRECTORY` and updates it as request handling discovers new `.uce` files, so proactive recompiles are not limited to the initial startup scan.
@@ -395,12 +407,14 @@ That script:
- writes Debian maintainer scripts for systemd reload/enable handling
- follows a more PHP-like/FHS deployment shape with immutable runtime files under `/usr/lib`, config under `/etc`, cache/state under `/var`, and the FastCGI socket under `/run/uce/`
### 5. Configure nginx for `.uce` and `.ws.uce`
### 5. Configure nginx for `.uce` and WebSocket upgrades
You need two nginx paths for `.ws.uce` endpoints:
Any `.uce` unit can expose `WS(Request& context)`. WebSocket upgrade requests for `.uce` paths should be routed to the runtime's HTTP/WebSocket listener.
- FastCGI for ordinary `.uce` requests and plain `.ws.uce` page renders
- HTTP proxying only for actual WebSocket upgrade traffic on `.ws.uce` endpoints
You need two transport paths for `.uce` endpoints:
- FastCGI for ordinary `.uce` page renders
- HTTP proxying only for WebSocket upgrade traffic on `.uce` endpoints
If you use WebSockets, add this `map` in the nginx `http` block:
@@ -417,7 +431,7 @@ Then use a server block along these lines:
server {
listen 80;
server_name example.com;
root /Code/uce.openfu.com/uce/site;
root /var/www/html;
index index.uce index.html;
@@ -426,16 +440,6 @@ server {
}
location ~ \.uce$ {
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param DOCUMENT_URI $uri;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_pass unix:/run/uce/fastcgi.sock;
}
location ~ \.ws\.uce$ {
error_page 418 = @uce_websocket;
if ($http_upgrade = "websocket") {
return 418;
@@ -453,7 +457,6 @@ server {
location @uce_websocket {
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
@@ -465,19 +468,19 @@ server {
Important details:
- `.ws.uce` must be matched before the more general `.uce` rule
- `fastcgi_pass` should point at the same socket path as `FCGI_SOCKET_PATH`
- `proxy_pass` should point at the runtime's `HTTP_PORT`
- ordinary `GET /page.ws.uce` page renders should stay on FastCGI
- only upgrade requests for `/page.ws.uce` should go through the HTTP/WebSocket listener
- `SCRIPT_FILENAME` should resolve to the actual `.uce` file on disk
- ordinary `GET /page.uce` page renders should stay on FastCGI
- only upgrade requests for `/page.uce` should go through the HTTP/WebSocket listener
- `SCRIPT_FILENAME` should resolve to the requested `.uce` file on disk
- `proxy_http_version 1.1` and the `Upgrade` / `Connection` headers are required for WebSockets
- socket-capable pages are ordinary `.uce` units; route client WebSocket upgrade requests to the HTTP/WebSocket listener
The `location /` block above is intentionally conservative and only serves real files from `site/`. If your app uses a front-controller pattern such as routing everything through `/index.uce`, change that block accordingly.
The `location /` block only serves files from `/var/www/html`. If your app uses a front-controller pattern such as routing everything through `/index.uce`, change that block accordingly.
### 6. Think about document root and private files
Point nginx at `site/`, not the repository root. The repo still contains source, scripts, packaging files, and operational assets that are not meant to be public.
Point nginx at `/var/www/html`, not the runtime repository root. The repo still contains source, scripts, packaging files, and operational assets that are not meant to be public.
At minimum, explicitly block internal directories that should never be served directly. For example:
@@ -487,7 +490,7 @@ location ~ ^/(src|scripts|etc|bin|work|dist|pkg)/ {
}
```
If nginx is rooted at `site/`, most of those paths will not be reachable anyway, which is the preferred setup.
If nginx is rooted at `/var/www/html`, most of those paths will not be reachable anyway, which is the preferred setup.
### 7. Reload nginx and verify the deployment
@@ -506,7 +509,7 @@ curl -i http://127.0.0.1/test/index.uce
curl -i http://127.0.0.1/doc/index.uce
```
If WebSockets are enabled, also verify a `.ws.uce` endpoint through nginx rather than talking to the runtime directly.
If WebSockets are enabled, also verify a `.uce` endpoint that defines `WS(Request& context)` through nginx rather than talking to the runtime directly.
### 8. Troubleshooting
@@ -515,7 +518,7 @@ Common failure modes:
- `502 Bad Gateway`
Usually means `uce.service` is down, the Unix socket path does not match, or the request crashed before sending a valid response.
- WebSocket upgrade fails
Check that nginx is routing `.ws.uce` to `proxy_pass`, not `fastcgi_pass`, and that `HTTP_PORT` is reachable on localhost.
Check that nginx is routing WebSocket upgrade requests to `proxy_pass`, not `fastcgi_pass`, and that `HTTP_PORT` is reachable on localhost.
- Requests compile but immediately crash
Check `journalctl -u uce.service`. Generated units carry an ABI metadata sidecar and should be recompiled automatically after runtime ABI changes, but clearing stale artifacts under `BIN_DIRECTORY` is still a useful last-resort recovery step if the cache has been damaged manually.
- nginx serves raw source or internal files
@@ -537,4 +540,4 @@ For up-to-date usage, prefer:
## AI Disclosure
This project is largely human-made, with all the typical idiosyncracies of my projects clearly visible. However, OpenAI Codex was used for code review and documentation. Claude Opus was used for UI design work, and I used VS Code's git commit message generator.
This project is largely human-made, with all the typical idiosyncracies of my projects clearly visible. However, OpenAI Codex was used for code review, debugging, API integration work, and documentation. Claude Opus was used for UI design work, and I used VS Code's git commit message generator.
-1026
View File
File diff suppressed because it is too large Load Diff
+15
View File
@@ -0,0 +1,15 @@
commit ad3d65cdacdcc5aed72fe12354f66190d7be454d
Initial commit
commit c047927b189860220428b42400ed18942150882c
initial import
commit 3e8f0f1fa780da7836128fdbc39bbe8c2ba97de8
initial import
commit 939009f9a11d0934226c830375057491f088d362
shell stuff, preprocessor directive
+4
View File
@@ -0,0 +1,4 @@
commit 18ca2368bc1d4a5ef7e4c8470a9b3e32f496fc86
VERY basic UTF8 parser
+12
View File
@@ -0,0 +1,12 @@
commit 17336fe649c52c162a4541968c8d24c0c2ab3eb9
Unicode stuff
commit 5df52146be3f5a45759459d5716aec4ef8d3c33a
split_space
commit 3a9dfba86cb7dadb1268fcfd689b14066779e6aa
fixed bug in #load
+4
View File
@@ -0,0 +1,4 @@
commit 0a6ebc60f0b8aacf8b01e14bd9b6447c750edeb5
precompile on startup
+14
View File
@@ -0,0 +1,14 @@
commit 6f6919f3f01686175001251e2fca215bccc4806f
backage builder
commit defc628204f95d0cdd16c95c89ea30efe922b31a
packaging
commit 9e5b7d39e3b8be9f4ad537a134b1081564ae4abc
Basic readme
commit 21fdeafb6c5b8bc80a9d93ed61c36e05fbe7b267
Basic documentation
+8
View File
@@ -0,0 +1,8 @@
commit 364d83b199afa6a341173a80791060a42c169f0c
doc update
commit 392b7224b16e41198b0425c5ae04f46e014e78c7
Merge branch 'main' of github.com:ThingamaNet/uce
+4
View File
@@ -0,0 +1,4 @@
commit 984453f336a9b2ad47689d161c89d7020006f75a
update sigh
+26
View File
@@ -0,0 +1,26 @@
commit cd8f07aaa78fc488eba3b0e435350e5c2823e53e
Refactor FastCGI server and compiler, enhance HTTP header parsing, and add WebSocket support
- Refactored FastCGIServer class to improve socket handling and added shutdown functionality.
- Updated compiler functions to streamline HTML and text literal processing.
- Enhanced split_kv function to support uppercase keys and added split_http_headers for better HTTP header parsing.
- Introduced new session management functions to validate and handle session IDs.
- Added WebSocket frame parsing and handling in the URI module.
- Created systemd service scripts for easier deployment and management of the UCE FastCGI runtime.
- Added new test cases for header handling, time parsing, and WebSocket functionality.
commit 86dc93864e8eab9227ac50b16f1fc7f561383f5b
Add WebSocket support and enhance chat functionality
- Implement WebSocket handling in the FastCGI server, allowing for real-time communication.
- Introduce functions for managing WebSocket connections, broadcasting messages, and sending to specific connections.
- Create a chat interface in the `websockets.ws.uce` file, including message handling and user notifications.
- Refactor existing code to accommodate WebSocket integration, ensuring compatibility with HTTP requests.
- Update documentation to reflect new features and usage instructions for WebSocket functionality.
commit 46d98a092fb606c87ab4b9eef32fe68eeca2b2cb
Enhance WebSocket support: add opcode handling, binary message support, and improve connection validation
+20
View File
@@ -0,0 +1,20 @@
commit be514d63d6ed709a40441db9dc809837081d54f6
trying to port web app starter from PHP
commit 2b5586d7dfffc5c1c40cac97e6d4d0e1f86f151a
decided in favor of dedicated COMPONENT() macro, updates to documentation
commit d1167aec3b4d7870db0b289ee0ea690f8e7e5cc9
getting closer to full port of web app starter
commit af642c8167383f9acf99274d3b3c94017d860dcf
getting closer to full port of web app starter
commit dc05f9faa5eab980f1f1fd44cee06f811cc748ff
PHP familiarity shortcuts
+9
View File
@@ -0,0 +1,9 @@
commit b53eb6e4f1b5fa4e2a4907022f591eb9a1860c33
Enhance template parser to handle C++ comments and refactor preprocessing logic
- Updated parser to correctly interpret C++ `//` and `/* ... */` comments within template code.
- Split preprocessing implementation into separate files for better organization.
- Added regression test for comment parsing in templates.
- Adjusted CSS styles for improved layout and readability in documentation.
+16
View File
@@ -0,0 +1,16 @@
commit 8223dcc6b3cb7f4d26c69097fac973c2671f0938
I think I need to change the documentation format
commit f7b066b374f1031d38e3d2b344f6680534c3a2b2
changing doc format and HTML literals
commit 14ebf10a229021bb5d0fb1eae42a69c042699318
changing doc format and HTML literals
commit 223cf4c6e1c7a205bf1a27bebdc6d04fd7557df0
website with slop placeholders
+4
View File
@@ -0,0 +1,4 @@
commit cd445f3c9b6aa079a969351d6eb742b8a4d24f6b
some cleanup
+4
View File
@@ -0,0 +1,4 @@
commit 9f7625c7fdd6a9d2a184dc2cdf9f8a719d219ec2
working on documentation and more API functions
+24
View File
@@ -0,0 +1,24 @@
commit 02e153a6a78bf8ded1c7380713023b4f9c3b656f
Add archive helpers and harden task runtime
commit d37517041de0c5ebe4dfd232093a78ff49cff383
Add custom server API and runtime limits
commit 0d8b74930c74cc8018cb1c0cfc7cfe6740b49cf6
Harden HTTP path headers sessions and archives
commit 41e9ca219fd684c75e78a5493e20c754c7899105
Streamline hardening helpers and expand coverage
commit 8b37e7ea1e58e36c97852f79ec121947a1ffea08
Fix direct HTTP status sanitizer fallback
commit 71ddcaf7d48ffd1343e597e4715b570c922a306c
Consolidate config and base64 helpers
+12
View File
@@ -0,0 +1,12 @@
commit 7f757654b658d049564f1725f54fde781575e304
fix: harden UCE runtime and starter
commit 20db6695890a3ac274217fd0ed9e0edf7c5eb093
fix: stabilize runtime follow-up regressions
commit b957a2373b8464ffcb7791cc6b0031c441d5b09c
chore: ignore Python cache artifacts
+28
View File
@@ -0,0 +1,28 @@
commit 941f5aea08941327eecbc43128d8f171a4064616
feat: add configurable UCE error pages
commit 7066da3cdef08b01dda64bca34caadc7000587b2
refactor: rename DTree to DValue
commit 7e2faf1472c84f302ce5b7f9674091a1a2b3930f
spike: validate WASM phase 0
commit 80285b7fb4acd15850218ee87c72ec8272332c05
feat: implement WASM phase 1 DValue ABI
commit 961df2d542ff6554e32da9ee28e73f5fc59b5975
phase 5
commit 89b5499c8f4e1a8acc8748e66c9dba9eb9679369
fix: harden WASM phase 5 harness
commit eb8f303f94f0f43699eca7cd6474eedaef1ed711
docs: remove unsupported second-plane language roadmap
+36
View File
@@ -0,0 +1,36 @@
commit 577aae076e3603137265332adfdded3c15cfa8f4
W3
commit afaa4dd7c0203f322d625593a0d1c96c37130005
feat: cut over to WASM backend
commit 5a56d4f39e4b30a9a4e43985d64906aa16a642e6
W5
commit c84fc86e6cdfe254a59d57b909f647e6c254ef76
W6
commit fe83c524118722c062295bc9575cb71eff64dacb
feat: membrane remaining wasm host surfaces
commit d6421cb8f3721e69e999b84335e9094ad96356e9
feat: extend wasm backend entrypoints
commit af6500d1343581abdac475dcb6855821afd8c8d0
fix: harden wasm w7 entrypoint paths
commit b1856c1725970b31027383a7ae79c2140a62a896
chore: narrow wasm backend entrypoint API
commit 6bb4f7f0ad801a0127956f49e54d2be722c6537f
test: port network suite to uce cli runner
+111
View File
@@ -0,0 +1,111 @@
commit 15e8d092bc13c425b5c7f7a19dfaf3c9728104f5
W7+
commit 8587fbc5aa0f38739182fa50383bb94c8d821ed7
wasm runtime: central WS broker, unified handlers, W7d holdouts, membrane completeness
- WS: a dedicated broker process owns HTTP_PORT + every connection; it forwards
renders to the worker pool over uce.sock (non-blocking) and applies ws_*
command batches flushed back at workspace teardown. Removes the now-dead
per-worker websocket executor (-509 lines).
- Dispatch: unify CLI / WebSocket / serve_http / page render through one
serve_via_wasm(entry_unit, handler) path; handler string -> __uce_<handler>
export symbol.
- W7d: rewrite zip.uce to the membrane return-value error contract (no C++
try/catch), error-reporting.uce to genuine wasm traps instead of throw, and
sharedunit.uce to unit_info(); empty the native-only token gate.
- Membrane: wire ls / mkdir / file_mtime through new uce_host_file_list /
uce_host_file_mkdir / uce_host_file_mtime hostcalls (resolve_guest_file gains
directory support). Fixes /doc/index.uce listing nothing; adds a regression
assertion that the index enumerates items.
- Docs: add docs/wasm-runtime-architecture.md; record the W7e staged native-
deletion plan in WASM-PROPOSAL.md.
commit 2debd338040cf97e82bb8ccbb5ec09caee089c17
W7e: wasm-preferred dispatch + compile-on-demand; harden unit ABI validator
- Dispatch (linux_fastcgi.cpp): route every request through wasm. On a
cold/stale artifact, compile the unit on demand (get_shared_unit, forced) and
serve wasm; native compiler_invoke* remains only as a fallback when wasm
cannot be made ready (compile failure / backend disabled). Applies to the 4
handle_complete branches and the CLI socket path.
- backend.cpp: delete the now-vestigial native-only fallback token gate
(wasm_backend_native_fallback_*), empty since W7d; should_handle now gates on
config + current artifact + healthy worker only.
- check_unit_wasm.py: skip the defense-in-depth llvm-nm allocator scan when
llvm-nm SIGSEGVs on a degenerate-but-valid module (e.g. a unit with no
exported handlers). Fixes site/demo/empty.uce, the last unit that could not
produce a .wasm; forbidden allocator *exports* are still rejected.
A pi-assisted review caught that the compile-freshness check keys off the .so
mtime only, so force_recompile is required to rebuild a missing/stale .wasm; a
unit already cached in-process whose .wasm later vanished still uses native
fallback (closed in W7e stage B). Native execution is otherwise bypassed for
all real traffic.
commit fb728d63bcd0d80bd5161537e81c30d27eb4df7b
W7e stage B: factor the .wasm artifact into unit compile-freshness
inspect_shared_unit_filesystem() tracked only the .so mtime, so a missing or
stale .wasm with a current .so never triggered a rebuild and the unit fell to
native indefinitely (the in-process cache also never invalidated). Account for
su->wasm_name when wasm unit compilation is enabled: a unit counts as compiled
only as of the OLDER of the two artifacts; a missing .wasm forces a recompile.
Closes the cached-vanished-.wasm gap noted in the stage A review.
commit cf513368730e5f337e64a9c9f1456e6a64f49d9c
W7e: delete the native unit pipeline (.so compile + dlopen execution)
Units now run exclusively on wasm; the native generated-C++ -> clang -> .so ->
dlopen path and its request-time fallback are removed.
- Dispatch (linux_fastcgi.cpp): the 4 handle_complete branches + the CLI-socket
path route every request through wasm (wasm_ready compiles cold/stale on
demand); a wasm-unavailable unit now yields a clean error page
(fail_wasm_unavailable / render_request_failure) instead of native execution.
compiler_invoke / _cli / _websocket / _serve_http deleted.
- compiler.cpp (-1274): removed the native .so compile (COMPILE_SCRIPT),
load_shared_unit, dlopen/dlsym/dlclose, compiler_load_shared_unit, and the
SharedUnit .so function-pointer fields (on_setup/on_render/on_component/
on_websocket/on_cli/on_once/on_init) in types.h/types.cpp. compile_shared_unit
now builds only the .wasm side-module; the .uce preprocessor/parser front-end
is kept (it emits the C++ the wasm compile consumes).
- unit_call()/component()/once/init now resolve across units through the wasm
host component resolver (uce_host_component_resolve) instead of native dlsym;
configured runtime error pages render through the wasm backend.
- Dropped the WASM_BACKEND_ENABLED feature flag and dead COMPILE_SCRIPT /
COMPILE_WASM_UNITS config; unit ABI freshness tied to UCE_UNIT_ABI_VERSION;
guard against serving a stale .wasm for a deleted source; retired the obsolete
W5 native-vs-wasm toggle script. Docs updated.
commit bfd6d338299dd9001fe2d4a66f82bbc2b3b07ed9
W7f: sweep dead/legacy/fallback leftovers after native-pipeline removal
Post-deletion cleanup (units run only on wasm):
- types.h/compiler.cpp: drop the native-era SharedUnit fields so_name,
bin_file_name, and the opt_so_optional cache-mode plumbing (no native
optional .so path remains). The per-unit compile lock is re-keyed from
so_name+.lock to wasm_name+.lock (still per-unit).
- unit_info() and to_string(SharedUnit*) no longer expose .so artifact fields.
- backend.h: drop the stale "+ fallback-token gate" comment.
- Docs/comments corrected to wasm-only reality: README, tests/README,
site/doc C++ preprocessor + error_pages + unit_info pages, site/info intro,
site/demo/unit-browser artifact card; the Phase-5 native-vs-wasm benchmark
harness (tests/wasm_benchmark.py) reframed for the wasm-only backend.
Audit confirmed no live references remain to so_handle, load_shared_unit,
compiler_load_shared_unit, compiler_invoke*/_cli/_websocket/_serve_http,
COMPILE_SCRIPT/COMPILE_WASM_UNITS, or the native export-symbol constants;
request_ref_handler/dv_call_handler are kept (live wasm funcref casts).
Swept via the pi agent (delegated to a gpt-5.3-codex-spark sub-model);
independently re-verified on the host: run_cli_tests --include-wasm-kill =>
87 passed, 0 failed, 0 skipped.
+58
View File
@@ -0,0 +1,58 @@
commit 560290ca1d18d4d5a0d204bb187d1aa68c3b5ac7
perf: cache the compiled core module so fresh workers deserialize, not recompile
Workers recycle every 8 requests (calls_until_termination=8). Each fresh worker
ran wasmtime::Module::compile() on the 6.8MB bin/wasm/core.wasm — a ~1.3s
Cranelift JIT — on its first request, so every ~8th request spiked to ~1.3s and
dominated suite wall-clock.
Cache the compiled artifact: on worker core-module load, if bin/wasm/core.cwasm
exists and is newer than core.wasm, load it via Module::deserialize_file() (mmap,
~ms); otherwise Module::compile() as before and atomically (temp+rename) write
the serialized artifact for the next worker. Deserialize failure / stale cache
falls back to a normal compile, so it is self-healing; rebuilding core.wasm
(newer mtime) invalidates the cache. Engine config (epoch_interruption,
signals_based_traps(false)) is unchanged, which the serialized format requires.
commit 83ab9e10f7c0f17805e40a9a3d2b4a3357e5280f
perf: extend the compiled-module disk cache to per-unit modules
Follow-up to 560290c. Per-unit .wasm modules were still Cranelift-compiled per
worker on first use (~40-70ms each), so with 8-call worker recycling fresh
workers re-JIT every unit they touch.
Refactor the core cache logic into a shared WasmWorker helper:
- cached_wasm_path(p): maps <...>.wasm -> <...>.cwasm.
- load_or_compile_cached_module(engine, cached, wasm, bytes, err): deserialize_file
the .cwasm when it is newer than the .wasm; otherwise Module::compile + serialize,
written atomically (temp+rename); deserialize failure falls back to compile.
Both the core module load and unit_module() now go through this helper, so unit
artifacts get the same <unit>.uce.cwasm cache the core got.
commit 4f84ac544d9c91a66d068d372055eec8e1723def
feat: request_perf() worker-side timing hostcall; restore demo System Info
Units run in the wasm sandbox, so my_pid/parent_pid/context.server->request_count
read as sandbox stubs — the demo System Info counters were broken, and there was
no authoritative server-side request timing available to unit code (client-side
measurement cannot see queue/dispatch latency).
Add a request_perf() unit API backed by a new uce_host_request_perf hostcall.
The native worker answers it live, returning a DValue:
worker_pid, parent_pid, request_count,
accept_us = (time_start - time_init)*1e6 (entry -> dispatch wait),
running_us = (now - time_start)*1e6 (since dispatch, live),
total_us = (now - time_init)*1e6 (since the request entered UCE),
workspace_birth_us.
time_init is captured at request entry (handle_request, with a handle_complete
fallback); a RequestPerfSnapshot {pids, request_count, time_init, time_start} is
threaded from wasm_backend_serve through wasm_worker_serve onto the workspace,
and the hostcall computes the live deltas at call time. Wired like uce_host_units
(sized DValue hostcall): core_hostcalls.syms + sys.cpp/sys.h request_perf().
site/demo/index.uce System Info now uses request_perf() and shows the real worker
PID, an incrementing per-worker request count, and the timing counters.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
BIN
View File
Binary file not shown.
-90
View File
@@ -1,90 +0,0 @@
# CLI Unit Test Port Plan
## Objective
Replace the Python-based network test runner and plugins with UCE unit tests invoked through the runtime's CLI socket path, keeping equivalent coverage for HTTP smoke, site suites, security checks, starter parity, TCP/WebSocket listener probes, and wasm kill checks. The final invocation should be a bash script that calls UCE CLI units; Python test files should be removed only after UCE coverage is in place and validated.
## Success Criteria
- [x] A bash command runs the full test suite through the CLI socket and exits nonzero on failure.
- [x] UCE CLI tests cover every current Python plugin behavior or explicitly document a deliberate replacement.
- [x] Existing W5/WASM gates use the UCE CLI test runner instead of `tests/run_network_tests.py`.
- [x] Python test runner/plugins are deleted after parity validation.
- [x] Full suite passes on `uce-dev` with `WASM_BACKEND_ENABLED=1`.
## Current State
- Status: verifying
- Last updated: 2026-06-13
- Source of truth: `/root/mount_ssh/uce-dev-root-htdocs-uce`
- Runtime/live target: `uce-dev:/Code/uce.openfu.com/uce`
## Goal Tree
Legend: `[ ]` not started, `[~]` in progress, `[x]` done, `[!]` blocked, `[-]` superseded
- [x] G1: Inventory Python test coverage and CLI constraints
- Why: replacement must preserve coverage before deleting Python.
- Done when: every plugin has a mapped UCE equivalent or blocker.
- Verify: coverage matrix in this document.
- [x] G1.1: Delegate design/coverage review to Spark.
- [x] G1.2: Inspect delegates and reconcile plan.
- [x] G2: Build UCE CLI test harness
- Why: Python runner features need a UCE-native replacement.
- Done when: one CLI unit can list/run tests, print pass/fail summary, and return failing CLI status.
- Verify: `scripts/run_cli_tests.sh --list` and `scripts/run_cli_tests.sh --include-wasm-kill`.
- [x] G2.1: Add reusable UCE assertion/reporting helpers.
- [x] G2.2: Add HTTP/TCP helper functions using UCE socket APIs.
- [x] G2.3: Add bash wrapper under `scripts/`.
- [x] G3: Port current plugin cases to UCE
- Why: only delete Python after equivalent UCE coverage exists.
- Done when: UCE suite covers demo, HTTP docs/starter, site suites, security, starter parity, TCP, wasm kill.
- Verify: UCE CLI full run passes and output names match coverage matrix.
- [x] G4: Replace Python gate usage and delete Python tests
- Why: user explicitly requested eliminating the Python suite.
- Done when: scripts no longer call `tests/run_network_tests.py`, Python test files removed, validation green.
- Verify: `rg 'run_network_tests|tests/plugins|python3 tests'` has no obsolete gate references except historical docs and benchmark/audit utilities.
- [~] G5: Document and validate
- Why: future agents/operators need the new test workflow.
- Done when: docs/project notes and in-repo docs mention the CLI test command and validation artifact.
- Verify: docs committed, full suite run artifact recorded.
## Coverage Matrix
- `uce_demo_smoke.py``cli_run_demo_smoke()` in `site/tests/cli_runner.uce` (43 demo pages).
- `uce_http_smoke.py``cli_run_http_smoke()` (docs and starter route/body checks, 14 cases).
- `uce_site_suite.py``cli_run_site_suite()` (manifest-driven published site suite pages, 13 cases).
- `uce_security_smoke.py``cli_run_security_smoke()` (direct HTTP traversal/header spoofing, CRLF header sanitization, session hardening, 4 cases).
- `uce_starter_parity.py``cli_run_starter_parity()` (starter view title/404 checks, 7 cases).
- `uce_tcp_smoke.py``cli_run_tcp_smoke()` (port 80 and 8080 reachability, 2 cases).
- `uce_wasm_kill.py``cli_run_wasm_kill()` gated by `--include-wasm-kill` (trap/loop/recurse + post-kill health checks, 3 cases).
## Execution Queue
1. Run final no-Python-suite validation after removing stale references.
2. Commit UCE and project-doc updates.
## Decisions
- 2026-06-13: Use UCE CLI socket invocation as the test entrypoint; bash wrappers are acceptable, Python runner/plugins are not.
- 2026-06-13: Do not delete Python tests until UCE replacement validates equivalent coverage on `uce-dev`.
## Assumptions
- UCE socket APIs are sufficient for HTTP/1.0 probes, TCP connect checks, and security header injection checks.
- Bash can provide filtering/list convenience if exact Python CLI parity is not needed.
## Blockers and Risks
- [ ] The new runner intentionally does not preserve the old Python runner's dynamic plugin/tag/regex filtering; add UCE-side selectors later if operators miss them.
- [ ] `tests/wasm_benchmark.py` and `tests/wasm_site_audit.py` remain Python utility scripts, not the network test suite; port separately if a strict no-Python tools policy is desired.
## Evidence and Verification Log
- 2026-06-13: Prior to this plan, `b1856c1 chore: narrow wasm backend entrypoint API` was committed after build and focused services validation.
- 2026-06-13: `scripts/run_cli_tests.sh --include-wasm-kill` on `uce-dev` passed `86 passed, 0 failed, 0 skipped` (`/tmp/uce/cli-tests-final.txt`).
## Change Log
- 2026-06-13: Created initial goal tree for CLI unit test port.
- 2026-06-13: Added `site/tests/cli_runner.uce`, `scripts/run_cli_tests.sh`, removed Python network runner/plugins, and repointed W5 network gates to the CLI runner.
@@ -1,6 +1,6 @@
diff --git a/vendor/miniz/miniz_tdef.c b/vendor/miniz/miniz_tdef.c
--- a/vendor/miniz/miniz_tdef.c
+++ b/vendor/miniz/miniz_tdef.c
diff --git a/src/3rdparty/miniz/miniz_tdef.c b/src/3rdparty/miniz/miniz_tdef.c
--- a/src/3rdparty/miniz/miniz_tdef.c
+++ b/src/3rdparty/miniz/miniz_tdef.c
@@
-static const mz_uint s_tdefl_num_probes[11];
+static const mz_uint s_tdefl_num_probes[11] = { 0, 1, 6, 32, 16, 32, 128, 256, 512, 768, 1500 };
-147
View File
@@ -1,147 +0,0 @@
# React Developer Affordances Todo
## Objective
Add practical value for developers coming from React frameworks while preserving UCE's server-first C++ model. Defer component syntax/children work, avoid global head/assets/islands in the runtime, and focus on function-library data helpers, diagnostics, docs, examples, demos, and a starter-local router with starter-local asset/island components.
## Success Criteria
- [x] Function library has useful collection/data-shaping helpers with docs and tests.
- [x] Compile/runtime diagnostics are more helpful, especially for generated-code and common preprocessor mistakes.
- [x] Docs include a concise React/Next/Remix orientation guide.
- [x] Starter example uses a centralized hierarchical/file-based router in `index.uce` efficiently.
- [x] Starter-local asset/island affordances live as component handlers in the starter, not global runtime APIs.
- [x] Network tests and relevant build checks pass on `k-uce`.
## Current State
- Status: complete
- Last updated: 2026-05-28
- Source of truth: `/root/mount_ssh/k-uce-root-htdocs-uce`
- Runtime/live target: `k-uce:/Code/uce.openfu.com/uce`; rebuilt and restarted `uce.service` on `k-uce`.
## Goal Tree
Legend: `[ ]` not started, `[~]` in progress, `[x]` done, `[!]` blocked, `[-]` superseded
- [x] G1: Add collection/data helpers to function library
- Why: React-framework developers routinely shape arrays/objects near render code.
- Done when: helpers are declared, implemented, documented, and covered by tests.
- Verify: build plus focused site/network tests.
- [x] G1.1: Identify current `StringList`/`DValue` idioms and choose helper surface.
- [x] G1.2: Implement minimal high-value helpers without broad template complexity.
- [x] G1.3: Add docs and examples for helpers.
- [x] G1.4: Add/extend tests.
- [x] G2: Improve developer diagnostics
- Why: React frameworks win by making failures easy to act on.
- Done when: compile/runtime error output includes actionable context and docs mention debugging flow.
- Verify: intentional broken page surfaces improved message.
- [x] G2.1: Inspect current compiler/runtime error rendering path.
- [x] G2.2: Add source excerpt / generated path / common-hint text where appropriate.
- [x] G2.3: Document diagnostics.
- [x] G3: Add React/Next/Remix orientation docs
- Why: mapping familiar concepts reduces onboarding cost without adding syntax.
- Done when: docs page exists and is linked from docs/README/demo surfaces.
- Verify: docs page renders live.
- [x] G4: Starter-local router and starter affordances
- Why: User specifically wants hierarchical/file-based routing beautifully in starter `index.uce`.
- Done when: starter routes go through a central router in `index.uce`, and starter-local asset/island component handlers exist and are used where sensible.
- Verify: key starter routes render 200.
- [x] G4.1: Inspect current starter routing.
- [x] G4.2: Refactor to clear route table / hierarchical file resolution in `index.uce`.
- [x] G4.3: Add starter-local `COMPONENT:asset` / `COMPONENT:island` style handlers in one unit.
- [x] G4.4: Use them efficiently in starter pages/layout.
- [x] G5: Demos and examples
- Why: Affordances must be visible to developers, not hidden in APIs.
- Done when: docs/demo/tests expose examples.
- Verify: demo URLs return 200 and tests pass.
- [x] G6: Verification and project docs
- Done when: build/test commands are run on `k-uce`, project notes updated, and adversarial review completed.
## Execution Queue
Complete.
## Decisions
- 2026-05-28: Defer component children/slots and JSX-like preprocessor syntax.
- 2026-05-28: Do not add global runtime head/assets/islands APIs; implement asset/island as starter-local components.
- 2026-05-28: Do not add a generic runtime file-based-routing system; demonstrate hierarchical/file routing inside starter `index.uce`.
- 2026-05-28: Keep collection helpers explicit (`list_*`, `dv_*`) instead of overloading generic names such as `map`/`sort`.
## Assumptions
- Current source-of-truth mount is live-editable; runtime validation requires SSH to `k-uce`.
- Existing site tests are the right place for function-library coverage.
## Blockers and Risks
- No current blockers.
- Future risk: if UCE grows a full parser or component-tag syntax, keep this pass's explicit component/router APIs as a stable lower-level fallback.
## Evidence and Verification Log
- 2026-05-28: Created plan after reviewing README, preprocessor docs, and function library headers.
- 2026-05-28: `ssh k-uce 'cd /Code/uce.openfu.com/uce && bash scripts/build_linux.sh'` succeeded.
- 2026-05-28: Restarted `uce.service` on `k-uce`.
- 2026-05-28: `tests/run_network_tests.py --match core` passed.
- 2026-05-28: Manual checks returned `200` for `/examples/uce-starter/index.uce`, `?dashboard`, `?workspace/projects`, `?themes`, `/demo/collections.uce`, `/doc/index.uce?p=coming_from_react`, `/doc/index.uce?p=list_map`, and `/doc/index.uce?p=dv_group_by`.
- 2026-05-28: Full internal network suite passed, 25/25.
- 2026-05-28: Temporary broken `/tests/diagnostic-probe.uce` returned `500` with formatted `UCE compile error` diagnostics; source and cache artifacts were removed afterward.
## Change Log
- 2026-05-28: Created initial goal tree.
- 2026-05-28: Implemented helpers, diagnostics, docs, demo, starter router, starter-local web affordance components, tests, and validation.
## Follow-up Cleanup 2026-05-29
- Removed duplicate route cleanup from `starter_router_candidates()` because `app_make_route()` is the single normalization point for `l_path`.
- Weeded out nearby duplicate/obsolete starter code:
- `starter_router_add_candidate(...)` now owns repeated candidate tree construction.
- Removed unused `app_resolve_view()` / `starter_resolve_view()` after moving routing into starter `index.uce`.
- Removed unused legacy registered asset rendering functions from `lib/app.uce`; registered assets now render through `components/theme/web_affordances.uce`.
- `app_init()` now reuses `app_base_url(context)` instead of repeating base URL derivation.
- `web_affordances.uce` now uses one `starter_render_asset_group(...)` loop for CSS and JS.
- Verification: rebuilt on `k-uce`, restarted `uce.service`, checked key starter routes, and ran `tests/run_network_tests.py --match core` successfully.
## Follow-up Routed Views 2026-05-29
- Changed starter route dispatch from `unit_render(...)` to `component(...)`.
- Converted all routed `site/examples/uce-starter/views/*.uce` files to `COMPONENT(Request& context)` rather than `RENDER(Request& context)` because they are central-router-only views.
- Updated the starter README and verified key starter routes. No service restart was required because only `.uce`/docs changed.
## Follow-up Canonical Starter URLs 2026-05-29
- Canonicalized starter self-links from `/examples/uce-starter/index.uce?...` to `/examples/uce-starter/?...` with `app_canonical_script_url(...)`.
- Updated the starter README to show canonical directory URLs.
- Touched the starter front controller to force the `#load`ed helper change into the cached generated unit.
- Verified canonical/direct starter routes and checked generated self-links. No service restart was required.
## Follow-up Not Found Component 2026-05-29
- Moved `starter_router_render_not_found` markup into `components/basic/notfound.uce`.
- Router now delegates 404 body rendering through `component("components/basic/notfound", props, context)`.
- Verified missing routes return `404` and normal dashboard route returns `200`. No service restart required.
## Follow-up Page Shell Component 2026-05-29
- Moved app page rendering into `themes/page.uce` as a component.
- Removed `app_render_page`, `app_theme_page_component`, and `starter_render_page` from `lib/app.uce`.
- Page template resolution now follows `context.call["app"]["page_type"]`: current theme first, common fallback second.
- Verified representative HTML routes and the JSON page-type fallback. No service restart required.
## Follow-up Deep Starter Context Cleanup 2026-05-29
- Removed repeated `starter_boot(context)` calls; root `index.uce` is the boot point.
- Removed `context.call["starter"]` duplicated state and JSON side-channel state.
- JSON routes now use only `context.call["app"]["page_type"]` plus normal captured output.
- Simplified `themes/page.uce` and `themes/common/page.json.uce` accordingly.
- Replaced `starter_*` alias helper usage with direct `app_*` helpers and removed alias wrappers except `StarterUser`.
- Verified key routes and core tests. No service restart required.
## Follow-up Route Context Flattening 2026-05-29
- Flattened `context.call["app"]["route"]` to `context.call["route"]`.
- Moved former `context.call["app"]["router"]` metadata into `context.call["route"]`.
- Verified representative starter HTML, 404, and JSON routes. No service restart required.
+674
View File
@@ -0,0 +1,674 @@
# UCE Runtime Setup
This guide describes how to run UCE behind nginx or Apache. UCE is a FastCGI application server for `.uce` units; the web server should serve static files directly and forward dynamic `.uce` requests to the UCE runtime.
## Deployment shape
A typical deployment has four pieces:
1. A checked-out or packaged UCE runtime tree.
2. `/etc/uce/settings.cfg`, read by the UCE runtime at startup.
3. `uce.service`, a systemd service that builds/starts/restarts the runtime.
4. nginx or Apache as the public HTTP server.
Recommended filesystem layout for a source checkout:
```text
/opt/uce/ UCE repository/runtime root
/var/www/html/ public web root served by nginx/Apache
/etc/uce/settings.cfg runtime configuration
/run/uce/fastcgi.sock FastCGI socket used by nginx/Apache
/run/uce/cli.sock local CLI/admin/test socket
/var/cache/uce/work generated source, wasm modules, caches
/var/lib/uce/uploads multipart upload scratch space
/var/lib/uce/sessions session files
```
For packaged installs, the runtime may live under `/usr/lib/uce` instead of `/opt/uce`. Keep the public web root at `/var/www/html` or another normal web-root path, not under the runtime source tree.
## Build requirements
On Debian/Ubuntu-like systems, install the distro packages first:
```bash
apt update
apt install -y clang build-essential libpcre2-dev mariadb-client libmariadb-dev curl rsync ca-certificates
```
UCE also requires two non-vendored dependencies. WASI SDK is load-bearing at runtime because UCE compiles units on demand during requests and during proactive startup scans.
- **Wasmtime C API / C++ headers** at `/opt/wasmtime` by default. `scripts/build_linux.sh` expects:
- `/opt/wasmtime/include/wasmtime.hh`
- `/opt/wasmtime/include/wasmtime/*.h`
- `/opt/wasmtime/lib/libwasmtime.so`
- **Pinned WASI SDK** at `/opt/wasi-sdk` by default. `scripts/build_core_wasm.sh` and request-time `scripts/compile_wasm_unit` expect:
- `/opt/wasi-sdk/bin/clang++`
- `/opt/wasi-sdk/bin/wasm-ld`
- `/opt/wasi-sdk/bin/llvm-objcopy`
You can use different install locations by setting environment variables before building and in the systemd service environment:
```bash
export WASMTIME_HOME=/path/to/wasmtime
export WASI_SDK=/path/to/wasi-sdk
```
Install one web server:
```bash
apt install -y nginx
# or
apt install -y apache2
```
Build UCE from the repository root:
```bash
cd /opt/uce
bash scripts/build_core_wasm.sh
bash scripts/build_linux.sh
```
Publish the starter site or your application files into the web root:
```bash
mkdir -p /var/www/html
rsync -a site/ /var/www/html/
```
The main binary is written to:
```text
bin/uce_fastcgi.linux.bin
```
### Installing Wasmtime and WASI SDK
The UCE build does not download these dependencies for you. Install Wasmtime through a compatible distro package or a pinned upstream C API archive. Install WASI SDK with UCE's pinned installer, or unpack the same pinned archive under `/opt/wasi-sdk`.
Do not use `curl | sh` installers in production setup scripts. Download archives from the upstream release pages, verify checksums/signatures when available, and record the exact versions in your deployment notes. Avoid installing a release published in the last few days unless you have reviewed it separately.
The expected directories are:
```text
/opt/wasmtime/include/wasmtime.hh
/opt/wasmtime/include/wasmtime/store.h
/opt/wasmtime/lib/libwasmtime.so
/opt/wasi-sdk/bin/clang++
/opt/wasi-sdk/bin/wasm-ld
/opt/wasi-sdk/bin/llvm-objcopy
```
Install the WASI SDK:
```bash
cd /opt/uce
scripts/install_wasi_sdk.sh
scripts/install_wasi_sdk.sh --check-only
```
The current pin is documented in `docs/wasi-sdk-toolchain.md`. The script verifies the archive SHA256 before installing and updates `/opt/wasi-sdk` to point at the pinned versioned directory.
For Wasmtime, use a compatible distro package or an upstream C API archive. Example flow using an archive you have already chosen and verified:
```bash
mkdir -p /opt /tmp/uce-deps
cd /tmp/uce-deps
# Download the Wasmtime C API archive for your architecture from the upstream
# release page, then verify its checksum before unpacking. The archive name
# normally contains "c-api".
sha256sum -c wasmtime-c-api.sha256
mkdir -p /opt/wasmtime
tar -xf wasmtime-*-c-api*.tar.* -C /opt/wasmtime --strip-components=1
```
After unpacking, verify the tools UCE needs. Also record the exact Wasmtime and WASI SDK versions used. The native build embeds an rpath for `$WASMTIME_HOME/lib`, so the service environment should use the same `WASMTIME_HOME` value used during build.
```bash
test -f /opt/wasmtime/include/wasmtime.hh
test -f /opt/wasmtime/lib/libwasmtime.so
/opt/wasi-sdk/bin/clang++ --version
/opt/wasi-sdk/bin/wasm-ld --version
/opt/wasi-sdk/bin/llvm-objcopy --version
```
If your paths differ, export the variables for manual builds:
```bash
WASMTIME_HOME=/usr/local/wasmtime WASI_SDK=/usr/local/wasi-sdk bash scripts/build_core_wasm.sh
WASMTIME_HOME=/usr/local/wasmtime WASI_SDK=/usr/local/wasi-sdk bash scripts/build_linux.sh
```
For systemd, add an override:
```bash
systemctl edit uce.service
```
```ini
[Service]
Environment=WASMTIME_HOME=/usr/local/wasmtime
Environment=WASI_SDK=/usr/local/wasi-sdk
```
Then reload and restart:
```bash
systemctl daemon-reload
systemctl restart uce.service
```
## Runtime configuration
Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths if your runtime is not under `/opt/uce`. Replace any checkout-specific paths such as `WASM_CORE_PATH` with `/opt/uce/bin/wasm/core.wasm` or your actual runtime path.
Minimum useful settings:
```ini
BIN_DIRECTORY=/var/cache/uce/work
TMP_UPLOAD_PATH=/var/lib/uce/uploads
SESSION_PATH=/var/lib/uce/sessions
FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
CLI_SOCKET_PATH=/run/uce/cli.sock
SITE_DIRECTORY=/var/www/html
HTTP_DOCUMENT_ROOT=/var/www/html
JIT_COMPILE_ON_REQUEST=1
PROACTIVE_COMPILE_ENABLED=1
PROACTIVE_COMPILE_CHECK_INTERVAL=60
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
WASM_BACKEND_VERBOSE=0
WASM_CORE_PATH=/opt/uce/bin/wasm/core.wasm
WASM_MEMORY_LIMIT_BYTES=536870912
WASM_EPOCH_DEADLINE_TICKS=200
WASM_EPOCH_PERIOD_MS=50
WORKER_COUNT=4
MAX_MEMORY=16777216
SESSION_TIME=2592000
HTTP_PORT=8080
```
Important settings:
- `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests. Set it explicitly and keep this value and the web-server `fastcgi_pass` path identical. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
- `CLI_SOCKET_PATH` is a local HTTP-over-Unix socket used by `scripts/uce-cli` and test/admin units.
- `SITE_DIRECTORY` is the public site tree to scan for `.uce` files. Use `/var/www/html` when the web root is outside the runtime tree; relative paths are resolved from the runtime working directory.
- `HTTP_DOCUMENT_ROOT` is the root used by the built-in HTTP/WebSocket listener when it resolves upgrade requests. Set it to the same web root as nginx/Apache.
- `BIN_DIRECTORY` stores generated C++, wasm artifacts, compile output, and runtime caches.
- `TMP_UPLOAD_PATH` and `SESSION_PATH` must be writable by the runtime.
- `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes. Bind/firewall it for local access only; nginx/Apache should be the public entry point.
- `WASM_COMPILE_SCRIPT` must point to `scripts/compile_wasm_unit` unless you provide an equivalent compiler. Relative paths are resolved from the runtime root/`COMPILER_SYS_PATH`. That script calls `scripts/check_unit_wasm.py` after linking each unit and uses the pinned WASI SDK on every deployment host.
- `WASM_CORE_PATH` must point at the built `core.wasm` file.
After editing settings, restart UCE:
```bash
systemctl restart uce.service
```
## systemd service
For source-checkout deployments, install the provided service helper:
```bash
cd /opt/uce
scripts/systemd/manage-uce-service.sh setup
```
That helper installs `scripts/systemd/uce.service`, creates runtime directories, enables the service, and starts it.
Useful commands:
```bash
scripts/systemd/manage-uce-service.sh status
scripts/systemd/manage-uce-service.sh restart
scripts/systemd/manage-uce-service.sh logs 200
```
Equivalent manual systemd service for a source checkout:
```ini
[Unit]
Description=UCE FastCGI Runtime
After=network-online.target mariadb.service memcached.service
Wants=network-online.target
[Service]
Type=simple
WorkingDirectory=/opt/uce
RuntimeDirectory=uce
StateDirectory=uce
CacheDirectory=uce
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock
ExecStartPre=/usr/bin/bash /opt/uce/scripts/build_linux.sh
ExecStart=/opt/uce/bin/uce_fastcgi.linux.bin
ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock
Restart=always
RestartSec=2
TimeoutStopSec=15
KillMode=mixed
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
```
Install it as `/etc/systemd/system/uce.service` and run:
```bash
systemctl daemon-reload
systemctl enable --now uce.service
```
### Debian package build
To build a Debian package from the repository root:
```bash
bash scripts/make_deb.sh 0.1.2
```
The Debian package creator bundles WASI SDK and Wasmtime by default when `/opt/wasi-sdk` and `/opt/wasmtime` are present. Verify the pinned SDK before building:
```bash
scripts/install_wasi_sdk.sh --check-only
bash scripts/make_deb.sh 0.1.2
```
This includes the resolved `/opt/wasi-sdk-...` tree, `/opt/wasi-sdk` symlink, resolved `/opt/wasmtime-...` tree, and `/opt/wasmtime` symlink in the package. It makes the package large, but keeps request-time unit compilation and runtime linking tied to the toolchain versions that passed the test suite. Set `UCE_DEB_BUNDLE_WASI_SDK=0` or `UCE_DEB_BUNDLE_WASMTIME=0` only if your deployment provides those exact dependencies separately.
### RPM package build
To build an RPM package from the repository root, install `rpmbuild` on the packaging host, verify the pinned SDK, then run:
```bash
scripts/install_wasi_sdk.sh --check-only
bash scripts/make_rpm.sh 0.1.2
```
The RPM creator mirrors the Debian package layout: runtime files under `/usr/lib/uce`, public files under `/var/www/html`, config under `/etc/uce/settings.cfg`, systemd unit under `/usr/lib/systemd/system/uce.service`, and bundled `/opt/wasi-sdk` plus `/opt/wasmtime` trees by default. Set `UCE_RPM_BUNDLE_WASI_SDK=0` or `UCE_RPM_BUNDLE_WASMTIME=0` only if your deployment provides those exact dependencies separately.
## How request routing works
### Static files
The web server should serve ordinary static files directly from the public web root, for example `/var/www/html`.
Examples:
```text
/style.css
/images/logo.png
/examples/uce-starter/js/site.js
```
These should not touch the UCE runtime.
### Normal `.uce` page requests
For a request such as:
```text
GET /doc/index.uce?p=component
```
nginx/Apache forwards the request to `FCGI_SOCKET_PATH` as FastCGI. The web server must provide CGI/FastCGI variables including:
- `SCRIPT_FILENAME` — full filesystem path to the `.uce` file.
- `DOCUMENT_ROOT` — public web root, normally `/var/www/html` or whatever nginx/Apache uses as `root`/`DocumentRoot`.
- `SCRIPT_NAME` — URL path to the script, such as `/doc/index.uce`.
- `DOCUMENT_URI` — normalized URI path without query string.
- `REQUEST_URI` — original request URI including query string.
- standard request variables such as method, query string, content type, body length, cookies, and headers.
UCE resolves the unit, compiles it to wasm if needed, creates a request workspace, and calls:
```cpp
RENDER(Request& context)
```
The unit writes output with template literals or `print()`. Response headers and status are set through `context.header` and `context.set_status()`.
### Component and sub-render calls
Inside a request, UCE code can call other units:
```cpp
component("components/card", props, context);
unit_render("other-page.uce", context);
```
These calls stay inside the UCE runtime. They are not new HTTP requests and do not go back through nginx or Apache.
### WebSocket pages
Any `.uce` unit can provide both an ordinary page render and WebSocket message handling:
```cpp
RENDER(Request& context) { ... } // normal page load
WS(Request& context) { ... } // later WebSocket messages
```
The nginx and Apache examples below split traffic by checking for a WebSocket upgrade request on `.uce` paths. A file such as `chat.uce` or `events.uce` can expose `WS(Request& context)`.
Routing split:
- Plain `GET /demo/chat.uce` should use FastCGI, just like any other page render.
- WebSocket upgrade requests for `/demo/chat.uce` should proxy to the UCE built-in HTTP/WebSocket listener at `HTTP_PORT`.
The built-in listener owns the socket lifecycle. When a message arrives, the broker forwards a render-style invocation back to the worker pool so `WS(Request& context)` runs inside the same wasm runtime model as normal pages.
### CLI requests
`CLI(Request& context)` handlers are not public web endpoints. They are invoked over `CLI_SOCKET_PATH`:
```bash
scripts/uce-cli /tests/cli.uce action=echo message=hello
curl --unix-socket /run/uce/cli.sock http://localhost/tests/cli.uce
```
Use CLI units for local tests, admin commands, and maintenance tools. Do not expose the CLI socket through nginx or Apache.
### Custom runtime HTTP servers
UCE code can start local custom HTTP listeners with `server_start_http()`. Those are runtime-managed listeners for app-specific local services. They are separate from the public nginx/Apache entry point and should be firewalled or bound locally unless you explicitly want them reachable.
## nginx configuration
### Required modules
A normal nginx build includes the needed FastCGI and proxy modules. Confirm nginx is installed and can load your config:
```bash
nginx -t
```
### WebSocket upgrade map
Put this in the nginx `http` block if using WebSockets:
```nginx
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
```
### Server block
Example site config:
```nginx
server {
listen 80;
server_name example.com;
root /var/www/html;
index index.uce index.html;
# Serve static files directly. Directory requests use index.uce when present.
location / {
try_files $uri $uri/ =404;
}
# UCE page requests use FastCGI. If the client asks to upgrade a .uce
# request to WebSocket, send that connection to the built-in listener.
location ~ \.uce$ {
error_page 418 = @uce_websocket;
if ($http_upgrade = "websocket") {
return 418;
}
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param DOCUMENT_URI $uri;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_pass unix:/run/uce/fastcgi.sock;
}
location @uce_websocket {
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_pass http://127.0.0.1:8080;
}
# Defense in depth if the root is changed later.
location ~ ^/(src|scripts|etc|bin|work|dist|pkg|docs|changelog)/ {
return 404;
}
}
```
Notes:
- `fastcgi_pass` must match `FCGI_SOCKET_PATH`.
- `proxy_pass` must match `HTTP_PORT`.
- The example routes WebSocket upgrades for `.uce` paths to the HTTP/WebSocket listener.
- The built-in HTTP/WebSocket listener resolves scripts from `HTTP_DOCUMENT_ROOT`; do not depend on client-supplied or proxied `Script-Filename` headers for routing.
- Ordinary `.uce` page loads continue to use FastCGI.
- Keep `root` pointed at `/var/www/html`, not the runtime repository root.
- If your app uses a front controller, replace `location /` with a `try_files` rule that ends at `/index.uce`.
Front-controller variant:
```nginx
location / {
try_files $uri $uri/ /index.uce?$query_string;
}
```
Reload nginx:
```bash
nginx -t
systemctl reload nginx
```
## Apache configuration
Apache can run UCE through `mod_proxy_fcgi` for FastCGI and `mod_proxy_wstunnel` or `mod_proxy_http` for WebSocket upgrades.
### Enable modules
On Debian/Ubuntu:
```bash
a2enmod proxy proxy_fcgi proxy_http proxy_wstunnel rewrite headers setenvif
systemctl restart apache2
```
### VirtualHost example
```apache
<VirtualHost *:80>
ServerName example.com
DocumentRoot /var/www/html
<Directory /var/www/html>
Require all granted
Options FollowSymLinks
AllowOverride None
DirectoryIndex index.uce index.html
</Directory>
# Do not expose repository internals if DocumentRoot changes later.
<LocationMatch "^/(src|scripts|etc|bin|work|dist|pkg|docs|changelog)/">
Require all denied
</LocationMatch>
RewriteEngine On
# WebSocket upgrade traffic for any .uce unit goes to UCE's built-in HTTP listener.
RewriteCond %{HTTP:Upgrade} =websocket [NC]
RewriteCond %{REQUEST_URI} \.uce(?:\?|$) [NC]
RewriteRule ^/(.*)$ ws://127.0.0.1:8080/$1 [P,L]
# Normal .uce page loads go to FastCGI.
<FilesMatch "\.uce$">
SetHandler "proxy:unix:/run/uce/fastcgi.sock|fcgi://localhost/"
</FilesMatch>
# Optional: make the key CGI variables explicit for UCE.
ProxyFCGISetEnvIf "true" DOCUMENT_ROOT "/var/www/html"
ProxyFCGISetEnvIf "true" DOCUMENT_URI "%{REQUEST_URI}"
</VirtualHost>
```
Apache notes:
- `SetHandler "proxy:unix:/run/uce/fastcgi.sock|fcgi://localhost/"` must use the same socket path as `FCGI_SOCKET_PATH`.
- The WebSocket rewrite rule must run before the FastCGI handler.
- Plain `.uce` page loads should not be proxied as WebSockets unless the client sends `Upgrade: websocket`.
- Apache's FastCGI environment differs by version and module configuration. If UCE cannot resolve a page, inspect the request environment and make sure `SCRIPT_FILENAME` points to the target file under the web root.
If your Apache version does not populate `SCRIPT_FILENAME` correctly through `SetHandler`, use `ProxyPassMatch` for `.uce` files instead:
```apache
ProxyPassMatch ^/(.*\.uce)$ unix:/run/uce/fastcgi.sock|fcgi://localhost/var/www/html/$1
```
Use only one FastCGI mapping style at a time (`SetHandler` or `ProxyPassMatch`) to avoid duplicate routing.
## Permissions
The web server needs permission to connect to `/run/uce/fastcgi.sock`. Common approaches:
- run UCE and the web server under compatible groups;
- add the web server user (`www-data` on Debian/Ubuntu) to the socket's group;
- adjust the service or runtime socket mode if needed.
The runtime creates the FastCGI socket and CLI socket under `/run/uce`. The CLI socket should remain local-only and should not be reachable from the public web server.
Writable paths for the runtime:
```text
/var/cache/uce/work
/var/lib/uce/uploads
/var/lib/uce/sessions
/run/uce
```
## Verification
Check service state:
```bash
systemctl status uce.service
journalctl -u uce.service -n 100 --no-pager
```
Check the FastCGI/web-server path:
```bash
curl -i http://127.0.0.1/doc/index.uce -H 'Host: example.com'
curl -i http://127.0.0.1/examples/uce-starter/ -H 'Host: example.com'
```
Check the local CLI path:
```bash
cd /opt/uce
scripts/uce-cli /tests/cli.uce action=echo message=hello
scripts/run_cli_tests.sh
```
Check WebSocket routing with a WebSocket client against a `.uce` endpoint that defines `WS(Request& context)` through nginx/Apache, not directly against `HTTP_PORT`:
```bash
python3 - <<'PY'
import base64, os, socket
host = "example.com"
path = "/chat.uce"
key = base64.b64encode(os.urandom(16)).decode()
request = (
f"GET {path} HTTP/1.1\r\n"
f"Host: {host}\r\n"
"Upgrade: websocket\r\n"
"Connection: Upgrade\r\n"
f"Sec-WebSocket-Key: {key}\r\n"
"Sec-WebSocket-Version: 13\r\n\r\n"
).encode()
sock = socket.create_connection(("127.0.0.1", 80), timeout=5)
sock.sendall(request)
print(sock.recv(4096).decode("latin1", "replace").split("\r\n", 1)[0])
sock.close()
PY
```
## Operational footguns
- Keep the FastCGI socket path consistent: `FCGI_SOCKET_PATH` and the web-server `fastcgi_pass` must match exactly. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
- Keep the public web root separate from the runtime source tree. The examples use `/opt/uce` for runtime files and `/var/www/html` for public files.
- Set `HTTP_DOCUMENT_ROOT` when the web root is outside the runtime working directory. The built-in HTTP/WebSocket listener resolves upgrade paths from this setting.
- Do not expose `CLI_SOCKET_PATH` or `HTTP_PORT` as public entry points. The public path should be nginx/Apache.
- Do not trust `Script-Filename` request headers from direct HTTP clients. The built-in HTTP listener resolves from `HTTP_DOCUMENT_ROOT` and rejects `..` path segments.
- WASI SDK is a deployment/runtime dependency, not just a developer build tool. UCE compiles units to wasm on demand during requests and during proactive startup scans, so each host must use the pinned SDK version documented in `docs/wasi-sdk-toolchain.md`.
- After toolchain or compile-script fixes, clear stale failed artifacts under `BIN_DIRECTORY`; otherwise a later request may report an old compile failure.
## Troubleshooting
### 502 Bad Gateway
Check:
- `systemctl status uce.service`
- `journalctl -u uce.service -n 200 --no-pager`
- socket path in web server config equals `FCGI_SOCKET_PATH`
- web server user can connect to the Unix socket
- `SCRIPT_FILENAME` resolves to an existing `.uce` file
### Raw `.uce` source is downloaded or displayed
The `.uce` request did not match the FastCGI rule. Check location/order rules and confirm the public root is `/var/www/html` or your chosen web-root path.
### Static files 404
Confirm the web server `root`/`DocumentRoot` is `/var/www/html` or your chosen web-root path and that `location /` or Apache directory rules allow static file reads.
### WebSocket page renders but upgrade fails
Check:
- the client sends `Upgrade: websocket`
- `.uce` upgrade traffic reaches `HTTP_PORT`
- nginx/Apache preserves `Upgrade` and `Connection` headers
- `HTTP_DOCUMENT_ROOT` matches the web-server root; if it points at the runtime tree while files live in `/var/www/html`, the built-in listener will return `404 script not found`
- firewall/network policy allows localhost access to `HTTP_PORT`
### Page compiles fail
Check the compile artifact paths shown in the UCE error response and service logs. Generated files and compile output live under `BIN_DIRECTORY`.
Common compile footguns:
- `WASM_COMPILE_SCRIPT` is unset or points at a removed script such as `scripts/compile`; set it to `scripts/compile_wasm_unit`.
- `scripts/check_unit_wasm.py` is missing or not executable; `scripts/compile_wasm_unit` calls it after linking each unit.
- `WASI_SDK` does not point at the pinned tree with `clang++`, `wasm-ld`, `llvm-objcopy`, and `llvm-nm`; run `scripts/install_wasi_sdk.sh --check-only`.
- `WASMTIME_HOME` does not point at a tree with Wasmtime headers and `libwasmtime.so`.
- A previous failed compile left stale `.compile.txt`, `.wasm-check.txt`, or partial `.wasm` files under `BIN_DIRECTORY`.
Failed compile output is persisted under the unit's generated path in `BIN_DIRECTORY` and may be reused until the source or compiler inputs change. First fix the source/toolchain issue and reload the page. If the cache itself is suspect, stop UCE, move only the affected unit artifact files or directory aside, and restart so the runtime recompiles from source. Avoid deleting the whole `BIN_DIRECTORY` unless you intentionally want a full rebuild.
### CLI commands fail
Check:
- `CLI_SOCKET_PATH` in `/etc/uce/settings.cfg`
- `/run/uce/cli.sock` exists
- `scripts/uce-cli --socket /run/uce/cli.sock /ping` works
- the target unit defines `CLI(Request& context)`
+55
View File
@@ -0,0 +1,55 @@
# WASI SDK Toolchain Pin
UCE treats WASI SDK as a deployment/runtime dependency, not just a developer build tool.
The runtime compiles `.uce` units to wasm on demand during requests and during the proactive compiler scan. That means every deployment host must have the same compiler/linker toolchain available, and the generated `.wasm`/`.cwasm` artifacts are tied to that toolchain version and UCE unit ABI version.
## Current pin
- Upstream: <https://github.com/WebAssembly/wasi-sdk>
- Release tag: `wasi-sdk-33`
- Version: `33.0`
- Linux x86_64 archive: `wasi-sdk-33.0-x86_64-linux.tar.gz`
- URL: `https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-33/wasi-sdk-33.0-x86_64-linux.tar.gz`
- SHA256: `0ba8b5bfaeb2adf3f29bab5841d76cf5318ab8e1642ea195f88baba1abd47bce`
- Expected install symlink: `/opt/wasi-sdk`
- Expected resolved path: `/opt/wasi-sdk-33.0-x86_64-linux`
Install or verify with:
```bash
scripts/install_wasi_sdk.sh
scripts/install_wasi_sdk.sh --check-only
```
## Required tools
UCE expects these executables on each deployment host:
```text
/opt/wasi-sdk/bin/clang++
/opt/wasi-sdk/bin/wasm-ld
/opt/wasi-sdk/bin/llvm-objcopy
/opt/wasi-sdk/bin/llvm-nm
```
`llvm-nm` is used by `scripts/check_unit_wasm.py`, which is called by `scripts/compile_wasm_unit` after linking each unit.
## Upgrade policy
Treat WASI SDK upgrades like runtime dependency upgrades:
1. Update `scripts/install_wasi_sdk.sh` version, URL, and SHA256.
2. Record the new release and checksum here.
3. Rebuild `bin/wasm/core.wasm` with `scripts/build_core_wasm.sh`.
4. Rebuild the native runtime with `scripts/build_linux.sh`.
5. Clear or invalidate stale unit wasm artifacts by bumping `UCE_UNIT_ABI_VERSION` when required, or by removing affected generated artifacts under `BIN_DIRECTORY`.
6. Run the full CLI suite including wasm kill tests:
```bash
scripts/run_cli_tests.sh --include-wasm-kill
```
## Known footgun
WASI SDK 33's `llvm-nm` was observed to crash on a degenerate but valid unit module with no exported handlers. `scripts/check_unit_wasm.py` treats that specific validator-tool crash as a skipped allocator-definition scan while still rejecting forbidden allocator exports and other ABI violations. This is one reason the toolchain is pinned instead of relying on whatever `/opt/wasi-sdk` happens to contain.
@@ -1,104 +0,0 @@
[
{
"backend": "native",
"target": "template-heavy-doc",
"url": "/doc/singlepage.uce",
"ok": true,
"status": 200,
"samples_ms": [
318.72209906578064,
340.9293442964554,
319.705568253994,
324.0259513258934,
350.93583166599274,
313.20811808109283,
317.0944079756737,
310.312956571579,
307.89367109537125,
308.1864267587662,
311.681292951107,
313.39504569768906,
314.3857270479202,
312.5988021492958,
313.25943768024445,
312.9996135830879,
334.4864323735237,
314.2518773674965,
309.7490146756172,
317.4726217985153
],
"median_ms": 313.8234615325928,
"mean_ms": 318.2647120207548,
"min_ms": 307.89367109537125,
"max_ms": 350.93583166599274,
"note": ""
},
{
"backend": "native",
"target": "sqlite-page",
"url": "/demo/sqlite.uce",
"ok": true,
"status": 200,
"samples_ms": [
3.4144148230552673,
6.088584661483765,
3.463640809059143,
3.498159348964691,
3.5023540258407593,
6.214611232280731,
3.563329577445984,
3.3720433712005615,
3.330707550048828,
3.3808723092079163,
3.322914242744446,
3.3655911684036255,
3.374151885509491,
3.358304500579834,
3.3307820558547974,
3.268897533416748,
3.360658884048462,
3.3061057329177856,
6.233863532543182,
3.432638943195343
],
"median_ms": 3.3775120973587036,
"mean_ms": 3.809131309390068,
"min_ms": 3.268897533416748,
"max_ms": 6.233863532543182,
"note": ""
},
{
"backend": "native",
"target": "component-heavy-starter",
"url": "/examples/uce-starter/?dashboard",
"ok": true,
"status": 200,
"samples_ms": [
74.01428371667862,
40.9795418381691,
40.995217859745026,
41.04568809270859,
40.724173188209534,
41.10313206911087,
40.59913754463196,
41.28593951463699,
41.10439121723175,
40.649913251399994,
42.06441342830658,
42.074643075466156,
43.475523591041565,
41.8255478143692,
41.068583726882935,
41.116394102573395,
41.09777510166168,
42.58237034082413,
41.312023997306824,
70.98117470741272
],
"median_ms": 41.11039265990257,
"mean_ms": 44.50499340891838,
"min_ms": 40.59913754463196,
"max_ms": 74.01428371667862,
"note": ""
}
]
@@ -1,19 +0,0 @@
# Phase 5 native baseline — 2026-06-12
Host: `k-uce` / `uce.openfu.com` via localhost with Host header.
This is a durable informational snapshot. The Phase 5 gate recomputes native medians during paired native/WASM runs; these numbers are not hard-coded budgets.
- Warmup suite: 82/82 passed; excludes `site tests tasks` to avoid perturbing task lifecycle state.
- Measured full network suite: 83/83 passed.
- Measured starter subset: 14/14 passed.
- Static audit default scan: 50 code findings, documentation prose excluded by default.
| target | median ms | mean ms | samples |
|---|---:|---:|---:|
| template-heavy-doc | 313.8 | 318.3 | 20 |
| sqlite-page | 3.4 | 3.8 | 20 |
| component-heavy-starter | 41.1 | 44.5 | 20 |
Raw benchmark JSON: `native-baseline-2026-06-12.json`.
Static audit snapshot: `site-static-audit-2026-06-12.md`.
@@ -1,56 +0,0 @@
# Phase 5 site static-state audit
Findings: 50 code, 0 documentation prose.
| file | line | severity | kind | code | note |
|---|---:|---|---|---|---|
| site/demo/index.uce | 76 | code | background task | `<? if(allow_server_demos) { render_card("task_repeat.uce", "Task Repeat", "Recurring task scheduling"); } ?>` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/once-init.uce | 1 | code | static local/global | `static s64 demo_worker_init_count = 0;` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/demo/once-init.uce | 2 | code | static local/global | `static s64 demo_component_hits = 0;` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/demo/once-init.uce | 4 | code | init hook | `INIT(Request& context)` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/demo/once-init.uce | 10 | code | once hook | `ONCE(Request& context)` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/demo/once-init.uce | 44 | code | once hook | `ONCE() and INIT()` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/demo/once-init.uce | 44 | code | init hook | `ONCE() and INIT()` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/demo/once-init.uce | 47 | code | once hook | `This page calls the same named component twice. `ONCE()` should only run once for the request, while `INIT()` should stay stable for the currently loaded worker copy.` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/demo/once-init.uce | 47 | code | init hook | `This page calls the same named component twice. `ONCE()` should only run once for the request, while `INIT()` should stay stable for the currently loaded worker copy.` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/demo/task-status.uce | 11 | code | background task | `String task_name = first(context.get["task-name"], "example-task");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task-status.uce | 13 | code | background task | `print("Task Name: ", task_name, "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task-status.uce | 14 | code | background task | `print("Task ID: ", task_pid(task_name), "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task-status.uce | 15 | code | background task | `print("Task Running: ", task_pid(task_name) == 0 ? "no" : "yes", "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 13 | code | background task | `String task_name = first(context.get["task-name"], "example-task");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 37 | code | background task | `<input type="text" name="task-name" value="<?= task_name ?>"/>` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 46 | code | background task | `load(document.getElementById('task-status'), 'task-status.uce?task-name=<?= uri_encode(task_name) ?>');` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 56 | code | background task | `print("Task Name: ", task_name, "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 57 | code | background task | `print("Task ID: ", task_pid(task_name), "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 58 | code | background task | `print("Task Running: ", task_pid(task_name) == 0 ? "no" : "yes", "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task.uce | 71 | code | background task | `print("New Task ID: ", task(task_name, []() {` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 13 | code | background task | `String task_name = first(context.get["task-name"], "example-task");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 37 | code | background task | `<input type="text" name="task-name" value="<?= task_name ?>"/>` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 46 | code | background task | `load(document.getElementById('task-status'), 'task-status.uce?task-name=<?= uri_encode(task_name) ?>');` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 56 | code | background task | `print("Task Name: ", task_name, "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 57 | code | background task | `print("Task ID: ", task_pid(task_name), "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 58 | code | background task | `print("Task Running: ", task_pid(task_name) == 0 ? "no" : "yes", "\n");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/demo/task_repeat.uce | 71 | code | background task | `print("New Task ID: ", task_repeat(task_name, 5, []() {` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/examples/uce-starter/components/data/widgets.uce | 3 | code | once hook | `ONCE(Request& context)` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/examples/uce-starter/components/workspace/primitives.uce | 3 | code | once hook | `ONCE(Request& context)` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/examples/uce-starter/lib/user.class.h | 13 | code | static local/global | `static String session_key()` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/examples/uce-starter/lib/user.class.h | 18 | code | static local/global | `static String normalize_email(String email)` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/examples/uce-starter/lib/user.class.h | 23 | code | static local/global | `static String hash_id(String raw)` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/examples/uce-starter/lib/user.class.h | 45 | code | static local/global | `static String password_hash(String password, String salt)` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/examples/uce-starter/lib/user.class.h | 53 | code | static local/global | `static DValue read_json_file(String file_name)` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/examples/uce-starter/lib/user.class.h | 64 | code | static local/global | `static bool write_json_file(String file_name, DValue data)` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/examples/uce-starter/views/dashboard.uce | 3 | code | once hook | `ONCE(Request& context)` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/info/index.uce | 269 | code | static local/global | `<li>nginx serves static files directly from `site/`</li>` | Check whether state is request-local, immutable, or intentionally persistent; unit statics reset per wasm workspace. |
| site/tests/preprocessor.uce | 3 | code | once hook | `ONCE(Request& context)` | Audit behavior under per-request wasm workspaces; ONCE/INIT may need host-side cache semantics if used for cross-request state. |
| site/tests/tasks.uce | 25 | code | background task | `pid_t repeat_existing = task_pid("site-tests-repeat");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 27 | code | background task | `task_kill(repeat_existing, 15);` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 37 | code | background task | `repeat_pid = task_repeat("site-tests-repeat", 1.0, []() {` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 55 | code | background task | `pid_t seen_short_pid = task_pid("site-tests-short");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 56 | code | background task | `pid_t seen_repeat_pid = task_pid("site-tests-repeat");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 57 | code | background task | `pid_t seen_timeout_pid = task_pid("site-tests-timeout");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 58 | code | background task | `pid_t seen_unsafe_key_pid = task_pid("site-tests/../unsafe key");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 59 | code | background task | `int short_alive = seen_short_pid == 0 ? -1 : task_kill(seen_short_pid, 0);` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 60 | code | background task | `int repeat_alive = seen_repeat_pid == 0 ? -1 : task_kill(seen_repeat_pid, 0);` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 72 | code | background task | `check("task_pid() + task_kill(pid, 0)", short_alive == 0, "kill(0) result=" + std::to_string(short_alive));` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 73 | code | background task | `check("task_repeat()", repeat_pid != 0 && seen_repeat_pid != 0, "started=" + std::to_string(repeat_pid) + " seen=" + std::to_string(seen_repeat_pid));` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
| site/tests/tasks.uce | 77 | code | background task | `check("task_kill() rejects negative pid", task_kill(-1, 0) == -1, "kill(-1, 0) rejected");` | Task APIs cross request lifetimes; verify they are host handles, not guest statics. |
-110
View File
@@ -1,110 +0,0 @@
# WASM Phase 1: DValue C ABI and UCEB1
Phase 1 freezes the native DValue ABI that the future WASM core and units use
as their shared structured-value contract. The implementation is in
`src/lib/dvalue.{h,cpp}` and is available in the native runtime before any WASM
backend is enabled.
## Opaque handle
```c
typedef struct DValue uce_dvalue;
```
`uce_dvalue*` is a borrowed pointer owned by the active request/workspace. It
must not be freed by ABI callers and it must not be retained beyond that
workspace lifetime.
## Accessors
```c
uce_dvalue* uce_dv_root(void);
uce_dvalue* uce_dv_get(uce_dvalue* value, const char* key, size_t key_len);
uce_dvalue* uce_dv_find(uce_dvalue* value, const char* key, size_t key_len);
const char* uce_dv_value(uce_dvalue* value, size_t* len_out);
void uce_dv_set_value(uce_dvalue* value, const char* bytes, size_t len);
size_t uce_dv_count(uce_dvalue* value);
int uce_dv_is_list(uce_dvalue* value);
```
- `uce_dv_root()` returns the current native request's `context.call` root.
The WASM core will later map this to the decoded request context root.
- `uce_dv_get()` creates the child if absent. `uce_dv_find()` returns `NULL`
if absent.
- String inputs and outputs are length-delimited and binary-safe.
- `uce_dv_value()` returns a borrowed pointer valid until the next ABI value
call on the same thread.
- Bad `NULL` inputs return `NULL`, zero, or no-op rather than trapping.
## Iteration
```c
typedef struct uce_dv_iter { size_t position; size_t reserved[3]; } uce_dv_iter;
uce_dv_iter uce_dv_iter_begin(uce_dvalue* value);
int uce_dv_iter_next(uce_dvalue* value, uce_dv_iter* iter,
const char** key_out, size_t* key_len_out,
uce_dvalue** child_out);
```
Map iteration follows DValue's native order. List-shaped maps iterate in numeric
index order (`0`, `1`, ...), matching `DValue::each()`, `dv_values()`, and the
serializers. The reserved iterator fields are caller-opaque and must be
zero-preserved by callers that copy the iterator; they provide ABI headroom for
future non-linear keyed-map iteration without changing the struct size.
## UCEB1 wire format
UCEB1 is the membrane/cross-instance binary DValue encoding.
```
document := "UCEB" version node
version := 0x01
node := flags scalar children
flags := u8 bitset; bit0 = list-shaped map
scalar := varuint length, bytes
children := varuint count, count * (key, node)
key := varuint length, bytes
```
Varuint is unsigned LEB128. Strings are byte sequences; the codec does not
assume NUL termination and preserves embedded NUL bytes. The Phase 1 layout stores scalar values as their native string representation
plus child nodes and the list-shape flag. Floating-point values use
`max_digits10` precision so numeric scalars can round-trip through the string
form. Pointer/reference identity is intentionally not part of the wire contract;
pointer nodes encode as an empty scalar rather than leaking process addresses.
An empty non-list map has no wire distinction from an empty scalar in UCEB1 v1.
Documents that contain both scalar bytes and child nodes are reserved for future
use; the v1 decoder accepts the children and ignores the scalar.
## Codec APIs
C++/UCE-visible helpers:
```cpp
String ucb_encode(const DValue& value);
DValue ucb_decode(const String& encoded);
bool ucb_decode(const String& encoded, DValue& out, String* error_out = 0);
```
C ABI helpers:
```c
size_t uce_dv_encode(uce_dvalue* value, char* buf, size_t cap);
uce_dvalue* uce_dv_decode(const char* buf, size_t len);
const char* uce_dv_last_error(void);
```
`uce_dv_encode()` returns the required byte length even when `buf` is `NULL` or
`cap` is zero. `uce_dv_decode()` returns a thread-local decoded root, or `NULL`
with `uce_dv_last_error()` populated. The returned decoded root is valid until
the next `uce_dv_decode()` call on the same thread. Decoding rejects documents
deeper than 1024 nested nodes so malformed input cannot recurse without bound.
## Test coverage
`site/tests/core.uce` covers:
- UCEB1 round-trip for maps, nested values, lists, empty lists, and embedded NUL
scalar bytes.
- C ABI get/find/value/count/list/iteration/encode/decode behavior.
+305
View File
@@ -0,0 +1,305 @@
# UCE WASM Runtime Architecture
Status: current as of the W7e native-pipeline removal (June 2026). This document
describes the **runtime architecture as built** — the process topology, the
wasm membrane, the unified request dispatch, and the central WebSocket broker.
Native `.so` unit execution/dlopen fallback has been removed; the parser and
preprocessor remain only as the front-end that emits C++ for wasm side-module
compilation.
The guiding principle: **request code never shares an address space or an
allocator with the runtime.** Every unit runs as a WebAssembly module inside a
per-request workspace behind a narrow host membrane. Long-lived connection
state (WebSocket connections, listening HTTP sockets) lives in dedicated native
broker processes that own no unit code — they hold the connection and forward
the actual unit invocation to a clean-engine worker, exactly like a normal
page render.
---
## 1. Process topology
A single native binary (`uce_fastcgi`) forks into a small set of long-lived
roles. None of them is special-cased per request mode; the differences are
purely *which socket a process listens on* and *which function inside a unit
gets invoked*.
```
┌────────────────────────────┐
nginx ──FastCGI──► worker pool (N processes) │ /run/uce.sock
(port 80 etc.) │ uniform unit renderers │ (FastCGI + CLI)
└─────────────▲──────────────┘
│ forward render (FastCGI, uce.sock)
browser ──raw HTTP / WS──► ┌────────┴─────────┐
(HTTP_PORT 8080) │ WS broker │ owns HTTP_PORT + every
│ (1 process) │ WS connection; renders
└────────▲─────────┘ nothing itself
│ ws_* command flush
│ (FastCGI, ws-broker.sock)
on-demand serve_http ──► ┌────────┴─────────┐
(per bind addr) │ custom-server │ owns one serve_http
│ dispatcher(s) │ bind addr; forwards to pool
└──────────────────┘
parent process: spawns/respawns all of the above + the proactive compiler.
```
| Process | Owns | Renders units? | Source |
|---|---|---|---|
| **Parent** | nothing; supervises children | no | `main()`, `init_base_process()` |
| **Worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH` (`/run/uce.sock`) + `CLI_SOCKET_PATH` | **yes** — the only processes that run wasm | `listen_for_connections()` |
| **WS broker** (×1) | `HTTP_PORT` + every live WS connection + `WS_BROKER_SOCKET_PATH` | no — forwards to the pool | `run_ws_broker()` |
| **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` |
| **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` |
**only workers instantiate Wasmtime and run unit code.**
Every connection-owning process (broker, serve_http dispatcher) forwards the
request invocation back to a worker over `/run/uce.sock` using the minimal
FastCGI client in [`src/lib/fcgi_forward.h`](../src/lib/fcgi_forward.h). This is
forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across
`fork()`, and the brokers fork from the parent that already touched the
runtime. So the brokers hold the long-lived connection and units respond to
events the same way they respond to a page request — through a clean worker.
---
## 2. The membrane and the DValue ABI
Units are compiled to WebAssembly and linked against a host import surface (the
"membrane"). Host and guest exchange structured values as **UCEB** (a compact
binary `DValue` encoding — `ucb_encode`/`ucb_decode`); strings cross as
`std::string` on the native side throughout (no raw `char*` ownership across
the boundary).
- `DValue` is the universal value type: scalars plus an ordered child map
(`_map`, a `std::map<String,DValue>`). `operator[]` is non-const (creates);
`.key(k)` is the const probe returning `const DValue*`; `.each(fn)` iterates
children yielding `const DValue&`.
- The request context (`params`/`get`/`post`/`cookies`/`session`, the raw body
`in`, and—for WS—the connection context) is marshalled into a single `ctx`
DValue, UCEB-encoded, and handed to the workspace. The response (body,
headers, status, and any `meta` such as `ws_commands`) comes back the same
way.
See [`docs/wasm-phase1-dvalue-abi.md`](wasm-phase1-dvalue-abi.md) for the wire
format details.
---
## 3. Units, handlers, and export naming
A `.uce` unit compiles to a wasm module that exports one symbol per **handler**.
There is no per-mode machinery — a "CLI unit", "WebSocket unit", and "page" are
the same compiled artifact invoked at different exports. The dispatcher passes a
**handler string**; the host maps it to an export symbol:
```
__uce_<base>[_<sanitize(suffix)>]
```
`handler_export_symbol()` (`src/wasm/worker.cpp`) splits on the first `:`:
| Handler string | Export symbol |
|---|---|
| `render` | `__uce_render` |
| `cli` | `__uce_cli` |
| `websocket` | `__uce_websocket` |
| `once` | `__uce_once` (optional; absence is not an error) |
| `serve_http` | `__uce_serve_http` |
| `serve_http:named` | `__uce_serve_http_named` |
| `component:CARD` | `__uce_component_CARD` |
| `exists` | probe only — resolves the unit, loads nothing |
`sanitize_symbol_suffix()` keeps `[A-Za-z0-9_]` (mirrors `ascii_safe_name`).
`wasm_resolve_target(unit, handler)` (`src/wasm/core.cpp`) resolves the source
path and looks up the export's funcref slot; `exists` lets callers probe a unit
without instantiating it.
`wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
backend is initialized and the requested artifact/handler is currently
available. If an artifact is cold or stale, dispatch compiles it on demand via
`get_shared_unit()` and rechecks. There is no native unit-execution fallback.
---
## 4. The workspace runtime
Each request gets a fresh **workspace** — a per-request wasm instance tree with
the membrane wired in. `wasm_worker_serve(worker, ctx, entry_unit, handler)`
(`src/wasm/worker.cpp`) is the single entry point for *every* mode:
1. Birth a workspace (CoW-snapshot-based where available).
2. Resolve `entry_unit` + `handler` to an export; components referenced at
runtime are resolved on demand via the `uce_host_component_resolve` hostcall
(`component_resolve()``__uce_<...>` slot), loading dependency modules
lazily and recording resolve counts/timings.
3. Invoke the export. Unit code calls host functions (filesystem, sqlite, regex,
markdown, time, tasks, `ws_*`, …) across the membrane.
4. Collect the response (`WasmResponse`: body, headers, status, `meta`).
Because the workspace owns no process-lifetime state, a unit crash or trap is
contained: it fails the one request and the worker stays healthy. Workers
suspend the native SIGSEGV/SIGILL recovery handler around the wasm call so that
Wasmtime's own trap signals are not escalated into a native fatal signal (see
`serve_via_wasm` in `handle_complete`).
---
## 5. Request dispatch (`handle_complete`)
`handle_complete()` (`src/linux_fastcgi.cpp`) is the worker's single dispatch
point. It selects a handler string from request params and calls the same
`serve_via_wasm(entry_unit, handler)` lambda for all of them:
```
UCE_WS == "1" → serve_via_wasm(entry_unit, "websocket")
request.resources.is_cli → serve_via_wasm(entry_unit, "cli")
UCE_SERVE_HTTP == "1" → serve_via_wasm(entry_unit, "serve_http"[:fn])
otherwise (page) → serve_via_wasm(entry_unit, "render")
```
The `UCE_*` params are set by whichever broker forwarded the request:
- **Page render**: FastCGI nginx → `/run/uce.sock` directly; no `UCE_*` flags → `render`.
- **CLI**: the CLI socket sets `is_cli`.
- **serve_http**: the custom-server dispatcher sets `UCE_SERVE_HTTP=1` plus
`UCE_SERVE_HTTP_FUNCTION` and rewrites `SCRIPT_FILENAME` to the configured
unit (`custom_server_http_complete`).
- **WebSocket**: the WS broker sets `UCE_WS=1` and carries the connection
identity as `UCE_WS_*` params (see §6). `handle_complete` rebuilds
`request.resources.websocket_*` and `request.connection` from them before
invoking `__uce_websocket`.
If a unit still cannot be served after the on-demand wasm compile, the worker
returns a clean 500 with the wasm/compile error; it does not execute native unit
code.
---
## 6. The central WebSocket broker
**One process owns the HTTP port
and every WebSocket connection**, so any unit's `ws_*` call can reach any or all
connections, and a unit-code crash (which happens in a worker) never drops live
connections.
### 6.1 Inbound: a WS frame → a worker render (non-blocking)
`ws_broker_ws_message(request, message, opcode)` fires when a complete
(reassembled) WS message arrives on a connection. It does **not** block the
broker loop:
1. Build FastCGI params: `SCRIPT_FILENAME`, `REQUEST_METHOD=GET`, a
`REQUEST_URI` (required — `handle_request()` rejects requests without one
*before* `on_complete` runs), `UCE_WS=1`, and the connection context as
`UCE_WS_CONNECTION_ID / SCOPE / OPCODE / BINARY / CONNECTIONS / STATE`.
2. The message rides as `UCE_WS_MESSAGE` (base64) with an **empty STDIN body**
a non-empty STDIN makes the FastCGI transport flush a premature response
before `on_complete` ever runs.
3. Connect to `/run/uce.sock` (non-blocking) and queue the encoded request in
`ws_broker_outbound[fd]`.
`ws_broker_drain_outbound()` runs after every `process(50)` tick: it finishes
writing each queued request, then drains and discards the reply (the unit's
output comes back via the command socket, not this reply), closing the fd when
the worker closes its end.
### 6.2 Outbound: `ws_*` commands flushed back to the broker
Any unit code — not just WebSocket handlers — may call `ws_send` / `ws_send_to`
/ `ws_close`. In the workspace these **record dispatch commands** rather than
touching a socket (the workspace owns no connections); `wasm-core`'s `ws_*`
(`src/lib/sys.cpp`) append to `websocket_dispatch_commands`, and
`finish_response_meta` (`src/wasm/core.cpp`) emits them as `ws_commands` (plus
`ws_connection_state` if the handler mutated per-connection state).
`wasm_backend_serve` (`src/wasm/backend.cpp`) flushes that batch at workspace
teardown — in **any** scenario, not just WS handlers — to the broker's command
socket (`WS_BROKER_SOCKET_PATH`, `/run/uce/ws-broker.sock`) via
`fcgi_forward_request` with `UCE_WS_DISPATCH=1`. This is the only path WS data
takes out of a workspace.
`ws_broker_apply_commands()` decodes the batch and applies each command against
the full registry it owns: `broadcast` (by scope), `send_to` (by connection id),
`close`. If the batch carries `connection_state`, it persists that onto the
matching live connection's `websocket_state`.
### 6.3 Un-upgraded HTTP on the WS port
The WS port can also receive ordinary (non-Upgrade) HTTP requests.
`ws_broker_complete()` routes by param: `UCE_WS_DISPATCH=1` → apply commands;
otherwise → `forward_request_to_worker()` — the *same* shared facility the
serve_http dispatcher uses, so there is no duplicated request-forwarding code.
### 6.4 The broker loop
`run_ws_broker()` drops the worker listeners it inherited
(`close_inherited_server_sockets`), installs permissive `on_request`/`on_data`
handlers (it renders nothing, so it accepts every request straight through to
`on_complete`), wires `on_complete=ws_broker_complete` and
`on_websocket_message=ws_broker_ws_message`, listens on `HTTP_PORT` and the
command socket, and loops `process(50)` + `drain_outbound()`. The design is
**non-blocking outbound dispatch + async command-socket flush, all in the
broker's single epoll loop** — the broker never blocks on a worker.
The parent respawns the broker if it dies (`ws_broker_alive` / `ensure_ws_broker`
in `main()`).
---
## 7. The serve_http facility
`serve_http` units are reachable on their own bind address via a custom-server
dispatcher (`custom_server_http_dispatcher_loop`). The dispatcher owns the bind
socket and, on each request, sets `UCE_SERVE_HTTP=1` + the configured unit/
function and calls `forward_request_to_worker()`. The worker then runs
`serve_via_wasm(entry_unit, "serve_http"[:fn])`. This is the same
hold-connection-forward-render model as the WS broker, sharing
`forward_request_to_worker` and `fcgi_forward.h`.
---
## 8. Build / object layout
The native side is split into separately-compiled objects so editing the wasm
runtime does not recompile the whole TU (`scripts/build_linux.sh`):
| Object | Contents |
|---|---|
| `bin/sqlite3.o` | sqlite amalgamation (cached) |
| `bin/wasm.o` | `src/wasm/wasm_module.cpp` (backend.cpp + worker.cpp + wasmtime) |
| `bin/main.o` | `src/linux_fastcgi.cpp` (includes `lib/uce_lib.cpp`) |
Linked into one `-rdynamic` binary. ODR hazards across the objects are handled
deliberately: `context` and `my_pid`/`parent_pid` are `extern` (guarded for the
wasm core vs. unit builds), `operator new`/`delete` live in `types.cpp`, and
header free-functions are `inline`. The wasm backend exposes only declarations
(`src/wasm/backend.h`) to `main.o`.
---
## 9. Configuration keys
| Key | Default | Meaning |
|---|---|---|
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
| `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). |
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. |
| `HTTP_PORT` | `8080` | Raw HTTP + WebSocket port — owned by the WS broker. |
| `WS_BROKER_SOCKET_PATH` | `/run/uce/ws-broker.sock` | Broker command socket for `ws_*` flushes. |
| `WORKER_COUNT` | `4` | Number of uniform worker processes. |
---
## 10. Testing
- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages.
- **WebSocket end-to-end**: a headless client performs a raw WS handshake to
`:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
`SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
broker → worker → broker → client chain across process boundaries.
-197
View File
@@ -1,197 +0,0 @@
# Phase 0 findings — toolchain & runtime spike
- **Status: EXIT CRITERION PASSED** (2026-06-12, on k-uce)
- **Runtime selected: Wasmtime** (v45.0.1, C API). WAMR rejected — evidence below.
- Exceptions decision (§11.1, error codes / `-fno-exceptions`) **confirmed viable**:
both the stubs and two real generated units compile with `-fno-exceptions`,
no try/catch blocker anywhere.
The exit criterion ran end-to-end: a core stub (libc/libc++ statically linked,
owns memory/allocator) and a unit stub (PIC, `dylink.0`) were linked **at
runtime** by `loader.cpp` and produced:
```
hello from unit; unit-data-segment-ok; counter=7; mapsum=3; core-string+unit[cb:42][got-func-ok][fn:42]
core_counter (in linear memory): before=7 after=8
PHASE0 EXIT CRITERION: PASS
```
which validates, in one render call: unit data-segment relocation
(`__memory_base`), GOT.mem read **and write** of a core global, `std::string`
/`std::map` in unit code on the core's heap, a heap C++ object created in core
and mutated/read by the unit, function pointers crossing unit→core→unit
through the shared table, GOT.func resolution, and a `std::function` lambda
allocated in the unit and invoked by core. That is the §3.4 contract
("DValue inside the workspace: no serialization, ever") demonstrated at the
ABI level.
## Toolchain pins
| What | Version | Where on k-uce |
|---|---|---|
| wasi-sdk | 33 (clang 22.1.0-wasi-sdk) | `/opt/wasi-sdk` |
| target triple | `wasm32-wasip1` (`wasm32-wasi` is deprecated) | — |
| Wasmtime C API | v45.0.1 (prebuilt x86_64-linux release) | `/opt/wasmtime` |
| WAMR (rejected) | WAMR-2.4.4, built from source | `/opt/wamr` |
| cmake / ninja | 3.31.6 / 1.12.1 (apt) | — |
## Runtime selection: why not WAMR
WAMR was tried first per §9 ("preferred ... use Wasmtime only if blocked").
We are blocked, on the load-bearing requirement itself:
1. **WAMR's wasm-c-api ignores imported memories and tables.** At unit
instantiation it logs `"doesn't support import memories and tables for
now, ignore them"` (`wasm_c_api.c`) and gives the instance its own
memory/table — which silently destroys the shared-workspace model.
2. **Host-side `wasm_table_grow` / `wasm_memory_grow` are explicitly
unsupported** ("Only allow growing a table via the opcode table.grow").
3. Its build banner lists *Import/Export of Mutable Globals* as unsupported —
the dylink ABI imports `__stack_pointer` and every `GOT.*` entry as a
mutable global.
Its multi-module feature is name-based auto-resolution, not host-orchestrated
dylink (no host-computed `__memory_base`/`__table_base`, no GOT). Making WAMR
fit means implementing import binding through the c-api layer and runtime
internals — a runtime-development project, not a patch.
**Wasmtime v45.0.1 passed everything on the first run** through the standard
`wasm.h` C API: host-created funcref table shared by both instances, exported
memory imported by the unit, host-created (mutable) globals, cross-instance
export→import wiring. The remaining §9 criteria also favor it: AOT artifacts
(`.cwasm` precompilation) for the unit cache, epoch interruption for CPU
limits, and built-in copy-on-write memory-image instantiation for the §6 core
snapshot (machinery we'd have had to build ourselves on WAMR).
Trade-off accepted (was already in §10): Rust codebase, heavier to
vendor/patch. Pin the release artifact (lib + headers, checksummed) the way
sqlite is vendored; building from source stays possible but is not the
default path. The C API .so is ~27 MB.
## Module build recipe (what `build_modules.sh` settled on)
Core (non-PIC reactor, owns libc/libc++/allocator):
```
clang++ --target=wasm32-wasip1 -mexec-model=reactor -O1 -fno-exceptions \
core.cpp -o core.wasm \
-Wl,--export-all -Wl,--import-table \
-Wl,--export=__stack_pointer -Wl,--export=__heap_base \
-Wl,--undefined=_ZTVN10__cxxabiv117__class_type_infoE
```
Unit (PIC side module):
```
clang++ --target=wasm32-wasip1 -fPIC -fvisibility=default \
-fvisibility-inlines-hidden -O1 -fno-exceptions -c unit.cpp
wasm-ld -shared --experimental-pic --unresolved-symbols=import-dynamic \
--Bsymbolic unit.o -o unit.wasm --export=<entry>
```
Hard-won flag findings:
1. **`--unresolved-symbols=import-dynamic`** is required for the side-module
link; undefined symbols then become `env.*` function imports and `GOT.*`
globals exactly per the Emscripten dylink ABI.
2. **`-fvisibility-inlines-hidden` is mandatory.** Without it one libc++
vague-linkage lambda (`std::map` tree-emplace internals, missing libc++'s
usual hide-from-ABI attribute) is emitted as *both* an export and an
import of the unit — a self-import the loader cannot satisfy at
instantiation time without lazy-binding trampolines. `--Bsymbolic` alone
did **not** bind it.
3. **Core symbol closure**: `--export-all` only exports what got *linked*.
The unit needed `__cxxabiv1::__class_type_info`'s vtable (RTTI machinery
behind `std::function`), which the core never references — forced in with
`--undefined=`. The production core needs a closure strategy:
`--whole-archive` for libc/libc++/libc++abi, or a curated keep-list. The
loader also implements the complementary fallback (resolve `GOT.mem` of
weak data from the unit's *own* exports post-instantiation, patching the
provisional GOT global).
4. **`--import-table` on the core** + a **host-created table** is the right
shape (see loader notes); `--export-table`/`--growable-table` was the
first attempt and died on WAMR's host-grow limitation, but host-created
stays the better design under Wasmtime too: the loader picks table size
(core's declared minimum + headroom) before any instantiation.
5. `wasm32-wasi` triple is deprecated in wasi-sdk 33 → use `wasm32-wasip1`.
## Loader notes (`loader.cpp`, ~450 lines, standard wasm-c-api)
Sequence proven: instantiate core (45 WASI imports satisfied with named trap
stubs — none was ever called) → parse `dylink.0` (`mem_info`: memsize/align,
tablesize) → `__memory_base` = call core's exported `malloc``__table_base`
= bump pointer starting at core's table-import minimum → build the unit's
import vector (memory/table/`__stack_pointer` shared from core; `env.*`
functions from core exports; `GOT.mem.*` as host mutable i32 globals holding
addresses read from core's exported data-symbol globals) → instantiate →
patch deferred GOT entries → `__wasm_apply_data_relocs``__wasm_call_ctors`
→ call the entry export.
- **Erratum (found in Phase 3): self-resolved `GOT.mem` values must add
`__memory_base`.** A PIC module's exported data symbols are i32 globals
holding offsets *relative to its `__memory_base`*, not absolute addresses —
the linker adds the base when patching deferred GOT entries (there is no
`__wasm_apply_global_relocs` export to do it). Copying the export verbatim
reads/writes core memory at low addresses and renders silently wrong values;
the Phase 3 fixture's `self-got`/`callback` markers exist to catch exactly
this. `GOT.mem` entries resolved from the *core's* exports are absolute
already (the core is non-PIC) and need no adjustment.
- **GOT.func is resolved guest-side**: the core exports a helper returning
`(intptr_t)&func` — taking the address forces a link-time elem entry and a
wasm function pointer *is* its table index. No host-side funcref injection
is needed at all (it was WAMR-unsupported; under Wasmtime it would work but
the guest-side registry is simpler and runtime-agnostic). The production
core should carry a name→funcptr registry (dlsym-shaped) for its API
surface.
- **wasi-libc gotcha**: `_initialize` has a double-init guard ending in
`__builtin_trap()`. WAMR runs `_initialize` automatically at instantiation
(so calling it again traps "unreachable"); Wasmtime does not (so you must
call it). Cost one debugging round; recorded here so it never costs another.
- Export-name `wasm_name_t` may include the trailing NUL in `size` (WAMR
did); trim when indexing exports by name.
## Real generated units (delegated grind — full log in `realunit-report.md`)
`site/demo/collections.uce.cpp` and `hello.uce.cpp` (taken verbatim from the
live unit cache at `/tmp/uce/work/...`) both compile and link as PIC side
modules with `dylink.0`, **no allocator definitions**, with only shim-level
intervention. `collections.wasm`: 42 KB, 52 imports — including exactly the
predicted `GOT.mem.context` for the global `Request*`. Friction points that
become Phase 2 work items:
1. **`types.h` defines global `operator new/delete` in every unit** — must be
gated (`#ifdef`) out of side-module builds; allocator ownership belongs to
the core (§3.2: "the one fatal misconfiguration").
2. **`sys.h` includes `<signal.h>`** → wasi needs `-D_WASI_EMULATED_SIGNAL`
(+ `-lwasi-emulated-signal` in the core) or an `#ifdef __wasm__` carve-out;
signals/fork/exec/sockets in `sys.h` have no wasi equivalent and move
behind hostcalls anyway (§5.1).
3. **Generated units include `uce_lib.h` by absolute path** — the
preprocessor should emit a logical include so the wasm build can supply
its own include order.
4. **Header-inline connector wrappers (MySQL etc.) get pulled into every
unit** regardless of use; the §3.3 membrane split (thin wasm-side shim,
host-side implementation) resolves this and shrinks unit import lists.
5. The real-unit compile used the pre-`-fvisibility-inlines-hidden` flag set
(parallel work); the final unit flag set above should be used from
Phase 2 on.
## Implications for the next phases
- **Phase 1 (DValue C ABI, native)**: unaffected by any of this; proceed as
written.
- **Phase 2 (core module + membrane)**: add the closure strategy
(whole-archive), the GOT.func name→funcptr registry, the `types.h`
allocator gate, the signal emulation define, and the preprocessor include
change. Compile `uce_lib` with the core recipe above.
- **Phase 3 (loader)**: `loader.cpp` here is the skeleton — dylink parsing,
base allocation, GOT resolution, and init sequencing are all proven; what
remains is the registry/dispatch layer, ABI stamping, and multi-unit
placement.
- **Phase 4**: use Wasmtime's epoch interruption for CPU limits and its
memory-image/CoW instantiation for the core snapshot rather than building
either by hand.
## Artifacts (on k-uce, not in git)
- `/tmp/uce/wasm-phase0/{core.wasm,unit.wasm,loader}` — exit-criterion run
- `/tmp/uce/wasm-phase0/realunit/` — real-unit compiles + shim tree + inspector dumps
- `/opt/wasi-sdk`, `/opt/wasmtime`, `/opt/wamr` — toolchains/runtimes
+6 -5
View File
@@ -11,6 +11,10 @@ FCGI_PORT=9993
# Example: curl --unix-socket /run/uce/cli.sock http://localhost/ping
CLI_SOCKET_PATH=/run/uce/cli.sock
# Built-in HTTP/WebSocket listener used for WebSocket Upgrade requests.
# Keep this behind nginx/Apache on localhost or firewall it from public access.
HTTP_PORT=8080
# OPTIONAL PROACTIVE COMPILE ROOT
# Leave empty to scan SITE_DIRECTORY relative to the runtime root.
PRECOMPILE_FILES_IN=
@@ -21,13 +25,10 @@ SITE_DIRECTORY=site
# ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT
JIT_COMPILE_ON_REQUEST=1
# OPTIONAL W2 WEBASSEMBLY SIDE-MODULE COMPILATION BESIDE NATIVE .so UNITS
COMPILE_WASM_UNITS=0
# WASM SIDE-MODULE COMPILER USED FOR .uce UNITS
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
# DEFAULT PAGE RENDER BACKEND. W5 defaults to the WASM backend with explicit
# native fallbacks for host-owned surfaces that are not membrane APIs yet.
WASM_BACKEND_ENABLED=1
# WASM RUNTIME SETTINGS. Unit execution is always routed through wasm.
WASM_BACKEND_VERBOSE=0
WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm
WASM_MEMORY_LIMIT_BYTES=536870912
+94
View File
@@ -0,0 +1,94 @@
#!/usr/bin/env python3
"""Guard the hand-maintained unit-facing API coverage manifest.
This intentionally avoids network/external services. It checks that public API
names we expose to wasm units are either mentioned by a site test or explicitly
marked internal/integration-only, and that active docs exist for doc-required
APIs. The manifest is deliberately source-controlled so a new public function
requires an explicit coverage decision.
"""
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
TEST_DIR = ROOT / "site" / "tests"
DOC_DIR = ROOT / "site" / "doc" / "pages"
# name, needs_doc, status. status: public | internal | integration
PUBLIC_APIS = [
("shell_exec", True, "public"), ("shell_escape", True, "public"),
("basename", True, "public"), ("dirname", True, "public"), ("path_join", True, "public"),
("path_real", True, "public"), ("path_is_within", True, "public"),
("file_open_locked", True, "public"), ("file_close_locked", True, "public"),
("file_release_process_locks", True, "public"), ("file_get_contents_locked_fd", True, "public"),
("file_put_contents_locked_fd", True, "public"), ("file_get_contents", True, "public"),
("file_put_contents", True, "public"), ("file_append_contents", False, "public"),
("cwd_get", True, "public"), ("cwd_set", True, "public"), ("process_start_directory", True, "public"),
("file_mtime", True, "public"), ("file_unlink", True, "public"), ("expand_path", True, "public"),
("ls", True, "public"), ("config_map_u64", True, "public"), ("config_map_f64", True, "public"),
("config_bool_value", True, "public"), ("config_map_bool", True, "public"),
("config_u64", True, "public"), ("config_f64", True, "public"), ("config_bool", True, "public"),
("request_perf", True, "public"), ("time_format_local", True, "public"),
("time_format_relative", True, "public"), ("time_parse", True, "public"),
("backtrace_frames_string", False, "public"), ("capture_backtrace_string", False, "public"),
("signal_name", False, "public"), ("memcache_escape_key", True, "public"),
("memcache_escape_keys", True, "public"), ("memcache_command", True, "public"),
("memcache_get_multiple", True, "public"), ("runtime_safe_key", True, "public"),
("float_val", True, "public"), ("nibble", True, "public"), ("json_consume_space", False, "public"),
("array_merge", True, "public"), ("safe_name", True, "public"), ("ascii_safe_name", True, "public"),
("to_json", False, "public"), ("remove", False, "public"), ("clear", False, "public"),
("gen_sha1", True, "public"), ("gen_noise32", True, "public"), ("gen_noise64", True, "public"),
("gen_noise01", True, "public"), ("gen_int", True, "public"), ("gen_float", True, "public"),
("draw_int", True, "public"), ("draw_float", True, "public"),
("encode_query", True, "public"), ("request_script_url", True, "public"),
("request_base_url", True, "public"), ("request_route_from_raw_path", True, "public"),
("cli_arg", True, "public"), ("unit_compile", True, "public"),
("cleanup_sqlite_connections", False, "internal"), ("cleanup_mysql_connections", False, "internal"),
("mysql_connect", True, "integration"), ("mysql_query", True, "integration"),
]
REMOVED_APIS = ["unit_load", "concat"]
def all_test_text() -> str:
parts = []
for path in TEST_DIR.glob("*.uce"):
parts.append(path.read_text(errors="ignore"))
return "\n".join(parts)
def doc_exists(name: str) -> bool:
path = DOC_DIR / f"{name}.txt"
return path.exists() and "Removed" not in path.read_text(errors="ignore")[:200]
def has_call(text: str, name: str) -> bool:
return f"{name}(" in text or f".{name}(" in text or f'"{name}"' in text
def main() -> int:
tests = all_test_text()
errors = []
for name, needs_doc, status in PUBLIC_APIS:
if status == "public" and not has_call(tests, name):
errors.append(f"missing test coverage: {name}")
if needs_doc and status in {"public", "integration"} and not doc_exists(name):
errors.append(f"missing active doc page: {name}")
compiler_h = (ROOT / "src" / "lib" / "compiler.h").read_text(errors="ignore")
if "#ifndef __UCE_WASM_UNIT__\nSharedUnit* unit_load" not in compiler_h:
errors.append("unit_load is not guarded out of wasm-unit exposure")
for name in REMOVED_APIS:
page = DOC_DIR / f"{name}.txt"
if name == "concat" and page.exists() and "Removed" not in page.read_text(errors="ignore")[:300]:
errors.append("concat doc is not tombstoned")
if name == "unit_load" and page.exists() and "native-only" not in page.read_text(errors="ignore"):
errors.append("unit_load doc is not native-only/tombstoned")
if errors:
print("API coverage manifest FAILED")
for error in errors:
print("- " + error)
return 1
print(f"API coverage manifest ok: {len(PUBLIC_APIS)} entries checked")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+1 -1
View File
@@ -1,5 +1,5 @@
#!/bin/bash
# Build the production W1 UCE WASM core from the real runtime carve-out.
# Build the production W1 UCE WASM core from the production runtime carve-out.
# Run on k-uce from any working directory.
set -euo pipefail
cd "$(dirname "$0")/.."
+12 -4
View File
@@ -9,11 +9,11 @@ GF="uce_fastcgi"
mkdir bin > /dev/null 2>&1
mkdir bin/tmp > /dev/null 2>&1
mkdir bin/assets > /dev/null 2>&1
mkdir bin/wasm > /dev/null 2>&1
mkdir work > /dev/null 2>&1
COMPILER="clang++"
# -rdynamic is a link-time flag (exports the binary's symbols so dlopen'd .uce
# units resolve the runtime against it); the -c compiles below do not need it.
# -rdynamic is a link-time flag; the -c compiles below do not need it.
FLAGS="-g -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math"
# Wasmtime C++ API — needed only by the wasm backend object (src/wasm).
@@ -30,8 +30,8 @@ SRCFLAGS="-D EXEC_NAME=\"$GF\" -D PLATFORM_NAME=\"linux\""
# bin/sqlite3.o vendored SQLite amalgamation (depends only on its own source)
# bin/wasm.o wasm backend + worker + wasmtime.hh (src/wasm)
# bin/main.o linux_fastcgi.cpp + the uce_lib core amalgamation
# All link into the single -rdynamic binary, so the dlopen unit model is
# unchanged. Delete bin/*.o to force a clean rebuild.
# All link into the single -rdynamic binary. Delete bin/*.o to force a clean
# rebuild.
# Rebuild $1 if it is missing or anything under the remaining find-args is newer.
needs_rebuild() {
@@ -41,6 +41,14 @@ needs_rebuild() {
return 1
}
# core.wasm: guest runtime loaded by the native wasm backend.
if needs_rebuild bin/wasm/core.wasm src/wasm/core.cpp src/lib src/wasm/core_hostcalls.syms src/wasm/core_libc_exports.syms scripts/build_core_wasm.sh; then
echo "Compiling wasm core..."
bash scripts/build_core_wasm.sh || exit 1
else
echo "Reusing bin/wasm/core.wasm"
fi
# SQLite: vendored C, depends only on its own source (not our headers).
if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then
echo "Compiling SQLite..."
@@ -140,6 +140,14 @@ def dylink_has_valid_mem_info(payload: bytes) -> bool:
def defined_symbols(path: Path, llvm_nm: str) -> list[str]:
proc = subprocess.run([llvm_nm, "--defined-only", str(path)], text=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if proc.returncode != 0:
# llvm-nm (wasi-sdk) can SIGSEGV on degenerate-but-valid modules — e.g. a
# unit with no exported handlers (`empty.uce`). A toolchain crash is not
# evidence of a forbidden allocator, so skip this defense-in-depth scan
# rather than fail the unit; forbidden allocator *exports* are still
# rejected by the export-section check above.
crashed = proc.returncode < 0 or "Stack dump:" in proc.stderr or "PLEASE submit a bug report" in proc.stderr
if crashed:
return []
raise RuntimeError(proc.stderr.strip() or "llvm-nm failed")
symbols = []
for line in proc.stdout.splitlines():
-47
View File
@@ -1,47 +0,0 @@
#!/bin/bash
cd "$(dirname "$0")"
cd ..
SRC_DIR="$1"
DEST_DIR="$2"
SRC_FN="$3"
PP_FN="$4"
SO_FN="$5"
LINK_OBJECTS="$6"
#echo "Source Dir: $SRC_DIR"
#echo "Dest Dir: $DEST_DIR"
#echo "Source File: $SRC_FN"
#echo "Preprocessed File: $PP_FN"
#echo "Dest File: $SO_FN"
mkdir -p "$DEST_DIR" > /dev/null 2>&1
export CPLUS_INCLUDE_PATH="${CPLUS_INCLUDE_PATH:+${CPLUS_INCLUDE_PATH}:}$SRC_DIR"
BUILDMODE="debug"
OPT_FLAG="O0"
COMPILER="clang++"
#COMPILER="g++"
FLAGS="-shared -g -rdynamic -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math -fPIC -Isrc/lib"
LIBS="-ldl -lm -lpthread"
SRCFLAGS="-D PLATFORM_NAME=\"linux\""
# echo "Compliling executable..."
$COMPILER "$DEST_DIR/$PP_FN" $SRCFLAGS $FLAGS $LIBS -o "$DEST_DIR/$SO_FN"
# separate .o file
#$COMPILER -c "$DEST_DIR/$PP_FN" $SRCFLAGS $FLAGS $LIBS -o "$DEST_DIR/$PP_FN.o"
#$COMPILER "$DEST_DIR/$PP_FN.o" "$LINK_OBJECTS" $SRCFLAGS $FLAGS $LIBS -o "$DEST_DIR/$SO_FN"
if [ $? -eq 0 ]
then
# ls -lh "$DEST_DIR"
exit 0
else
exit 1
fi
+5 -5
View File
@@ -21,9 +21,9 @@ COMMON_FLAGS=(
--target=wasm32-wasip1
-fPIC -fvisibility=default -fvisibility-inlines-hidden
-O1 -g -std=c++20
# -w as in scripts/compile: warnings are not failures. The server captures
# this script's output and treats any non-empty result as a compile failure
# (then drops the .wasm), so a successful build must be silent.
# The server captures this script's output and treats any non-empty result as
# a compile failure (then drops the .wasm), so a successful build must be
# silent; warnings are intentionally suppressed.
-w
# must match the core build ABI: units with RTTI/EH enabled import
# typeinfo/unwind symbols the -fno-rtti/-fno-exceptions core cannot provide
@@ -50,7 +50,7 @@ build_pch_if_needed() {
return 0
fi
mkdir -p "$PCH_DIR"
if [ -s "$PCH_FN" ]; then
if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then
return 0
fi
"$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \
@@ -92,6 +92,6 @@ fi
"$SDK/bin/llvm-objcopy" --add-section=uce.abi="$ABI_TMP" "$DEST_DIR/$WASM_FN"
python3 scripts/wasm/check_unit_wasm.py "$DEST_DIR/$WASM_FN" --abi-version "$ABI_VERSION" --llvm-nm "$SDK/bin/llvm-nm"
python3 scripts/check_unit_wasm.py "$DEST_DIR/$WASM_FN" --abi-version "$ABI_VERSION" --llvm-nm "$SDK/bin/llvm-nm"
rm -f "$OBJ_FN" "$ABI_TMP"
+1 -1
View File
@@ -3,7 +3,7 @@ Version: @VERSION@
Section: web
Priority: optional
Architecture: @ARCH@
Depends: bash, clang, libmariadb-dev | default-libmysqlclient-dev, systemd
Depends: bash, python3, curl, systemd, libpcre2-8-0, zlib1g, libssl3, libstdc++6, libgcc-s1, libmariadb3 | libmysqlclient21 | libmysqlclient24
Maintainer: Udo Schroeter <udo@openfu.com>
Installed-Size: @INSTALLED_SIZE@
Description: UCE FastCGI runtime and live-compiling C++ web environment
+1
View File
@@ -5,6 +5,7 @@ if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true
if [ "$1" = "configure" ]; then
systemctl enable uce.service >/dev/null 2>&1 || true
systemctl restart uce.service >/dev/null 2>&1 || true
fi
fi
-1
View File
@@ -11,7 +11,6 @@ StateDirectory=uce
CacheDirectory=uce
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock
ExecStartPre=/usr/bin/bash /usr/lib/uce/scripts/build_linux.sh
ExecStart=/usr/lib/uce/bin/uce_fastcgi.linux.bin
ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock
Restart=always
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
set -euo pipefail
repo_root=$(git rev-parse --show-toplevel)
cd "$repo_root"
out_dir="${1:-changelog}"
mkdir -p "$out_dir"
rm -f "$out_dir"/*.log
while IFS= read -r commit; do
commit_date=$(git show -s --format=%cs "$commit")
out_file="$out_dir/$commit_date.log"
{
printf 'commit %s\n' "$commit"
git show -s --format=%B "$commit"
printf '\n'
} >> "$out_file"
done < <(git rev-list --reverse HEAD)
printf 'Wrote changelog files to %s\n' "$out_dir"
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
set -euo pipefail
# Install the pinned WASI SDK used by UCE's request-time wasm compiler.
# This is a runtime dependency: UCE compiles .uce units on demand and during
# proactive startup scans, so every deployment host needs the same toolchain.
WASI_SDK_VERSION="${WASI_SDK_VERSION:-33.0}"
WASI_SDK_RELEASE_TAG="${WASI_SDK_RELEASE_TAG:-wasi-sdk-33}"
WASI_SDK_ARCHIVE="${WASI_SDK_ARCHIVE:-wasi-sdk-33.0-x86_64-linux.tar.gz}"
WASI_SDK_SHA256="${WASI_SDK_SHA256:-0ba8b5bfaeb2adf3f29bab5841d76cf5318ab8e1642ea195f88baba1abd47bce}"
WASI_SDK_URL="${WASI_SDK_URL:-https://github.com/WebAssembly/wasi-sdk/releases/download/${WASI_SDK_RELEASE_TAG}/${WASI_SDK_ARCHIVE}}"
INSTALL_BASE="${INSTALL_BASE:-/opt}"
INSTALL_DIR="${WASI_SDK_INSTALL_DIR:-${INSTALL_BASE}/wasi-sdk-${WASI_SDK_VERSION}-x86_64-linux}"
SYMLINK_PATH="${WASI_SDK_SYMLINK:-${INSTALL_BASE}/wasi-sdk}"
CACHE_DIR="${WASI_SDK_CACHE_DIR:-/tmp/uce-deps}"
usage() {
cat <<EOF
Usage: scripts/install_wasi_sdk.sh [--check-only]
Installs pinned WASI SDK ${WASI_SDK_VERSION} for UCE request-time unit compilation.
Environment overrides:
WASI_SDK_VERSION ${WASI_SDK_VERSION}
WASI_SDK_RELEASE_TAG ${WASI_SDK_RELEASE_TAG}
WASI_SDK_ARCHIVE ${WASI_SDK_ARCHIVE}
WASI_SDK_SHA256 ${WASI_SDK_SHA256}
WASI_SDK_URL ${WASI_SDK_URL}
WASI_SDK_INSTALL_DIR ${INSTALL_DIR}
WASI_SDK_SYMLINK ${SYMLINK_PATH}
WASI_SDK_CACHE_DIR ${CACHE_DIR}
EOF
}
check_only=0
if [[ "${1:-}" == "--help" || "${1:-}" == "-h" ]]; then
usage
exit 0
elif [[ "${1:-}" == "--check-only" ]]; then
check_only=1
elif [[ $# -gt 0 ]]; then
usage >&2
exit 2
fi
require_command() {
if ! command -v "$1" >/dev/null 2>&1; then
echo "Required command not found: $1" >&2
exit 1
fi
}
verify_tree() {
local root="$1"
for tool in clang++ wasm-ld llvm-objcopy llvm-nm; do
if [[ ! -x "$root/bin/$tool" ]]; then
echo "Missing WASI SDK tool: $root/bin/$tool" >&2
return 1
fi
done
"$root/bin/clang++" --version | head -n 1
}
if [[ $check_only -eq 1 ]]; then
verify_tree "${WASI_SDK:-$SYMLINK_PATH}"
exit 0
fi
require_command curl
require_command sha256sum
require_command tar
require_command mkdir
require_command ln
mkdir -p "$CACHE_DIR" "$INSTALL_BASE"
archive_path="$CACHE_DIR/$WASI_SDK_ARCHIVE"
if [[ ! -f "$archive_path" ]]; then
echo "Downloading $WASI_SDK_URL"
curl -fL --proto '=https' --tlsv1.2 -o "$archive_path.tmp" "$WASI_SDK_URL"
mv "$archive_path.tmp" "$archive_path"
fi
printf '%s %s\n' "$WASI_SDK_SHA256" "$archive_path" | sha256sum -c -
rm -rf -- "$INSTALL_DIR.tmp"
mkdir -p "$INSTALL_DIR.tmp"
tar -xf "$archive_path" -C "$INSTALL_DIR.tmp" --strip-components=1
verify_tree "$INSTALL_DIR.tmp"
rm -rf -- "$INSTALL_DIR"
mv "$INSTALL_DIR.tmp" "$INSTALL_DIR"
ln -sfn "$INSTALL_DIR" "$SYMLINK_PATH"
echo "Installed WASI SDK at $INSTALL_DIR"
echo "Updated symlink $SYMLINK_PATH -> $INSTALL_DIR"
verify_tree "$SYMLINK_PATH"
+97 -13
View File
@@ -5,16 +5,21 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
DEB_ASSET_DIR="$SCRIPT_DIR/deb"
PACKAGE_NAME="uce"
REVISION="${UCE_DEB_REVISION:-1}"
REVISION="${UCE_DEB_REVISION:-}"
usage() {
cat <<'EOF'
Usage:
scripts/make_deb.sh VERSION
scripts/make_deb.sh [VERSION]
When VERSION is omitted, scripts/make_deb.sh reads VERSION from version.txt.
Environment:
UCE_DEB_REVISION Debian package revision suffix (default: 1)
UCE_DEB_ARCH Override package architecture
UCE_DEB_REVISION Optional Debian package revision suffix
UCE_DEB_ARCH Override package architecture
UCE_DEB_WEBROOT Public web root staged into the package (default: /var/www/html)
UCE_DEB_BUNDLE_WASI_SDK Bundle pinned /opt/wasi-sdk into the package (default: 1)
UCE_DEB_BUNDLE_WASMTIME Bundle /opt/wasmtime into the package (default: 1)
EOF
}
@@ -39,7 +44,7 @@ resolve_arch() {
validate_version() {
local version="$1"
if [[ ! "$version" =~ ^[0-9][A-Za-z0-9.+:~]*$ ]]; then
if [[ ! "$version" =~ ^[0-9][A-Za-z0-9.+:~-]*$ ]]; then
echo "Invalid Debian version string: $version" >&2
exit 1
fi
@@ -47,12 +52,37 @@ validate_version() {
copy_payload() {
local destination="$1"
local webroot="$2"
local stage_dir="$3"
local path
for path in LICENSE README.md codesearch bin scripts site src; do
for path in LICENSE README.md codesearch bin scripts src docs; do
cp -a "$REPO_ROOT/$path" "$destination/"
done
mkdir -p "$destination/etc"
mkdir -p "$destination/etc" "$stage_dir$webroot"
cp -a "$REPO_ROOT/etc/uce" "$destination/etc/"
cp -a "$REPO_ROOT/site/." "$stage_dir$webroot/"
}
write_packaged_settings() {
local output_file="$1"
local webroot="$2"
python3 - "$REPO_ROOT/etc/uce/settings.cfg" "$output_file" "$webroot" <<'PY'
from pathlib import Path
import sys
src, dst, webroot = sys.argv[1:4]
s = Path(src).read_text()
replacements = {
"SITE_DIRECTORY=site": f"SITE_DIRECTORY={webroot}",
"WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm": "WASM_CORE_PATH=/usr/lib/uce/bin/wasm/core.wasm",
"HTTP_DOCUMENT_ROOT=": f"HTTP_DOCUMENT_ROOT={webroot}",
}
for old, new in replacements.items():
if old in s:
s = s.replace(old, new)
if "WASM_COMPILE_SCRIPT=" not in s:
s += "\nWASM_COMPILE_SCRIPT=scripts/compile_wasm_unit\n"
Path(dst).write_text(s)
PY
}
write_control_file() {
@@ -69,20 +99,68 @@ write_control_file() {
"$DEB_ASSET_DIR/control.in" > "$output_file"
}
bundle_wasi_sdk() {
local stage_dir="$1"
local wasi_root="${WASI_SDK:-/opt/wasi-sdk}"
if [[ "${UCE_DEB_BUNDLE_WASI_SDK:-1}" != "1" ]]; then
return
fi
if [[ ! -x "$wasi_root/bin/clang++" || ! -x "$wasi_root/bin/wasm-ld" || ! -x "$wasi_root/bin/llvm-nm" ]]; then
echo "UCE_DEB_BUNDLE_WASI_SDK=1 but WASI_SDK does not point at a complete SDK: $wasi_root" >&2
exit 1
fi
local resolved
resolved="$(readlink -f "$wasi_root")"
local base
base="$(basename "$resolved")"
mkdir -p "$stage_dir/opt"
cp -a "$resolved" "$stage_dir/opt/$base"
ln -sfn "$base" "$stage_dir/opt/wasi-sdk"
}
bundle_wasmtime() {
local stage_dir="$1"
local wasmtime_root="${WASMTIME_HOME:-/opt/wasmtime}"
if [[ "${UCE_DEB_BUNDLE_WASMTIME:-1}" != "1" ]]; then
return
fi
if [[ ! -f "$wasmtime_root/include/wasmtime.hh" || ! -f "$wasmtime_root/lib/libwasmtime.so" ]]; then
echo "UCE_DEB_BUNDLE_WASMTIME=1 but WASMTIME_HOME does not point at a complete C API tree: $wasmtime_root" >&2
exit 1
fi
local resolved
resolved="$(readlink -f "$wasmtime_root")"
local base
base="$(basename "$resolved")"
mkdir -p "$stage_dir/opt"
cp -a "$resolved" "$stage_dir/opt/$base"
ln -sfn "$base" "$stage_dir/opt/wasmtime"
}
write_md5sums() {
local stage_dir="$1"
(
cd "$stage_dir"
find usr etc lib -type f -print0 | sort -z | xargs -0 md5sum > DEBIAN/md5sums
find usr etc lib opt -type f -print0 2>/dev/null | sort -z | xargs -0 --no-run-if-empty md5sum > DEBIAN/md5sums
)
}
if [[ $# -ne 1 ]]; then
if [[ $# -gt 1 ]]; then
usage >&2
exit 1
fi
VERSION="$1"
if [[ $# -eq 1 ]]; then
VERSION="$1"
else
if [[ ! -r "$REPO_ROOT/version.txt" ]]; then
echo "Missing version.txt and no VERSION argument supplied" >&2
exit 1
fi
# shellcheck disable=SC1091
. "$REPO_ROOT/version.txt"
VERSION="${VERSION:-}"
fi
validate_version "$VERSION"
require_command bash
@@ -101,11 +179,15 @@ require_command cp
require_command sort
ARCH="$(resolve_arch)"
PACKAGE_VERSION="${VERSION}-${REVISION}"
PACKAGE_VERSION="$VERSION"
if [[ -n "$REVISION" ]]; then
PACKAGE_VERSION="${PACKAGE_VERSION}-${REVISION}"
fi
PACKAGE_BASENAME="${PACKAGE_NAME}_${PACKAGE_VERSION}_${ARCH}"
STAGE_DIR="$REPO_ROOT/pkg/$PACKAGE_BASENAME"
DEBIAN_DIR="$STAGE_DIR/DEBIAN"
INSTALL_ROOT="$STAGE_DIR/usr/lib/uce"
WEBROOT="${UCE_DEB_WEBROOT:-/var/www/html}"
DIST_DIR="$REPO_ROOT/dist"
OUTPUT_DEB="$DIST_DIR/$PACKAGE_BASENAME.deb"
@@ -117,9 +199,11 @@ bash "$REPO_ROOT/scripts/build_linux.sh"
rm -rf -- "$STAGE_DIR"
mkdir -p "$DEBIAN_DIR" "$INSTALL_ROOT" "$STAGE_DIR/etc/uce" "$STAGE_DIR/lib/systemd/system" "$DIST_DIR"
copy_payload "$INSTALL_ROOT"
copy_payload "$INSTALL_ROOT" "$WEBROOT" "$STAGE_DIR"
bundle_wasi_sdk "$STAGE_DIR"
bundle_wasmtime "$STAGE_DIR"
install -m 0644 "$REPO_ROOT/etc/uce/settings.cfg" "$STAGE_DIR/etc/uce/settings.cfg"
write_packaged_settings "$STAGE_DIR/etc/uce/settings.cfg" "$WEBROOT"
install -m 0644 "$DEB_ASSET_DIR/uce.service" "$STAGE_DIR/lib/systemd/system/uce.service"
install -m 0644 "$DEB_ASSET_DIR/conffiles" "$DEBIAN_DIR/conffiles"
install -m 0755 "$DEB_ASSET_DIR/postinst" "$DEBIAN_DIR/postinst"
+260
View File
@@ -0,0 +1,260 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
PACKAGE_NAME="uce"
RELEASE="${UCE_RPM_RELEASE:-1}"
WEBROOT="${UCE_RPM_WEBROOT:-/var/www/html}"
usage() {
cat <<'EOF'
Usage:
scripts/make_rpm.sh [VERSION]
When VERSION is omitted, scripts/make_rpm.sh reads VERSION, MAJOR, and RELEASE from version.txt.
Environment:
UCE_RPM_RELEASE Override RPM release suffix (default: RELEASE from version.txt)
UCE_RPM_ARCH Override RPM architecture
UCE_RPM_WEBROOT Public web root staged into the package (default: /var/www/html)
UCE_RPM_BUNDLE_WASI_SDK Bundle pinned /opt/wasi-sdk into the package (default: 1)
UCE_RPM_BUNDLE_WASMTIME Bundle /opt/wasmtime into the package (default: 1)
EOF
}
require_command() {
if ! command -v "$1" >/dev/null 2>&1; then
echo "Required command not found: $1" >&2
exit 1
fi
}
validate_version() {
local version="$1"
if [[ ! "$version" =~ ^[0-9][A-Za-z0-9._+~]*$ ]]; then
echo "Invalid RPM version string: $version" >&2
exit 1
fi
}
resolve_arch() {
if [[ -n "${UCE_RPM_ARCH:-}" ]]; then
printf '%s\n' "$UCE_RPM_ARCH"
return
fi
uname -m
}
copy_payload() {
local destination="$1"
local webroot="$2"
local stage_dir="$3"
local path
for path in LICENSE README.md codesearch bin scripts src docs; do
cp -a "$REPO_ROOT/$path" "$destination/"
done
mkdir -p "$destination/etc" "$stage_dir$webroot"
cp -a "$REPO_ROOT/etc/uce" "$destination/etc/"
cp -a "$REPO_ROOT/site/." "$stage_dir$webroot/"
}
write_packaged_settings() {
local output_file="$1"
local webroot="$2"
python3 - "$REPO_ROOT/etc/uce/settings.cfg" "$output_file" "$webroot" <<'PY'
from pathlib import Path
import sys
src, dst, webroot = sys.argv[1:4]
s = Path(src).read_text()
replacements = {
"SITE_DIRECTORY=site": f"SITE_DIRECTORY={webroot}",
"WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm": "WASM_CORE_PATH=/usr/lib/uce/bin/wasm/core.wasm",
"HTTP_DOCUMENT_ROOT=": f"HTTP_DOCUMENT_ROOT={webroot}",
}
for old, new in replacements.items():
if old in s:
s = s.replace(old, new)
if "WASM_COMPILE_SCRIPT=" not in s:
s += "\nWASM_COMPILE_SCRIPT=scripts/compile_wasm_unit\n"
Path(dst).write_text(s)
PY
}
bundle_wasi_sdk() {
local stage_dir="$1"
local wasi_root="${WASI_SDK:-/opt/wasi-sdk}"
if [[ "${UCE_RPM_BUNDLE_WASI_SDK:-1}" != "1" ]]; then
return
fi
if [[ ! -x "$wasi_root/bin/clang++" || ! -x "$wasi_root/bin/wasm-ld" || ! -x "$wasi_root/bin/llvm-nm" ]]; then
echo "UCE_RPM_BUNDLE_WASI_SDK=1 but WASI_SDK does not point at a complete SDK: $wasi_root" >&2
exit 1
fi
local resolved base
resolved="$(readlink -f "$wasi_root")"
base="$(basename "$resolved")"
mkdir -p "$stage_dir/opt"
cp -a "$resolved" "$stage_dir/opt/$base"
ln -sfn "$base" "$stage_dir/opt/wasi-sdk"
}
bundle_wasmtime() {
local stage_dir="$1"
local wasmtime_root="${WASMTIME_HOME:-/opt/wasmtime}"
if [[ "${UCE_RPM_BUNDLE_WASMTIME:-1}" != "1" ]]; then
return
fi
if [[ ! -f "$wasmtime_root/include/wasmtime.hh" || ! -f "$wasmtime_root/lib/libwasmtime.so" ]]; then
echo "UCE_RPM_BUNDLE_WASMTIME=1 but WASMTIME_HOME does not point at a complete C API tree: $wasmtime_root" >&2
exit 1
fi
local resolved base
resolved="$(readlink -f "$wasmtime_root")"
base="$(basename "$resolved")"
mkdir -p "$stage_dir/opt"
cp -a "$resolved" "$stage_dir/opt/$base"
ln -sfn "$base" "$stage_dir/opt/wasmtime"
}
if [[ $# -gt 1 ]]; then
usage >&2
exit 1
fi
FULL_VERSION=""
if [[ $# -eq 1 ]]; then
FULL_VERSION="$1"
RPM_VERSION="${FULL_VERSION%%-*}"
RPM_RELEASE_PART="${FULL_VERSION#*-}"
if [[ "$RPM_RELEASE_PART" == "$FULL_VERSION" ]]; then
RPM_RELEASE_PART="$RELEASE"
fi
else
if [[ ! -r "$REPO_ROOT/version.txt" ]]; then
echo "Missing version.txt and no VERSION argument supplied" >&2
exit 1
fi
# shellcheck disable=SC1091
. "$REPO_ROOT/version.txt"
FULL_VERSION="${VERSION:-}"
RPM_VERSION="${MAJOR:-${FULL_VERSION%%-*}}"
RPM_RELEASE_PART="${RELEASE:-${FULL_VERSION#*-}}"
fi
if [[ -z "$FULL_VERSION" || -z "$RPM_VERSION" || -z "$RPM_RELEASE_PART" ]]; then
echo "Could not derive RPM version from VERSION/MAJOR/RELEASE" >&2
exit 1
fi
RPM_RELEASE_PART="${RPM_RELEASE_PART//-/_}"
VERSION="$RPM_VERSION"
RELEASE="${UCE_RPM_RELEASE:-$RPM_RELEASE_PART}"
validate_version "$VERSION"
require_command bash
require_command rpmbuild
require_command find
require_command cp
require_command tar
require_command sed
require_command python3
require_command readlink
ARCH="$(resolve_arch)"
PACKAGE_VERSION="$FULL_VERSION"
BUILD_ROOT="$REPO_ROOT/pkg/rpm-build"
STAGE_DIR="$BUILD_ROOT/stage"
RPMBUILD_DIR="$BUILD_ROOT/rpmbuild"
INSTALL_ROOT="$STAGE_DIR/usr/lib/uce"
SPEC_FILE="$RPMBUILD_DIR/SPECS/$PACKAGE_NAME.spec"
DIST_DIR="$REPO_ROOT/dist"
bash "$REPO_ROOT/scripts/build_linux.sh"
rm -rf -- "$BUILD_ROOT"
mkdir -p "$INSTALL_ROOT" "$STAGE_DIR/etc/uce" "$STAGE_DIR/usr/lib/systemd/system" "$STAGE_DIR/var/cache/uce" "$STAGE_DIR/var/lib/uce" "$RPMBUILD_DIR"/{BUILD,RPMS,SOURCES,SPECS,SRPMS} "$DIST_DIR"
copy_payload "$INSTALL_ROOT" "$WEBROOT" "$STAGE_DIR"
bundle_wasi_sdk "$STAGE_DIR"
bundle_wasmtime "$STAGE_DIR"
write_packaged_settings "$STAGE_DIR/etc/uce/settings.cfg" "$WEBROOT"
install -m 0644 "$REPO_ROOT/scripts/deb/uce.service" "$STAGE_DIR/usr/lib/systemd/system/uce.service"
(
cd "$STAGE_DIR"
tar --sort=name --mtime='UTC 2026-01-01' --owner=0 --group=0 --numeric-owner -czf "$RPMBUILD_DIR/SOURCES/$PACKAGE_NAME-$VERSION.tar.gz" .
)
cat > "$SPEC_FILE" <<EOF
%global __os_install_post %{nil}
Name: $PACKAGE_NAME
Version: $VERSION
Release: $RELEASE%{?dist}
Summary: UCE FastCGI runtime and live-compiling C++ web environment
License: GPL-3.0-or-later
URL: https://example.com/uce
BuildArch: $ARCH
Requires: bash
Requires: python3
Requires: curl
Requires: systemd
Requires: pcre2
Requires: zlib
Requires: openssl-libs
Requires: libstdc++
Requires: mariadb-connector-c
%description
UCE is an experimental C/C++ web runtime with FastCGI request handling,
on-demand wasm unit compilation, and a packaged site/doc/test tree.
%prep
rm -rf %{_builddir}/$PACKAGE_NAME-$VERSION
mkdir -p %{_builddir}/$PACKAGE_NAME-$VERSION
cd %{_builddir}/$PACKAGE_NAME-$VERSION
tar -xzf %{_sourcedir}/$PACKAGE_NAME-$VERSION.tar.gz
%build
%install
rm -rf %{buildroot}
mkdir -p %{buildroot}
cp -a %{_builddir}/$PACKAGE_NAME-$VERSION/. %{buildroot}/
%post
if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true
systemctl enable uce.service >/dev/null 2>&1 || true
systemctl restart uce.service >/dev/null 2>&1 || true
fi
%preun
if [ "\$1" = "0" ] && command -v systemctl >/dev/null 2>&1; then
systemctl disable --now uce.service >/dev/null 2>&1 || true
fi
%postun
if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true
fi
%files
%license /usr/lib/uce/LICENSE
%doc /usr/lib/uce/README.md
%config(noreplace) /etc/uce/settings.cfg
/usr/lib/uce
/usr/lib/systemd/system/uce.service
$WEBROOT
%dir /var/cache/uce
%dir /var/lib/uce
/opt/*
%changelog
* Mon Jun 15 2026 UCE Packager <root@localhost> - $PACKAGE_VERSION
- Package UCE runtime with pinned deployment toolchains.
EOF
rpmbuild --define "_topdir $RPMBUILD_DIR" -bb "$SPEC_FILE"
find "$RPMBUILD_DIR/RPMS" -type f -name '*.rpm' -print -exec cp -a {} "$DIST_DIR/" \;
find "$DIST_DIR" -maxdepth 1 -type f -name "${PACKAGE_NAME}-${VERSION}-${RELEASE}*.rpm" -print
-118
View File
@@ -1,118 +0,0 @@
#!/usr/bin/env python3
"""Mechanical one-shot rename script for UCE's legacy value-tree API.
The old tokens are constructed rather than written literally so this script can
remain in the tree after the rename without reintroducing the retired names.
"""
from __future__ import annotations
import os
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
OLD_TYPE = "D" + "Tree"
OLD_LOWER = "d" + "tree"
NEW_TYPE = "DValue"
NEW_LOWER = "dvalue"
DIR_EXCLUDES = {
".git",
".hg",
".svn",
"3rdparty",
"bin",
"node_modules",
"pkg",
"tmp",
"work",
"__pycache__",
}
BINARY_SUFFIXES = {
".bin",
".gif",
".ico",
".jpg",
".jpeg",
".lock",
".o",
".pdf",
".png",
".pyc",
".so",
".sqlite",
".webp",
".zip",
}
REPLACEMENTS = (
(OLD_TYPE, NEW_TYPE),
(OLD_TYPE.upper(), NEW_TYPE.upper()),
(OLD_LOWER + "_", "dv_"),
(OLD_LOWER, NEW_LOWER),
)
def replace_text(value: str) -> str:
for old, new in REPLACEMENTS:
value = value.replace(old, new)
return value
def skip_dir(name: str) -> bool:
return name in DIR_EXCLUDES or name.startswith(".fuse")
def skip_file(path: Path) -> bool:
return path.name.startswith(".fuse") or path.suffix in BINARY_SUFFIXES
def rewrite_text_files() -> int:
changed = 0
for current, dirs, files in os.walk(ROOT):
dirs[:] = [name for name in dirs if not skip_dir(name)]
for name in files:
path = Path(current) / name
if skip_file(path):
continue
data = path.read_bytes()
if b"\0" in data:
continue
try:
original = data.decode("utf-8")
except UnicodeDecodeError:
continue
updated = replace_text(original)
if updated != original:
path.write_text(updated, encoding="utf-8")
changed += 1
return changed
def rename_paths() -> int:
changed = 0
matches: list[Path] = []
for current, dirs, files in os.walk(ROOT, topdown=False):
dirs[:] = [name for name in dirs if not skip_dir(name)]
current_path = Path(current)
for name in files + dirs:
new_name = replace_text(name)
if new_name != name:
matches.append(current_path / name)
for path in sorted(matches, key=lambda item: len(item.parts), reverse=True):
if not path.exists():
continue
target = path.with_name(replace_text(path.name))
if target.exists():
raise RuntimeError(f"cannot rename {path} -> {target}: target exists")
path.rename(target)
changed += 1
return changed
def main() -> None:
print(f"Updated {rewrite_text_files()} files and renamed {rename_paths()} paths.")
if __name__ == "__main__":
main()
-25
View File
@@ -1,25 +0,0 @@
#!/bin/bash
# Build the W1 host smoke driver. Run on k-uce.
set -euo pipefail
cd "$(dirname "$0")/../.."
OUT=${UCE_WASM_OUT:-/tmp/uce/wasm-w1}
WASMTIME_HOME=${WASMTIME_HOME:-/opt/wasmtime}
WASMTIME_INCLUDE=${WASMTIME_INCLUDE:-$WASMTIME_HOME/include}
WASMTIME_LIB=${WASMTIME_LIB:-$WASMTIME_HOME/lib}
mkdir -p "$OUT"
if [ ! -d "$WASMTIME_INCLUDE" ] || [ ! -d "$WASMTIME_LIB" ]; then
echo "Wasmtime C API not found; set WASMTIME_HOME or WASMTIME_INCLUDE/WASMTIME_LIB" >&2
exit 1
fi
g++ -std=c++17 -O2 -Wall -Wextra \
-I"$WASMTIME_INCLUDE" \
src/wasm/w1_smoke.cpp \
-L"$WASMTIME_LIB" \
-Wl,-rpath,"$WASMTIME_LIB" \
-lwasmtime \
-o "$OUT/w1_smoke"
ls -lh "$OUT/w1_smoke"
-62
View File
@@ -1,62 +0,0 @@
#!/bin/bash
# Batch-build W2 wasm side modules for already-known/generated UCE units.
set -euo pipefail
cd "$(dirname "$0")/../.."
BIN_DIR=${UCE_BIN_DIRECTORY:-/tmp/uce/work}
KNOWN_FILE=${UCE_KNOWN_UNITS_FILE:-$BIN_DIR/known-uce-files.txt}
MIN_UNITS=${UCE_W2_MIN_UNITS:-1}
if [ "$#" -gt 0 ]; then
UNITS=("$@")
else
if [ ! -f "$KNOWN_FILE" ]; then
echo "known unit registry not found: $KNOWN_FILE" >&2
exit 1
fi
mapfile -t UNITS < <(grep -v '^[[:space:]]*$' "$KNOWN_FILE")
fi
count=0
checked=0
skipped=0
# Native-only units that cannot be wasm side modules (yet).
# - error-reporting.uce deliberately throws to exercise the native exception
# path; the wasm backend replaces that machinery with traps (§11.1).
# - tests/zip.uce uses try/catch around the zip library, which is carved out
# of the wasm core until it moves behind a hostcall (W4+ membrane work).
SKIP_PATTERN=${UCE_W2_SKIP:-(error-reporting|tests/zip)\.uce$}
for unit in "${UNITS[@]}"; do
case "$unit" in
*.uce|*.ws.uce) ;;
*) continue ;;
esac
if [[ "$unit" =~ $SKIP_PATTERN ]]; then
continue
fi
src_dir=$(dirname "$unit")
base=$(basename "$unit")
dest_dir="$BIN_DIR$src_dir"
pp_fn="$base.cpp"
wasm_fn="$base.wasm"
if [ ! -f "$dest_dir/$pp_fn" ]; then
echo "missing preprocessed unit: $dest_dir/$pp_fn" >&2
exit 1
fi
if [ -s "$dest_dir/$wasm_fn" ] && [ "$dest_dir/$wasm_fn" -nt "$dest_dir/$pp_fn" ] && [ "$dest_dir/$wasm_fn" -nt "$unit" ]; then
scripts/wasm/check_unit_wasm.py "$dest_dir/$wasm_fn"
skipped=$((skipped + 1))
else
scripts/compile_wasm_unit "$src_dir" "$dest_dir" "$unit" "$pp_fn" "$wasm_fn"
count=$((count + 1))
fi
checked=$((checked + 1))
done
if [ "$checked" -lt "$MIN_UNITS" ]; then
echo "checked only $checked wasm units, expected at least $MIN_UNITS" >&2
exit 1
fi
echo "W2 batch wasm units: checked=$checked compiled=$count reused=$skipped"
-19
View File
@@ -1,19 +0,0 @@
#!/bin/bash
# Build the W3 workspace-runtime CLI driver. Run on k-uce.
set -euo pipefail
cd "$(dirname "$0")/../.."
OUT=${UCE_WASM_OUT:-/tmp/uce/wasm-w3}
WASMTIME_HOME=${WASMTIME_HOME:-/opt/wasmtime}
mkdir -p "$OUT"
g++ -std=c++20 -O1 -g -w \
-Isrc/lib -Isrc/wasm \
-I"$WASMTIME_HOME/include" \
src/wasm/w3_driver.cpp \
-L"$WASMTIME_HOME/lib" \
-Wl,-rpath,"$WASMTIME_HOME/lib" \
-lwasmtime -lpcre2-8 -lpthread -ldl \
-o "$OUT/w3_driver"
ls -lh "$OUT/w3_driver"
-94
View File
@@ -1,94 +0,0 @@
#!/bin/bash
# W5 parity/performance gate for the config-selectable WASM backend.
# Runs on k-uce. Requires root because the live service reads /etc/uce/settings.cfg.
set -euo pipefail
cd "$(dirname "$0")/../.."
if [ "${EUID:-$(id -u)}" -ne 0 ]; then
echo "run_w5.sh must run as root so it can switch /etc/uce/settings.cfg and restart uce.service" >&2
exit 1
fi
OUT=${UCE_W5_OUT:-/tmp/uce/wasm-w5}
CONFIG=${UCE_CONFIG:-/etc/uce/settings.cfg}
mkdir -p "$OUT"
BACKUP="$OUT/settings.cfg.before-w5"
cp "$CONFIG" "$BACKUP"
restore_on_error() {
if [ "${UCE_W5_KEEP_BACKEND:-0}" != "1" ]; then
cp "$BACKUP" "$CONFIG"
systemctl restart uce.service >/dev/null 2>&1 || true
fi
}
trap restore_on_error EXIT
set_backend() {
local enabled="$1"
local tmp
tmp=$(mktemp)
awk -F= -v enabled="$enabled" '
BEGIN { found = 0 }
/^WASM_BACKEND_ENABLED=/ { print "WASM_BACKEND_ENABLED=" enabled; found = 1; next }
{ print }
END { if(!found) print "WASM_BACKEND_ENABLED=" enabled }
' "$CONFIG" > "$tmp"
for kv in \
'WASM_BACKEND_VERBOSE=0' \
'WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm' \
'WASM_MEMORY_LIMIT_BYTES=536870912' \
'WASM_EPOCH_DEADLINE_TICKS=200' \
'WASM_EPOCH_PERIOD_MS=50'
do
key=${kv%%=*}
if ! grep -q "^${key}=" "$tmp"; then
printf '%s\n' "$kv" >> "$tmp"
fi
done
cat "$tmp" > "$CONFIG"
rm -f "$tmp"
systemctl restart uce.service >/dev/null
sleep 1
}
summary_total() {
awk '/^Summary:/ { print $2; found=1 } END { if(!found) print 0 }' "$1"
}
# Native reference baseline.
set_backend 0
scripts/run_cli_tests.sh > "$OUT/native-warmup.txt" || true
scripts/run_cli_tests.sh > "$OUT/native-network.txt"
python3 tests/wasm_benchmark.py --out-dir "$OUT/native-benchmark" --samples "${UCE_W5_BENCH_SAMPLES:-20}" --timeout 30 >/dev/null
# WASM default backend with W5 native fallbacks for host-owned surfaces.
set_backend 1
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-warmup.txt" || true
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-network.txt"
python3 tests/wasm_benchmark.py \
--out-dir "$OUT/benchmark" \
--backend-label wasm \
--compare-native-json "$OUT/native-benchmark/benchmark.json" \
--samples "${UCE_W5_BENCH_SAMPLES:-20}" \
--timeout 30
python3 tests/wasm_site_audit.py --out-dir "$OUT" >/dev/null
network_cases=$(summary_total "$OUT/wasm-network.txt")
if [ "$network_cases" -lt 80 ]; then
echo "W5 HARNESS: FAIL"
echo "wasm network ran $network_cases cases, expected at least 80"
exit 1
fi
echo 'W5 HARNESS: PASS'
echo "network_cases=$network_cases benchmark_rows=see benchmark.json"
echo "reports=$OUT"
if [ "${UCE_W5_KEEP_BACKEND:-0}" = "1" ]; then
trap - EXIT
printf 'WASM backend left enabled in %s\n' "$CONFIG"
else
cp "$BACKUP" "$CONFIG"
systemctl restart uce.service >/dev/null
trap - EXIT
fi
+26 -11
View File
@@ -1,21 +1,36 @@
#include "demo_guard.h"
// Fault-injection demo. Under the wasm runtime every unit fault is a guest
// trap that the workspace turns into a clean 500 without harming the worker;
// these modes exercise the three distinct trap causes (mirrors tests/wasm-kill).
u64 error_reporting_recurse(volatile u64 depth)
{
volatile u64 next = depth + 1;
return(next + error_reporting_recurse(next));
}
RENDER(Request& context)
{
if(!test_demo_request_allowed(context))
{
test_demo_render_restricted_html(context, "Error reporting", "intentionally crash or abort request workers");
test_demo_render_restricted_html(context, "Error reporting", "crash or abort request workers for diagnostics");
return;
}
String mode = context.get["mode"];
if(mode == "exception")
throw std::runtime_error("Intentional test exception from /test/error-reporting.uce");
if(mode == "abort")
raise(SIGABRT);
if(mode == "segfault")
raise(SIGSEGV);
if(mode == "trap")
__builtin_trap();
if(mode == "recurse")
{
volatile u64 sink = error_reporting_recurse(0);
(void)sink;
}
if(mode == "loop")
{
volatile u64 i = 0;
while(i >= 0)
i++;
}
<>
<link rel="stylesheet" href='style.css?v=<?= time() ?>'></link>
@@ -23,11 +38,11 @@ RENDER(Request& context)
<a href="index.uce">UCE Test</a>:
Error reporting
</h1>
<p>These actions intentionally trigger failures so you can verify that UCE returns a usable `500` response instead of dropping the upstream connection.</p>
<p>These actions trigger guest traps so you can verify that UCE returns a usable `500` response from a healthy worker instead of dropping the upstream connection.</p>
<ul>
<li><a href="?mode=exception">Trigger uncaught exception</a></li>
<li><a href="?mode=abort">Trigger SIGABRT</a></li>
<li><a href="?mode=segfault">Trigger SIGSEGV</a></li>
<li><a href="?mode=trap">Trigger an explicit trap (`__builtin_trap`)</a></li>
<li><a href="?mode=recurse">Trigger stack exhaustion (unbounded recursion)</a></li>
<li><a href="?mode=loop">Trigger a runaway loop (epoch interrupt)</a></li>
</ul>
</>
}
+10 -4
View File
@@ -70,7 +70,7 @@ RENDER(Request& context)
<? render_card("once-init.uce", "ONCE / INIT", "Unit lifecycle hooks for worker load and request entry"); ?>
<? render_card("markdown.uce", "Markdown", "Markdown parsing with components"); ?>
<? render_card("script.uce", "Script", "UCE script integration"); ?>
<? render_card("websockets.ws.uce", "WebSockets", "Real-time WebSocket chat"); ?>
<? render_card("websockets.ws.uce", "WebSockets", "Live WebSocket chat"); ?>
<? render_card("unit-browser.uce", "Unit Browser", "units_list(), unit_info(), unit_compile()"); ?>
<? if(allow_server_demos) { render_card("task.uce", "Task", "Background task execution"); } ?>
<? if(allow_server_demos) { render_card("task_repeat.uce", "Task Repeat", "Recurring task scheduling"); } ?>
@@ -84,10 +84,16 @@ RENDER(Request& context)
<div class="system-info">
<h3>System Info</h3>
<pre><?
print("Worker PID: ", my_pid, "\n");
print("Parent PID: ", parent_pid, "\n");
DValue perf = request_perf();
print("Worker PID: ", (u64)perf["worker_pid"].to_u64(), "\n");
print("Parent PID: ", (u64)perf["parent_pid"].to_u64(), "\n");
print("Request #: ", (u64)perf["request_count"].to_u64(), "\n");
print("Accept us: ", (f64)perf["accept_us"].to_f64(), "\n");
print("Running us: ", (f64)perf["running_us"].to_f64(), "\n");
print("Total us: ", (f64)perf["total_us"].to_f64(), "\n");
if(perf["workspace_birth_us"].type != 'S')
print("Workspace birth us: ", (u64)perf["workspace_birth_us"].to_u64(), "\n");
print("Output buffer size: ", context.ob->str().length(), "\n");
print("Request #", context.server->request_count, "\n");
?></pre>
</div>
<? } ?>
+1 -1
View File
@@ -1,6 +1,6 @@
# Markdown Demo
This page exercises **strong**, *emphasis*, ~~strikethrough~~, `code spans`, and a bare URL: https://uce.openfu.com/doc/index.uce
This page exercises **strong**, *emphasis*, ~~strikethrough~~, `code spans`, and a bare URL: https://example.com/doc/index.uce
## Task List
+5 -3
View File
@@ -1,7 +1,9 @@
RENDER(Request& context)
{
auto p = compiler_load_shared_unit(&context, "post.uce");
if(p)
print(to_string(p));
// The native SharedUnit loader has been retired. Unit metadata is now
// exposed through the wasm membrane via unit_info() (uce_host_units), which
// resolves and introspects another unit without dlopen-ing a native .so.
DValue info = unit_info("post.uce");
print(json_encode(info));
}
+3 -3
View File
@@ -157,16 +157,16 @@ RENDER(Request& context)
</div>
<div class="detail-card">
<strong>Runtime Flags</strong>
<span>loaded <?= unit_flag_label(selected_info["loaded"]) ?>, stale <?= unit_flag_label(selected_info["stale"]) ?>, current <?= unit_flag_label(selected_info["current_unit"]) ?></span>
<span>wasm <?= unit_flag_label(selected_info["wasm_available"]) ?>, stale <?= unit_flag_label(selected_info["stale"]) ?>, current <?= unit_flag_label(selected_info["current_unit"]) ?></span>
</div>
<div class="detail-card">
<strong>Artifacts</strong>
<code><?= selected_info["so_name"].to_string() ?></code>
<code><?= selected_info["wasm_name"].to_string() ?></code>
</div>
<div class="detail-card">
<strong>Timestamps</strong>
<span>Source file: <?= time_format_relative(selected_info["source_mtime"].to_u64()) ?><br>
Binary: <?= time_format_relative(selected_info["compiled_mtime"].to_u64()) ?><br>
Wasm: <?= time_format_relative(selected_info["compiled_mtime"].to_u64()) ?><br>
Meta info: <?= time_format_relative(selected_info["metadata_mtime"].to_u64()) ?></span>
</div>
</div>
+3
View File
@@ -1,5 +1,8 @@
Runtime
backtrace_frames_string
capture_backtrace_string
error_pages
signal_name
unit_info
units_list
unit_compile
+5 -1
View File
@@ -1,11 +1,13 @@
String Functions
ascii_safe_name
base64_decode
base64_encode
contains
concat
filter
first
join
json_consume_space
json_encode
list_every
list_find
@@ -20,6 +22,8 @@ str_ends_with
str_starts_with
substr
split
split_http_headers
split_kv
split_space
split_utf8
replace
+1
View File
@@ -4,6 +4,7 @@ basename
dirname
expand_path
file_append
file_append_contents
file_exists
file_get_contents
file_mtime
+1
View File
@@ -1,5 +1,6 @@
Time and Date Functions
usleep
time_precise
time
time_format_local
+1
View File
@@ -12,5 +12,6 @@ route_path_normalize
route_path_sanitize
session_id_create
parse_query
parse_uri
uri_decode
uri_encode
+1 -1
View File
@@ -15,7 +15,7 @@ INIT(Request& context)
:content
Defines a worker-load hook for the current `.uce` unit.
When a worker loads the unit's compiled shared object into memory, the runtime checks whether the unit exposes `INIT(Request& context)`. If it does, the hook runs once for that load before the unit begins serving later requests from that in-memory copy.
When a worker instantiates the unit's compiled wasm module, the runtime checks whether the unit exposes `INIT(Request& context)`. If it does, the hook runs once for that instance before the unit begins serving requests from that worker-local copy.
Because UCE usually loads units on demand during a request, `INIT()` still receives a valid `Request& context`. Use it for worker-local initialization, not for request-local state that should reset each request.
+1 -1
View File
@@ -32,7 +32,7 @@ For a normal direct page request, `context.props` starts empty.
If the page is invoked from another UCE file via `unit_render(file_name, context)`, the callee receives that same `context`.
Pages intended to serve WebSocket traffic may expose both `RENDER(Request& context)` and `WS(Request& context)`. Files may also define `COMPONENT()` handlers when they intentionally need both page and component behavior in one unit.
Pages that serve WebSocket traffic may expose both `RENDER(Request& context)` and `WS(Request& context)`. Files may also define `COMPONENT()` handlers when one unit needs both page and component behavior.
In that case:
+2 -2
View File
@@ -12,9 +12,9 @@ WS(Request& context)
>1_RENDER
:content
Defines the WebSocket message handler for the current `.ws.uce` page.
Defines the WebSocket message handler for the current `.uce` page.
The same page may expose both `RENDER(Request& context)` and `WS(Request& context)`. `RENDER(Request& context)` serves the initial HTTP response, while `WS(Request& context)` is called whenever a complete WebSocket message arrives for that page.
Any `.uce` unit may expose both `RENDER(Request& context)` and `WS(Request& context)`. `RENDER(Request& context)` serves the initial HTTP response, while `WS(Request& context)` is called whenever a complete WebSocket message arrives for that page. Configure nginx/Apache to route WebSocket upgrade requests for `.uce` paths to the runtime's HTTP/WebSocket listener.
UCE reassembles fragmented messages before calling `WS(Request& context)`. Text and binary frames are both delivered. The current payload is available directly on `context.in`, message metadata is mirrored into `context.params["WS_..."]`, and connection-local state lives on `context.connection`.
+6 -6
View File
@@ -14,7 +14,7 @@ unit_call
:content
UCE runs a small custom source-to-source preprocessor before Clang sees a `.uce` or `.ws.uce` file.
The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, with orchestration in `src/lib/compiler.cpp`. It does not try to parse all of C++. Instead, it performs a narrow character-wise rewrite that understands literal output, inline code islands, `#load`, and `EXPORT` harvesting, then writes a generated `.cpp` file and compiles that file into a shared object.
The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, with orchestration in `src/lib/compiler.cpp`. It does not try to parse all of C++. Instead, it performs a narrow character-wise rewrite that understands literal output, inline code islands, `#load`, and `EXPORT` harvesting, then writes a generated `.cpp` file and compiles that file into a WebAssembly side module.
## Syntax
@@ -32,7 +32,7 @@ The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, wi
## Pipeline
- The generated file starts by including the logical runtime header `uce_lib.h`; native and WASM compile scripts provide the include path.
- The generated file starts by including the logical runtime header `uce_lib.h`; the wasm unit compile script provides the include path.
- It then inlines the configured setup template from `SETUP_TEMPLATE` (by default `scripts/setup.h.template`), which defines the internal hook `__uce_set_current_request(Request*)`.
- It inserts `#line 1` before page code so compiler diagnostics point back to the original `.uce` file.
- Each literal region is rewritten into one or more `print(R"...( ... )...");` calls using a safe raw-string delimiter selected for that literal content.
@@ -47,8 +47,8 @@ The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, wi
- Lines beginning with `RENDER:NAME(...)` are rewritten into exported `__uce_render_NAME(...)` functions.
- Lines beginning with `COMPONENT:NAME(...)` are rewritten into exported `__uce_component_NAME(...)` functions for the component helpers.
- The final generated source is written to `BIN_DIRECTORY + src_path + "/" + source_file + ".cpp"`.
- `scripts/compile` then compiles that generated `.cpp` into `source_file + ".so"` with `clang++ -shared -std=c++20 ...`.
- When a worker loads the compiled unit into memory, the runtime checks for `INIT(Request& context)` and calls it once for that worker-side load.
- `scripts/compile_wasm_unit` then compiles that generated `.cpp` into `source_file + ".wasm"` as a PIC WebAssembly side module.
- When a worker instantiates the compiled unit, the runtime checks for `INIT(Request& context)` and calls it once for that worker-side instance.
- On each request, the first time a given unit is entered through `RENDER()`, `CLI()`, or any `COMPONENT...` handler, the runtime checks for `ONCE(Request& context)` and calls it before the selected handler.
## Generated Files
@@ -56,7 +56,7 @@ The template rewriting implementation lives in `src/lib/compiler-parser.cpp`, wi
For a source file like `/some/path/page.uce`, the preprocessor produces:
- generated C++: `BIN_DIRECTORY/some/path/page.uce.cpp`
- shared object: `BIN_DIRECTORY/some/path/page.uce.so`
- wasm side module: `BIN_DIRECTORY/some/path/page.uce.wasm`
- export list: `BIN_DIRECTORY/some/path/page.uce.exports.txt`
## Examples
@@ -153,7 +153,7 @@ The page template can then render `context.call["fragments"]["head"]` inside `<h
- `EXPORT` harvesting only triggers when the current line starts with `EXPORT` at column 1 and is followed by whitespace.
- Relative `#load` paths are expanded against the including unit's source directory.
- `unit_render()` and `unit_call()` are runtime APIs. `#load` is a compile-time composition feature.
- `INIT()` runs when the shared object is loaded into a worker during a request-triggered load, so it still receives a valid `Request& context`.
- `INIT()` runs when the wasm unit is instantiated by a worker during a request-triggered load, so it still receives a valid `Request& context`.
- `ONCE()` is tracked per request and per resolved unit file. A file entered multiple times in one request only runs `ONCE()` once.
## Limitations
@@ -0,0 +1,18 @@
:sig
String backtrace_frames_string(void* const* frames, size_t size, u32 skip_frames = 0)
:params
frames : frame pointer array returned by native backtrace collection
size : number of frames in the array
skip_frames : number of newest frames to omit
return value : formatted backtrace string
:see
>runtime
capture_backtrace_string
signal_name
:content
Formats a captured native backtrace frame array.
Most page code should use `capture_backtrace_string()` instead. Use this helper when you already have raw frame pointers from lower-level diagnostic code.
+30
View File
@@ -0,0 +1,30 @@
:sig
String base64_decode(String raw, bool& ok)
:params
raw : Base64 encoded string
ok : set to `true` when decoding succeeds; set to `false` for invalid input
return value : decoded binary-safe string, or an empty string when decoding fails
:see
>string
base64_encode
:content
Decodes a Base64 string.
Pass a `bool` variable for `ok` so callers can distinguish invalid input from a valid empty decoded value.
Example:
```uce
bool ok = false;
String decoded = base64_decode("aGVsbG8=", ok);
// ok == true
// decoded == "hello"
```
Related:
- PHP: `base64_decode($value, true)`
- JavaScript / Node.js: `Buffer.from(value, "base64")`
+27
View File
@@ -0,0 +1,27 @@
:sig
String base64_encode(String raw)
:params
raw : binary-safe source string
return value : Base64 encoded string
:see
>string
base64_decode
:content
Encodes a string with Base64.
UCE strings can contain binary data, so `raw` may include NUL bytes and non-text bytes.
Example:
```uce
String encoded = base64_encode("hello");
// encoded == "aGVsbG8="
```
Related:
- PHP: `base64_encode()`
- JavaScript / Node.js: `Buffer.from(value).toString("base64")`
@@ -0,0 +1,24 @@
:sig
String capture_backtrace_string(u32 max_frames = 32, u32 skip_frames = 0)
:params
max_frames : maximum number of frames to capture
skip_frames : number of newest frames to omit
return value : formatted backtrace string
:see
>runtime
backtrace_frames_string
signal_name
:content
Captures and formats a native backtrace for the current call stack.
This helper is for diagnostics. It is used by runtime error reporting paths and can also be useful in local debugging pages.
Example:
```uce
String trace = capture_backtrace_string(16, 0);
print("<pre>", html_escape(trace), "</pre>");
```
+4 -4
View File
@@ -27,7 +27,7 @@ UCE is server-first C++ with a small template preprocessor. It does not try to b
- `ONCE(Request& context)` is per-request setup for a unit before its first render/component entry.
- `INIT(Request& context)` is worker-local setup when a unit is loaded.
- `<?= expression ?>` is escaped interpolation; prefer it for user-visible text.
- `<?: expression ?>` is trusted raw markup output, closer to a deliberate `dangerouslySetInnerHTML` decision.
- `<?: expression ?>` writes trusted raw markup, similar to `dangerouslySetInnerHTML` in React.
- `unit_render()` renders another page unit; `component()` returns component HTML as a string.
## Routes and Layouts
@@ -51,11 +51,11 @@ DValue app_items = dv_filter(menu, [](DValue item, String key) { return(item["se
DValue by_section = dv_group_by(menu, [](DValue item, String key) { return(item["section"].to_string()); });
```
Use these when the transformation communicates intent. Prefer explicit loops when side effects or multi-step validation are the main concern.
Use these when a short transformation is clearer than a loop. Prefer explicit loops for side effects or multi-step validation.
## Assets and Islands
Global runtime APIs for assets and islands are intentionally not part of UCE core. The starter emits CSS and JavaScript from the owning unit's `ONCE(Request& context)` hook, with a few shared sibling asset components when multiple components need the same files. The only starter web-affordance helper left is `COMPONENT:island` in `components/theme/web_affordances.uce` for small progressive-enhancement modules. This keeps app policy in the app without an asset registry layer.
UCE core does not provide a global asset or island registry. The starter emits CSS and JavaScript from the owning unit's `ONCE(Request& context)` hook, with a few shared sibling asset components when multiple components need the same files. The starter's `COMPONENT:island` helper in `components/theme/web_affordances.uce` covers small progressive-enhancement modules while keeping app policy in the app.
## Debugging
@@ -66,4 +66,4 @@ When a unit fails to compile, UCE reports the source path, generated C++ path, c
- No client-side virtual DOM is built into UCE.
- No global file-router is imposed by the runtime.
- No JSX-like component tags are required for this workflow.
- Component children/slot syntax is intentionally deferred; use explicit props and component calls for now.
- Component children/slot syntax is not part of UCE yet; use explicit props and component calls for now.
+1 -1
View File
@@ -11,7 +11,7 @@ component_resolve
:content
Checks whether a component file can be resolved from the current page context.
Resolution tries the exact name first and then the `components/` shorthand form.
Resolution uses the same host resolver as `component_resolve()`: candidate bases include absolute targets, the entry unit directory, the current unit directory, and the site root; each base tries exact, `.uce`, `components/name`, and `components/name.uce` forms.
If `name` contains a colon, only the file portion is used for existence checks.
+1 -1
View File
@@ -11,7 +11,7 @@ component_render
:content
Resolves a component name to the concrete `.uce` file path that will be loaded.
Resolution tries the exact file name first, then the same name with `.uce` appended, and then the same two forms under the `components/` prefix.
Resolution searches host-side candidate bases in order: absolute target when supplied, entry unit directory, current unit directory, and site root. Within each base it tries the exact file name, the same name with `.uce` appended, and the same two forms under the `components/` prefix.
If `name` contains a colon, only the file portion is used for resolution.
+2 -13
View File
@@ -1,16 +1,5 @@
:sig
String concat(...vals)
:params
...val : one or more values that should be concatenated
:see
>string
concat(...vals)
:content
Returns a string with all parameters concatenated into one result.
## Related Concepts
- PHP: string concatenation with `.` or helpers like `implode()`
- JavaScript / Node.js: string concatenation with `+`, template literals, or `Array.prototype.join()`
Removed. `concat()` is not a current UCE API. Use string `+`, output streams/`print()`, or `join()` for lists.
+13
View File
@@ -0,0 +1,13 @@
:sig
bool config_bool(String key, bool fallback = true)
:params
key : server configuration key
fallback : value returned when missing
:see
>sys
>config_bool_value
:content
Reads a boolean value from the active server configuration.
+13
View File
@@ -0,0 +1,13 @@
:sig
bool config_bool_value(String raw, bool fallback = true)
:params
raw : raw string value
fallback : value returned for an empty string
:see
>sys
>config_bool
:content
Parses common configuration booleans. Empty uses the fallback; `0`, `false`, `no`, and `off` are false; other non-empty values are true.
+13
View File
@@ -0,0 +1,13 @@
:sig
f64 config_f64(String key, f64 fallback)
:params
key : server configuration key
fallback : value returned when missing or invalid
:see
>sys
>config_map_f64
:content
Reads a floating-point value from the active server configuration.
+14
View File
@@ -0,0 +1,14 @@
:sig
bool config_map_bool(StringMap& cfg, String key, bool fallback = true)
:params
cfg : configuration map
key : entry to parse
fallback : value returned when missing
:see
>sys
>config_bool_value
:content
Reads a boolean from a string map using `config_bool_value()` semantics.
+14
View File
@@ -0,0 +1,14 @@
:sig
f64 config_map_f64(StringMap& cfg, String key, f64 fallback)
:params
cfg : configuration map
key : entry to parse
fallback : value returned when missing or invalid
:see
>sys
>config_f64
:content
Reads a floating-point value from a string map with fallback handling.
+14
View File
@@ -0,0 +1,14 @@
:sig
u64 config_map_u64(StringMap& cfg, String key, u64 fallback)
:params
cfg : configuration map
key : entry to parse
fallback : value returned when missing or invalid
:see
>sys
>config_u64
:content
Reads an unsigned integer from a string map with fallback handling.
+13
View File
@@ -0,0 +1,13 @@
:sig
u64 config_u64(String key, u64 fallback)
:params
key : server configuration key
fallback : value returned when missing or invalid
:see
>sys
>config_map_u64
:content
Reads an unsigned integer from the active server configuration.
+1 -1
View File
@@ -8,7 +8,7 @@ path : the new working directory
>sys
:content
Sets a new working directory.
Sets the host worker process current directory. In wasm this is a hostcall, so restore the previous directory when using it inside request code.
## Related Concepts
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Keeps children for which f returns true. List-like input stays list-like.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
DValue visible = dv_filter(items, [](DValue item, String key) { return(item["hidden"].to_bool() == false); });
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Groups children into list-like buckets by the string returned from f.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
DValue by_section = dv_group_by(menu, [](DValue item, String key) { return(item["section"].to_string()); });
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Returns map keys from a DValue. Scalar values produce an empty list.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
StringList keys = dv_keys(context.cfg["menu"]);
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Transforms each child. List-like input stays list-like; map input keeps keys.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
DValue titles = dv_map(items, [](DValue item, String key) { DValue out; out = item["title"].to_string(); return(out); });
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Copies a DValue map except for selected keys.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
DValue safe_user = dv_omit(user, {"password_hash"});
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Copies only selected keys from a DValue map.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
DValue public_user = dv_pick(user, {"name", "avatar"});
+1 -1
View File
@@ -12,7 +12,7 @@ filter
:content
Returns child values as a list-like DValue.
These helpers are intentionally small data-shaping conveniences for render code, routers, and configuration trees. They are useful when porting habits from React/Next/Remix code where lists of routes, navigation items, cards, or records are transformed close to the rendering boundary.
These helpers keep common data-shaping code close to render code, routers, and configuration trees. They are useful for route lists, navigation items, cards, and records that need simple transformations before rendering.
```cpp
DValue menu_items = dv_values(context.cfg["menu"]);
+1 -1
View File
@@ -42,7 +42,7 @@ RENDER(Request& context)
}
```
The compiling page is only used while `PROACTIVE_COMPILE_ENABLED` is on (the default) — otherwise nothing would finish the build, and the runtime falls back to the blocking compile.
The compiling page is only used while `PROACTIVE_COMPILE_ENABLED` is on (the default) — otherwise nothing would finish the build asynchronously, and the runtime waits for the on-request wasm compile.
## page_compiler_error
+26
View File
@@ -0,0 +1,26 @@
:sig
bool file_append_contents(String file_name, String content)
:params
file_name : path to open or create
content : bytes to append
return value : `true` when the append succeeds
:see
>sys
file_append
file_put_contents
file_get_contents
:content
Appends one string to a file.
`file_append()` is the variadic convenience wrapper for ordinary page code. Use `file_append_contents()` when you already have one string buffer.
Example:
```uce
bool ok = file_append_contents("/tmp/uce-log.txt", "line\n");
```
The file is created if it does not exist.
+12
View File
@@ -0,0 +1,12 @@
:sig
void file_close_locked(int fd)
:params
fd : handle returned by `file_open_locked()`
:see
>sys
>file_open_locked
:content
Releases the flock and closes a locked file handle. Wasm handles are opaque and request-local.
@@ -0,0 +1,14 @@
:sig
String file_get_contents_locked_fd(int fd)
:params
fd : handle returned by `file_open_locked()`
return value : complete file contents, or an empty string on error/empty file
:see
>sys
>file_open_locked
>file_put_contents_locked_fd
:content
Reads the full contents of a locked file handle from the start of the file.
+18
View File
@@ -0,0 +1,18 @@
:sig
int file_open_locked(String file_name, int open_flags, int lock_type = LOCK_SH, int create_mode = 0644, f64 wait_timeout_seconds = 3.0, String purpose = "")
:params
file_name : path to open
open_flags : host open(2) flags
lock_type : `LOCK_SH` or `LOCK_EX`
create_mode : mode used when creating
return value : opaque locked file handle, or -1
:see
>sys
>file_close_locked
>file_get_contents_locked_fd
>file_put_contents_locked_fd
:content
Opens and locks a file on the host. In wasm units the returned integer is an opaque worker-owned handle that is valid only for the current request.
@@ -0,0 +1,15 @@
:sig
bool file_put_contents_locked_fd(int fd, String content)
:params
fd : handle returned by `file_open_locked()`
content : bytes to write
return value : true on complete write
:see
>sys
>file_open_locked
>file_get_contents_locked_fd
:content
Truncates and rewrites the file behind a locked file handle.
@@ -0,0 +1,12 @@
:sig
void file_release_process_locks(String reason = "")
:params
reason : diagnostic reason for releasing locks
:see
>sys
>file_open_locked
:content
Releases locked file handles owned by the current process/workspace. Wasm uses this to close all request-local locked file handles.

Some files were not shown because too many files have changed in this diff Show More