post-W7 cleanup
This commit is contained in:
root
@@ -1,90 +0,0 @@
|
||||
# CLI Unit Test Port Plan
|
||||
|
||||
## Objective
|
||||
|
||||
Replace the Python-based network test runner and plugins with UCE unit tests invoked through the runtime's CLI socket path, keeping equivalent coverage for HTTP smoke, site suites, security checks, starter parity, TCP/WebSocket listener probes, and wasm kill checks. The final invocation should be a bash script that calls UCE CLI units; Python test files should be removed only after UCE coverage is in place and validated.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [x] A bash command runs the full test suite through the CLI socket and exits nonzero on failure.
|
||||
- [x] UCE CLI tests cover every current Python plugin behavior or explicitly document a deliberate replacement.
|
||||
- [x] Existing W5/WASM gates use the UCE CLI test runner instead of `tests/run_network_tests.py`.
|
||||
- [x] Python test runner/plugins are deleted after parity validation.
|
||||
- [x] Full suite passes on `uce-dev` with wasm-only unit execution.
|
||||
|
||||
## Current State
|
||||
|
||||
- Status: verifying
|
||||
- Last updated: 2026-06-13
|
||||
- Source of truth: `/root/mount_ssh/uce-dev-root-htdocs-uce`
|
||||
- Runtime/live target: `uce-dev:/Code/uce.openfu.com/uce`
|
||||
|
||||
## Goal Tree
|
||||
|
||||
Legend: `[ ]` not started, `[~]` in progress, `[x]` done, `[!]` blocked, `[-]` superseded
|
||||
|
||||
- [x] G1: Inventory Python test coverage and CLI constraints
|
||||
- Why: replacement must preserve coverage before deleting Python.
|
||||
- Done when: every plugin has a mapped UCE equivalent or blocker.
|
||||
- Verify: coverage matrix in this document.
|
||||
- [x] G1.1: Delegate design/coverage review to Spark.
|
||||
- [x] G1.2: Inspect delegates and reconcile plan.
|
||||
- [x] G2: Build UCE CLI test harness
|
||||
- Why: Python runner features need a UCE-native replacement.
|
||||
- Done when: one CLI unit can list/run tests, print pass/fail summary, and return failing CLI status.
|
||||
- Verify: `scripts/run_cli_tests.sh --list` and `scripts/run_cli_tests.sh --include-wasm-kill`.
|
||||
- [x] G2.1: Add reusable UCE assertion/reporting helpers.
|
||||
- [x] G2.2: Add HTTP/TCP helper functions using UCE socket APIs.
|
||||
- [x] G2.3: Add bash wrapper under `scripts/`.
|
||||
- [x] G3: Port current plugin cases to UCE
|
||||
- Why: only delete Python after equivalent UCE coverage exists.
|
||||
- Done when: UCE suite covers demo, HTTP docs/starter, site suites, security, starter parity, TCP, wasm kill.
|
||||
- Verify: UCE CLI full run passes and output names match coverage matrix.
|
||||
- [x] G4: Replace Python gate usage and delete Python tests
|
||||
- Why: user explicitly requested eliminating the Python suite.
|
||||
- Done when: scripts no longer call `tests/run_network_tests.py`, Python test files removed, validation green.
|
||||
- Verify: `rg 'run_network_tests|tests/plugins|python3 tests'` has no obsolete gate references except historical docs and benchmark/audit utilities.
|
||||
- [~] G5: Document and validate
|
||||
- Why: future agents/operators need the new test workflow.
|
||||
- Done when: docs/project notes and in-repo docs mention the CLI test command and validation artifact.
|
||||
- Verify: docs committed, full suite run artifact recorded.
|
||||
|
||||
## Coverage Matrix
|
||||
|
||||
- `uce_demo_smoke.py` → `cli_run_demo_smoke()` in `site/tests/cli_runner.uce` (43 demo pages).
|
||||
- `uce_http_smoke.py` → `cli_run_http_smoke()` (docs and starter route/body checks, 14 cases).
|
||||
- `uce_site_suite.py` → `cli_run_site_suite()` (manifest-driven published site suite pages, 13 cases).
|
||||
- `uce_security_smoke.py` → `cli_run_security_smoke()` (direct HTTP traversal/header spoofing, CRLF header sanitization, session hardening, 4 cases).
|
||||
- `uce_starter_parity.py` → `cli_run_starter_parity()` (starter view title/404 checks, 7 cases).
|
||||
- `uce_tcp_smoke.py` → `cli_run_tcp_smoke()` (port 80 and 8080 reachability, 2 cases).
|
||||
- `uce_wasm_kill.py` → `cli_run_wasm_kill()` gated by `--include-wasm-kill` (trap/loop/recurse + post-kill health checks, 3 cases).
|
||||
|
||||
## Execution Queue
|
||||
|
||||
1. Run final no-Python-suite validation after removing stale references.
|
||||
2. Commit UCE and project-doc updates.
|
||||
|
||||
## Decisions
|
||||
|
||||
- 2026-06-13: Use UCE CLI socket invocation as the test entrypoint; bash wrappers are acceptable, Python runner/plugins are not.
|
||||
- 2026-06-13: Do not delete Python tests until UCE replacement validates equivalent coverage on `uce-dev`.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- UCE socket APIs are sufficient for HTTP/1.0 probes, TCP connect checks, and security header injection checks.
|
||||
- Bash can provide filtering/list convenience if exact Python CLI parity is not needed.
|
||||
|
||||
## Blockers and Risks
|
||||
|
||||
- [ ] The new runner intentionally does not preserve the old Python runner's dynamic plugin/tag/regex filtering; add UCE-side selectors later if operators miss them.
|
||||
- [ ] `tests/wasm_benchmark.py` and `tests/wasm_site_audit.py` remain Python utility scripts, not the network test suite; port separately if a strict no-Python tools policy is desired.
|
||||
|
||||
## Evidence and Verification Log
|
||||
|
||||
- 2026-06-13: Prior to this plan, `b1856c1 chore: narrow wasm backend entrypoint API` was committed after build and focused services validation.
|
||||
- 2026-06-13: `scripts/run_cli_tests.sh --include-wasm-kill` on `uce-dev` passed `86 passed, 0 failed, 0 skipped` (`/tmp/uce/cli-tests-final.txt`).
|
||||
|
||||
## Change Log
|
||||
|
||||
- 2026-06-13: Created initial goal tree for CLI unit test port.
|
||||
- 2026-06-13: Added `site/tests/cli_runner.uce`, `scripts/run_cli_tests.sh`, removed Python network runner/plugins, and repointed W5 network gates to the CLI runner.
|
||||
+565
@@ -0,0 +1,565 @@
|
||||
# UCE Runtime Setup
|
||||
|
||||
This guide describes how to run UCE behind nginx or Apache. UCE is a FastCGI application server for `.uce` units; the web server should serve static files directly and forward dynamic `.uce` requests to the UCE runtime.
|
||||
|
||||
## Deployment shape
|
||||
|
||||
A typical deployment has four pieces:
|
||||
|
||||
1. A checked-out or packaged UCE runtime tree.
|
||||
2. `/etc/uce/settings.cfg`, read by the UCE runtime at startup.
|
||||
3. `uce.service`, a systemd service that builds/starts/restarts the runtime.
|
||||
4. nginx or Apache as the public HTTP server.
|
||||
|
||||
Recommended filesystem layout for a source checkout:
|
||||
|
||||
```text
|
||||
/opt/uce/ UCE repository/runtime root
|
||||
/var/www/html/ public web root served by nginx/Apache
|
||||
/etc/uce/settings.cfg runtime configuration
|
||||
/run/uce/fastcgi.sock FastCGI socket used by nginx/Apache
|
||||
/run/uce/cli.sock local CLI/admin/test socket
|
||||
/var/cache/uce/work generated source, wasm modules, caches
|
||||
/var/lib/uce/uploads multipart upload scratch space
|
||||
/var/lib/uce/sessions session files
|
||||
```
|
||||
|
||||
For packaged installs, the runtime may live under `/usr/lib/uce` instead of `/opt/uce`. Keep the public web root at `/var/www/html` or another normal web-root path, not under the runtime source tree.
|
||||
|
||||
## Build requirements
|
||||
|
||||
On Debian/Ubuntu-like systems, install the distro packages first:
|
||||
|
||||
```bash
|
||||
apt update
|
||||
apt install -y clang build-essential libpcre2-dev mariadb-client libmariadb-dev curl rsync ca-certificates
|
||||
```
|
||||
|
||||
UCE also requires two toolchains that are not vendored in this repository:
|
||||
|
||||
- **Wasmtime C API / C++ headers** at `/opt/wasmtime` by default. `scripts/build_linux.sh` expects:
|
||||
- `/opt/wasmtime/include/wasmtime.hh`
|
||||
- `/opt/wasmtime/include/wasmtime/*.h`
|
||||
- `/opt/wasmtime/lib/libwasmtime.so`
|
||||
- **WASI SDK** at `/opt/wasi-sdk` by default. `scripts/build_core_wasm.sh` and `scripts/compile_wasm_unit` expect:
|
||||
- `/opt/wasi-sdk/bin/clang++`
|
||||
- `/opt/wasi-sdk/bin/wasm-ld`
|
||||
- `/opt/wasi-sdk/bin/llvm-objcopy`
|
||||
|
||||
You can use different install locations by setting environment variables before building and in the systemd service environment:
|
||||
|
||||
```bash
|
||||
export WASMTIME_HOME=/path/to/wasmtime
|
||||
export WASI_SDK=/path/to/wasi-sdk
|
||||
```
|
||||
|
||||
Install one web server:
|
||||
|
||||
```bash
|
||||
apt install -y nginx
|
||||
# or
|
||||
apt install -y apache2
|
||||
```
|
||||
|
||||
Build UCE from the repository root:
|
||||
|
||||
```bash
|
||||
cd /opt/uce
|
||||
bash scripts/build_core_wasm.sh
|
||||
bash scripts/build_linux.sh
|
||||
```
|
||||
|
||||
Publish the starter site or your application files into the web root:
|
||||
|
||||
```bash
|
||||
mkdir -p /var/www/html
|
||||
rsync -a site/ /var/www/html/
|
||||
```
|
||||
|
||||
The main binary is written to:
|
||||
|
||||
```text
|
||||
bin/uce_fastcgi.linux.bin
|
||||
```
|
||||
|
||||
### Installing Wasmtime and WASI SDK
|
||||
|
||||
The UCE build does not download these dependencies for you. Install them through your distro if it provides compatible development packages, or unpack pinned release archives under `/opt/wasmtime` and `/opt/wasi-sdk`.
|
||||
|
||||
Do not use `curl | sh` installers in production setup scripts. Download archives from the upstream release pages, verify checksums/signatures when available, and keep the exact versions recorded in your deployment notes.
|
||||
|
||||
The expected directory shape is:
|
||||
|
||||
```text
|
||||
/opt/wasmtime/include/wasmtime.hh
|
||||
/opt/wasmtime/include/wasmtime/store.h
|
||||
/opt/wasmtime/lib/libwasmtime.so
|
||||
|
||||
/opt/wasi-sdk/bin/clang++
|
||||
/opt/wasi-sdk/bin/wasm-ld
|
||||
/opt/wasi-sdk/bin/llvm-objcopy
|
||||
```
|
||||
|
||||
If your paths differ, export the variables for manual builds:
|
||||
|
||||
```bash
|
||||
WASMTIME_HOME=/usr/local/wasmtime WASI_SDK=/usr/local/wasi-sdk bash scripts/build_core_wasm.sh
|
||||
WASMTIME_HOME=/usr/local/wasmtime WASI_SDK=/usr/local/wasi-sdk bash scripts/build_linux.sh
|
||||
```
|
||||
|
||||
For systemd, add an override:
|
||||
|
||||
```bash
|
||||
systemctl edit uce.service
|
||||
```
|
||||
|
||||
```ini
|
||||
[Service]
|
||||
Environment=WASMTIME_HOME=/usr/local/wasmtime
|
||||
Environment=WASI_SDK=/usr/local/wasi-sdk
|
||||
```
|
||||
|
||||
Then reload and restart:
|
||||
|
||||
```bash
|
||||
systemctl daemon-reload
|
||||
systemctl restart uce.service
|
||||
```
|
||||
|
||||
## Runtime configuration
|
||||
|
||||
Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths if your runtime is not under `/opt/uce`.
|
||||
|
||||
Minimum useful settings:
|
||||
|
||||
```ini
|
||||
BIN_DIRECTORY=/var/cache/uce/work
|
||||
TMP_UPLOAD_PATH=/var/lib/uce/uploads
|
||||
SESSION_PATH=/var/lib/uce/sessions
|
||||
|
||||
FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
|
||||
CLI_SOCKET_PATH=/run/uce/cli.sock
|
||||
|
||||
SITE_DIRECTORY=/var/www/html
|
||||
JIT_COMPILE_ON_REQUEST=1
|
||||
PROACTIVE_COMPILE_ENABLED=1
|
||||
PROACTIVE_COMPILE_CHECK_INTERVAL=60
|
||||
|
||||
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
|
||||
WASM_BACKEND_VERBOSE=0
|
||||
WASM_CORE_PATH=/opt/uce/bin/wasm/core.wasm
|
||||
WASM_MEMORY_LIMIT_BYTES=536870912
|
||||
WASM_EPOCH_DEADLINE_TICKS=200
|
||||
WASM_EPOCH_PERIOD_MS=50
|
||||
|
||||
WORKER_COUNT=4
|
||||
MAX_MEMORY=16777216
|
||||
SESSION_TIME=2592000
|
||||
|
||||
HTTP_PORT=8080
|
||||
```
|
||||
|
||||
Important settings:
|
||||
|
||||
- `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests.
|
||||
- `CLI_SOCKET_PATH` is a local HTTP-over-Unix socket used by `scripts/uce-cli` and test/admin units.
|
||||
- `SITE_DIRECTORY` is the public site tree to scan for `.uce` files. Use `/var/www/html` when the web root is outside the runtime tree; relative paths are resolved from the runtime working directory.
|
||||
- `BIN_DIRECTORY` stores generated C++, wasm artifacts, compile output, and runtime caches.
|
||||
- `TMP_UPLOAD_PATH` and `SESSION_PATH` must be writable by the runtime.
|
||||
- `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes.
|
||||
- `WASM_CORE_PATH` must point at the built `core.wasm` file.
|
||||
|
||||
After editing settings, restart UCE:
|
||||
|
||||
```bash
|
||||
systemctl restart uce.service
|
||||
```
|
||||
|
||||
## systemd service
|
||||
|
||||
For source-checkout deployments, install the provided service helper:
|
||||
|
||||
```bash
|
||||
cd /opt/uce
|
||||
scripts/systemd/manage-uce-service.sh setup
|
||||
```
|
||||
|
||||
That helper installs `scripts/systemd/uce.service`, creates runtime directories, enables the service, and starts it.
|
||||
|
||||
Useful commands:
|
||||
|
||||
```bash
|
||||
scripts/systemd/manage-uce-service.sh status
|
||||
scripts/systemd/manage-uce-service.sh restart
|
||||
scripts/systemd/manage-uce-service.sh logs 200
|
||||
```
|
||||
|
||||
Equivalent manual systemd service for a source checkout:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=UCE FastCGI Runtime
|
||||
After=network-online.target mariadb.service memcached.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/opt/uce
|
||||
RuntimeDirectory=uce
|
||||
StateDirectory=uce
|
||||
CacheDirectory=uce
|
||||
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
|
||||
ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock
|
||||
ExecStartPre=/usr/bin/bash /opt/uce/scripts/build_linux.sh
|
||||
ExecStart=/opt/uce/bin/uce_fastcgi.linux.bin
|
||||
ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
TimeoutStopSec=15
|
||||
KillMode=mixed
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Install it as `/etc/systemd/system/uce.service` and run:
|
||||
|
||||
```bash
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now uce.service
|
||||
```
|
||||
|
||||
## How request routing works
|
||||
|
||||
### Static files
|
||||
|
||||
The web server should serve ordinary static files directly from the public web root, for example `/var/www/html`.
|
||||
|
||||
Examples:
|
||||
|
||||
```text
|
||||
/style.css
|
||||
/images/logo.png
|
||||
/examples/uce-starter/js/site.js
|
||||
```
|
||||
|
||||
These should not touch the UCE runtime.
|
||||
|
||||
### Normal `.uce` page requests
|
||||
|
||||
For a request such as:
|
||||
|
||||
```text
|
||||
GET /doc/index.uce?p=component
|
||||
```
|
||||
|
||||
nginx/Apache forwards the request to `FCGI_SOCKET_PATH` as FastCGI. The web server must provide CGI/FastCGI variables including:
|
||||
|
||||
- `SCRIPT_FILENAME` — full filesystem path to the `.uce` file.
|
||||
- `DOCUMENT_ROOT` — public web root, normally `<runtime-root>/site`.
|
||||
- `SCRIPT_NAME` — URL path to the script, such as `/doc/index.uce`.
|
||||
- `DOCUMENT_URI` — normalized URI path without query string.
|
||||
- `REQUEST_URI` — original request URI including query string.
|
||||
- standard request variables such as method, query string, content type, body length, cookies, and headers.
|
||||
|
||||
UCE resolves the unit, compiles it to wasm if needed, creates a request workspace, and calls:
|
||||
|
||||
```cpp
|
||||
RENDER(Request& context)
|
||||
```
|
||||
|
||||
The unit writes output with template literals or `print()`. Response headers and status are set through `context.header` and `context.set_status()`.
|
||||
|
||||
### Component and sub-render calls
|
||||
|
||||
Inside a request, UCE code can call other units:
|
||||
|
||||
```cpp
|
||||
component("components/card", props, context);
|
||||
unit_render("other-page.uce", context);
|
||||
```
|
||||
|
||||
These calls stay inside the UCE runtime. They are not new HTTP requests and do not go back through nginx or Apache.
|
||||
|
||||
### WebSocket pages
|
||||
|
||||
Any `.uce` unit can provide both an ordinary page render and WebSocket message handling:
|
||||
|
||||
```cpp
|
||||
RENDER(Request& context) { ... } // normal page load
|
||||
WS(Request& context) { ... } // later WebSocket messages
|
||||
```
|
||||
|
||||
The nginx and Apache examples below split traffic by checking for a WebSocket upgrade request on `.uce` paths. A file such as `chat.uce` or `events.uce` can expose `WS(Request& context)`.
|
||||
|
||||
Routing split:
|
||||
|
||||
- Plain `GET /demo/chat.uce` should use FastCGI, just like any other page render.
|
||||
- WebSocket upgrade requests for `/demo/chat.uce` should proxy to the UCE built-in HTTP/WebSocket listener at `HTTP_PORT`.
|
||||
|
||||
The built-in listener owns the socket lifecycle. When a message arrives, the broker forwards a render-style invocation back to the worker pool so `WS(Request& context)` runs inside the same wasm runtime model as normal pages.
|
||||
|
||||
### CLI requests
|
||||
|
||||
`CLI(Request& context)` handlers are not public web endpoints. They are invoked over `CLI_SOCKET_PATH`:
|
||||
|
||||
```bash
|
||||
scripts/uce-cli /tests/cli.uce action=echo message=hello
|
||||
curl --unix-socket /run/uce/cli.sock http://localhost/tests/cli.uce
|
||||
```
|
||||
|
||||
Use CLI units for local tests, admin commands, and maintenance tools. Do not expose the CLI socket through nginx or Apache.
|
||||
|
||||
### Custom runtime HTTP servers
|
||||
|
||||
UCE code can start local custom HTTP listeners with `server_start_http()`. Those are runtime-managed listeners for app-specific local services. They are separate from the public nginx/Apache entry point and should be firewalled or bound locally unless you explicitly want them reachable.
|
||||
|
||||
## nginx configuration
|
||||
|
||||
### Required modules
|
||||
|
||||
A normal nginx build includes the needed FastCGI and proxy modules. Confirm nginx is installed and can load your config:
|
||||
|
||||
```bash
|
||||
nginx -t
|
||||
```
|
||||
|
||||
### WebSocket upgrade map
|
||||
|
||||
Put this in the nginx `http` block if using WebSockets:
|
||||
|
||||
```nginx
|
||||
map $http_upgrade $connection_upgrade {
|
||||
default upgrade;
|
||||
'' close;
|
||||
}
|
||||
```
|
||||
|
||||
### Server block
|
||||
|
||||
Example site config:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name example.com;
|
||||
|
||||
root /var/www/html;
|
||||
index index.uce index.html;
|
||||
|
||||
# Serve static files directly. Directory requests use index.uce when present.
|
||||
location / {
|
||||
try_files $uri $uri/ =404;
|
||||
}
|
||||
|
||||
# UCE page requests use FastCGI. If the client asks to upgrade a .uce
|
||||
# request to WebSocket, send that connection to the built-in listener.
|
||||
location ~ \.uce$ {
|
||||
error_page 418 = @uce_websocket;
|
||||
if ($http_upgrade = "websocket") {
|
||||
return 418;
|
||||
}
|
||||
|
||||
include fastcgi_params;
|
||||
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
|
||||
fastcgi_param DOCUMENT_ROOT $document_root;
|
||||
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
|
||||
fastcgi_param DOCUMENT_URI $uri;
|
||||
fastcgi_param REQUEST_URI $request_uri;
|
||||
fastcgi_pass unix:/run/uce/fastcgi.sock;
|
||||
}
|
||||
|
||||
location @uce_websocket {
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $connection_upgrade;
|
||||
proxy_pass http://127.0.0.1:8080;
|
||||
}
|
||||
|
||||
# Defense in depth if the root is changed later.
|
||||
location ~ ^/(src|scripts|etc|bin|work|dist|pkg|docs|changelog)/ {
|
||||
return 404;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `fastcgi_pass` must match `FCGI_SOCKET_PATH`.
|
||||
- `proxy_pass` must match `HTTP_PORT`.
|
||||
- The example routes WebSocket upgrades for `.uce` paths to the HTTP/WebSocket listener.
|
||||
- Ordinary `.uce` page loads continue to use FastCGI.
|
||||
- Keep `root` pointed at `/var/www/html`, not the runtime repository root.
|
||||
- If your app uses a front controller, replace `location /` with a `try_files` rule that ends at `/index.uce`.
|
||||
|
||||
Front-controller variant:
|
||||
|
||||
```nginx
|
||||
location / {
|
||||
try_files $uri $uri/ /index.uce?$query_string;
|
||||
}
|
||||
```
|
||||
|
||||
Reload nginx:
|
||||
|
||||
```bash
|
||||
nginx -t
|
||||
systemctl reload nginx
|
||||
```
|
||||
|
||||
## Apache configuration
|
||||
|
||||
Apache can run UCE through `mod_proxy_fcgi` for FastCGI and `mod_proxy_wstunnel` or `mod_proxy_http` for WebSocket upgrades.
|
||||
|
||||
### Enable modules
|
||||
|
||||
On Debian/Ubuntu:
|
||||
|
||||
```bash
|
||||
a2enmod proxy proxy_fcgi proxy_http proxy_wstunnel rewrite headers setenvif
|
||||
systemctl restart apache2
|
||||
```
|
||||
|
||||
### VirtualHost example
|
||||
|
||||
```apache
|
||||
<VirtualHost *:80>
|
||||
ServerName example.com
|
||||
DocumentRoot /var/www/html
|
||||
|
||||
<Directory /var/www/html>
|
||||
Require all granted
|
||||
Options FollowSymLinks
|
||||
AllowOverride None
|
||||
DirectoryIndex index.uce index.html
|
||||
</Directory>
|
||||
|
||||
# Do not expose repository internals if DocumentRoot changes later.
|
||||
<LocationMatch "^/(src|scripts|etc|bin|work|dist|pkg|docs|changelog)/">
|
||||
Require all denied
|
||||
</LocationMatch>
|
||||
|
||||
RewriteEngine On
|
||||
|
||||
# WebSocket upgrade traffic for any .uce unit goes to UCE's built-in HTTP listener.
|
||||
RewriteCond %{HTTP:Upgrade} =websocket [NC]
|
||||
RewriteCond %{REQUEST_URI} \.uce(?:\?|$) [NC]
|
||||
RewriteRule ^/(.*)$ ws://127.0.0.1:8080/$1 [P,L]
|
||||
|
||||
# Normal .uce page loads go to FastCGI.
|
||||
<FilesMatch "\.uce$">
|
||||
SetHandler "proxy:unix:/run/uce/fastcgi.sock|fcgi://localhost/"
|
||||
</FilesMatch>
|
||||
|
||||
# Optional: make the key CGI variables explicit for UCE.
|
||||
ProxyFCGISetEnvIf "true" DOCUMENT_ROOT "/var/www/html"
|
||||
ProxyFCGISetEnvIf "true" DOCUMENT_URI "%{REQUEST_URI}"
|
||||
</VirtualHost>
|
||||
```
|
||||
|
||||
Apache notes:
|
||||
|
||||
- `SetHandler "proxy:unix:/run/uce/fastcgi.sock|fcgi://localhost/"` must use the same socket path as `FCGI_SOCKET_PATH`.
|
||||
- The WebSocket rewrite rule must run before the FastCGI handler.
|
||||
- Plain `.uce` page loads should not be proxied as WebSockets unless the client sends `Upgrade: websocket`.
|
||||
- Apache's FastCGI environment differs by version and module configuration. If UCE cannot resolve a page, inspect the request environment and make sure `SCRIPT_FILENAME` points to the target file under the web root.
|
||||
|
||||
If your Apache version does not populate `SCRIPT_FILENAME` correctly through `SetHandler`, use `ProxyPassMatch` for `.uce` files instead:
|
||||
|
||||
```apache
|
||||
ProxyPassMatch ^/(.*\.uce)$ unix:/run/uce/fastcgi.sock|fcgi://localhost/var/www/html/$1
|
||||
```
|
||||
|
||||
Use only one FastCGI mapping style at a time (`SetHandler` or `ProxyPassMatch`) to avoid duplicate routing.
|
||||
|
||||
## Permissions
|
||||
|
||||
The web server needs permission to connect to `/run/uce/fastcgi.sock`. Common approaches:
|
||||
|
||||
- run UCE and the web server under compatible groups;
|
||||
- add the web server user (`www-data` on Debian/Ubuntu) to the socket's group;
|
||||
- adjust the service or runtime socket mode if needed.
|
||||
|
||||
The runtime creates the FastCGI socket and CLI socket under `/run/uce`. The CLI socket should remain local-only and should not be reachable from the public web server.
|
||||
|
||||
Writable paths for the runtime:
|
||||
|
||||
```text
|
||||
/var/cache/uce/work
|
||||
/var/lib/uce/uploads
|
||||
/var/lib/uce/sessions
|
||||
/run/uce
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
Check service state:
|
||||
|
||||
```bash
|
||||
systemctl status uce.service
|
||||
journalctl -u uce.service -n 100 --no-pager
|
||||
```
|
||||
|
||||
Check the FastCGI/web-server path:
|
||||
|
||||
```bash
|
||||
curl -i http://127.0.0.1/doc/index.uce -H 'Host: example.com'
|
||||
curl -i http://127.0.0.1/examples/uce-starter/ -H 'Host: example.com'
|
||||
```
|
||||
|
||||
Check the local CLI path:
|
||||
|
||||
```bash
|
||||
cd /opt/uce
|
||||
scripts/uce-cli /tests/cli.uce action=echo message=hello
|
||||
scripts/run_cli_tests.sh
|
||||
```
|
||||
|
||||
Check WebSocket routing with a WebSocket client against a `.uce` endpoint that defines `WS(Request& context)` through nginx/Apache, not directly against `HTTP_PORT`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### 502 Bad Gateway
|
||||
|
||||
Check:
|
||||
|
||||
- `systemctl status uce.service`
|
||||
- `journalctl -u uce.service -n 200 --no-pager`
|
||||
- socket path in web server config equals `FCGI_SOCKET_PATH`
|
||||
- web server user can connect to the Unix socket
|
||||
- `SCRIPT_FILENAME` resolves to an existing `.uce` file
|
||||
|
||||
### Raw `.uce` source is downloaded or displayed
|
||||
|
||||
The `.uce` request did not match the FastCGI rule. Check location/order rules and confirm the public root is `/var/www/html` or your chosen web-root path.
|
||||
|
||||
### Static files 404
|
||||
|
||||
Confirm the web server `root`/`DocumentRoot` is `/var/www/html` or your chosen web-root path and that `location /` or Apache directory rules allow static file reads.
|
||||
|
||||
### WebSocket page renders but upgrade fails
|
||||
|
||||
Check:
|
||||
|
||||
- the client sends `Upgrade: websocket`
|
||||
- `.uce` upgrade traffic reaches `HTTP_PORT`
|
||||
- nginx/Apache preserves `Upgrade` and `Connection` headers
|
||||
- firewall/network policy allows localhost access to `HTTP_PORT`
|
||||
|
||||
### Page compiles fail
|
||||
|
||||
Check the compile artifact paths shown in the UCE error response and service logs. Generated files and compile output live under `BIN_DIRECTORY`. A safe last-resort recovery step is to stop UCE, move the affected generated artifact directory aside, and restart so the runtime recompiles from source.
|
||||
|
||||
### CLI commands fail
|
||||
|
||||
Check:
|
||||
|
||||
- `CLI_SOCKET_PATH` in `/etc/uce/settings.cfg`
|
||||
- `/run/uce/cli.sock` exists
|
||||
- `scripts/uce-cli --socket /run/uce/cli.sock /ping` works
|
||||
- the target unit defines `CLI(Request& context)`
|
||||
@@ -1,110 +0,0 @@
|
||||
# WASM Phase 1: DValue C ABI and UCEB1
|
||||
|
||||
Phase 1 freezes the native DValue ABI that the future WASM core and units use
|
||||
as their shared structured-value contract. The implementation is in
|
||||
`src/lib/dvalue.{h,cpp}` and is available in the native runtime before any WASM
|
||||
backend is enabled.
|
||||
|
||||
## Opaque handle
|
||||
|
||||
```c
|
||||
typedef struct DValue uce_dvalue;
|
||||
```
|
||||
|
||||
`uce_dvalue*` is a borrowed pointer owned by the active request/workspace. It
|
||||
must not be freed by ABI callers and it must not be retained beyond that
|
||||
workspace lifetime.
|
||||
|
||||
## Accessors
|
||||
|
||||
```c
|
||||
uce_dvalue* uce_dv_root(void);
|
||||
uce_dvalue* uce_dv_get(uce_dvalue* value, const char* key, size_t key_len);
|
||||
uce_dvalue* uce_dv_find(uce_dvalue* value, const char* key, size_t key_len);
|
||||
const char* uce_dv_value(uce_dvalue* value, size_t* len_out);
|
||||
void uce_dv_set_value(uce_dvalue* value, const char* bytes, size_t len);
|
||||
size_t uce_dv_count(uce_dvalue* value);
|
||||
int uce_dv_is_list(uce_dvalue* value);
|
||||
```
|
||||
|
||||
- `uce_dv_root()` returns the current native request's `context.call` root.
|
||||
The WASM core will later map this to the decoded request context root.
|
||||
- `uce_dv_get()` creates the child if absent. `uce_dv_find()` returns `NULL`
|
||||
if absent.
|
||||
- String inputs and outputs are length-delimited and binary-safe.
|
||||
- `uce_dv_value()` returns a borrowed pointer valid until the next ABI value
|
||||
call on the same thread.
|
||||
- Bad `NULL` inputs return `NULL`, zero, or no-op rather than trapping.
|
||||
|
||||
## Iteration
|
||||
|
||||
```c
|
||||
typedef struct uce_dv_iter { size_t position; size_t reserved[3]; } uce_dv_iter;
|
||||
|
||||
uce_dv_iter uce_dv_iter_begin(uce_dvalue* value);
|
||||
int uce_dv_iter_next(uce_dvalue* value, uce_dv_iter* iter,
|
||||
const char** key_out, size_t* key_len_out,
|
||||
uce_dvalue** child_out);
|
||||
```
|
||||
|
||||
Map iteration follows DValue's native order. List-shaped maps iterate in numeric
|
||||
index order (`0`, `1`, ...), matching `DValue::each()`, `dv_values()`, and the
|
||||
serializers. The reserved iterator fields are caller-opaque and must be
|
||||
zero-preserved by callers that copy the iterator; they provide ABI headroom for
|
||||
future non-linear keyed-map iteration without changing the struct size.
|
||||
|
||||
## UCEB1 wire format
|
||||
|
||||
UCEB1 is the membrane/cross-instance binary DValue encoding.
|
||||
|
||||
```
|
||||
document := "UCEB" version node
|
||||
version := 0x01
|
||||
node := flags scalar children
|
||||
flags := u8 bitset; bit0 = list-shaped map
|
||||
scalar := varuint length, bytes
|
||||
children := varuint count, count * (key, node)
|
||||
key := varuint length, bytes
|
||||
```
|
||||
|
||||
Varuint is unsigned LEB128. Strings are byte sequences; the codec does not
|
||||
assume NUL termination and preserves embedded NUL bytes. The Phase 1 layout stores scalar values as their native string representation
|
||||
plus child nodes and the list-shape flag. Floating-point values use
|
||||
`max_digits10` precision so numeric scalars can round-trip through the string
|
||||
form. Pointer/reference identity is intentionally not part of the wire contract;
|
||||
pointer nodes encode as an empty scalar rather than leaking process addresses.
|
||||
An empty non-list map has no wire distinction from an empty scalar in UCEB1 v1.
|
||||
Documents that contain both scalar bytes and child nodes are reserved for future
|
||||
use; the v1 decoder accepts the children and ignores the scalar.
|
||||
|
||||
## Codec APIs
|
||||
|
||||
C++/UCE-visible helpers:
|
||||
|
||||
```cpp
|
||||
String ucb_encode(const DValue& value);
|
||||
DValue ucb_decode(const String& encoded);
|
||||
bool ucb_decode(const String& encoded, DValue& out, String* error_out = 0);
|
||||
```
|
||||
|
||||
C ABI helpers:
|
||||
|
||||
```c
|
||||
size_t uce_dv_encode(uce_dvalue* value, char* buf, size_t cap);
|
||||
uce_dvalue* uce_dv_decode(const char* buf, size_t len);
|
||||
const char* uce_dv_last_error(void);
|
||||
```
|
||||
|
||||
`uce_dv_encode()` returns the required byte length even when `buf` is `NULL` or
|
||||
`cap` is zero. `uce_dv_decode()` returns a thread-local decoded root, or `NULL`
|
||||
with `uce_dv_last_error()` populated. The returned decoded root is valid until
|
||||
the next `uce_dv_decode()` call on the same thread. Decoding rejects documents
|
||||
deeper than 1024 nested nodes so malformed input cannot recurse without bound.
|
||||
|
||||
## Test coverage
|
||||
|
||||
`site/tests/core.uce` covers:
|
||||
|
||||
- UCEB1 round-trip for maps, nested values, lists, empty lists, and embedded NUL
|
||||
scalar bytes.
|
||||
- C ABI get/find/value/count/list/iteration/encode/decode behavior.
|
||||
@@ -5,9 +5,7 @@ describes the **runtime architecture as built** — the process topology, the
|
||||
wasm membrane, the unified request dispatch, and the central WebSocket broker.
|
||||
Native `.so` unit execution/dlopen fallback has been removed; the parser and
|
||||
preprocessor remain only as the front-end that emits C++ for wasm side-module
|
||||
compilation. For the historical motivation and the phased migration plan, see
|
||||
[`WASM-PROPOSAL.md`](../WASM-PROPOSAL.md); this file is the steady-state
|
||||
reference that proposal points at.
|
||||
compilation.
|
||||
|
||||
The guiding principle: **request code never shares an address space or an
|
||||
allocator with the runtime.** Every unit runs as a WebAssembly module inside a
|
||||
@@ -55,9 +53,9 @@ gets invoked*.
|
||||
| **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` |
|
||||
| **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` |
|
||||
|
||||
The key invariant: **only workers instantiate Wasmtime and run unit code.**
|
||||
**only workers instantiate Wasmtime and run unit code.**
|
||||
Every connection-owning process (broker, serve_http dispatcher) forwards the
|
||||
real invocation back to a worker over `/run/uce.sock` using the minimal
|
||||
request invocation back to a worker over `/run/uce.sock` using the minimal
|
||||
FastCGI client in [`src/lib/fcgi_forward.h`](../src/lib/fcgi_forward.h). This is
|
||||
forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across
|
||||
`fork()`, and the brokers fork from the parent that already touched the
|
||||
@@ -163,7 +161,7 @@ otherwise (page) → serve_via_wasm(entry_unit, "render")
|
||||
|
||||
The `UCE_*` params are set by whichever broker forwarded the request:
|
||||
|
||||
- **Page render**: nginx → `/run/uce.sock` directly; no `UCE_*` flags → `render`.
|
||||
- **Page render**: FastCGI nginx → `/run/uce.sock` directly; no `UCE_*` flags → `render`.
|
||||
- **CLI**: the CLI socket sets `is_cli`.
|
||||
- **serve_http**: the custom-server dispatcher sets `UCE_SERVE_HTTP=1` plus
|
||||
`UCE_SERVE_HTTP_FUNCTION` and rewrites `SCRIPT_FILENAME` to the configured
|
||||
@@ -181,21 +179,12 @@ code.
|
||||
|
||||
## 6. The central WebSocket broker
|
||||
|
||||
The broker is the architectural centerpiece. **One process owns the HTTP port
|
||||
**One process owns the HTTP port
|
||||
and every WebSocket connection**, so any unit's `ws_*` call can reach any or all
|
||||
connections, and a unit-code crash (which happens in a worker) never drops live
|
||||
connections.
|
||||
|
||||
### 6.1 Why central, and why it forwards
|
||||
|
||||
Earlier designs gave each worker its connections and captured `ws_*` output to
|
||||
return inline. That was wrong: connections couldn't outlive a worker, and one
|
||||
unit could only talk to connections it happened to own. The broker fixes both —
|
||||
it is the sole connection registry and data broker between connected clients and
|
||||
the units that handle them. It renders nothing itself; like every other
|
||||
connection owner it forwards unit invocation to the worker pool.
|
||||
|
||||
### 6.2 Inbound: a WS frame → a worker render (non-blocking)
|
||||
### 6.1 Inbound: a WS frame → a worker render (non-blocking)
|
||||
|
||||
`ws_broker_ws_message(request, message, opcode)` fires when a complete
|
||||
(reassembled) WS message arrives on a connection. It does **not** block the
|
||||
@@ -216,7 +205,7 @@ writing each queued request, then drains and discards the reply (the unit's
|
||||
output comes back via the command socket, not this reply), closing the fd when
|
||||
the worker closes its end.
|
||||
|
||||
### 6.3 Outbound: `ws_*` commands flushed back to the broker
|
||||
### 6.2 Outbound: `ws_*` commands flushed back to the broker
|
||||
|
||||
Any unit code — not just WebSocket handlers — may call `ws_send` / `ws_send_to`
|
||||
/ `ws_close`. In the workspace these **record dispatch commands** rather than
|
||||
@@ -236,14 +225,14 @@ the full registry it owns: `broadcast` (by scope), `send_to` (by connection id),
|
||||
`close`. If the batch carries `connection_state`, it persists that onto the
|
||||
matching live connection's `websocket_state`.
|
||||
|
||||
### 6.4 Un-upgraded HTTP on the WS port
|
||||
### 6.3 Un-upgraded HTTP on the WS port
|
||||
|
||||
The WS port can also receive ordinary (non-Upgrade) HTTP requests.
|
||||
`ws_broker_complete()` routes by param: `UCE_WS_DISPATCH=1` → apply commands;
|
||||
otherwise → `forward_request_to_worker()` — the *same* shared facility the
|
||||
serve_http dispatcher uses, so there is no duplicated request-forwarding code.
|
||||
|
||||
### 6.5 The broker loop
|
||||
### 6.4 The broker loop
|
||||
|
||||
`run_ws_broker()` drops the worker listeners it inherited
|
||||
(`close_inherited_server_sockets`), installs permissive `on_request`/`on_data`
|
||||
@@ -255,9 +244,7 @@ command socket, and loops `process(50)` + `drain_outbound()`. The design is
|
||||
broker's single epoll loop** — the broker never blocks on a worker.
|
||||
|
||||
The parent respawns the broker if it dies (`ws_broker_alive` / `ensure_ws_broker`
|
||||
in `main()`). A broker restart loses live connections (acceptable: crashes
|
||||
happen in workers, so the broker stays up in practice), but no unit state is at
|
||||
risk because the broker holds none.
|
||||
in `main()`).
|
||||
|
||||
---
|
||||
|
||||
@@ -309,11 +296,10 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
|
||||
- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
|
||||
in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages.
|
||||
Current baseline: **87 passed, 0 failed**.
|
||||
|
||||
- **WebSocket end-to-end**: a headless client performs a raw WS handshake to
|
||||
`:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
|
||||
`SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
|
||||
broker → worker → broker → client chain across process boundaries.
|
||||
|
||||
All builds, runs, and installs happen on the dev host (`k-uce` / uce-dev) over
|
||||
SSH; the local checkout is edit-only.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user