W7e: delete the native unit pipeline (.so compile + dlopen execution)

Units now run exclusively on wasm; the native generated-C++ -> clang -> .so ->
dlopen path and its request-time fallback are removed.

- Dispatch (linux_fastcgi.cpp): the 4 handle_complete branches + the CLI-socket
  path route every request through wasm (wasm_ready compiles cold/stale on
  demand); a wasm-unavailable unit now yields a clean error page
  (fail_wasm_unavailable / render_request_failure) instead of native execution.
  compiler_invoke / _cli / _websocket / _serve_http deleted.
- compiler.cpp (-1274): removed the native .so compile (COMPILE_SCRIPT),
  load_shared_unit, dlopen/dlsym/dlclose, compiler_load_shared_unit, and the
  SharedUnit .so function-pointer fields (on_setup/on_render/on_component/
  on_websocket/on_cli/on_once/on_init) in types.h/types.cpp. compile_shared_unit
  now builds only the .wasm side-module; the .uce preprocessor/parser front-end
  is kept (it emits the C++ the wasm compile consumes).
- unit_call()/component()/once/init now resolve across units through the wasm
  host component resolver (uce_host_component_resolve) instead of native dlsym;
  configured runtime error pages render through the wasm backend.
- Dropped the WASM_BACKEND_ENABLED feature flag and dead COMPILE_SCRIPT /
  COMPILE_WASM_UNITS config; unit ABI freshness tied to UCE_UNIT_ABI_VERSION;
  guard against serving a stale .wasm for a deleted source; retired the obsolete
  W5 native-vs-wasm toggle script. Docs updated.

Implemented via the pi agent (with 3 delegated sub-reviews); independently
re-verified on the build host: run_cli_tests --include-wasm-kill => 87 passed,
0 failed, 0 skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
root
2026-06-14 23:20:50 +00:00
co-authored by Claude Opus 4.8
parent fb728d63bc
commit cf51336873
16 changed files with 352 additions and 1344 deletions
+1 -1
View File
@@ -10,7 +10,7 @@ Replace the Python-based network test runner and plugins with UCE unit tests inv
- [x] UCE CLI tests cover every current Python plugin behavior or explicitly document a deliberate replacement. - [x] UCE CLI tests cover every current Python plugin behavior or explicitly document a deliberate replacement.
- [x] Existing W5/WASM gates use the UCE CLI test runner instead of `tests/run_network_tests.py`. - [x] Existing W5/WASM gates use the UCE CLI test runner instead of `tests/run_network_tests.py`.
- [x] Python test runner/plugins are deleted after parity validation. - [x] Python test runner/plugins are deleted after parity validation.
- [x] Full suite passes on `uce-dev` with `WASM_BACKEND_ENABLED=1`. - [x] Full suite passes on `uce-dev` with wasm-only unit execution.
## Current State ## Current State
+14 -13
View File
@@ -1,9 +1,11 @@
# UCE WASM Runtime Architecture # UCE WASM Runtime Architecture
Status: current as of the W7 cutover (June 2026). This document describes the Status: current as of the W7e native-pipeline removal (June 2026). This document
**runtime architecture as built** — the process topology, the wasm membrane, describes the **runtime architecture as built** — the process topology, the
the unified request dispatch, and the central WebSocket broker. For the wasm membrane, the unified request dispatch, and the central WebSocket broker.
historical motivation and the phased migration plan, see Native `.so` unit execution/dlopen fallback has been removed; the parser and
preprocessor remain only as the front-end that emits C++ for wasm side-module
compilation. For the historical motivation and the phased migration plan, see
[`WASM-PROPOSAL.md`](../WASM-PROPOSAL.md); this file is the steady-state [`WASM-PROPOSAL.md`](../WASM-PROPOSAL.md); this file is the steady-state
reference that proposal points at. reference that proposal points at.
@@ -116,10 +118,10 @@ __uce_<base>[_<sanitize(suffix)>]
path and looks up the export's funcref slot; `exists` lets callers probe a unit path and looks up the export's funcref slot; `exists` lets callers probe a unit
without instantiating it. without instantiating it.
`wasm_backend_should_handle(request, entry_unit)` gates whether the wasm path `wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
takes the request (vs. the legacy native compiler path, still selectable while backend is initialized and the requested artifact/handler is currently
the native pipeline is retired). With `WASM_BACKEND_ENABLED=1` this is the available. If an artifact is cold or stale, dispatch compiles it on demand via
default for all unit execution. `get_shared_unit()` and rechecks. There is no native unit-execution fallback.
--- ---
@@ -171,9 +173,9 @@ The `UCE_*` params are set by whichever broker forwarded the request:
`request.resources.websocket_*` and `request.connection` from them before `request.resources.websocket_*` and `request.connection` from them before
invoking `__uce_websocket`. invoking `__uce_websocket`.
Each branch has a native-compiler fallback (`compiler_invoke*`) used only when If a unit still cannot be served after the on-demand wasm compile, the worker
`wasm_backend_should_handle` declines — the holdout path during native returns a clean 500 with the wasm/compile error; it does not execute native unit
retirement. code.
--- ---
@@ -294,7 +296,6 @@ header free-functions are `inline`. The wasm backend exposes only declarations
| Key | Default | Meaning | | Key | Default | Meaning |
|---|---|---| |---|---|---|
| `WASM_BACKEND_ENABLED` | `1` | Route unit execution through the wasm path. |
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). | | `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
| `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). | | `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). |
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. | | `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. |
@@ -308,7 +309,7 @@ header free-functions are `inline`. The wasm backend exposes only declarations
- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the - **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages. in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages.
Current baseline: **86 passed, 0 failed**. Current baseline: **87 passed, 0 failed**.
- **WebSocket end-to-end**: a headless client performs a raw WS handshake to - **WebSocket end-to-end**: a headless client performs a raw WS handshake to
`:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving `:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
`SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full `SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
+2 -5
View File
@@ -21,13 +21,10 @@ SITE_DIRECTORY=site
# ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT # ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT
JIT_COMPILE_ON_REQUEST=1 JIT_COMPILE_ON_REQUEST=1
# OPTIONAL W2 WEBASSEMBLY SIDE-MODULE COMPILATION BESIDE NATIVE .so UNITS # WASM SIDE-MODULE COMPILER USED FOR .uce UNITS
COMPILE_WASM_UNITS=0
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
# DEFAULT PAGE RENDER BACKEND. W5 defaults to the WASM backend with explicit # WASM RUNTIME SETTINGS. Unit execution is always routed through wasm.
# native fallbacks for host-owned surfaces that are not membrane APIs yet.
WASM_BACKEND_ENABLED=1
WASM_BACKEND_VERBOSE=0 WASM_BACKEND_VERBOSE=0
WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm
WASM_MEMORY_LIMIT_BYTES=536870912 WASM_MEMORY_LIMIT_BYTES=536870912
+12 -4
View File
@@ -9,11 +9,11 @@ GF="uce_fastcgi"
mkdir bin > /dev/null 2>&1 mkdir bin > /dev/null 2>&1
mkdir bin/tmp > /dev/null 2>&1 mkdir bin/tmp > /dev/null 2>&1
mkdir bin/assets > /dev/null 2>&1 mkdir bin/assets > /dev/null 2>&1
mkdir bin/wasm > /dev/null 2>&1
mkdir work > /dev/null 2>&1 mkdir work > /dev/null 2>&1
COMPILER="clang++" COMPILER="clang++"
# -rdynamic is a link-time flag (exports the binary's symbols so dlopen'd .uce # -rdynamic is a link-time flag; the -c compiles below do not need it.
# units resolve the runtime against it); the -c compiles below do not need it.
FLAGS="-g -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math" FLAGS="-g -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math"
# Wasmtime C++ API — needed only by the wasm backend object (src/wasm). # Wasmtime C++ API — needed only by the wasm backend object (src/wasm).
@@ -30,8 +30,8 @@ SRCFLAGS="-D EXEC_NAME=\"$GF\" -D PLATFORM_NAME=\"linux\""
# bin/sqlite3.o vendored SQLite amalgamation (depends only on its own source) # bin/sqlite3.o vendored SQLite amalgamation (depends only on its own source)
# bin/wasm.o wasm backend + worker + wasmtime.hh (src/wasm) # bin/wasm.o wasm backend + worker + wasmtime.hh (src/wasm)
# bin/main.o linux_fastcgi.cpp + the uce_lib core amalgamation # bin/main.o linux_fastcgi.cpp + the uce_lib core amalgamation
# All link into the single -rdynamic binary, so the dlopen unit model is # All link into the single -rdynamic binary. Delete bin/*.o to force a clean
# unchanged. Delete bin/*.o to force a clean rebuild. # rebuild.
# Rebuild $1 if it is missing or anything under the remaining find-args is newer. # Rebuild $1 if it is missing or anything under the remaining find-args is newer.
needs_rebuild() { needs_rebuild() {
@@ -41,6 +41,14 @@ needs_rebuild() {
return 1 return 1
} }
# core.wasm: guest runtime loaded by the native wasm backend.
if needs_rebuild bin/wasm/core.wasm src/wasm/core.cpp src/lib src/wasm/core_hostcalls.syms src/wasm/core_libc_exports.syms scripts/build_core_wasm.sh; then
echo "Compiling wasm core..."
bash scripts/build_core_wasm.sh || exit 1
else
echo "Reusing bin/wasm/core.wasm"
fi
# SQLite: vendored C, depends only on its own source (not our headers). # SQLite: vendored C, depends only on its own source (not our headers).
if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then
echo "Compiling SQLite..." echo "Compiling SQLite..."
+1 -1
View File
@@ -50,7 +50,7 @@ build_pch_if_needed() {
return 0 return 0
fi fi
mkdir -p "$PCH_DIR" mkdir -p "$PCH_DIR"
if [ -s "$PCH_FN" ]; then if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then
return 0 return 0
fi fi
"$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \ "$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \
+15 -90
View File
@@ -1,94 +1,19 @@
#!/bin/bash #!/bin/bash
# W5 parity/performance gate for the config-selectable WASM backend. # Retired W5 parity/performance gate.
# Runs on k-uce. Requires root because the live service reads /etc/uce/settings.cfg. #
# W7e removed the config-selectable native backend and all native unit
# execution. There is no longer a valid WASM_BACKEND_ENABLED=0 baseline to
# switch to here; doing so would only create a misleading service state.
set -euo pipefail set -euo pipefail
cd "$(dirname "$0")/../.." cat >&2 <<'EOF'
run_w5.sh is retired: UCE unit execution is wasm-only after W7e.
Use the supported regression gate instead:
if [ "${EUID:-$(id -u)}" -ne 0 ]; then bash scripts/build_linux.sh && systemctl restart uce.service && sleep 3 \
echo "run_w5.sh must run as root so it can switch /etc/uce/settings.cfg and restart uce.service" >&2 && bash scripts/run_cli_tests.sh --include-wasm-kill
exit 1
fi
OUT=${UCE_W5_OUT:-/tmp/uce/wasm-w5} For performance comparisons, use tests/wasm_benchmark.py against the current
CONFIG=${UCE_CONFIG:-/etc/uce/settings.cfg} wasm backend; native-baseline artifacts under docs/wasm-baselines are historical
mkdir -p "$OUT" only.
BACKUP="$OUT/settings.cfg.before-w5" EOF
cp "$CONFIG" "$BACKUP" exit 2
restore_on_error() {
if [ "${UCE_W5_KEEP_BACKEND:-0}" != "1" ]; then
cp "$BACKUP" "$CONFIG"
systemctl restart uce.service >/dev/null 2>&1 || true
fi
}
trap restore_on_error EXIT
set_backend() {
local enabled="$1"
local tmp
tmp=$(mktemp)
awk -F= -v enabled="$enabled" '
BEGIN { found = 0 }
/^WASM_BACKEND_ENABLED=/ { print "WASM_BACKEND_ENABLED=" enabled; found = 1; next }
{ print }
END { if(!found) print "WASM_BACKEND_ENABLED=" enabled }
' "$CONFIG" > "$tmp"
for kv in \
'WASM_BACKEND_VERBOSE=0' \
'WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm' \
'WASM_MEMORY_LIMIT_BYTES=536870912' \
'WASM_EPOCH_DEADLINE_TICKS=200' \
'WASM_EPOCH_PERIOD_MS=50'
do
key=${kv%%=*}
if ! grep -q "^${key}=" "$tmp"; then
printf '%s\n' "$kv" >> "$tmp"
fi
done
cat "$tmp" > "$CONFIG"
rm -f "$tmp"
systemctl restart uce.service >/dev/null
sleep 1
}
summary_total() {
awk '/^Summary:/ { print $2; found=1 } END { if(!found) print 0 }' "$1"
}
# Native reference baseline.
set_backend 0
scripts/run_cli_tests.sh > "$OUT/native-warmup.txt" || true
scripts/run_cli_tests.sh > "$OUT/native-network.txt"
python3 tests/wasm_benchmark.py --out-dir "$OUT/native-benchmark" --samples "${UCE_W5_BENCH_SAMPLES:-20}" --timeout 30 >/dev/null
# WASM default backend with W5 native fallbacks for host-owned surfaces.
set_backend 1
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-warmup.txt" || true
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-network.txt"
python3 tests/wasm_benchmark.py \
--out-dir "$OUT/benchmark" \
--backend-label wasm \
--compare-native-json "$OUT/native-benchmark/benchmark.json" \
--samples "${UCE_W5_BENCH_SAMPLES:-20}" \
--timeout 30
python3 tests/wasm_site_audit.py --out-dir "$OUT" >/dev/null
network_cases=$(summary_total "$OUT/wasm-network.txt")
if [ "$network_cases" -lt 80 ]; then
echo "W5 HARNESS: FAIL"
echo "wasm network ran $network_cases cases, expected at least 80"
exit 1
fi
echo 'W5 HARNESS: PASS'
echo "network_cases=$network_cases benchmark_rows=see benchmark.json"
echo "reports=$OUT"
if [ "${UCE_W5_KEEP_BACKEND:-0}" = "1" ]; then
trap - EXIT
printf 'WASM backend left enabled in %s\n' "$CONFIG"
else
cp "$BACKUP" "$CONFIG"
systemctl restart uce.service >/dev/null
trap - EXIT
fi
+4 -1
View File
@@ -564,7 +564,10 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
parsed_content.resize(parsed_content.length() - current_line.length()); parsed_content.resize(parsed_content.length() - current_line.length());
nibble(current_line, "\""); nibble(current_line, "\"");
String unit_name = nibble(current_line, "\""); String unit_name = nibble(current_line, "\"");
SharedUnit* sub_su = compiler_load_shared_unit(context, unit_name, su->src_path, true); String resolved_unit = unit_name;
if(resolved_unit != "" && resolved_unit[0] != '/')
resolved_unit = expand_path(resolved_unit, su->src_path);
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit(context, resolved_unit, true));
if(sub_su) if(sub_su)
parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n"); parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n");
} }
+156 -1118
View File
File diff suppressed because it is too large Load Diff
-7
View File
@@ -13,16 +13,9 @@ String preprocess_shared_unit(Request* context, SharedUnit* su);
String compiler_generated_cpp_path(Request* context, String source_file); String compiler_generated_cpp_path(Request* context, String source_file);
String compiler_generated_cpp_path(SharedUnit* su); String compiler_generated_cpp_path(SharedUnit* su);
void setup_unit_paths(Request* context, SharedUnit* su, String file_name); void setup_unit_paths(Request* context, SharedUnit* su, String file_name);
void load_shared_unit(Request* context, SharedUnit* su);
void compile_shared_unit(Request* context, SharedUnit* su); void compile_shared_unit(Request* context, SharedUnit* su);
SharedUnit* get_shared_unit(Request* context, String file_name, bool opt_so_optional = false); SharedUnit* get_shared_unit(Request* context, String file_name, bool opt_so_optional = false);
void compiler_invoke(Request* context, String file_name);
void compiler_invoke_websocket(Request* context, String file_name);
void compiler_invoke_cli(Request* context, String file_name);
void compiler_invoke_serve_http(Request* context, String file_name, String handler_name = "");
SharedUnit* compiler_load_shared_unit(Request* context, String file_name, String current_path = "", bool opt_so_optional = false);
String compiler_error_page_unit(Request* context, String config_key); String compiler_error_page_unit(Request* context, String config_key);
bool compiler_render_error_page(Request* context, String config_key, s32 status_code, String status_reason, DValue error_info);
bool compiler_unit_compile_pending(Request* context, String file_name); bool compiler_unit_compile_pending(Request* context, String file_name);
String compiler_site_directory(Request* context); String compiler_site_directory(Request* context);
StringList compiler_scan_site_units(Request* context); StringList compiler_scan_site_units(Request* context);
-3
View File
@@ -1314,10 +1314,7 @@ StringMap make_server_settings()
StringMap cfg; StringMap cfg;
cfg["BIN_DIRECTORY"] = "/tmp/uce/work"; cfg["BIN_DIRECTORY"] = "/tmp/uce/work";
cfg["COMPILE_SCRIPT"] = "scripts/compile";
cfg["WASM_COMPILE_SCRIPT"] = "scripts/compile_wasm_unit"; cfg["WASM_COMPILE_SCRIPT"] = "scripts/compile_wasm_unit";
cfg["COMPILE_WASM_UNITS"] = "0";
cfg["WASM_BACKEND_ENABLED"] = "1";
cfg["WASM_BACKEND_VERBOSE"] = "0"; cfg["WASM_BACKEND_VERBOSE"] = "0";
cfg["WASM_CORE_PATH"] = ""; cfg["WASM_CORE_PATH"] = "";
cfg["WASM_MEMORY_LIMIT_BYTES"] = std::to_string(512ull * 1024 * 1024); cfg["WASM_MEMORY_LIMIT_BYTES"] = std::to_string(512ull * 1024 * 1024);
-6
View File
@@ -79,12 +79,6 @@ String http_status_reason(s32 code)
SharedUnit::~SharedUnit() SharedUnit::~SharedUnit()
{ {
#ifndef __UCE_WASM_CORE__
if(so_handle)
{
dlclose(so_handle);
}
#endif
} }
String nibble(String div, String& haystack) String nibble(String div, String& haystack)
-13
View File
@@ -74,7 +74,6 @@ struct SharedUnit {
String compile_output_file_name; String compile_output_file_name;
String setup_file_name; String setup_file_name;
StringList api_declarations; StringList api_declarations;
std::map<String, void*> api_functions;
String src_path; String src_path;
String bin_path; String bin_path;
@@ -84,16 +83,6 @@ struct SharedUnit {
String wasm_file_name; String wasm_file_name;
String pre_file_name; String pre_file_name;
void* so_handle = 0;
request_handler on_setup = 0;
request_ref_handler on_render = 0;
request_ref_handler on_component = 0;
request_ref_handler on_websocket = 0;
request_ref_handler on_cli = 0;
request_ref_handler on_once = 0;
request_ref_handler on_init = 0;
String compiler_messages; String compiler_messages;
String compile_status = "unknown"; String compile_status = "unknown";
String compile_error_status = ""; String compile_error_status = "";
@@ -153,8 +142,6 @@ String nibble(String div, String& haystack);
#include "dvalue.h" #include "dvalue.h"
void compiler_invoke(Request* context, String file_name);
struct Request { struct Request {
ServerState* server = 0; ServerState* server = 0;
+67 -29
View File
@@ -65,6 +65,51 @@ void clear_request_output(Request& request)
request.ob_start(); request.ob_start();
} }
bool render_wasm_error_page(Request& request, String config_key, s32 status_code, String status_reason, DValue error_info)
{
if(!(request.params["REQUEST_METHOD"] != "" && request.ob && request.ob_stack.size() > 0))
return(false);
if(!request.server || request.resources.error_page_active)
return(false);
String unit_file = compiler_error_page_unit(&request, config_key);
if(unit_file == "")
return(false);
String previous_response_code = request.response_code;
s32 previous_status = request.flags.status;
String previous_content_type = request.header["Content-Type"];
request.resources.error_page_active = true;
request.call["error"] = error_info;
request.set_status(status_code, status_reason);
request.header["Content-Type"] = first(request.server->config["CONTENT_TYPE"], "text/html; charset=utf-8");
String unit = compiler_normalize_unit_path(&request, unit_file);
if(!wasm_backend_should_handle(request, unit))
get_shared_unit(&request, unit, true);
ob_start();
String wasm_error = "";
if(wasm_backend_should_handle(request, unit))
wasm_error = wasm_backend_serve(request, unit, "render");
else
wasm_error = "error page wasm unit unavailable after compile: " + unit_file;
String html = ob_get_close();
request.resources.error_page_active = false;
if(wasm_error != "")
{
printf("(!) configured %s page %s failed to render: %s\n", config_key.c_str(), unit_file.c_str(), trim(wasm_error).c_str());
request.response_code = previous_response_code;
request.flags.status = previous_status;
request.header["Content-Type"] = previous_content_type;
request.call.remove("error");
return(false);
}
print(html);
return(true);
}
void render_request_failure(Request& request, String title, String details, String trace, int status_code = 500) void render_request_failure(Request& request, String title, String details, String trace, int status_code = 500)
{ {
request.response_code = request_status_line(request, status_code, "Internal Server Error"); request.response_code = request_status_line(request, status_code, "Internal Server Error");
@@ -92,7 +137,7 @@ void render_request_failure(Request& request, String title, String details, Stri
error_info["signal_name"] = signal_name((int)request_fault_signal); error_info["signal_name"] = signal_name((int)request_fault_signal);
} }
error_info["trace"] = trace; error_info["trace"] = trace;
if(compiler_render_error_page(&request, "page_runtime_error", status_code, "Internal Server Error", error_info)) if(render_wasm_error_page(request, "page_runtime_error", status_code, "Internal Server Error", error_info))
{ {
request.err = "UCE runtime error: " + title + (details != "" ? " (" + details + ")" : ""); request.err = "UCE runtime error: " + title + (details != "" ? " (" + details + ")" : "");
restore_active_request(previous_context); restore_active_request(previous_context);
@@ -368,10 +413,9 @@ int handle_cli_complete(FastCGIRequest& request)
// The CLI socket path installs no native fault handler, so the // The CLI socket path installs no native fault handler, so the
// wasm worker (whose traps are signal-based) can run directly. // wasm worker (whose traps are signal-based) can run directly.
String cli_unit = compiler_normalize_unit_path(&request, script_filename); String cli_unit = compiler_normalize_unit_path(&request, script_filename);
// W7e: compile a cold/stale unit on demand instead of executing // W7e: compile a cold/stale unit on demand; native execution has
// native. force_recompile: the compile-needed check ignores the // been removed, so a unit that still cannot be served by wasm is a
// .wasm (keys off .so mtime), so a missing/stale .wasm with a // request failure instead of a fallback path.
// current .so must be forced to rebuild — see wasm_ready note.
if(!wasm_backend_should_handle(request, cli_unit)) if(!wasm_backend_should_handle(request, cli_unit))
get_shared_unit(&request, cli_unit, true); get_shared_unit(&request, cli_unit, true);
if(wasm_backend_should_handle(request, cli_unit)) if(wasm_backend_should_handle(request, cli_unit))
@@ -384,7 +428,10 @@ int handle_cli_complete(FastCGIRequest& request)
} }
} }
else else
compiler_invoke_cli(&request, script_filename); {
request.set_status(500, "Internal Server Error");
print("UCE CLI wasm unit unavailable after compile: ", script_filename, "\n");
}
} }
} }
} }
@@ -484,27 +531,21 @@ int handle_complete(FastCGIRequest& request) {
String entry_unit = compiler_normalize_unit_path(&request, request.params["SCRIPT_FILENAME"]); String entry_unit = compiler_normalize_unit_path(&request, request.params["SCRIPT_FILENAME"]);
// W7e: every unit runs on wasm. When the artifact is missing or stale // W7e: every unit runs on wasm. When the artifact is missing or stale
// (cold worker, or source edited since the last compile), // (cold worker, or source edited since the last compile), compile the
// wasm_backend_should_handle() returns false; rather than fall back to // unit on demand — get_shared_unit() builds the .wasm side-module — and
// native execution we compile the unit on demand — get_shared_unit() // recheck. Native execution has been removed, so a unit that still cannot
// builds the .wasm side-module — and recheck. Native execution remains // be served by wasm becomes a clean 500 request failure.
// only as a safety net for a genuine wasm-compile failure (or the wasm
// backend being disabled), both of which leave should_handle() false.
// Only reached on the cold/stale path (should_handle false); warm
// requests short-circuit before this, so there is no steady-state
// recompile cost. force_recompile is needed because the compile-needed
// check keys off the .so mtime only (shared_unit_compile_check /
// compiled_time ignores the .wasm), so an uncached unit with a current
// .so but missing .wasm would otherwise not rebuild. NOTE: this does not
// cover a unit already cached in-process whose .wasm later vanished
// (compiler_reusable_cached_unit short-circuits before the recompile) —
// that synthetic case still uses native fallback and is closed properly
// in W7e stage B (make the .wasm part of the compile-freshness check).
auto wasm_ready = [&](const String& unit) -> bool { auto wasm_ready = [&](const String& unit) -> bool {
if(!wasm_backend_should_handle(request, unit)) if(!wasm_backend_should_handle(request, unit))
get_shared_unit(&request, unit, true); get_shared_unit(&request, unit, true);
return(wasm_backend_should_handle(request, unit)); return(wasm_backend_should_handle(request, unit));
}; };
auto fail_wasm_unavailable = [&](const String& handler) {
failure_title = "wasm unit unavailable after compile";
failure_details = handler + " handler could not be served by wasm";
failure_trace = "source: " + request.params["SCRIPT_FILENAME"];
};
if(request.params["UCE_WS"] == "1") if(request.params["UCE_WS"] == "1")
{ {
// A WS message the broker forwarded here: rebuild the connection // A WS message the broker forwarded here: rebuild the connection
@@ -529,14 +570,14 @@ int handle_complete(FastCGIRequest& request) {
if(wasm_ready(entry_unit)) if(wasm_ready(entry_unit))
serve_via_wasm(entry_unit, "websocket"); serve_via_wasm(entry_unit, "websocket");
else else
compiler_invoke_websocket(&request, request.params["SCRIPT_FILENAME"]); fail_wasm_unavailable("websocket");
} }
else if(request.resources.is_cli) else if(request.resources.is_cli)
{ {
if(wasm_ready(entry_unit)) if(wasm_ready(entry_unit))
serve_via_wasm(entry_unit, "cli"); serve_via_wasm(entry_unit, "cli");
else else
compiler_invoke_cli(&request, request.params["SCRIPT_FILENAME"]); fail_wasm_unavailable("cli");
} }
else if(request.params["UCE_SERVE_HTTP"] == "1") else if(request.params["UCE_SERVE_HTTP"] == "1")
{ {
@@ -549,12 +590,12 @@ int handle_complete(FastCGIRequest& request) {
serve_via_wasm(entry_unit, fn == "" ? String("serve_http") : "serve_http:" + fn); serve_via_wasm(entry_unit, fn == "" ? String("serve_http") : "serve_http:" + fn);
} }
else else
compiler_invoke_serve_http(&request, request.params["SCRIPT_FILENAME"], request.params["UCE_SERVE_HTTP_FUNCTION"]); fail_wasm_unavailable("serve_http");
} }
else if(wasm_ready(entry_unit)) else if(wasm_ready(entry_unit))
serve_via_wasm(entry_unit, "render"); serve_via_wasm(entry_unit, "render");
else else
compiler_invoke(&request, request.params["SCRIPT_FILENAME"]); fail_wasm_unavailable("render");
} }
catch(const std::exception& e) catch(const std::exception& e)
{ {
@@ -1270,9 +1311,6 @@ void init_base_process()
server_state.config["COMPILER_SYS_PATH"] = cwd_get(); server_state.config["COMPILER_SYS_PATH"] = cwd_get();
printf("Compiler base path: %s\n", server_state.config["COMPILER_SYS_PATH"].c_str()); printf("Compiler base path: %s\n", server_state.config["COMPILER_SYS_PATH"].c_str());
server_state.config["COMPILE_SCRIPT"] =
server_state.config["COMPILER_SYS_PATH"] + "/" + server_state.config["COMPILE_SCRIPT"];
if(server_state.config["FCGI_PORT"] != "") if(server_state.config["FCGI_PORT"] != "")
server.listen(int_val(server_state.config["FCGI_PORT"])); server.listen(int_val(server_state.config["FCGI_PORT"]));
+13 -23
View File
@@ -2,12 +2,8 @@
// //
// Included into the native server TU (src/linux_fastcgi.cpp) after uce_lib.cpp, // Included into the native server TU (src/linux_fastcgi.cpp) after uce_lib.cpp,
// so it shares String/DValue/config and the UCEB1 codec. Provides a // so it shares String/DValue/config and the UCEB1 codec. Provides a
// config-selectable page-render backend: when WASM_BACKEND_ENABLED and a wasm // Always-on wasm backend: every unit request is served through a per-request
// artifact exists for the entry unit, the request is served through a // wasm workspace. The legacy native dlopen execution path has been removed.
// per-request wasm workspace instead of the native dlopen path.
//
// The seam is narrow on purpose — only the page render branch in
// handle_complete() changes. CLI, serve_http, and websocket stay native.
#include "../lib/wasm_trace.h" #include "../lib/wasm_trace.h"
// The native server TU has the real connectors (sqlite/mysql) compiled in, so // The native server TU has the real connectors (sqlite/mysql) compiled in, so
@@ -31,9 +27,7 @@ static bool g_wasm_init_attempted = false;
bool wasm_backend_configured(Request* context) bool wasm_backend_configured(Request* context)
{ {
if(!context || !context->server) return(context && context->server);
return(false);
return(config_bool("WASM_BACKEND_ENABLED", false));
} }
// Lazily bring up the per-process worker on first use inside a forked child // Lazily bring up the per-process worker on first use inside a forked child
@@ -90,22 +84,19 @@ static bool wasm_artifact_exists(Request* context, const String& entry_unit)
struct stat wasm_st; struct stat wasm_st;
if(stat(wasm_path.c_str(), &wasm_st) != 0 || !S_ISREG(wasm_st.st_mode)) if(stat(wasm_path.c_str(), &wasm_st) != 0 || !S_ISREG(wasm_st.st_mode))
return(false); return(false);
// The wasm path bypasses the native JIT recompile, so a source edit would // Require the artifact to be newer than the source. If it is stale,
// otherwise be served from a stale .wasm. Require the artifact to be newer // dispatch compiles the unit on demand and then rechecks this predicate.
// than the source; if it is stale, fall back to native — which JIT-rebuilds
// both .so and .wasm — so the next request gets a fresh artifact.
struct stat src_st; struct stat src_st;
if(stat(entry_unit.c_str(), &src_st) == 0 && wasm_st.st_mtime < src_st.st_mtime) if(stat(entry_unit.c_str(), &src_st) != 0 || !S_ISREG(src_st.st_mode))
return(false);
if(wasm_st.st_mtime < src_st.st_mtime)
return(false); return(false);
return(true); return(true);
} }
// True if this request should be served by the wasm backend. Every unit now // True if this request can be served by the wasm backend now. Every unit runs
// runs on wasm (W7d retired the last native-only surfaces), so this only gates // on wasm; cold/stale artifacts return false so dispatch can compile on demand
// on config, the presence of a current wasm artifact, and a healthy worker — // and then recheck.
// the remaining native fallback exists solely to cover a cold/stale artifact
// (a unit not yet compiled, or whose source is newer than its .wasm). W7e
// removes that last fallback once cold/stale triggers an on-demand wasm compile.
bool wasm_backend_should_handle(Request& request, const String& entry_unit) bool wasm_backend_should_handle(Request& request, const String& entry_unit)
{ {
if(!wasm_backend_configured(&request)) if(!wasm_backend_configured(&request))
@@ -181,9 +172,8 @@ String wasm_backend_serve(Request& request, const String& entry_unit, const Stri
fcgi_forward_request(broker_socket, dispatch_params, ucb_encode(batch), 5); fcgi_forward_request(broker_socket, dispatch_params, ucb_encode(batch), 5);
} }
// A cli/serve_http unit that does not export the requested handler is a 404, // A cli/serve_http unit that does not export the requested handler is a 404.
// matching native compiler_invoke_cli. For page render a missing handler is // For page render a missing handler is simply an empty body.
// simply an empty body (native parity).
if(!response.handler_present && handler != "render") if(!response.handler_present && handler != "render")
{ {
request.set_status(404, handler == "cli" ? "CLI Entry Point Not Found" : "Handler Not Found"); request.set_status(404, handler == "cli" ? "CLI Entry Point Not Found" : "Handler Not Found");
+59 -29
View File
@@ -139,32 +139,6 @@ bool unit_compile(String path)
} }
static DValue wasm_unit_call_result; static DValue wasm_unit_call_result;
DValue* unit_call(String file_name, String function_name, DValue* call_param)
{
DValue request;
request["op"] = "call";
request["file"] = file_name;
request["function"] = function_name;
if(call_param)
request["param"] = *call_param;
DValue response = wasm_units_call(request);
String output = response["output"].to_string();
if(output != "")
print(output);
DValue* result = response.key("result");
if(result)
{
wasm_unit_call_result = *result;
return(&wasm_unit_call_result);
}
return(0);
}
SharedUnit* compiler_load_shared_unit(Request* context, String file_name, String current_path, bool opt_so_optional)
{
(void)context; (void)file_name; (void)current_path; (void)opt_so_optional;
return(0);
}
static DValue wasm_mysql_call(DValue request) static DValue wasm_mysql_call(DValue request)
{ {
@@ -484,9 +458,9 @@ extern "C" int32_t uce_host_component_resolve(
// but a single workspace can render the same component many times) // but a single workspace can render the same component many times)
static std::map<String, s32> wasm_component_slots; static std::map<String, s32> wasm_component_slots;
// These mirror small page-runtime pieces of compiler.cpp, which is carved // These mirror small page-runtime pieces that cannot include compiler.cpp in
// out of the wasm core wholesale (it is the native toolchain: parser, clang // the wasm core (compiler.cpp owns parser/clang/cache bookkeeping for the host
// driver, dlopen). Kept byte-identical where possible. // build). Kept byte-identical where possible.
String component_normalize_path(String name) String component_normalize_path(String name)
{ {
name = trim(name); name = trim(name);
@@ -595,6 +569,62 @@ static void wasm_run_once(const String& resolved, Request& request)
request.resources.current_unit_file = previous_unit; request.resources.current_unit_file = previous_unit;
} }
DValue* unit_call(String file_name, String function_name, DValue* call_param)
{
String macro = to_upper(trim(function_name));
String handler = "";
if(macro == "RENDER" || macro.rfind("RENDER:", 0) == 0)
handler = "render" + (macro.length() > 7 ? ":" + trim(function_name.substr(function_name.find(":") + 1)) : String(""));
else if(macro == "COMPONENT" || macro.rfind("COMPONENT:", 0) == 0)
handler = "component" + (macro.length() > 10 ? ":" + trim(function_name.substr(function_name.find(":") + 1)) : String(""));
else if(macro == "ONCE")
handler = "once";
else if(macro == "INIT")
handler = "init";
if(handler != "")
{
String resolved;
s32 slot = wasm_resolve_target(file_name, handler, &resolved);
if(!slot)
{
print("Error: unit_call() function '", function_name, "' not found");
return(0);
}
RequestPropsScope props_scope(context, (call_param ? *call_param : DValue()));
if((handler == "render" || handler.rfind("render:", 0) == 0 || handler == "component" || handler.rfind("component:", 0) == 0) && resolved != "")
wasm_run_once(resolved, *context);
String previous_unit = context->resources.current_unit_file;
if(resolved != "")
context->resources.current_unit_file = resolved;
request_ref_handler handler_fn = (request_ref_handler)(uintptr_t)slot;
handler_fn(*context);
context->resources.current_unit_file = previous_unit;
return(0);
}
String resolved;
s32 slot = wasm_resolve_target(file_name, "export:" + function_name, &resolved);
if(!slot)
{
print("Error: unit_call() function '", function_name, "' not found");
return(0);
}
String previous_unit = context->resources.current_unit_file;
if(resolved != "")
context->resources.current_unit_file = resolved;
dv_call_handler handler_fn = (dv_call_handler)(uintptr_t)slot;
DValue* result = handler_fn(call_param);
context->resources.current_unit_file = previous_unit;
if(result)
{
wasm_unit_call_result = *result;
return(&wasm_unit_call_result);
}
return(0);
}
void component_render(String name, DValue props, Request& request) void component_render(String name, DValue props, Request& request)
{ {
String file_name, render_name; String file_name, render_name;
+8 -1
View File
@@ -1046,6 +1046,9 @@ private:
// __uce_<base>[_<suffix>] for a handler spec like "component:CARD" / "render". // __uce_<base>[_<suffix>] for a handler spec like "component:CARD" / "render".
static String handler_export_symbol(const String& handler) static String handler_export_symbol(const String& handler)
{ {
String export_prefix = "export:";
if(handler.rfind(export_prefix, 0) == 0)
return(handler.substr(export_prefix.size()));
auto colon = handler.find(":"); auto colon = handler.find(":");
String symbol = "__uce_" + (colon == String::npos ? handler : handler.substr(0, colon)); String symbol = "__uce_" + (colon == String::npos ? handler : handler.substr(0, colon));
if(colon != String::npos) if(colon != String::npos)
@@ -1086,6 +1089,9 @@ private:
return(1); return(1);
} }
if(!file_exists_host(worker.unit_wasm_path(resolved)) || compiler_unit_needs_recompile(context, resolved, 0))
get_shared_unit(context, resolved, true);
size_t unit_index = 0; size_t unit_index = 0;
String error = load_unit(resolved, unit_index); String error = load_unit(resolved, unit_index);
if(error != "") if(error != "")
@@ -1791,7 +1797,7 @@ private:
return(std::monostate()); return(std::monostate());
})); }));
if(mod == "env" && name == "uce_host_component_resolve") if(mod == "env" && name == "uce_host_component_resolve")
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> { return(add([self](Caller caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
String target, handler, current, resolved; String target, handler, current, resolved;
self->hostcall_read(args[0].i32(), args[1].i32(), target); self->hostcall_read(args[0].i32(), args[1].i32(), target);
self->hostcall_read(args[2].i32(), args[3].i32(), handler); self->hostcall_read(args[2].i32(), args[3].i32(), handler);
@@ -1806,6 +1812,7 @@ private:
self->hostcall_write(args[6].i32(), resolved); self->hostcall_write(args[6].i32(), resolved);
} }
results[0] = Val(slot); results[0] = Val(slot);
caller.context().set_epoch_deadline(self->worker.cfg.epoch_deadline_ticks);
return(std::monostate()); return(std::monostate());
})); }));