W7e: delete the native unit pipeline (.so compile + dlopen execution)
Units now run exclusively on wasm; the native generated-C++ -> clang -> .so -> dlopen path and its request-time fallback are removed. - Dispatch (linux_fastcgi.cpp): the 4 handle_complete branches + the CLI-socket path route every request through wasm (wasm_ready compiles cold/stale on demand); a wasm-unavailable unit now yields a clean error page (fail_wasm_unavailable / render_request_failure) instead of native execution. compiler_invoke / _cli / _websocket / _serve_http deleted. - compiler.cpp (-1274): removed the native .so compile (COMPILE_SCRIPT), load_shared_unit, dlopen/dlsym/dlclose, compiler_load_shared_unit, and the SharedUnit .so function-pointer fields (on_setup/on_render/on_component/ on_websocket/on_cli/on_once/on_init) in types.h/types.cpp. compile_shared_unit now builds only the .wasm side-module; the .uce preprocessor/parser front-end is kept (it emits the C++ the wasm compile consumes). - unit_call()/component()/once/init now resolve across units through the wasm host component resolver (uce_host_component_resolve) instead of native dlsym; configured runtime error pages render through the wasm backend. - Dropped the WASM_BACKEND_ENABLED feature flag and dead COMPILE_SCRIPT / COMPILE_WASM_UNITS config; unit ABI freshness tied to UCE_UNIT_ABI_VERSION; guard against serving a stale .wasm for a deleted source; retired the obsolete W5 native-vs-wasm toggle script. Docs updated. Implemented via the pi agent (with 3 delegated sub-reviews); independently re-verified on the build host: run_cli_tests --include-wasm-kill => 87 passed, 0 failed, 0 skipped. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
fb728d63bc
commit
cf51336873
@ -10,7 +10,7 @@ Replace the Python-based network test runner and plugins with UCE unit tests inv
|
||||
- [x] UCE CLI tests cover every current Python plugin behavior or explicitly document a deliberate replacement.
|
||||
- [x] Existing W5/WASM gates use the UCE CLI test runner instead of `tests/run_network_tests.py`.
|
||||
- [x] Python test runner/plugins are deleted after parity validation.
|
||||
- [x] Full suite passes on `uce-dev` with `WASM_BACKEND_ENABLED=1`.
|
||||
- [x] Full suite passes on `uce-dev` with wasm-only unit execution.
|
||||
|
||||
## Current State
|
||||
|
||||
|
||||
@ -1,9 +1,11 @@
|
||||
# UCE WASM Runtime Architecture
|
||||
|
||||
Status: current as of the W7 cutover (June 2026). This document describes the
|
||||
**runtime architecture as built** — the process topology, the wasm membrane,
|
||||
the unified request dispatch, and the central WebSocket broker. For the
|
||||
historical motivation and the phased migration plan, see
|
||||
Status: current as of the W7e native-pipeline removal (June 2026). This document
|
||||
describes the **runtime architecture as built** — the process topology, the
|
||||
wasm membrane, the unified request dispatch, and the central WebSocket broker.
|
||||
Native `.so` unit execution/dlopen fallback has been removed; the parser and
|
||||
preprocessor remain only as the front-end that emits C++ for wasm side-module
|
||||
compilation. For the historical motivation and the phased migration plan, see
|
||||
[`WASM-PROPOSAL.md`](../WASM-PROPOSAL.md); this file is the steady-state
|
||||
reference that proposal points at.
|
||||
|
||||
@ -116,10 +118,10 @@ __uce_<base>[_<sanitize(suffix)>]
|
||||
path and looks up the export's funcref slot; `exists` lets callers probe a unit
|
||||
without instantiating it.
|
||||
|
||||
`wasm_backend_should_handle(request, entry_unit)` gates whether the wasm path
|
||||
takes the request (vs. the legacy native compiler path, still selectable while
|
||||
the native pipeline is retired). With `WASM_BACKEND_ENABLED=1` this is the
|
||||
default for all unit execution.
|
||||
`wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
|
||||
backend is initialized and the requested artifact/handler is currently
|
||||
available. If an artifact is cold or stale, dispatch compiles it on demand via
|
||||
`get_shared_unit()` and rechecks. There is no native unit-execution fallback.
|
||||
|
||||
---
|
||||
|
||||
@ -171,9 +173,9 @@ The `UCE_*` params are set by whichever broker forwarded the request:
|
||||
`request.resources.websocket_*` and `request.connection` from them before
|
||||
invoking `__uce_websocket`.
|
||||
|
||||
Each branch has a native-compiler fallback (`compiler_invoke*`) used only when
|
||||
`wasm_backend_should_handle` declines — the holdout path during native
|
||||
retirement.
|
||||
If a unit still cannot be served after the on-demand wasm compile, the worker
|
||||
returns a clean 500 with the wasm/compile error; it does not execute native unit
|
||||
code.
|
||||
|
||||
---
|
||||
|
||||
@ -294,7 +296,6 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `WASM_BACKEND_ENABLED` | `1` | Route unit execution through the wasm path. |
|
||||
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
|
||||
| `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). |
|
||||
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. |
|
||||
@ -308,7 +309,7 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
|
||||
- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
|
||||
in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages.
|
||||
Current baseline: **86 passed, 0 failed**.
|
||||
Current baseline: **87 passed, 0 failed**.
|
||||
- **WebSocket end-to-end**: a headless client performs a raw WS handshake to
|
||||
`:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
|
||||
`SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
|
||||
|
||||
@ -21,13 +21,10 @@ SITE_DIRECTORY=site
|
||||
# ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT
|
||||
JIT_COMPILE_ON_REQUEST=1
|
||||
|
||||
# OPTIONAL W2 WEBASSEMBLY SIDE-MODULE COMPILATION BESIDE NATIVE .so UNITS
|
||||
COMPILE_WASM_UNITS=0
|
||||
# WASM SIDE-MODULE COMPILER USED FOR .uce UNITS
|
||||
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
|
||||
|
||||
# DEFAULT PAGE RENDER BACKEND. W5 defaults to the WASM backend with explicit
|
||||
# native fallbacks for host-owned surfaces that are not membrane APIs yet.
|
||||
WASM_BACKEND_ENABLED=1
|
||||
# WASM RUNTIME SETTINGS. Unit execution is always routed through wasm.
|
||||
WASM_BACKEND_VERBOSE=0
|
||||
WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm
|
||||
WASM_MEMORY_LIMIT_BYTES=536870912
|
||||
|
||||
@ -9,11 +9,11 @@ GF="uce_fastcgi"
|
||||
mkdir bin > /dev/null 2>&1
|
||||
mkdir bin/tmp > /dev/null 2>&1
|
||||
mkdir bin/assets > /dev/null 2>&1
|
||||
mkdir bin/wasm > /dev/null 2>&1
|
||||
mkdir work > /dev/null 2>&1
|
||||
|
||||
COMPILER="clang++"
|
||||
# -rdynamic is a link-time flag (exports the binary's symbols so dlopen'd .uce
|
||||
# units resolve the runtime against it); the -c compiles below do not need it.
|
||||
# -rdynamic is a link-time flag; the -c compiles below do not need it.
|
||||
FLAGS="-g -w -Wall -$OPT_FLAG -std=c++20 -fpermissive -ffast-math"
|
||||
|
||||
# Wasmtime C++ API — needed only by the wasm backend object (src/wasm).
|
||||
@ -30,8 +30,8 @@ SRCFLAGS="-D EXEC_NAME=\"$GF\" -D PLATFORM_NAME=\"linux\""
|
||||
# bin/sqlite3.o vendored SQLite amalgamation (depends only on its own source)
|
||||
# bin/wasm.o wasm backend + worker + wasmtime.hh (src/wasm)
|
||||
# bin/main.o linux_fastcgi.cpp + the uce_lib core amalgamation
|
||||
# All link into the single -rdynamic binary, so the dlopen unit model is
|
||||
# unchanged. Delete bin/*.o to force a clean rebuild.
|
||||
# All link into the single -rdynamic binary. Delete bin/*.o to force a clean
|
||||
# rebuild.
|
||||
|
||||
# Rebuild $1 if it is missing or anything under the remaining find-args is newer.
|
||||
needs_rebuild() {
|
||||
@ -41,6 +41,14 @@ needs_rebuild() {
|
||||
return 1
|
||||
}
|
||||
|
||||
# core.wasm: guest runtime loaded by the native wasm backend.
|
||||
if needs_rebuild bin/wasm/core.wasm src/wasm/core.cpp src/lib src/wasm/core_hostcalls.syms src/wasm/core_libc_exports.syms scripts/build_core_wasm.sh; then
|
||||
echo "Compiling wasm core..."
|
||||
bash scripts/build_core_wasm.sh || exit 1
|
||||
else
|
||||
echo "Reusing bin/wasm/core.wasm"
|
||||
fi
|
||||
|
||||
# SQLite: vendored C, depends only on its own source (not our headers).
|
||||
if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then
|
||||
echo "Compiling SQLite..."
|
||||
|
||||
@ -50,7 +50,7 @@ build_pch_if_needed() {
|
||||
return 0
|
||||
fi
|
||||
mkdir -p "$PCH_DIR"
|
||||
if [ -s "$PCH_FN" ]; then
|
||||
if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then
|
||||
return 0
|
||||
fi
|
||||
"$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \
|
||||
|
||||
@ -1,94 +1,19 @@
|
||||
#!/bin/bash
|
||||
# W5 parity/performance gate for the config-selectable WASM backend.
|
||||
# Runs on k-uce. Requires root because the live service reads /etc/uce/settings.cfg.
|
||||
# Retired W5 parity/performance gate.
|
||||
#
|
||||
# W7e removed the config-selectable native backend and all native unit
|
||||
# execution. There is no longer a valid WASM_BACKEND_ENABLED=0 baseline to
|
||||
# switch to here; doing so would only create a misleading service state.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/../.."
|
||||
cat >&2 <<'EOF'
|
||||
run_w5.sh is retired: UCE unit execution is wasm-only after W7e.
|
||||
Use the supported regression gate instead:
|
||||
|
||||
if [ "${EUID:-$(id -u)}" -ne 0 ]; then
|
||||
echo "run_w5.sh must run as root so it can switch /etc/uce/settings.cfg and restart uce.service" >&2
|
||||
exit 1
|
||||
fi
|
||||
bash scripts/build_linux.sh && systemctl restart uce.service && sleep 3 \
|
||||
&& bash scripts/run_cli_tests.sh --include-wasm-kill
|
||||
|
||||
OUT=${UCE_W5_OUT:-/tmp/uce/wasm-w5}
|
||||
CONFIG=${UCE_CONFIG:-/etc/uce/settings.cfg}
|
||||
mkdir -p "$OUT"
|
||||
BACKUP="$OUT/settings.cfg.before-w5"
|
||||
cp "$CONFIG" "$BACKUP"
|
||||
|
||||
restore_on_error() {
|
||||
if [ "${UCE_W5_KEEP_BACKEND:-0}" != "1" ]; then
|
||||
cp "$BACKUP" "$CONFIG"
|
||||
systemctl restart uce.service >/dev/null 2>&1 || true
|
||||
fi
|
||||
}
|
||||
trap restore_on_error EXIT
|
||||
|
||||
set_backend() {
|
||||
local enabled="$1"
|
||||
local tmp
|
||||
tmp=$(mktemp)
|
||||
awk -F= -v enabled="$enabled" '
|
||||
BEGIN { found = 0 }
|
||||
/^WASM_BACKEND_ENABLED=/ { print "WASM_BACKEND_ENABLED=" enabled; found = 1; next }
|
||||
{ print }
|
||||
END { if(!found) print "WASM_BACKEND_ENABLED=" enabled }
|
||||
' "$CONFIG" > "$tmp"
|
||||
for kv in \
|
||||
'WASM_BACKEND_VERBOSE=0' \
|
||||
'WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm' \
|
||||
'WASM_MEMORY_LIMIT_BYTES=536870912' \
|
||||
'WASM_EPOCH_DEADLINE_TICKS=200' \
|
||||
'WASM_EPOCH_PERIOD_MS=50'
|
||||
do
|
||||
key=${kv%%=*}
|
||||
if ! grep -q "^${key}=" "$tmp"; then
|
||||
printf '%s\n' "$kv" >> "$tmp"
|
||||
fi
|
||||
done
|
||||
cat "$tmp" > "$CONFIG"
|
||||
rm -f "$tmp"
|
||||
systemctl restart uce.service >/dev/null
|
||||
sleep 1
|
||||
}
|
||||
|
||||
summary_total() {
|
||||
awk '/^Summary:/ { print $2; found=1 } END { if(!found) print 0 }' "$1"
|
||||
}
|
||||
|
||||
# Native reference baseline.
|
||||
set_backend 0
|
||||
scripts/run_cli_tests.sh > "$OUT/native-warmup.txt" || true
|
||||
scripts/run_cli_tests.sh > "$OUT/native-network.txt"
|
||||
python3 tests/wasm_benchmark.py --out-dir "$OUT/native-benchmark" --samples "${UCE_W5_BENCH_SAMPLES:-20}" --timeout 30 >/dev/null
|
||||
|
||||
# WASM default backend with W5 native fallbacks for host-owned surfaces.
|
||||
set_backend 1
|
||||
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-warmup.txt" || true
|
||||
scripts/run_cli_tests.sh --include-wasm-kill > "$OUT/wasm-network.txt"
|
||||
python3 tests/wasm_benchmark.py \
|
||||
--out-dir "$OUT/benchmark" \
|
||||
--backend-label wasm \
|
||||
--compare-native-json "$OUT/native-benchmark/benchmark.json" \
|
||||
--samples "${UCE_W5_BENCH_SAMPLES:-20}" \
|
||||
--timeout 30
|
||||
python3 tests/wasm_site_audit.py --out-dir "$OUT" >/dev/null
|
||||
|
||||
network_cases=$(summary_total "$OUT/wasm-network.txt")
|
||||
if [ "$network_cases" -lt 80 ]; then
|
||||
echo "W5 HARNESS: FAIL"
|
||||
echo "wasm network ran $network_cases cases, expected at least 80"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo 'W5 HARNESS: PASS'
|
||||
echo "network_cases=$network_cases benchmark_rows=see benchmark.json"
|
||||
echo "reports=$OUT"
|
||||
|
||||
if [ "${UCE_W5_KEEP_BACKEND:-0}" = "1" ]; then
|
||||
trap - EXIT
|
||||
printf 'WASM backend left enabled in %s\n' "$CONFIG"
|
||||
else
|
||||
cp "$BACKUP" "$CONFIG"
|
||||
systemctl restart uce.service >/dev/null
|
||||
trap - EXIT
|
||||
fi
|
||||
For performance comparisons, use tests/wasm_benchmark.py against the current
|
||||
wasm backend; native-baseline artifacts under docs/wasm-baselines are historical
|
||||
only.
|
||||
EOF
|
||||
exit 2
|
||||
|
||||
@ -564,7 +564,10 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
|
||||
parsed_content.resize(parsed_content.length() - current_line.length());
|
||||
nibble(current_line, "\"");
|
||||
String unit_name = nibble(current_line, "\"");
|
||||
SharedUnit* sub_su = compiler_load_shared_unit(context, unit_name, su->src_path, true);
|
||||
String resolved_unit = unit_name;
|
||||
if(resolved_unit != "" && resolved_unit[0] != '/')
|
||||
resolved_unit = expand_path(resolved_unit, su->src_path);
|
||||
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit(context, resolved_unit, true));
|
||||
if(sub_su)
|
||||
parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n");
|
||||
}
|
||||
|
||||
1274
src/lib/compiler.cpp
1274
src/lib/compiler.cpp
File diff suppressed because it is too large
Load Diff
@ -13,16 +13,9 @@ String preprocess_shared_unit(Request* context, SharedUnit* su);
|
||||
String compiler_generated_cpp_path(Request* context, String source_file);
|
||||
String compiler_generated_cpp_path(SharedUnit* su);
|
||||
void setup_unit_paths(Request* context, SharedUnit* su, String file_name);
|
||||
void load_shared_unit(Request* context, SharedUnit* su);
|
||||
void compile_shared_unit(Request* context, SharedUnit* su);
|
||||
SharedUnit* get_shared_unit(Request* context, String file_name, bool opt_so_optional = false);
|
||||
void compiler_invoke(Request* context, String file_name);
|
||||
void compiler_invoke_websocket(Request* context, String file_name);
|
||||
void compiler_invoke_cli(Request* context, String file_name);
|
||||
void compiler_invoke_serve_http(Request* context, String file_name, String handler_name = "");
|
||||
SharedUnit* compiler_load_shared_unit(Request* context, String file_name, String current_path = "", bool opt_so_optional = false);
|
||||
String compiler_error_page_unit(Request* context, String config_key);
|
||||
bool compiler_render_error_page(Request* context, String config_key, s32 status_code, String status_reason, DValue error_info);
|
||||
bool compiler_unit_compile_pending(Request* context, String file_name);
|
||||
String compiler_site_directory(Request* context);
|
||||
StringList compiler_scan_site_units(Request* context);
|
||||
|
||||
@ -1314,10 +1314,7 @@ StringMap make_server_settings()
|
||||
StringMap cfg;
|
||||
|
||||
cfg["BIN_DIRECTORY"] = "/tmp/uce/work";
|
||||
cfg["COMPILE_SCRIPT"] = "scripts/compile";
|
||||
cfg["WASM_COMPILE_SCRIPT"] = "scripts/compile_wasm_unit";
|
||||
cfg["COMPILE_WASM_UNITS"] = "0";
|
||||
cfg["WASM_BACKEND_ENABLED"] = "1";
|
||||
cfg["WASM_BACKEND_VERBOSE"] = "0";
|
||||
cfg["WASM_CORE_PATH"] = "";
|
||||
cfg["WASM_MEMORY_LIMIT_BYTES"] = std::to_string(512ull * 1024 * 1024);
|
||||
|
||||
@ -79,12 +79,6 @@ String http_status_reason(s32 code)
|
||||
|
||||
SharedUnit::~SharedUnit()
|
||||
{
|
||||
#ifndef __UCE_WASM_CORE__
|
||||
if(so_handle)
|
||||
{
|
||||
dlclose(so_handle);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
|
||||
String nibble(String div, String& haystack)
|
||||
|
||||
@ -74,7 +74,6 @@ struct SharedUnit {
|
||||
String compile_output_file_name;
|
||||
String setup_file_name;
|
||||
StringList api_declarations;
|
||||
std::map<String, void*> api_functions;
|
||||
|
||||
String src_path;
|
||||
String bin_path;
|
||||
@ -84,16 +83,6 @@ struct SharedUnit {
|
||||
String wasm_file_name;
|
||||
String pre_file_name;
|
||||
|
||||
void* so_handle = 0;
|
||||
|
||||
request_handler on_setup = 0;
|
||||
request_ref_handler on_render = 0;
|
||||
request_ref_handler on_component = 0;
|
||||
request_ref_handler on_websocket = 0;
|
||||
request_ref_handler on_cli = 0;
|
||||
request_ref_handler on_once = 0;
|
||||
request_ref_handler on_init = 0;
|
||||
|
||||
String compiler_messages;
|
||||
String compile_status = "unknown";
|
||||
String compile_error_status = "";
|
||||
@ -153,8 +142,6 @@ String nibble(String div, String& haystack);
|
||||
|
||||
#include "dvalue.h"
|
||||
|
||||
void compiler_invoke(Request* context, String file_name);
|
||||
|
||||
struct Request {
|
||||
|
||||
ServerState* server = 0;
|
||||
|
||||
@ -65,6 +65,51 @@ void clear_request_output(Request& request)
|
||||
request.ob_start();
|
||||
}
|
||||
|
||||
bool render_wasm_error_page(Request& request, String config_key, s32 status_code, String status_reason, DValue error_info)
|
||||
{
|
||||
if(!(request.params["REQUEST_METHOD"] != "" && request.ob && request.ob_stack.size() > 0))
|
||||
return(false);
|
||||
if(!request.server || request.resources.error_page_active)
|
||||
return(false);
|
||||
String unit_file = compiler_error_page_unit(&request, config_key);
|
||||
if(unit_file == "")
|
||||
return(false);
|
||||
|
||||
String previous_response_code = request.response_code;
|
||||
s32 previous_status = request.flags.status;
|
||||
String previous_content_type = request.header["Content-Type"];
|
||||
|
||||
request.resources.error_page_active = true;
|
||||
request.call["error"] = error_info;
|
||||
request.set_status(status_code, status_reason);
|
||||
request.header["Content-Type"] = first(request.server->config["CONTENT_TYPE"], "text/html; charset=utf-8");
|
||||
|
||||
String unit = compiler_normalize_unit_path(&request, unit_file);
|
||||
if(!wasm_backend_should_handle(request, unit))
|
||||
get_shared_unit(&request, unit, true);
|
||||
|
||||
ob_start();
|
||||
String wasm_error = "";
|
||||
if(wasm_backend_should_handle(request, unit))
|
||||
wasm_error = wasm_backend_serve(request, unit, "render");
|
||||
else
|
||||
wasm_error = "error page wasm unit unavailable after compile: " + unit_file;
|
||||
String html = ob_get_close();
|
||||
request.resources.error_page_active = false;
|
||||
|
||||
if(wasm_error != "")
|
||||
{
|
||||
printf("(!) configured %s page %s failed to render: %s\n", config_key.c_str(), unit_file.c_str(), trim(wasm_error).c_str());
|
||||
request.response_code = previous_response_code;
|
||||
request.flags.status = previous_status;
|
||||
request.header["Content-Type"] = previous_content_type;
|
||||
request.call.remove("error");
|
||||
return(false);
|
||||
}
|
||||
print(html);
|
||||
return(true);
|
||||
}
|
||||
|
||||
void render_request_failure(Request& request, String title, String details, String trace, int status_code = 500)
|
||||
{
|
||||
request.response_code = request_status_line(request, status_code, "Internal Server Error");
|
||||
@ -92,7 +137,7 @@ void render_request_failure(Request& request, String title, String details, Stri
|
||||
error_info["signal_name"] = signal_name((int)request_fault_signal);
|
||||
}
|
||||
error_info["trace"] = trace;
|
||||
if(compiler_render_error_page(&request, "page_runtime_error", status_code, "Internal Server Error", error_info))
|
||||
if(render_wasm_error_page(request, "page_runtime_error", status_code, "Internal Server Error", error_info))
|
||||
{
|
||||
request.err = "UCE runtime error: " + title + (details != "" ? " (" + details + ")" : "");
|
||||
restore_active_request(previous_context);
|
||||
@ -368,10 +413,9 @@ int handle_cli_complete(FastCGIRequest& request)
|
||||
// The CLI socket path installs no native fault handler, so the
|
||||
// wasm worker (whose traps are signal-based) can run directly.
|
||||
String cli_unit = compiler_normalize_unit_path(&request, script_filename);
|
||||
// W7e: compile a cold/stale unit on demand instead of executing
|
||||
// native. force_recompile: the compile-needed check ignores the
|
||||
// .wasm (keys off .so mtime), so a missing/stale .wasm with a
|
||||
// current .so must be forced to rebuild — see wasm_ready note.
|
||||
// W7e: compile a cold/stale unit on demand; native execution has
|
||||
// been removed, so a unit that still cannot be served by wasm is a
|
||||
// request failure instead of a fallback path.
|
||||
if(!wasm_backend_should_handle(request, cli_unit))
|
||||
get_shared_unit(&request, cli_unit, true);
|
||||
if(wasm_backend_should_handle(request, cli_unit))
|
||||
@ -384,7 +428,10 @@ int handle_cli_complete(FastCGIRequest& request)
|
||||
}
|
||||
}
|
||||
else
|
||||
compiler_invoke_cli(&request, script_filename);
|
||||
{
|
||||
request.set_status(500, "Internal Server Error");
|
||||
print("UCE CLI wasm unit unavailable after compile: ", script_filename, "\n");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -484,27 +531,21 @@ int handle_complete(FastCGIRequest& request) {
|
||||
|
||||
String entry_unit = compiler_normalize_unit_path(&request, request.params["SCRIPT_FILENAME"]);
|
||||
// W7e: every unit runs on wasm. When the artifact is missing or stale
|
||||
// (cold worker, or source edited since the last compile),
|
||||
// wasm_backend_should_handle() returns false; rather than fall back to
|
||||
// native execution we compile the unit on demand — get_shared_unit()
|
||||
// builds the .wasm side-module — and recheck. Native execution remains
|
||||
// only as a safety net for a genuine wasm-compile failure (or the wasm
|
||||
// backend being disabled), both of which leave should_handle() false.
|
||||
// Only reached on the cold/stale path (should_handle false); warm
|
||||
// requests short-circuit before this, so there is no steady-state
|
||||
// recompile cost. force_recompile is needed because the compile-needed
|
||||
// check keys off the .so mtime only (shared_unit_compile_check /
|
||||
// compiled_time ignores the .wasm), so an uncached unit with a current
|
||||
// .so but missing .wasm would otherwise not rebuild. NOTE: this does not
|
||||
// cover a unit already cached in-process whose .wasm later vanished
|
||||
// (compiler_reusable_cached_unit short-circuits before the recompile) —
|
||||
// that synthetic case still uses native fallback and is closed properly
|
||||
// in W7e stage B (make the .wasm part of the compile-freshness check).
|
||||
// (cold worker, or source edited since the last compile), compile the
|
||||
// unit on demand — get_shared_unit() builds the .wasm side-module — and
|
||||
// recheck. Native execution has been removed, so a unit that still cannot
|
||||
// be served by wasm becomes a clean 500 request failure.
|
||||
auto wasm_ready = [&](const String& unit) -> bool {
|
||||
if(!wasm_backend_should_handle(request, unit))
|
||||
get_shared_unit(&request, unit, true);
|
||||
return(wasm_backend_should_handle(request, unit));
|
||||
};
|
||||
auto fail_wasm_unavailable = [&](const String& handler) {
|
||||
failure_title = "wasm unit unavailable after compile";
|
||||
failure_details = handler + " handler could not be served by wasm";
|
||||
failure_trace = "source: " + request.params["SCRIPT_FILENAME"];
|
||||
};
|
||||
|
||||
if(request.params["UCE_WS"] == "1")
|
||||
{
|
||||
// A WS message the broker forwarded here: rebuild the connection
|
||||
@ -529,14 +570,14 @@ int handle_complete(FastCGIRequest& request) {
|
||||
if(wasm_ready(entry_unit))
|
||||
serve_via_wasm(entry_unit, "websocket");
|
||||
else
|
||||
compiler_invoke_websocket(&request, request.params["SCRIPT_FILENAME"]);
|
||||
fail_wasm_unavailable("websocket");
|
||||
}
|
||||
else if(request.resources.is_cli)
|
||||
{
|
||||
if(wasm_ready(entry_unit))
|
||||
serve_via_wasm(entry_unit, "cli");
|
||||
else
|
||||
compiler_invoke_cli(&request, request.params["SCRIPT_FILENAME"]);
|
||||
fail_wasm_unavailable("cli");
|
||||
}
|
||||
else if(request.params["UCE_SERVE_HTTP"] == "1")
|
||||
{
|
||||
@ -549,12 +590,12 @@ int handle_complete(FastCGIRequest& request) {
|
||||
serve_via_wasm(entry_unit, fn == "" ? String("serve_http") : "serve_http:" + fn);
|
||||
}
|
||||
else
|
||||
compiler_invoke_serve_http(&request, request.params["SCRIPT_FILENAME"], request.params["UCE_SERVE_HTTP_FUNCTION"]);
|
||||
fail_wasm_unavailable("serve_http");
|
||||
}
|
||||
else if(wasm_ready(entry_unit))
|
||||
serve_via_wasm(entry_unit, "render");
|
||||
else
|
||||
compiler_invoke(&request, request.params["SCRIPT_FILENAME"]);
|
||||
fail_wasm_unavailable("render");
|
||||
}
|
||||
catch(const std::exception& e)
|
||||
{
|
||||
@ -1270,9 +1311,6 @@ void init_base_process()
|
||||
server_state.config["COMPILER_SYS_PATH"] = cwd_get();
|
||||
printf("Compiler base path: %s\n", server_state.config["COMPILER_SYS_PATH"].c_str());
|
||||
|
||||
server_state.config["COMPILE_SCRIPT"] =
|
||||
server_state.config["COMPILER_SYS_PATH"] + "/" + server_state.config["COMPILE_SCRIPT"];
|
||||
|
||||
if(server_state.config["FCGI_PORT"] != "")
|
||||
server.listen(int_val(server_state.config["FCGI_PORT"]));
|
||||
|
||||
|
||||
@ -2,12 +2,8 @@
|
||||
//
|
||||
// Included into the native server TU (src/linux_fastcgi.cpp) after uce_lib.cpp,
|
||||
// so it shares String/DValue/config and the UCEB1 codec. Provides a
|
||||
// config-selectable page-render backend: when WASM_BACKEND_ENABLED and a wasm
|
||||
// artifact exists for the entry unit, the request is served through a
|
||||
// per-request wasm workspace instead of the native dlopen path.
|
||||
//
|
||||
// The seam is narrow on purpose — only the page render branch in
|
||||
// handle_complete() changes. CLI, serve_http, and websocket stay native.
|
||||
// Always-on wasm backend: every unit request is served through a per-request
|
||||
// wasm workspace. The legacy native dlopen execution path has been removed.
|
||||
|
||||
#include "../lib/wasm_trace.h"
|
||||
// The native server TU has the real connectors (sqlite/mysql) compiled in, so
|
||||
@ -31,9 +27,7 @@ static bool g_wasm_init_attempted = false;
|
||||
|
||||
bool wasm_backend_configured(Request* context)
|
||||
{
|
||||
if(!context || !context->server)
|
||||
return(false);
|
||||
return(config_bool("WASM_BACKEND_ENABLED", false));
|
||||
return(context && context->server);
|
||||
}
|
||||
|
||||
// Lazily bring up the per-process worker on first use inside a forked child
|
||||
@ -90,22 +84,19 @@ static bool wasm_artifact_exists(Request* context, const String& entry_unit)
|
||||
struct stat wasm_st;
|
||||
if(stat(wasm_path.c_str(), &wasm_st) != 0 || !S_ISREG(wasm_st.st_mode))
|
||||
return(false);
|
||||
// The wasm path bypasses the native JIT recompile, so a source edit would
|
||||
// otherwise be served from a stale .wasm. Require the artifact to be newer
|
||||
// than the source; if it is stale, fall back to native — which JIT-rebuilds
|
||||
// both .so and .wasm — so the next request gets a fresh artifact.
|
||||
// Require the artifact to be newer than the source. If it is stale,
|
||||
// dispatch compiles the unit on demand and then rechecks this predicate.
|
||||
struct stat src_st;
|
||||
if(stat(entry_unit.c_str(), &src_st) == 0 && wasm_st.st_mtime < src_st.st_mtime)
|
||||
if(stat(entry_unit.c_str(), &src_st) != 0 || !S_ISREG(src_st.st_mode))
|
||||
return(false);
|
||||
if(wasm_st.st_mtime < src_st.st_mtime)
|
||||
return(false);
|
||||
return(true);
|
||||
}
|
||||
|
||||
// True if this request should be served by the wasm backend. Every unit now
|
||||
// runs on wasm (W7d retired the last native-only surfaces), so this only gates
|
||||
// on config, the presence of a current wasm artifact, and a healthy worker —
|
||||
// the remaining native fallback exists solely to cover a cold/stale artifact
|
||||
// (a unit not yet compiled, or whose source is newer than its .wasm). W7e
|
||||
// removes that last fallback once cold/stale triggers an on-demand wasm compile.
|
||||
// True if this request can be served by the wasm backend now. Every unit runs
|
||||
// on wasm; cold/stale artifacts return false so dispatch can compile on demand
|
||||
// and then recheck.
|
||||
bool wasm_backend_should_handle(Request& request, const String& entry_unit)
|
||||
{
|
||||
if(!wasm_backend_configured(&request))
|
||||
@ -181,9 +172,8 @@ String wasm_backend_serve(Request& request, const String& entry_unit, const Stri
|
||||
fcgi_forward_request(broker_socket, dispatch_params, ucb_encode(batch), 5);
|
||||
}
|
||||
|
||||
// A cli/serve_http unit that does not export the requested handler is a 404,
|
||||
// matching native compiler_invoke_cli. For page render a missing handler is
|
||||
// simply an empty body (native parity).
|
||||
// A cli/serve_http unit that does not export the requested handler is a 404.
|
||||
// For page render a missing handler is simply an empty body.
|
||||
if(!response.handler_present && handler != "render")
|
||||
{
|
||||
request.set_status(404, handler == "cli" ? "CLI Entry Point Not Found" : "Handler Not Found");
|
||||
|
||||
@ -139,32 +139,6 @@ bool unit_compile(String path)
|
||||
}
|
||||
|
||||
static DValue wasm_unit_call_result;
|
||||
DValue* unit_call(String file_name, String function_name, DValue* call_param)
|
||||
{
|
||||
DValue request;
|
||||
request["op"] = "call";
|
||||
request["file"] = file_name;
|
||||
request["function"] = function_name;
|
||||
if(call_param)
|
||||
request["param"] = *call_param;
|
||||
DValue response = wasm_units_call(request);
|
||||
String output = response["output"].to_string();
|
||||
if(output != "")
|
||||
print(output);
|
||||
DValue* result = response.key("result");
|
||||
if(result)
|
||||
{
|
||||
wasm_unit_call_result = *result;
|
||||
return(&wasm_unit_call_result);
|
||||
}
|
||||
return(0);
|
||||
}
|
||||
|
||||
SharedUnit* compiler_load_shared_unit(Request* context, String file_name, String current_path, bool opt_so_optional)
|
||||
{
|
||||
(void)context; (void)file_name; (void)current_path; (void)opt_so_optional;
|
||||
return(0);
|
||||
}
|
||||
|
||||
static DValue wasm_mysql_call(DValue request)
|
||||
{
|
||||
@ -484,9 +458,9 @@ extern "C" int32_t uce_host_component_resolve(
|
||||
// but a single workspace can render the same component many times)
|
||||
static std::map<String, s32> wasm_component_slots;
|
||||
|
||||
// These mirror small page-runtime pieces of compiler.cpp, which is carved
|
||||
// out of the wasm core wholesale (it is the native toolchain: parser, clang
|
||||
// driver, dlopen). Kept byte-identical where possible.
|
||||
// These mirror small page-runtime pieces that cannot include compiler.cpp in
|
||||
// the wasm core (compiler.cpp owns parser/clang/cache bookkeeping for the host
|
||||
// build). Kept byte-identical where possible.
|
||||
String component_normalize_path(String name)
|
||||
{
|
||||
name = trim(name);
|
||||
@ -595,6 +569,62 @@ static void wasm_run_once(const String& resolved, Request& request)
|
||||
request.resources.current_unit_file = previous_unit;
|
||||
}
|
||||
|
||||
DValue* unit_call(String file_name, String function_name, DValue* call_param)
|
||||
{
|
||||
String macro = to_upper(trim(function_name));
|
||||
String handler = "";
|
||||
if(macro == "RENDER" || macro.rfind("RENDER:", 0) == 0)
|
||||
handler = "render" + (macro.length() > 7 ? ":" + trim(function_name.substr(function_name.find(":") + 1)) : String(""));
|
||||
else if(macro == "COMPONENT" || macro.rfind("COMPONENT:", 0) == 0)
|
||||
handler = "component" + (macro.length() > 10 ? ":" + trim(function_name.substr(function_name.find(":") + 1)) : String(""));
|
||||
else if(macro == "ONCE")
|
||||
handler = "once";
|
||||
else if(macro == "INIT")
|
||||
handler = "init";
|
||||
|
||||
if(handler != "")
|
||||
{
|
||||
String resolved;
|
||||
s32 slot = wasm_resolve_target(file_name, handler, &resolved);
|
||||
if(!slot)
|
||||
{
|
||||
print("Error: unit_call() function '", function_name, "' not found");
|
||||
return(0);
|
||||
}
|
||||
RequestPropsScope props_scope(context, (call_param ? *call_param : DValue()));
|
||||
if((handler == "render" || handler.rfind("render:", 0) == 0 || handler == "component" || handler.rfind("component:", 0) == 0) && resolved != "")
|
||||
wasm_run_once(resolved, *context);
|
||||
String previous_unit = context->resources.current_unit_file;
|
||||
if(resolved != "")
|
||||
context->resources.current_unit_file = resolved;
|
||||
request_ref_handler handler_fn = (request_ref_handler)(uintptr_t)slot;
|
||||
handler_fn(*context);
|
||||
context->resources.current_unit_file = previous_unit;
|
||||
return(0);
|
||||
}
|
||||
|
||||
String resolved;
|
||||
s32 slot = wasm_resolve_target(file_name, "export:" + function_name, &resolved);
|
||||
if(!slot)
|
||||
{
|
||||
print("Error: unit_call() function '", function_name, "' not found");
|
||||
return(0);
|
||||
}
|
||||
|
||||
String previous_unit = context->resources.current_unit_file;
|
||||
if(resolved != "")
|
||||
context->resources.current_unit_file = resolved;
|
||||
dv_call_handler handler_fn = (dv_call_handler)(uintptr_t)slot;
|
||||
DValue* result = handler_fn(call_param);
|
||||
context->resources.current_unit_file = previous_unit;
|
||||
if(result)
|
||||
{
|
||||
wasm_unit_call_result = *result;
|
||||
return(&wasm_unit_call_result);
|
||||
}
|
||||
return(0);
|
||||
}
|
||||
|
||||
void component_render(String name, DValue props, Request& request)
|
||||
{
|
||||
String file_name, render_name;
|
||||
|
||||
@ -1046,6 +1046,9 @@ private:
|
||||
// __uce_<base>[_<suffix>] for a handler spec like "component:CARD" / "render".
|
||||
static String handler_export_symbol(const String& handler)
|
||||
{
|
||||
String export_prefix = "export:";
|
||||
if(handler.rfind(export_prefix, 0) == 0)
|
||||
return(handler.substr(export_prefix.size()));
|
||||
auto colon = handler.find(":");
|
||||
String symbol = "__uce_" + (colon == String::npos ? handler : handler.substr(0, colon));
|
||||
if(colon != String::npos)
|
||||
@ -1086,6 +1089,9 @@ private:
|
||||
return(1);
|
||||
}
|
||||
|
||||
if(!file_exists_host(worker.unit_wasm_path(resolved)) || compiler_unit_needs_recompile(context, resolved, 0))
|
||||
get_shared_unit(context, resolved, true);
|
||||
|
||||
size_t unit_index = 0;
|
||||
String error = load_unit(resolved, unit_index);
|
||||
if(error != "")
|
||||
@ -1791,7 +1797,7 @@ private:
|
||||
return(std::monostate());
|
||||
}));
|
||||
if(mod == "env" && name == "uce_host_component_resolve")
|
||||
return(add([self](Caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
|
||||
return(add([self](Caller caller, Span<const Val> args, Span<Val> results) -> Result<std::monostate, Trap> {
|
||||
String target, handler, current, resolved;
|
||||
self->hostcall_read(args[0].i32(), args[1].i32(), target);
|
||||
self->hostcall_read(args[2].i32(), args[3].i32(), handler);
|
||||
@ -1806,6 +1812,7 @@ private:
|
||||
self->hostcall_write(args[6].i32(), resolved);
|
||||
}
|
||||
results[0] = Val(slot);
|
||||
caller.context().set_epoch_deadline(self->worker.cfg.epoch_deadline_ticks);
|
||||
return(std::monostate());
|
||||
}));
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user