post-W7 cleanup

docs/setup.md

@@ -41,6 +41,7 @@ UCE also requires two toolchains that are not vendored in this repository:
   - `/opt/wasmtime/include/wasmtime.hh`
   - `/opt/wasmtime/include/wasmtime/*.h`
   - `/opt/wasmtime/lib/libwasmtime.so`
+
 - **WASI SDK** at `/opt/wasi-sdk` by default. `scripts/build_core_wasm.sh` and `scripts/compile_wasm_unit` expect:
   - `/opt/wasi-sdk/bin/clang++`
   - `/opt/wasi-sdk/bin/wasm-ld`
@@ -86,7 +87,7 @@ bin/uce_fastcgi.linux.bin
 
 The UCE build does not download these dependencies for you. Install them through your distro if it provides compatible development packages, or unpack pinned release archives under `/opt/wasmtime` and `/opt/wasi-sdk`.
 
-Do not use `curl | sh` installers in production setup scripts. Download archives from the upstream release pages, verify checksums/signatures when available, and keep the exact versions recorded in your deployment notes.
+Do not use `curl | sh` installers in production setup scripts. Download archives from the upstream release pages, verify checksums/signatures when available, and record the exact versions in your deployment notes. Avoid installing a release published in the last few days unless you have reviewed it separately.
 
 The expected directory shape is:
 
@@ -100,6 +101,36 @@ The expected directory shape is:
 /opt/wasi-sdk/bin/llvm-objcopy
 ```
 
+Example install flow using pinned archive files you have already chosen and verified:
+
+```bash
+mkdir -p /opt /tmp/uce-deps
+cd /tmp/uce-deps
+
+# Download the Wasmtime C API archive for your architecture from the upstream
+# release page, then verify its checksum before unpacking. The archive name
+# normally contains "c-api".
+sha256sum -c wasmtime-c-api.sha256
+mkdir -p /opt/wasmtime
+tar -xf wasmtime-*-c-api*.tar.* -C /opt/wasmtime --strip-components=1
+
+# Download the WASI SDK archive for your architecture from the upstream release
+# page, then verify its checksum before unpacking.
+sha256sum -c wasi-sdk.sha256
+mkdir -p /opt/wasi-sdk
+tar -xf wasi-sdk-*.tar.* -C /opt/wasi-sdk --strip-components=1
+```
+
+After unpacking, verify the tools UCE needs. Also record the exact Wasmtime and WASI SDK versions used. The native build embeds an rpath for `$WASMTIME_HOME/lib`, so the service environment should use the same `WASMTIME_HOME` value used during build.
+
+```bash
+test -f /opt/wasmtime/include/wasmtime.hh
+test -f /opt/wasmtime/lib/libwasmtime.so
+/opt/wasi-sdk/bin/clang++ --version
+/opt/wasi-sdk/bin/wasm-ld --version
+/opt/wasi-sdk/bin/llvm-objcopy --version
+```
+
 If your paths differ, export the variables for manual builds:
 
 ```bash
@@ -128,7 +159,7 @@ systemctl restart uce.service
 
 ## Runtime configuration
 
-Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths if your runtime is not under `/opt/uce`.
+Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths if your runtime is not under `/opt/uce`. Replace any checkout-specific paths such as `WASM_CORE_PATH` with `/opt/uce/bin/wasm/core.wasm` or your actual runtime path.
 
 Minimum useful settings:
 
@@ -141,6 +172,7 @@ FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
 CLI_SOCKET_PATH=/run/uce/cli.sock
 
 SITE_DIRECTORY=/var/www/html
+HTTP_DOCUMENT_ROOT=/var/www/html
 JIT_COMPILE_ON_REQUEST=1
 PROACTIVE_COMPILE_ENABLED=1
 PROACTIVE_COMPILE_CHECK_INTERVAL=60
@@ -161,12 +193,14 @@ HTTP_PORT=8080
 
 Important settings:
 
-- `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests.
+- `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests. Set it explicitly and keep this value and the web-server `fastcgi_pass` path identical. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
 - `CLI_SOCKET_PATH` is a local HTTP-over-Unix socket used by `scripts/uce-cli` and test/admin units.
 - `SITE_DIRECTORY` is the public site tree to scan for `.uce` files. Use `/var/www/html` when the web root is outside the runtime tree; relative paths are resolved from the runtime working directory.
+- `HTTP_DOCUMENT_ROOT` is the root used by the built-in HTTP/WebSocket listener when it resolves upgrade requests. Set it to the same web root as nginx/Apache.
 - `BIN_DIRECTORY` stores generated C++, wasm artifacts, compile output, and runtime caches.
 - `TMP_UPLOAD_PATH` and `SESSION_PATH` must be writable by the runtime.
-- `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes.
+- `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes. Bind/firewall it for local access only; nginx/Apache should be the public entry point.
+- `WASM_COMPILE_SCRIPT` must point to `scripts/compile_wasm_unit` unless you provide an equivalent compiler. Relative paths are resolved from the runtime root/`COMPILER_SYS_PATH`. That script calls `scripts/check_unit_wasm.py` after linking each unit.
 - `WASM_CORE_PATH` must point at the built `core.wasm` file.
 
 After editing settings, restart UCE:
@@ -258,7 +292,7 @@ GET /doc/index.uce?p=component
 nginx/Apache forwards the request to `FCGI_SOCKET_PATH` as FastCGI. The web server must provide CGI/FastCGI variables including:
 
 - `SCRIPT_FILENAME` — full filesystem path to the `.uce` file.
-- `DOCUMENT_ROOT` — public web root, normally `<runtime-root>/site`.
+- `DOCUMENT_ROOT` — public web root, normally `/var/www/html` or whatever nginx/Apache uses as `root`/`DocumentRoot`.
 - `SCRIPT_NAME` — URL path to the script, such as `/doc/index.uce`.
 - `DOCUMENT_URI` — normalized URI path without query string.
 - `REQUEST_URI` — original request URI including query string.
@@ -393,6 +427,7 @@ Notes:
 - `fastcgi_pass` must match `FCGI_SOCKET_PATH`.
 - `proxy_pass` must match `HTTP_PORT`.
 - The example routes WebSocket upgrades for `.uce` paths to the HTTP/WebSocket listener.
+- The built-in HTTP/WebSocket listener resolves scripts from `HTTP_DOCUMENT_ROOT`; do not depend on client-supplied or proxied `Script-Filename` headers for routing.
 - Ordinary `.uce` page loads continue to use FastCGI.
 - Keep `root` pointed at `/var/www/html`, not the runtime repository root.
 - If your app uses a front controller, replace `location /` with a `try_files` rule that ends at `/index.uce`.
@@ -520,7 +555,37 @@ scripts/uce-cli /tests/cli.uce action=echo message=hello
 scripts/run_cli_tests.sh
 ```
 
-Check WebSocket routing with a WebSocket client against a `.uce` endpoint that defines `WS(Request& context)` through nginx/Apache, not directly against `HTTP_PORT`.
+Check WebSocket routing with a WebSocket client against a `.uce` endpoint that defines `WS(Request& context)` through nginx/Apache, not directly against `HTTP_PORT`:
+
+```bash
+python3 - <<'PY'
+import base64, os, socket
+host = "example.com"
+path = "/chat.uce"
+key = base64.b64encode(os.urandom(16)).decode()
+request = (
+    f"GET {path} HTTP/1.1\r\n"
+    f"Host: {host}\r\n"
+    "Upgrade: websocket\r\n"
+    "Connection: Upgrade\r\n"
+    f"Sec-WebSocket-Key: {key}\r\n"
+    "Sec-WebSocket-Version: 13\r\n\r\n"
+).encode()
+sock = socket.create_connection(("127.0.0.1", 80), timeout=5)
+sock.sendall(request)
+print(sock.recv(4096).decode("latin1", "replace").split("\r\n", 1)[0])
+sock.close()
+PY
+```
+
+## Operational footguns
+
+- Keep the FastCGI socket path consistent: `FCGI_SOCKET_PATH` and the web-server `fastcgi_pass` must match exactly. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
+- Keep the public web root separate from the runtime source tree. The examples use `/opt/uce` for runtime files and `/var/www/html` for public files.
+- Set `HTTP_DOCUMENT_ROOT` when the web root is outside the runtime working directory. The built-in HTTP/WebSocket listener resolves upgrade paths from this setting.
+- Do not expose `CLI_SOCKET_PATH` or `HTTP_PORT` as public entry points. The public path should be nginx/Apache.
+- Do not trust `Script-Filename` request headers from direct HTTP clients. The built-in HTTP listener resolves from `HTTP_DOCUMENT_ROOT` and rejects `..` path segments.
+- After toolchain or compile-script fixes, clear stale failed artifacts under `BIN_DIRECTORY`; otherwise a later request may report an old compile failure.
 
 ## Troubleshooting
 
@@ -549,11 +614,22 @@ Check:
 - the client sends `Upgrade: websocket`
 - `.uce` upgrade traffic reaches `HTTP_PORT`
 - nginx/Apache preserves `Upgrade` and `Connection` headers
+- `HTTP_DOCUMENT_ROOT` matches the web-server root; if it points at the runtime tree while files live in `/var/www/html`, the built-in listener will return `404 script not found`
 - firewall/network policy allows localhost access to `HTTP_PORT`
 
 ### Page compiles fail
 
-Check the compile artifact paths shown in the UCE error response and service logs. Generated files and compile output live under `BIN_DIRECTORY`. A safe last-resort recovery step is to stop UCE, move the affected generated artifact directory aside, and restart so the runtime recompiles from source.
+Check the compile artifact paths shown in the UCE error response and service logs. Generated files and compile output live under `BIN_DIRECTORY`.
+
+Common compile footguns:
+
+- `WASM_COMPILE_SCRIPT` is unset or points at a removed script such as `scripts/compile`; set it to `scripts/compile_wasm_unit`.
+- `scripts/check_unit_wasm.py` is missing or not executable; `scripts/compile_wasm_unit` calls it after linking each unit.
+- `WASI_SDK` does not point at a tree with `clang++`, `wasm-ld`, and `llvm-objcopy`.
+- `WASMTIME_HOME` does not point at a tree with Wasmtime headers and `libwasmtime.so`.
+- A previous failed compile left stale `.compile.txt`, `.wasm-check.txt`, or partial `.wasm` files under `BIN_DIRECTORY`.
+
+Failed compile output is persisted under the unit's generated path in `BIN_DIRECTORY` and may be reused until the source or compiler inputs change. First fix the source/toolchain issue and reload the page. If the cache itself is suspect, stop UCE, move only the affected unit artifact files or directory aside, and restart so the runtime recompiles from source. Avoid deleting the whole `BIN_DIRECTORY` unless you intentionally want a full rebuild.
 
 ### CLI commands fail
 

src/fastcgi/src/fcgicc.cc

@@ -812,12 +812,20 @@ FastCGIServer::process_http_request(FastCGIRequest& request, String& data)
 
 	if(resolve_http_script_filename && request.params["SCRIPT_FILENAME"] == "" && request.params["DOCUMENT_URI"] != "")
 	{
-		String document_root = http_script_root();
+		String document_root = first(http_document_root, http_script_root());
 		String document_uri = strip_leading_slashes(request.params["DOCUMENT_URI"]);
+		for(String part : split(document_uri, "/"))
+		{
+			if(part == "..")
+			{
+				reject_http_connection(*client_sockets[request.resources.client_socket], "HTTP/1.1 404 Not Found", "script not found\n");
+				return;
+			}
+		}
 		String candidate = path_join(document_root, document_uri);
 		String real_root = path_real(document_root);
 		String real_candidate = path_real(candidate);
-		if(real_root == "" || real_candidate == "" || !path_is_within(candidate, document_root))
+		if(real_root == "" || real_candidate == "" || !path_is_within(real_candidate, real_root))
 		{
 			reject_http_connection(*client_sockets[request.resources.client_socket], "HTTP/1.1 404 Not Found", "script not found\n");
 			return;

src/fastcgi/src/fcgicc.h

@@ -62,6 +62,7 @@ public:
 	void process_forever();
 	int calls_until_termination = 8; // set this to -1 to never terminate
 	bool resolve_http_script_filename = true;
+	String http_document_root = "";
 
 	typedef unsigned RequestID;
 	typedef std::map<RequestID, FastCGIRequest*> RequestList;

src/linux_fastcgi.cpp

@@ -982,6 +982,7 @@ void run_ws_broker()
 	close_inherited_server_sockets();   // drop the worker listeners we inherited
 	install_process_fault_handlers();
 	ws_broker.calls_until_termination = -1;
+	ws_broker.http_document_root = first(server_state.config["HTTP_DOCUMENT_ROOT"], path_join(server_state.config["COMPILER_SYS_PATH"], server_state.config["SITE_DIRECTORY"]));
 	// The broker renders nothing itself, so accept every request through to
 	// on_complete (it either forwards to the pool or applies a ws_* batch).
 	ws_broker.on_request = [](FastCGIRequest&) { return 0; };