Compare commits
80
Commits
51cbb19f7d
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
09f743fdcf | ||
|
|
36d6207bf2 | ||
|
|
d0efab7db0 | ||
|
|
3d155203bd | ||
|
|
68b73343a2 | ||
|
|
10e6565037 | ||
|
|
0c75703186 | ||
|
|
91c745c44b | ||
|
|
e22cbf8882 | ||
|
|
80eaf0b69b | ||
|
|
4d68377704 | ||
|
|
a48f6af502 | ||
|
|
88e2ad022d | ||
|
|
93cbc83b82 | ||
|
|
cd993b34e9 | ||
|
|
881dba078a | ||
|
|
15fdfe725d | ||
|
|
a9e24cffc4 | ||
|
|
ccbe945d62 | ||
|
|
4f010d37a0 | ||
|
|
d95fb38183 | ||
|
|
a7576f3db7 | ||
|
|
fd6d472187 | ||
|
|
eb3d2fe9d3 | ||
|
|
14f4920ae8 | ||
|
|
33a7b00c78 | ||
|
|
d4ae51f54b | ||
|
|
322ffd006f | ||
|
|
3a6db7779b | ||
|
|
f379480783 | ||
|
|
1df3becdb2 | ||
|
|
3cbe6f50d4 | ||
|
|
39d4b41e2b | ||
|
|
22efea8ecd | ||
|
|
b9b70659a3 | ||
|
|
39636acf2e | ||
|
|
1952c7d762 | ||
|
|
810c19b042 | ||
|
|
418c5812dc | ||
|
|
0e0c0b6ccc | ||
|
|
495a8ae5f0 | ||
|
|
eda124b15f | ||
|
|
69e8051fd3 | ||
|
|
2a2d802daa | ||
|
|
0517f06925 | ||
|
|
f745ce9413 | ||
|
|
556025a779 | ||
|
|
a5ad5042b7 | ||
|
|
5e140d6a5a | ||
|
|
516ed94032 | ||
|
|
a043fb8fc7 | ||
|
|
339031b6c3 | ||
|
|
72fd0dd53a | ||
|
|
f89e33e241 | ||
|
|
6e4ef8ed7a | ||
|
|
1ebf94b135 | ||
|
|
497a5e89d2 | ||
|
|
da7c607091 | ||
|
|
e176ff09d9 | ||
|
|
075ce0bc07 | ||
|
|
1c78ea4f22 | ||
|
|
37b03eb4d5 | ||
|
|
914cf17f23 | ||
|
|
9beec0dd03 | ||
|
|
7e0b8337ab | ||
|
|
2dd23766e0 | ||
|
|
c28b446541 | ||
|
|
556bb06f81 | ||
|
|
3e423f49dd | ||
|
|
da04af1cbf | ||
|
|
88c637f5a4 | ||
|
|
dbb8ae2c4b | ||
|
|
8dbff881a1 | ||
|
|
5a5450caa8 | ||
|
|
c15fdc3346 | ||
|
|
c9f43d8279 | ||
|
|
b0d475e777 | ||
|
|
672e5b66ef | ||
|
|
1ad05475e8 | ||
|
|
aeab8df208 |
+12
-3
@@ -1,6 +1,11 @@
|
||||
# DOCS-TODO — `/doc` documentation overhaul
|
||||
# Completed `/doc` documentation overhaul (historical plan)
|
||||
|
||||
Briefing for the doc-system overhaul. Read this fully before touching anything.
|
||||
This June 2026 plan is complete and is retained only as implementation history.
|
||||
It is not the current task queue and its one-off execution rules do not govern
|
||||
new UCE work. Current documentation has 269 rendered pages and is covered by
|
||||
the sharded doc gate in `scripts/run_cli_tests.sh`.
|
||||
|
||||
The original briefing follows.
|
||||
The approved design and the two locked decisions are: **(1) examples are LIVE and
|
||||
SELF-VERIFYING** — real UCE code executed at render time, showing source + actual
|
||||
captured output, gated by the test suite; **(2) FULL SWEEP** of all 257 pages.
|
||||
@@ -163,7 +168,11 @@ and the suite stays green.
|
||||
```
|
||||
Expect the current 91 to grow by the new doc gate(s), 0 failed.
|
||||
|
||||
## Hard rules (non-negotiable)
|
||||
## Historical execution rules (no longer active)
|
||||
|
||||
These rules applied only while the overhaul was being carried out in June 2026.
|
||||
They are preserved to explain that work and must not override current repository
|
||||
or operator instructions.
|
||||
|
||||
- **NEVER `git commit`/push/tag.** Not pi, not its subagents, not anyone. Leave a clean
|
||||
working tree and report.
|
||||
|
||||
@@ -14,9 +14,9 @@ Implement the DOCS-TODO.md overhaul for the UCE `/doc` system on the sshfs sourc
|
||||
|
||||
## Current State
|
||||
|
||||
- Status: verifying; Phase 1 + gate complete, StringList conversion complete, Phase 2 example/filler sweep complete.
|
||||
- Last updated: 2026-06-16
|
||||
- Source of truth: `/root/mount_ssh/uce-dev-root-htdocs-uce`
|
||||
- Status: complete; retained as the implementation record for the June 2026 overhaul.
|
||||
- Last updated: 2026-07-20
|
||||
- Source of truth: `/root/mount_ssh/uce-upstream`
|
||||
- Runtime/live target: `root@10.4.2.110:/Code/uce.openfu.com/uce`
|
||||
|
||||
## Goal Tree
|
||||
@@ -38,10 +38,10 @@ Legend: `[ ]` not started, `[~]` in progress, `[x]` done, `[!]` blocked, `[-]` s
|
||||
- Done when: CLI runner checks every doc page and examples.
|
||||
- Verify: full host gate passes.
|
||||
|
||||
## Execution Queue
|
||||
## Completed Execution Queue
|
||||
|
||||
1. G2 area batching/delegation now that the mechanism is proven.
|
||||
2. Keep the host suite green after each area batch.
|
||||
1. Phase 2 area batching and delegation completed.
|
||||
2. The host suite and sharded 269-page documentation gate remain the ongoing regression boundary.
|
||||
|
||||
## Decisions
|
||||
|
||||
@@ -55,6 +55,7 @@ Legend: `[ ]` not started, `[~]` in progress, `[x]` done, `[!]` blocked, `[-]` s
|
||||
- 2026-06-16: Converted retiring `list_unique/list_sort/list_some/list_every/list_find` free functions to `StringList` methods, updated callers and method docs; full host gate passed: `Summary: 353 passed, 0 failed, 0 skipped`.
|
||||
- 2026-06-16: Added live examples across types, string, regex, time, markup, noise, uri, and session area pages; full host gate passed: `Summary: 353 passed, 0 failed, 0 skipped`.
|
||||
- 2026-06-16: Added `:example` blocks to all remaining doc pages, removed full PHP/JS Related Concepts filler, removed duplicate content code fences except resource-bound pages; full host gate passed: `Summary: 353 passed, 0 failed, 0 skipped`.
|
||||
- 2026-07-20: Marked the finished plan historical so it no longer presents completed work or one-off execution rules as current instructions.
|
||||
|
||||
## Change Log
|
||||
|
||||
|
||||
+130
-3
@@ -46,6 +46,8 @@ UCE also requires two non-vendored dependencies. WASI SDK is load-bearing at run
|
||||
- `/opt/wasi-sdk/bin/clang++`
|
||||
- `/opt/wasi-sdk/bin/wasm-ld`
|
||||
- `/opt/wasi-sdk/bin/llvm-objcopy`
|
||||
- `/opt/wasi-sdk/bin/llvm-nm`
|
||||
- `/opt/wasi-sdk/bin/llvm-dwarfdump`
|
||||
|
||||
You can use different install locations by setting environment variables before building and in the systemd service environment:
|
||||
|
||||
@@ -100,6 +102,8 @@ The expected directories are:
|
||||
/opt/wasi-sdk/bin/clang++
|
||||
/opt/wasi-sdk/bin/wasm-ld
|
||||
/opt/wasi-sdk/bin/llvm-objcopy
|
||||
/opt/wasi-sdk/bin/llvm-nm
|
||||
/opt/wasi-sdk/bin/llvm-dwarfdump
|
||||
```
|
||||
|
||||
Install the WASI SDK:
|
||||
@@ -135,6 +139,8 @@ test -f /opt/wasmtime/lib/libwasmtime.so
|
||||
/opt/wasi-sdk/bin/clang++ --version
|
||||
/opt/wasi-sdk/bin/wasm-ld --version
|
||||
/opt/wasi-sdk/bin/llvm-objcopy --version
|
||||
/opt/wasi-sdk/bin/llvm-nm --version
|
||||
/opt/wasi-sdk/bin/llvm-dwarfdump --version
|
||||
```
|
||||
|
||||
If your paths differ, export the variables for manual builds:
|
||||
@@ -163,6 +169,13 @@ systemctl daemon-reload
|
||||
systemctl restart uce.service
|
||||
```
|
||||
|
||||
When `uce.socket` is active, restart only `uce.service` (or use
|
||||
`scripts/systemd/manage-uce-service.sh restart`). The socket unit must retain
|
||||
ownership of the FastCGI listener while workers change. Restarting
|
||||
`uce.socket` alongside the service removes that listener briefly and can expose
|
||||
HTTP 502 responses through nginx. Restart the socket unit only to repair the
|
||||
socket itself, with an explicit outage window.
|
||||
|
||||
## Runtime configuration
|
||||
|
||||
Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths to your checkout root. Replace checkout-specific settings such as `WASM_CORE_PATH` with `<UCE_REPO_ROOT>/bin/wasm/core.wasm` or leave it relative to `COMPILER_SYS_PATH` as appropriate.
|
||||
@@ -179,11 +192,16 @@ FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
|
||||
FCGI_SOCKET_MODE=0666
|
||||
CLI_SOCKET_PATH=/run/uce/cli.sock
|
||||
CLI_SOCKET_MODE=0600
|
||||
CLI_WORKER_COUNT=2
|
||||
CLI_WORKER_MAX_REQUESTS=8
|
||||
|
||||
SITE_DIRECTORY=/var/www/html
|
||||
HTTP_DOCUMENT_ROOT=/var/www/html
|
||||
JIT_COMPILE_ON_REQUEST=1
|
||||
SHOW_DYNAMIC_COMPILE_ERRORS=1
|
||||
SERVE_LAST_KNOWN_GOOD=0
|
||||
PROACTIVE_COMPILE_ENABLED=1
|
||||
PROACTIVE_COMPILE_JOBS=2
|
||||
PROACTIVE_COMPILE_CHECK_INTERVAL=60
|
||||
|
||||
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
|
||||
@@ -192,7 +210,10 @@ WASM_CORE_PATH=<UCE_REPO_ROOT>/bin/wasm/core.wasm
|
||||
WASM_MEMORY_LIMIT_BYTES=536870912
|
||||
WASM_EPOCH_DEADLINE_TICKS=200
|
||||
WASM_EPOCH_PERIOD_MS=50
|
||||
WASM_INVOCATION_TIMEOUT_MS=30000
|
||||
WASM_SERIALIZE_TIMEOUT_SECONDS=120
|
||||
MYSQL_PERSISTENT_POOL_SIZE=8
|
||||
MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS=300
|
||||
|
||||
WORKER_COUNT=4
|
||||
MAX_MEMORY=16777216
|
||||
@@ -206,17 +227,59 @@ Important settings:
|
||||
|
||||
- `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests. Set it explicitly and keep this value and the web-server `fastcgi_pass` path identical. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
|
||||
- `CLI_SOCKET_PATH` is a local HTTP-over-Unix socket used by `scripts/uce-cli` and test/admin units. Keep it private (`CLI_SOCKET_MODE=0600`) unless you intentionally delegate admin/test execution to a trusted Unix group (`0660`).
|
||||
- `CLI_WORKER_COUNT` adds a transport-isolated CLI/test renderer pool. Use at least `2` on a live site so broad test/admin unit loads cannot evict or page out public FastCGI workers' hot modules and a CLI unit can make one nested CLI call without self-deadlocking. `0` preserves the legacy shared pool; dedicated CLI workers are additional to `WORKER_COUNT` and retain their own Wasmtime engine, module cache, and persistent connector pool.
|
||||
- `CLI_WORKER_MAX_REQUESTS` recycles each dedicated CLI worker after this many completed connections (default `8`, maximum `1024`); `0` disables recycling. Public FastCGI workers are not request-count recycled, preserving their warmed module caches. Recycling bounds the module set retained by broad test and administration runs while allowing one CLI invocation/test group to finish uninterrupted.
|
||||
- `FCGI_SOCKET_MODE` and `CLI_SOCKET_MODE` are octal permission modes applied after socket bind. Prefer tightening `FCGI_SOCKET_MODE` to `0660` when nginx/Apache can share a trusted group with the UCE worker.
|
||||
- `SITE_DIRECTORY` is the public site tree to scan for `.uce` files. Use `/var/www/html` when the web root is outside the runtime tree; relative paths are resolved from the runtime working directory. Installed regression gate scripts derive their temporary test root from this setting unless `UCE_TEST_SITE_DIRECTORY` is explicitly provided.
|
||||
- `HTTP_DOCUMENT_ROOT` is the root used by the built-in HTTP/WebSocket listener when it resolves upgrade requests. Set it to the same web root as nginx/Apache.
|
||||
- `BIN_DIRECTORY` stores generated C++, wasm artifacts, compile output, and runtime caches.
|
||||
- `BIN_DIRECTORY` stores runtime state plus ABI-scoped unit generations. Unit
|
||||
C++, wasm, serialized modules, source maps, and compile diagnostics live in
|
||||
`units-c<compiler ABI>-w<core ABI>` so an upgrade cannot mix generations.
|
||||
- `TMP_UPLOAD_PATH` and `SESSION_PATH` must be writable by the runtime.
|
||||
- `SESSION_COOKIE_SECURE=1` adds the `Secure` attribute to UCE-managed session cookies and should be used for HTTPS-only deployments. Leave it `0` only for local/plain-HTTP development.
|
||||
- `MYSQL_PERSISTENT_POOL_SIZE` caps credential-keyed connections retained by each Wasm worker. The default `8` is clamped to `64`; set it to `0` to restore request-lifetime connections. Cached sessions are reset before reuse.
|
||||
- `MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS` retires a pooled connection at the first request boundary after it has been idle for this interval (default `300`, maximum `86400`). Set it to `0` to retain idle connections until capacity eviction, reset failure, worker exit, or database-side closure. Keep a positive value below the database server's idle timeout to avoid attempting stale sessions after traffic resumes. A completely idle UCE worker has no request boundary and therefore does not proactively close sockets.
|
||||
- `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes. Bind/firewall it for local access only; nginx/Apache should be the public entry point.
|
||||
- `WS_BROKER_OUTBOUND_TIMEOUT_SECONDS` controls how long a forwarded WS message can remain queued in the broker before being dropped (default `30`). Set to `0` to disable the timeout.
|
||||
- `WASM_COMPILE_SCRIPT` must point to `scripts/compile_wasm_unit` unless you provide an equivalent compiler. Relative paths are resolved from the runtime root/`COMPILER_SYS_PATH`. That script calls `scripts/check_unit_wasm.py` after linking each unit and uses the pinned WASI SDK on every deployment host.
|
||||
- `SHOW_DYNAMIC_COMPILE_ERRORS=1` makes a failed dynamic `component()`, `unit_render()`, or `unit_call()` show the bounded compiler diagnostic instead of only a generic missing-handler message. Set it to `0` on deployments where source paths and compiler output must not reach HTTP responses.
|
||||
- `SERVE_LAST_KNOWN_GOOD=1` lets HTTP GET/HEAD/OPTIONS requests keep using a compatible complete unit artifact while the proactive compiler builds changed source. It defaults to `0`; CLI and mutation requests always use current code or fail closed. The option requires an enabled proactive compiler with a positive check interval. Failed background builds preserve the prior Wasm, source map, and serialized module until a successful atomic publication replaces them. Missing source or an incompatible compiler/core ABI is never served as last-known-good.
|
||||
- `PROACTIVE_COMPILE_JOBS` selects 1–16 low-priority full-site scanner processes (default `2`). Each canonical unit path has one scanner owner. When last-known-good serving is enabled, the separate higher-priority demand compiler remains reserved for stale units requested over HTTP, so total background compile concurrency can reach this value plus one.
|
||||
- `WASM_CORE_PATH` must point at the built `core.wasm` file.
|
||||
- `WASM_EPOCH_DEADLINE_TICKS` and `WASM_EPOCH_PERIOD_MS` bound one
|
||||
uninterrupted guest CPU segment. `WASM_INVOCATION_TIMEOUT_MS` is the
|
||||
separate absolute wall-clock bound for app-owned unit loading,
|
||||
initialization, the selected handler, and all nested component/unit calls.
|
||||
The three values must be positive integers; the ticker period is capped at
|
||||
`1000` ms and the invocation timeout at `86400000` ms. Invalid values prevent
|
||||
the Wasm backend from starting. The invocation timeout defaults to `30000` and is
|
||||
enforced to the epoch ticker's period resolution. Blocking host helpers
|
||||
retain their own shorter limits and are capped to the remaining invocation
|
||||
budget where the underlying operation is cancellable. Cold entry compilation,
|
||||
explicit `unit_compile()`, dynamic component JIT, transitive `#load` work,
|
||||
compiler and registry lock waits, and the configured runtime-error page all
|
||||
consume the same request budget. A timed-out compiler process group is killed
|
||||
without replacing the prior artifact set or persisting a compile-failure
|
||||
backoff. Request compiles require a zero child exit status and complete staged
|
||||
output; generated C++, exports, source map, Wasm, metadata, cached-module
|
||||
invalidation, and diagnostics are published or rolled back as one guarded
|
||||
generation. Proactive and offline precompile work remain independent of
|
||||
request invocation deadlines. Serialized-module metadata scanning checks the
|
||||
absolute deadline between section headers and before and after every bounded
|
||||
4 KiB positional read. Cache-miss full-artifact reads use the same checks
|
||||
around 64 KiB positional chunks. A single in-progress regular-file syscall
|
||||
cannot be cooperatively interrupted; later reads and parsing cannot overrun the budget.
|
||||
Wasmtime's serialized-module deserialization call is also synchronous and
|
||||
cannot be interrupted; if it stalls, the timeout is reported after that call
|
||||
returns rather than at the nominal wall-clock boundary.
|
||||
Proactive and offline serialization instead runs in a short-lived child
|
||||
bounded by `WASM_SERIALIZE_TIMEOUT_SECONDS` (default `120`, clamped to
|
||||
`1`–`3600`), so each unit releases its pooling allocator arenas and threads
|
||||
on exit instead of accumulating them in the long-lived scanner. Serialization
|
||||
holds the unit compile lock and rechecks the wasm artifact identity before
|
||||
publication; timed-out/failed children leave no dead temporary artifact.
|
||||
Initial/final descriptor identity, unique selected metadata sections, and
|
||||
strict 64-bit LEB high-bit validation reject changed or ambiguous artifacts.
|
||||
|
||||
After editing settings, restart UCE:
|
||||
|
||||
@@ -244,6 +307,49 @@ scripts/systemd/manage-uce-service.sh restart
|
||||
scripts/systemd/manage-uce-service.sh logs 200
|
||||
```
|
||||
|
||||
The server binary accepts only these process modes:
|
||||
|
||||
```bash
|
||||
bin/uce_fastcgi.linux.bin # start the server
|
||||
bin/uce_fastcgi.linux.bin --precompile
|
||||
bin/uce_fastcgi.linux.bin --serialize-module /absolute/unit.uce.wasm
|
||||
bin/uce_fastcgi.linux.bin --help
|
||||
```
|
||||
|
||||
Help and invalid arguments exit before configuration is loaded or any listener
|
||||
is opened. Invalid or extra arguments return status 2. This ordering prevents a
|
||||
diagnostic invocation from replacing or removing a configured Unix socket.
|
||||
|
||||
Managed restart builds and precompiles the complete candidate ABI generation
|
||||
before it switches the service. Precompile uses two low-priority processes by
|
||||
default; set `PRECOMPILE_JOBS` in `/etc/uce/settings.cfg` to tune the bounded
|
||||
1–16 process count for the host. A failed worker, compile, serialization, or
|
||||
result report aborts the switch and leaves the current service running. The
|
||||
managed invocation is bounded to 900 seconds by default; set
|
||||
`UCE_PRECOMPILE_TIMEOUT` on `manage-uce-service.sh restart` to select another
|
||||
GNU `timeout` duration. Timeout exit 124 also aborts before the service switch.
|
||||
|
||||
A trusted direct `--precompile` process may set `UCE_PRECOMPILE_FILES_IN` and
|
||||
`UCE_PRECOMPILE_BIN_DIRECTORY` to isolate that invocation's source scan and
|
||||
artifact registry. Both overrides apply only to precompile mode; normal server
|
||||
workers continue to use `/etc/uce/settings.cfg`. The parallel precompile
|
||||
regression uses private roots so a running proactive compiler cannot consume
|
||||
or publish its controlled race fixtures.
|
||||
|
||||
Direct `--precompile` is a listener-free whole-generation gate, not a
|
||||
single-unit readiness probe. It scans the configured source root, waits for
|
||||
unit publication locks, and has no internal wall-clock timeout. Automation
|
||||
must wrap it in a finite external timeout, as the managed restart does, before
|
||||
starting separately deadline-bounded CLI or HTTP invocations.
|
||||
|
||||
Prefer the managed restart when the service runs as an unprivileged user: it
|
||||
precompiles as that same user. Request-time publication still accepts readable
|
||||
artifacts produced by a trusted administrator. When Linux protected-hardlink
|
||||
policy rejects the normal rollback snapshot, UCE copies the prior artifact
|
||||
under the unit lock; generation markers are replaced atomically instead of
|
||||
requiring write access to the existing inode. Existing readable lock files can
|
||||
be locked without write access, including the shared-PCH lock.
|
||||
|
||||
Equivalent manual systemd service for a source checkout (`<UCE_REPO>` = checkout root):
|
||||
|
||||
```ini
|
||||
@@ -639,6 +745,20 @@ if(valid && password_needs_rehash(encoded))
|
||||
|
||||
`password_hash()` returns a self-contained `$uce$scrypt$...` encoding with a random 16-byte salt and the bounded scrypt parameters `N=65536`, `r=8`, `p=1`. `password_verify()` accepts only structurally valid encodings with bounded cost parameters and compares the derived key in constant time. `password_needs_rehash()` reports malformed, legacy, or non-current parameters so applications can upgrade a credential after a successful legacy verification. Treat an empty hash as an operational failure and never store it. Application-level password length policy, rate limiting, and legacy-format verification remain the application's responsibility.
|
||||
|
||||
## Structured cryptographic operations
|
||||
|
||||
`crypto_operation(request)` is the bounded, algorithm-selected API for structured
|
||||
asymmetric operations. The initial allowlist supports `operation=key_generate`
|
||||
and `operation=jwt_sign` with `algorithm=ES256`. Key generation returns P-256
|
||||
public/private JWKs plus the RFC 7638 thumbprint (`kid`). JWT signing accepts only
|
||||
a consistent P-256 private JWK, forces `alg=ES256`, and emits a compact JWT with
|
||||
a 64-byte JOSE signature. It also supports `cbor_decode`, `cose_es256_parse`, and `es256_verify` with `algorithm=ES256`: inputs are canonical unpadded base64url fields; requests are capped at 32 KiB, while CBOR input is definite-length only and capped at 16 KiB decoded (21,846 base64url characters), 256 nodes, and depth 16; it rejects duplicate keys and trailing bytes, and returns an explicit typed tree (`unsigned`, `negative`, `bytes`, `text`, `array`, `map`) rather than DValue-coerced map keys. COSE accepts only EC2/-7/P-256 with exact 32-byte coordinates and an OpenSSL-valid point. Verification accepts only complete canonical DER ECDSA over `message_base64url` and returns `valid`; malformed material fails closed. V1 has no attestation, Ed25519, or RS256. Unknown operations and algorithms fail closed.
|
||||
|
||||
Existing typed digest, HMAC, password, randomness, and constant-time comparison
|
||||
functions remain separate. `crypto_operation()` exposes no raw signing, arbitrary
|
||||
digest/curve selection, encryption, network fetch, or token exchange. Store
|
||||
private JWKs as secrets.
|
||||
|
||||
## Operational footguns
|
||||
|
||||
- Keep the FastCGI socket path consistent: `FCGI_SOCKET_PATH` and the web-server `fastcgi_pass` must match exactly. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
|
||||
@@ -687,11 +807,18 @@ Common compile footguns:
|
||||
|
||||
- `WASM_COMPILE_SCRIPT` is unset or points at a removed script such as `scripts/compile`; set it to `scripts/compile_wasm_unit`.
|
||||
- `scripts/check_unit_wasm.py` is missing or not executable; `scripts/compile_wasm_unit` calls it after linking each unit.
|
||||
- `WASI_SDK` does not point at the pinned tree with `clang++`, `wasm-ld`, `llvm-objcopy`, and `llvm-nm`; run `scripts/install_wasi_sdk.sh --check-only`.
|
||||
- `WASI_SDK` does not point at the pinned tree with `clang++`, `wasm-ld`, `llvm-objcopy`, `llvm-nm`, and `llvm-dwarfdump`; run `scripts/install_wasi_sdk.sh --check-only`.
|
||||
- `WASMTIME_HOME` does not point at a tree with Wasmtime headers and `libwasmtime.so`.
|
||||
- A previous failed compile left stale `.compile.txt`, `.wasm-check.txt`, or partial `.wasm` files under `BIN_DIRECTORY`.
|
||||
|
||||
Failed compile output is persisted under the unit's generated path in `BIN_DIRECTORY` and may be reused until the source or compiler inputs change. First fix the source/toolchain issue and reload the page. If the cache itself is suspect, stop UCE, move only the affected unit artifact files or directory aside, and restart so the runtime recompiles from source. Avoid deleting the whole `BIN_DIRECTORY` unless you intentionally want a full rebuild.
|
||||
Failed compile output is persisted under the unit's ABI-generation path in
|
||||
`BIN_DIRECTORY` and may be reused until the source or compiler inputs change.
|
||||
The managed `restart` command first runs the new binary's `--precompile` mode as
|
||||
the configured service user while the old service remains live. It restarts
|
||||
systemd only after every scanned unit compiles and serializes successfully.
|
||||
First fix source/toolchain failures and retry; do not remove the prior generation,
|
||||
which remains the rollback path. Avoid deleting the whole `BIN_DIRECTORY` unless
|
||||
you intentionally want a full rebuild and have accepted losing rollback artifacts.
|
||||
|
||||
### CLI commands fail
|
||||
|
||||
|
||||
@@ -31,9 +31,19 @@ UCE expects these executables on each deployment host:
|
||||
/opt/wasi-sdk/bin/wasm-ld
|
||||
/opt/wasi-sdk/bin/llvm-objcopy
|
||||
/opt/wasi-sdk/bin/llvm-nm
|
||||
/opt/wasi-sdk/bin/llvm-dwarfdump
|
||||
```
|
||||
|
||||
`llvm-nm` is used by `scripts/check_unit_wasm.py`, which is called by `scripts/compile_wasm_unit` after linking each unit.
|
||||
`llvm-dwarfdump` is used at unit compile time to extract the compact, out-of-band
|
||||
source map before debug sections are stripped from the runtime artifact. The
|
||||
map is consulted only on a wasm trap.
|
||||
|
||||
The native compiler passes the ABI-generation root to
|
||||
`scripts/compile_wasm_unit`. Unless `UCE_WASM_PCH_DIR` is explicitly set, the
|
||||
unit PCH lives beneath that writable generation rather than a
|
||||
process-user-dependent `/tmp` path. Managed precompile therefore behaves the
|
||||
same inside and outside the service unit.
|
||||
|
||||
## Upgrade policy
|
||||
|
||||
@@ -43,7 +53,9 @@ Treat WASI SDK upgrades like runtime dependency upgrades:
|
||||
2. Record the new release and checksum here.
|
||||
3. Rebuild `bin/wasm/core.wasm` with `scripts/build_core_wasm.sh`.
|
||||
4. Rebuild the native runtime with `scripts/build_linux.sh`.
|
||||
5. Clear or invalidate stale unit wasm artifacts by bumping `UCE_UNIT_ABI_VERSION` when required, or by removing affected generated artifacts under `BIN_DIRECTORY`.
|
||||
5. Bump the compiler or core ABI constant in `src/wasm/abi.h` when required.
|
||||
The managed restart precompiles the resulting isolated unit generation before
|
||||
switching workers; do not overwrite or delete the previous generation.
|
||||
6. Run the full CLI suite including wasm kill tests:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# UCE WASM Runtime Architecture
|
||||
|
||||
Status: current as of the W7e native-pipeline removal (June 2026). This document
|
||||
describes the **runtime architecture as built** — the process topology, the
|
||||
wasm membrane, the unified request dispatch, and the central WebSocket broker.
|
||||
Status: current as of the worker-role isolation and bounded serializer changes
|
||||
(July 2026). This document describes the **runtime architecture as built**. It
|
||||
covers the process topology, wasm membrane, request dispatch, and WebSocket broker.
|
||||
Native `.so` unit execution/dlopen fallback has been removed; the parser and
|
||||
preprocessor remain only as the front-end that emits C++ for wasm side-module
|
||||
compilation.
|
||||
@@ -26,8 +26,8 @@ gets invoked*.
|
||||
|
||||
```
|
||||
┌────────────────────────────┐
|
||||
nginx ──FastCGI──► worker pool (N processes) │ $FCGI_SOCKET_PATH (example `/run/uce/fastcgi.sock`)
|
||||
(port 80 etc.) │ uniform unit renderers │ (FastCGI + CLI)
|
||||
nginx ──FastCGI──► public workers (N processes)│ $FCGI_SOCKET_PATH (example `/run/uce/fastcgi.sock`)
|
||||
(port 80 etc.) │ public unit renderers │
|
||||
└─────────────▲──────────────┘
|
||||
│ forward render (FastCGI, FCGI_SOCKET_PATH)
|
||||
│
|
||||
@@ -42,13 +42,19 @@ gets invoked*.
|
||||
│ dispatcher(s) │ bind addr; forwards to pool
|
||||
└──────────────────┘
|
||||
|
||||
trusted CLI/test calls ──► ┌──────────────────┐
|
||||
(CLI socket) │ CLI workers │ isolated module and
|
||||
│ (M processes) │ connector caches
|
||||
└──────────────────┘
|
||||
|
||||
parent process: spawns/respawns all of the above + the proactive compiler.
|
||||
```
|
||||
|
||||
| Process | Owns | Renders units? | Source |
|
||||
|---|---|---|---|
|
||||
| **Parent** | nothing; supervises children | no | `main()`, `init_base_process()` |
|
||||
| **Worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH` (configured socket path; example `/run/uce/fastcgi.sock`) + `CLI_SOCKET_PATH` | **yes** — the only processes that run wasm | `listen_for_connections()` |
|
||||
| **Public worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH`; also `CLI_SOCKET_PATH` only when `CLI_WORKER_COUNT=0` | **yes** — runs public FastCGI wasm | `listen_for_connections()` |
|
||||
| **CLI worker** (×`CLI_WORKER_COUNT`) | `CLI_SOCKET_PATH` only | **yes** — isolates trusted CLI/test wasm and its module cache | `listen_for_connections()` |
|
||||
| **WS broker** (×1) | `HTTP_PORT` + every live WS connection + `WS_BROKER_SOCKET_PATH` | no — forwards to the pool | `run_ws_broker()` |
|
||||
| **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` |
|
||||
| **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` |
|
||||
@@ -62,6 +68,15 @@ forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across
|
||||
runtime. So the brokers hold the long-lived connection and units respond to
|
||||
events the same way they respond to a page request — through a clean worker.
|
||||
|
||||
The broker-to-worker hop explicitly sets `GATEWAY_INTERFACE=CGI/1.1`. After
|
||||
the Wasm core decodes request parameters it derives the guest's default status
|
||||
syntax from that marker (`Status:` for FastCGI, `HTTP/1.1` for direct HTTP).
|
||||
The FastCGI transport clears completed stdout/stderr as soon as their records
|
||||
are queued and emits no stream records after `FCGI_END_REQUEST`. Together these
|
||||
boundaries prevent a forwarded dynamic response from being parsed as body and
|
||||
wrapped in a second HTTP response. The TCP and custom-server tests reject an
|
||||
inner status line and require the exact dynamic body.
|
||||
|
||||
---
|
||||
|
||||
## 2. The membrane and the DValue ABI
|
||||
@@ -78,9 +93,11 @@ the boundary).
|
||||
children yielding `const DValue&`.
|
||||
- The request context (`params`/`get`/`post`/`cookies`/`session`, the raw body
|
||||
`in`, and—for WS—the connection context) is marshalled into a single `ctx`
|
||||
DValue, UCEB-encoded, and handed to the workspace. The response (body,
|
||||
headers, status, and any `meta` such as `ws_commands`) comes back the same
|
||||
way.
|
||||
DValue and UCEB-encoded. Immutable server configuration is UCEB-encoded once
|
||||
per worker as a flat string map; both byte ranges are written into one guest
|
||||
buffer. The guest decodes the flat map directly into its fresh `Server` and
|
||||
the dynamic tree into its fresh `Request`. The response (body, headers,
|
||||
status, and any `meta` such as `ws_commands`) comes back as UCEB.
|
||||
|
||||
See [`docs/wasm-phase1-dvalue-abi.md`](wasm-phase1-dvalue-abi.md) for the wire
|
||||
format details.
|
||||
@@ -120,10 +137,30 @@ __uce_<base>[_<sanitize(suffix)>]
|
||||
| `exists` | probe only — resolves the unit, loads nothing |
|
||||
|
||||
`sanitize_symbol_suffix()` keeps `[A-Za-z0-9_]` (mirrors `ascii_safe_name`).
|
||||
Unit code is hidden by default so the linker can discard loaded helpers that the
|
||||
unit does not use. Handler macros and the explicit `EXPORT` directive are the
|
||||
only application symbols promoted to default visibility; generated request
|
||||
binding and constructor exports remain part of the runtime ABI. Debug sections
|
||||
are removed after linking while the wasm name section is retained for runtime
|
||||
traces. This keeps each side module's code and public ABI local to its actual
|
||||
handler responsibility instead of duplicating every helper from a loaded app
|
||||
library.
|
||||
|
||||
`wasm_resolve_target(unit, handler)` (`src/wasm/core.cpp`) resolves the source
|
||||
path and looks up the export's funcref slot; `exists` lets callers probe a unit
|
||||
without instantiating it.
|
||||
|
||||
The request entry unit is already loaded by the host, so its selected handler
|
||||
and optional `ONCE` export are placed directly into the workspace table and
|
||||
passed to `uce_wasm_invoke_loaded_entry()`. Dynamic component and unit calls
|
||||
continue through `wasm_resolve_target()`, where relative-path resolution is
|
||||
required. Handler cache keys include the calling unit so identically named
|
||||
relative targets in different directories cannot alias. Once a unit has passed
|
||||
freshness checks and loaded into a request workspace, later handler lookups
|
||||
reuse that immutable request-local instance; `component_loaded_reuse_count`
|
||||
reports those lookups. Opt-in verbose response headers split entry invocation
|
||||
into load, presence lookup, table linking, and core dispatch time.
|
||||
|
||||
`wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
|
||||
backend is initialized and the requested artifact/handler is currently
|
||||
available. If an artifact is cold or stale, dispatch compiles it on demand via
|
||||
@@ -138,8 +175,17 @@ slots, then runs relocations, constructors, and request-pointer binding. This
|
||||
removes repeated Wasmtime import-type traversal without sharing request state or
|
||||
changing core-first, unit-load-order symbol resolution.
|
||||
Each FastCGI child initializes its process-local Wasmtime engine after fork and
|
||||
before entering the accept loop. This preserves Wasmtime's fork boundary while
|
||||
preventing the first request assigned to each worker from paying engine startup.
|
||||
before entering the accept loop. It also creates the linker, resolves the core's
|
||||
host imports into a store-independent Wasmtime `InstancePre`, and births then
|
||||
drops one empty workspace. This preserves Wasmtime's fork boundary while
|
||||
preventing the first request assigned to each worker from paying engine, linker,
|
||||
or pre-instantiation startup.
|
||||
Server configuration is immutable by that point. The worker therefore retains
|
||||
one native `DValue` view and one UCEB encoding of it. Each request transfers
|
||||
those cached bytes and the guest decodes the flat scalar map directly into its
|
||||
fresh `Server`, avoiding both repeated native encoding and an intermediate guest
|
||||
`DValue` tree. Request parameters, body, cookies, session, call data, and
|
||||
response state are never retained this way.
|
||||
Startup duration or failure is written to the service log. The serialized core
|
||||
module lives in the configured writable cache root rather than beside the
|
||||
possibly root-owned deployed `core.wasm`; freshness still uses the deployed
|
||||
@@ -152,6 +198,9 @@ the service and its post-fork workers restart. Direct launches without systemd
|
||||
activation retain the existing configured Unix/TCP listener behavior.
|
||||
The service preserves its runtime directory across service restarts because the
|
||||
socket unit, not the service unit, owns a listener path inside that directory.
|
||||
Normal deployments therefore restart only `uce.service`; restarting
|
||||
`uce.socket` at the same time removes the listener and forfeits the zero-gap
|
||||
handoff.
|
||||
On termination the parent asks render workers to close their listeners, finish
|
||||
accepted connections within the bounded worker drain interval, and only then
|
||||
exits. This prevents an accepted FastCGI request from being reset at handoff;
|
||||
@@ -160,11 +209,33 @@ The graceful signal handler belongs to the parent and render workers. Generic
|
||||
`task()` children restore default termination signals after fork so
|
||||
`task_kill()` and `server_stop()` retain their immediate stop contract.
|
||||
|
||||
Epoch interruption measures uninterrupted guest CPU segments. The common
|
||||
hostcall membrane re-arms the store deadline after every native call, excluding
|
||||
blocking I/O, process waits, hashing, and other host work without weakening a
|
||||
guest loop that makes no hostcalls. Keeping this at the membrane also covers new
|
||||
hostcalls without per-import timeout bookkeeping.
|
||||
Epoch interruption measures uninterrupted guest CPU segments. A separate
|
||||
absolute workspace invocation deadline starts before app-owned entry-unit
|
||||
readiness and cold compilation, loading, and initialization. It remains
|
||||
unchanged through the selected handler, ONCE, every nested component/unit call,
|
||||
and configured runtime-error rendering. The common hostcall membrane checks
|
||||
that deadline before and after every native call and re-arms the store with the
|
||||
smaller of the remaining absolute budget and the CPU-segment budget. A cheap
|
||||
hostcall loop therefore cannot renew an invocation indefinitely. Blocking host
|
||||
helpers retain operation-specific limits and cap them to the remaining
|
||||
invocation budget where the underlying operation is cancellable. Forked task
|
||||
callbacks receive a fresh invocation deadline capped by the task lifetime.
|
||||
The Memcached ASCII hostcall applies one deadline to the complete command write
|
||||
and protocol-framed response read. Retrieval payload lengths are parsed, so a
|
||||
payload containing `END` cannot terminate the response; multi-value and `stats`
|
||||
responses may span socket chunks up to the 8 MiB response bound. Timeout,
|
||||
malformed framing, or overflow shuts down the retained descriptor before
|
||||
returning an empty result, preventing unread response bytes from poisoning the
|
||||
next command on that connection. The native fallback uses the same parser and
|
||||
deadline contract.
|
||||
Synchronous compiler locks, transitive `#load` compilation, and compiler child
|
||||
processes consume that same deadline. Compiler children run in a dedicated
|
||||
process group; timeout kills the group, retains the previous generation, and
|
||||
never records a caller-budget timeout as a persisted source failure. Bounded
|
||||
compiles require a valid zero child exit, cap captured output, and stage
|
||||
generated C++, exports, source maps, Wasm, and metadata. Publication removes
|
||||
the prior serialized module and diagnostics inside the same rollback-protected
|
||||
generation while repeatedly checking the live request deadline.
|
||||
|
||||
`request_perf()` reports worker module-cache hits and misses and divides a miss
|
||||
into artifact lookup, wasm read, custom-section parse, serialized-module
|
||||
@@ -174,31 +245,87 @@ source, bytes, and phase durations so a cold outlier identifies its exact unit
|
||||
and whether it used worker memory, serialized code, compilation, or failed.
|
||||
Absolute source paths and source contents are not exposed. This keeps cold-worker
|
||||
module latency distinguishable without exposing source paths.
|
||||
The same snapshot divides pre-dispatch WASM readiness into entry normalization,
|
||||
mutation freshness, artifact stat, source-generation lookup, dependency
|
||||
freshness, and worker availability. `ready_freshness_full_check_us` and
|
||||
`ready_freshness_cache_hit_count` distinguish full graph validation from a
|
||||
positive worker-local hit; `ready_freshness_us` remains inclusive.
|
||||
`ready_check_count` distinguishes the warm one-check path from an on-demand
|
||||
compile and recheck; repeated snapshot reads retain the initial values.
|
||||
Read-only HTTP entry checks may reuse a positive result for at most ten seconds
|
||||
when the source generation and exact Wasm, metadata, and setup-template
|
||||
identities are unchanged. CLI and mutation requests always validate the complete
|
||||
graph. Misses, expiry, missing tokens, and changed identities also run the full
|
||||
check. Exact repeated load paths are then deduplicated before canonicalization,
|
||||
while distinct aliases are resolved independently so symlink retargets remain
|
||||
visible by the next hard validation.
|
||||
When a current serialized module exists, the worker scans wasm section headers
|
||||
and reads only `dylink.0` and `uce.abi`; it does not fault the multi-megabyte code
|
||||
and data bodies into every new worker. A missing/stale/invalid serialized module
|
||||
still reads, validates, compiles, and republishes the complete wasm artifact.
|
||||
through a bounded 4 KiB positional buffer and retains only `dylink.0`, `uce.abi`,
|
||||
and the tiny `uce.module` identity. It skips code and data bodies rather than
|
||||
issuing byte-at-a-time reads or faulting those bodies into every new worker.
|
||||
The request profile reports the physical read-ahead bytes and positional read
|
||||
count. The scanner checks the same non-renewable invocation deadline before
|
||||
each section and before and after every buffer refill; expiry is returned as the
|
||||
canonical `UCE_INVOCATION_TIMEOUT` error. One in-progress regular-file
|
||||
`pread()` remains the irreducible synchronous boundary. Initial descriptor
|
||||
validation matches device, inode, mode, nanosecond timestamps, and size from
|
||||
the preceding lookup; final validation rejects in-place truncation or mutation.
|
||||
Selected metadata sections must be unique and individually remain at most 1 MiB,
|
||||
which also bounds aggregate retained metadata. Both streamed and full-artifact
|
||||
LEB readers reject nonzero unused bits in the tenth `u64` byte. A
|
||||
missing/stale/invalid serialized module
|
||||
still reads the complete wasm artifact in deadline-checked 64 KiB positional
|
||||
chunks, validates it, compiles it, and republishes the serialization. The full
|
||||
read uses the same initial/final descriptor identity guard.
|
||||
The proactive compiler also creates that serialization immediately after source
|
||||
compilation, keeping first-worker native compilation off the request path.
|
||||
Cold module compilation and deserialization are host work, so `load_unit()`
|
||||
refreshes the epoch deadline before its first guest call. Otherwise a component
|
||||
whose compilation outlasted the guest CPU budget would immediately trap in the
|
||||
following allocator/relocation call even though no guest loop consumed it.
|
||||
Wasmtime's serialized-module deserialization API is synchronous and has no
|
||||
cooperative cancellation point. The absolute invocation timeout is checked
|
||||
immediately after it returns, so a stalled deserialization may overrun the
|
||||
nominal wall boundary before UCE reports the timeout.
|
||||
|
||||
Unit artifacts live beneath an ABI-generation directory such as
|
||||
`BIN_DIRECTORY/units-c13-w7`: `c13` is the compiler/unit-metadata ABI and `w7`
|
||||
is the runtime core ABI. Old and new service binaries therefore never publish
|
||||
or read the same unit path during an ABI transition. The managed restart builds
|
||||
and serializes the complete next generation before stopping the old service;
|
||||
failure aborts the switch and leaves the running generation intact. Old
|
||||
generation directories are retained as an explicit rollback path. Two
|
||||
low-priority precompile processes are used by default so independent units do
|
||||
not serialize deployment time while live workers retain scheduler priority.
|
||||
`PRECOMPILE_JOBS` or `UCE_PRECOMPILE_JOBS` may select 1–16 processes. Unit
|
||||
publication and the shared generation PCH use separate advisory locks, and any
|
||||
worker/reporting failure rejects the candidate generation.
|
||||
|
||||
The proactive compiler and request workers coordinate through per-unit file
|
||||
locks and a lock-protected demand-priority queue under `BIN_DIRECTORY`. Unit
|
||||
locks and a lock-protected demand-priority queue under `BIN_DIRECTORY`. The
|
||||
full-site scan uses two low-priority processes by default, bounded from 1–16 by
|
||||
`PROACTIVE_COMPILE_JOBS`; a stable canonical-path partition gives every unit one
|
||||
scanner owner and keeps retry/backoff state local. A separate higher-priority
|
||||
compiler process exclusively drains the demand queue. This matters when the
|
||||
scanners are already inside long transitive C++ compiles: a requested stale
|
||||
component can rebuild immediately instead of waiting for an unrelated compile.
|
||||
All compiler processes use the same per-unit lock, so concurrent demand and scan
|
||||
discovery cannot publish duplicate artifacts. The priority worker is idle when
|
||||
there is no demand and never scans the site on its own. Unit
|
||||
compilation writes and validates a process-unique temporary wasm file, then
|
||||
publishes it with an atomic rename. When proactive compilation is enabled,
|
||||
read-only HTTP requests keep using the last complete artifact while requesting
|
||||
that stale unit at the head of the compiler queue. Non-read requests never run
|
||||
a stale entry artifact: they return `503 Service Unavailable` with
|
||||
`Retry-After: 1`, allowing the client to retry after the priority rebuild.
|
||||
CLI and explicit compile paths remain synchronous. A failed rebuild removes
|
||||
availability and surfaces the compiler error rather than serving the old unit
|
||||
indefinitely. This freshness contract includes components resolved lazily from
|
||||
an otherwise-current CLI entry unit: if the proactive compiler already owns a
|
||||
stale child's lock, the CLI request joins that compile and waits for the fresh
|
||||
artifact. Read-only HTTP may use the child's last complete artifact instead.
|
||||
publishes it with an atomic rename. By default, requests do not execute stale
|
||||
code: a missing or changed unit is rebuilt synchronously and a failed dynamic
|
||||
build surfaces its bounded compiler diagnostic. `SERVE_LAST_KNOWN_GOOD=1`
|
||||
opts ordinary read-only HTTP requests into using the last complete,
|
||||
ABI-compatible artifact while requesting that stale unit at the head of the
|
||||
compiler queue. The proactive compiler must be enabled for this mode. Failed
|
||||
background builds preserve that artifact and its source map and serialized
|
||||
module until a later successful atomic publication replaces them. Missing
|
||||
source, incompatible ABI generations, CLI, WebSocket dispatch, and mutation
|
||||
requests never use last-known-good code. A stale mutation returns `503 Service
|
||||
Unavailable` with `Retry-After: 1`; CLI and explicit compile paths remain
|
||||
synchronous. If a proactive compiler already owns a stale child's lock, a CLI
|
||||
request joins that compile and waits for the current artifact.
|
||||
|
||||
Before preprocessing, the compiler verifies that the worker can actually read
|
||||
the source. An unreadable path is reported as a source-read failure with a
|
||||
@@ -210,6 +337,19 @@ intentional unreadable-source regression request as an expected source-read
|
||||
failure, keeping production log scans focused on real runtime failures while
|
||||
preserving the failing compile result and diagnostic artifacts.
|
||||
|
||||
Unit linking retains DWARF only long enough for
|
||||
`scripts/build_unit_source_map.py` to extract a compact address/file/line table.
|
||||
The published `.wasm` is debug-stripped and the table is stored beside it as
|
||||
`.wasm.source-map`, keyed to the exact temporary module identity recorded in
|
||||
the wasm's `uce.module` custom section. Normal module loading never reads this
|
||||
sidecar. On a Wasmtime trap, the worker uses structured frame module offsets to
|
||||
load only the matching map and appends source locations to the error. A missing,
|
||||
stale, or malformed map is deliberately non-fatal: the ordinary named wasm
|
||||
backtrace remains available. Generated C++ uses a `#line` directive naming the
|
||||
original `.uce` file, so application frames resolve to application source rather
|
||||
than the generated cache file. Artifact invalidation removes the wasm, serialized
|
||||
module, and source map together.
|
||||
|
||||
---
|
||||
|
||||
## 4. The workspace runtime
|
||||
@@ -218,7 +358,15 @@ Each request gets a fresh **workspace** — a per-request wasm instance tree wit
|
||||
the membrane wired in. `wasm_worker_serve(worker, ctx, entry_unit, handler)`
|
||||
(`src/wasm/worker.cpp`) is the single entry point for *every* mode:
|
||||
|
||||
1. Birth a request-scoped workspace with fresh per-request state. Current production workers instantiate a fresh workspace for each request; any future snapshot/CoW optimization must preserve that request-isolation contract.
|
||||
1. Birth a request-scoped workspace with fresh per-request state. Current
|
||||
production workers create a fresh Store and instantiate the store-independent
|
||||
`InstancePre`; the resulting core instance owns a fresh exported, growable
|
||||
function table, memory, globals, constructors, and request tree. The worker
|
||||
retains only Wasmtime's compiled module, linker, pre-instantiation plan, and
|
||||
store-independent host-function definitions. Every callback resolves the one
|
||||
active request workspace at invocation. Legacy cores that import their table
|
||||
remain supported through the prior per-request import path. Any future
|
||||
snapshot/CoW optimization must preserve this request-isolation contract.
|
||||
2. Resolve `entry_unit` + `handler` to an export; components referenced at
|
||||
runtime are resolved on demand via the `uce_host_component_resolve` hostcall
|
||||
(`component_resolve()` → `__uce_<...>` slot), loading dependency modules
|
||||
@@ -240,6 +388,25 @@ module cache, adding a visible cold-start request. Request state remains bounded
|
||||
by the fresh workspace and the normal per-request database/resource cleanup;
|
||||
faulted workers still exit and are replaced by the parent.
|
||||
|
||||
A warm entry artifact is checked for compiler/source freshness once before
|
||||
dispatch. A missing or stale artifact is compiled (or demand-prioritized) and
|
||||
then checked again before execution. The second check belongs only to that
|
||||
state-changing branch; repeating it immediately after a successful warm check
|
||||
adds no freshness guarantee.
|
||||
|
||||
The internal request envelope carries application `context.call` as UCEB2,
|
||||
each request `StringMap` as a flat UCEB2 map, scalar session/input metadata as
|
||||
bounded byte segments, and optional WebSocket state as UCEB2. The guest
|
||||
validates the complete envelope before moving those values into a fresh
|
||||
`Request`; transport-only params, cookies, session data, and entry metadata no
|
||||
longer become duplicate `context.call` children. The historical by-value
|
||||
`DValue::operator=(DValue)` symbol remains exported so warmed side-module
|
||||
artifacts stay ABI-compatible. `request_perf()` subdivides birth into policy,
|
||||
import materialization, core instantiation, export/table lookup, and initialization,
|
||||
and context transfer into bytes, host encode, guest allocation/write,
|
||||
guest decode/application, and free. The byte profile separately reports the
|
||||
worker-cached server-configuration portion.
|
||||
|
||||
### Task callbacks and workspace lifetime
|
||||
|
||||
`task()` and `task_repeat()` are fork-backed. The `uce_host_task_spawn` hostcall
|
||||
@@ -397,6 +564,9 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
|
||||
| `WASM_EPOCH_DEADLINE_TICKS` | `200` | Maximum uninterrupted guest CPU segment in epoch ticks; must be positive. |
|
||||
| `WASM_EPOCH_PERIOD_MS` | `50` | Worker epoch-ticker period and timeout resolution; range `1`–`1000` ms. |
|
||||
| `WASM_INVOCATION_TIMEOUT_MS` | `30000` | Absolute app-owned unit load/init/handler/nested-call deadline; range `1`–`86400000` ms and nested calls cannot renew it. |
|
||||
| `FCGI_SOCKET_PATH` | runtime-configured (`/run/uce/fastcgi.sock` in this doc) | Worker pool FastCGI socket (brokers forward here). |
|
||||
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI/admin socket. Keep private; reference `CLI_SOCKET_MODE` is `0600`. |
|
||||
| `FCGI_SOCKET_MODE` | `0666` | Permission mode applied to `FCGI_SOCKET_PATH` after bind; set tighter if nginx/Apache can use a trusted group. |
|
||||
@@ -404,7 +574,15 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
| `HTTP_PORT` | `8080` | Raw HTTP + WebSocket port — owned by the WS broker. |
|
||||
| `WS_BROKER_SOCKET_PATH` | `/run/uce/ws-broker.sock` | Broker command socket for `ws_*` flushes. |
|
||||
| `WS_BROKER_OUTBOUND_TIMEOUT_SECONDS` | `30` | Max lifetime in seconds for queued WS broker forwards before drop. |
|
||||
| `WORKER_COUNT` | `4` | Number of uniform worker processes. |
|
||||
| `WORKER_COUNT` | `4` | Number of public FastCGI worker processes. |
|
||||
| `CLI_WORKER_COUNT` | `0` built-in; `2` in the reference config | Additional CLI-only workers. Two permit one nested CLI invocation without self-deadlock; a positive count prevents test/admin module-cache churn from paging out public workers, while zero preserves the legacy shared pool. |
|
||||
| `CLI_WORKER_MAX_REQUESTS` | `8` | Completed CLI connections before a dedicated CLI worker is recycled (maximum `1024`; `0` disables). Public workers are not request-count recycled. |
|
||||
| `WASM_SERIALIZE_TIMEOUT_SECONDS` | `120` | Deadline for each short-lived serialized-module child; process exit reclaims pooling allocator arenas from proactive scanners. |
|
||||
|
||||
Proactive scanners never construct a Wasmtime serialization engine themselves.
|
||||
Each candidate is serialized by the executable's bounded `--serialize-module`
|
||||
child while holding the unit compile lock. Publication rechecks device, inode,
|
||||
size, mtime, and ctime, and the scanner removes dead child temporary artifacts.
|
||||
|
||||
---
|
||||
|
||||
@@ -423,7 +601,11 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
placeholder-example errors. The dependency-invalidation gate changes a transitive
|
||||
`#load`, then replaces a warmed worker artifact while preserving its
|
||||
whole-second mtime to prove both compiler and worker caches invalidate it. It
|
||||
also rejects an unreadable unit without publishing a wasm artifact, restores
|
||||
also breaks a loaded dependency, verifies the parent compile fails, restores
|
||||
the dependency byte-for-byte, and requires the parent to recover. Failed
|
||||
builds persist the input signature that actually failed, so metadata from an
|
||||
older successful artifact cannot indefinitely defer that rebuild.
|
||||
The gate also rejects an unreadable unit without publishing a wasm artifact, restores
|
||||
its permissions and proves the next CLI request compiles it, then sends 48
|
||||
requests and asserts the observed worker PID set does not exceed
|
||||
`WORKER_COUNT`, guarding against accidental reintroduction of request-count
|
||||
@@ -431,6 +613,29 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
the generated C++ snapshot has been written. The compiler must either retry
|
||||
and serve the post-edit source or fail closed; it must never stamp current
|
||||
source metadata onto wasm produced from an older snapshot.
|
||||
- **Metadata scanner deadline**: an isolated one-worker gate uses a targeted
|
||||
positional-read interposer to prove canonical expiry after one delayed 4 KiB
|
||||
metadata read and after one delayed cache-miss 64 KiB full-artifact read,
|
||||
concurrent truncation and same-size mutation rejection, malformed tenth-byte
|
||||
LEB rejection, duplicate selected-section rejection, and same-worker recovery.
|
||||
- **Core compatibility**: the production core owns and exports its growable
|
||||
function table, enabling `InstancePre`. A separately built legacy
|
||||
`--import-table` core must still pass the demo and 64-request pool-isolation
|
||||
groups before the compatibility fallback changes.
|
||||
`scripts/test_wasm_compile_timeout.sh` runs an isolated one-worker runtime
|
||||
with a two-second request budget. It covers cold entry JIT, guest
|
||||
`unit_compile()`, dynamic component and transitive `#load` compilation,
|
||||
held unit and registry locks, silent nonzero and missing-output compiler
|
||||
results, compiler descendants, staged-output timeout, prior-generation
|
||||
hashes, same-worker recovery, configured-error-page boundedness, residue,
|
||||
foreign-owned offline-precompile artifacts, and deadline-independent offline
|
||||
precompile. Normal rollback snapshots are hard links. If Linux ownership or
|
||||
link policy rejects that fast path, the compiler copies the prior artifact
|
||||
under the same unit lock and keeps the same all-or-nothing publication.
|
||||
Source-generation markers likewise publish through a same-directory rename,
|
||||
so a runtime user can replace a readable marker created by an administrator.
|
||||
Existing foreign-owned unit, registry, generation, and PCH lock files are
|
||||
opened read-only for `flock`; new locks are still created read-write.
|
||||
`scripts/test_cold_component_deadline.sh` separately compiles a deliberately
|
||||
cold component that exceeds the development epoch window and proves the
|
||||
parent request still renders it. The focused shell gates create temporary
|
||||
@@ -449,8 +654,10 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
||||
the caller after a repeated parent invocation. Resolved component paths are
|
||||
canonical absolute paths: equivalent spellings containing `.` or `..` must
|
||||
share one source, compile, artifact, and module-cache identity. The compiler
|
||||
independently enforces the same canonical identity, and persisted failures
|
||||
are reused only when their recorded source path matches the current request.
|
||||
independently enforces the same canonical identity. Within one resolution,
|
||||
identical entry/site bases and identical raw/`.uce` candidate spellings are
|
||||
probed only once while first-match order is preserved. Persisted failures are
|
||||
reused only when their recorded source path matches the current request.
|
||||
`scripts/test_nested_component_props.sh` passes a large prop tree through an
|
||||
outer component, invokes 300 nested components, and verifies both inner and
|
||||
caller props are restored. Its warm-request ceiling guards the request-scope
|
||||
|
||||
@@ -14,6 +14,12 @@ FCGI_PORT=9993
|
||||
CLI_SOCKET_PATH=/run/uce/cli.sock
|
||||
# Keep the CLI/admin socket private by default; set 0660 only for a trusted group.
|
||||
CLI_SOCKET_MODE=0600
|
||||
# Keep CLI/test module caches out of public FastCGI workers. Set to 0 only for
|
||||
# the legacy shared pool; dedicated workers are additional to WORKER_COUNT.
|
||||
CLI_WORKER_COUNT=2
|
||||
# Recycle dedicated CLI workers before broad test/admin runs retain every unit.
|
||||
# Zero disables recycling; public FastCGI workers are never request-count recycled.
|
||||
CLI_WORKER_MAX_REQUESTS=8
|
||||
|
||||
# Built-in HTTP/WebSocket listener used for WebSocket Upgrade requests.
|
||||
# Keep this behind nginx/Apache on localhost or firewall it from public access.
|
||||
@@ -27,26 +33,50 @@ WS_BROKER_OUTBOUND_TIMEOUT_SECONDS=30
|
||||
# Leave empty to scan SITE_DIRECTORY relative to the runtime root.
|
||||
PRECOMPILE_FILES_IN=
|
||||
|
||||
# NUMBER OF LOW-PRIORITY PROCESSES USED BY MANAGED GENERATION PRECOMPILE
|
||||
# Two keeps one half of a typical four-vCPU host available to live workers.
|
||||
PRECOMPILE_JOBS=2
|
||||
|
||||
# PUBLIC SITE DIRECTORY USED FOR STARTUP SCAN WHEN PRECOMPILE_FILES_IN IS EMPTY
|
||||
SITE_DIRECTORY=site
|
||||
|
||||
# ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT
|
||||
JIT_COMPILE_ON_REQUEST=1
|
||||
|
||||
# SHOW THE BOUNDED COMPILER DIAGNOSTIC WHEN DYNAMIC DISPATCH TARGETS A UNIT
|
||||
# THAT FAILED TO COMPILE. WHEN DISABLED, THE CALL REPORTS THE GENERIC MISSING
|
||||
# COMPONENT OR UNIT ERROR.
|
||||
SHOW_DYNAMIC_COMPILE_ERRORS=1
|
||||
|
||||
# KEEP SERVING A COMPATIBLE LAST-KNOWN-GOOD ARTIFACT TO HTTP READ REQUESTS
|
||||
# WHILE THE PROACTIVE COMPILER BUILDS A CHANGED UNIT. MUTATIONS AND CLI NEVER
|
||||
# SERVE STALE CODE. REQUIRES THE PROACTIVE COMPILER AND DEFAULTS FAIL-CLOSED.
|
||||
SERVE_LAST_KNOWN_GOOD=0
|
||||
|
||||
# WASM SIDE-MODULE COMPILER USED FOR .uce UNITS
|
||||
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
|
||||
|
||||
# WASM RUNTIME SETTINGS. Unit execution is always routed through wasm.
|
||||
WASM_BACKEND_VERBOSE=0
|
||||
WASM_PROFILE_HOSTCALL_CPU=0
|
||||
WASM_PROFILE_THREAD_RUNTIME=0
|
||||
WASM_CORE_PATH=bin/wasm/core.wasm
|
||||
WASM_MEMORY_LIMIT_BYTES=536870912
|
||||
WASM_EPOCH_DEADLINE_TICKS=200
|
||||
WASM_EPOCH_PERIOD_MS=50
|
||||
WASM_INVOCATION_TIMEOUT_MS=30000
|
||||
# Bound each short-lived proactive/precompile serialized-module child.
|
||||
WASM_SERIALIZE_TIMEOUT_SECONDS=120
|
||||
MYSQL_PERSISTENT_POOL_SIZE=8
|
||||
MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS=300
|
||||
|
||||
# ENABLE THE BACKGROUND PROACTIVE COMPILER LOOP
|
||||
PROACTIVE_COMPILE_ENABLED=1
|
||||
|
||||
# NUMBER OF LOW-PRIORITY FULL-SITE SCANNER PROCESSES (1-16)
|
||||
# The separate demand-priority compiler is not counted here.
|
||||
PROACTIVE_COMPILE_JOBS=2
|
||||
|
||||
# AFTER A FAILED COMPILE, UCE NOW REUSES THE PERSISTED COMPILER OUTPUT UNTIL
|
||||
# THE SOURCE OR COMPILER INPUTS CHANGE. THIS SETTING IS KEPT FOR COMPATIBILITY.
|
||||
COMPILE_FAILURE_RETRY_SECONDS=10
|
||||
|
||||
@@ -39,6 +39,7 @@ PUBLIC_APIS = [
|
||||
("hmac_sha256", True, "public"), ("hmac_sha256_hex", True, "public"), ("random_bytes", True, "public"),
|
||||
("crypto_equal", True, "public"), ("password_hash", True, "public"),
|
||||
("password_verify", True, "public"), ("password_needs_rehash", True, "public"),
|
||||
("crypto_operation", True, "public"),
|
||||
("gen_noise32", True, "public"), ("gen_noise64", True, "public"),
|
||||
("gen_noise01", True, "public"), ("gen_int", True, "public"), ("gen_float", True, "public"),
|
||||
("draw_int", True, "public"), ("draw_float", True, "public"),
|
||||
|
||||
@@ -21,7 +21,8 @@ fi
|
||||
-Wl,--export-all \
|
||||
-Wl,--export=__heap_base \
|
||||
-Wl,--export=__stack_pointer \
|
||||
-Wl,--import-table \
|
||||
-Wl,--export-table \
|
||||
-Wl,--growable-table \
|
||||
-Wl,-z,stack-size=8388608 \
|
||||
-Wl,--allow-undefined-file=src/wasm/core_hostcalls.syms \
|
||||
-Wl,--no-entry \
|
||||
|
||||
+28
-17
@@ -1,4 +1,5 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
cd ..
|
||||
|
||||
@@ -6,11 +7,14 @@ BUILDMODE=${2:-"debug"}
|
||||
OPT_FLAG="O0"
|
||||
GF="uce_fastcgi"
|
||||
|
||||
mkdir bin > /dev/null 2>&1
|
||||
mkdir bin/tmp > /dev/null 2>&1
|
||||
mkdir bin/assets > /dev/null 2>&1
|
||||
mkdir bin/wasm > /dev/null 2>&1
|
||||
mkdir work > /dev/null 2>&1
|
||||
mkdir -p bin/tmp bin/assets bin/wasm work
|
||||
exec 9>bin/.build.lock
|
||||
flock 9
|
||||
build_tmp_files=()
|
||||
cleanup_build_tmp() {
|
||||
((${#build_tmp_files[@]} == 0)) || rm -f "${build_tmp_files[@]}"
|
||||
}
|
||||
trap cleanup_build_tmp EXIT
|
||||
|
||||
COMPILER="clang++"
|
||||
# -rdynamic is a link-time flag; the -c compiles below do not need it.
|
||||
@@ -52,13 +56,16 @@ fi
|
||||
# SQLite: vendored C, depends only on its own source (not our headers).
|
||||
if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then
|
||||
echo "Compiling SQLite..."
|
||||
tmp="bin/sqlite3.o.tmp.$$"
|
||||
build_tmp_files+=("$tmp")
|
||||
clang -g -O2 -fPIC \
|
||||
-DSQLITE_THREADSAFE=1 \
|
||||
-DSQLITE_OMIT_LOAD_EXTENSION=1 \
|
||||
-DSQLITE_DQS=0 \
|
||||
-DSQLITE_DEFAULT_FOREIGN_KEYS=1 \
|
||||
-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1 \
|
||||
-c src/3rdparty/sqlite/sqlite3.c -o bin/sqlite3.o 2>&1 || exit 1
|
||||
-c src/3rdparty/sqlite/sqlite3.c -o "$tmp" 2>&1
|
||||
mv "$tmp" bin/sqlite3.o
|
||||
else
|
||||
echo "Reusing bin/sqlite3.o"
|
||||
fi
|
||||
@@ -67,7 +74,10 @@ fi
|
||||
# declarations (not the lib .cpp — those are compiled into main.o).
|
||||
if needs_rebuild bin/wasm.o src/wasm src/lib/*.h; then
|
||||
echo "Compiling wasm backend..."
|
||||
time -p $COMPILER -c src/wasm/wasm_module.cpp $SRCFLAGS $FLAGS $WASM_FLAGS -o bin/wasm.o 2>&1 || exit 1
|
||||
tmp="bin/wasm.o.tmp.$$"
|
||||
build_tmp_files+=("$tmp")
|
||||
time -p $COMPILER -c src/wasm/wasm_module.cpp $SRCFLAGS $FLAGS $WASM_FLAGS -o "$tmp" 2>&1
|
||||
mv "$tmp" bin/wasm.o
|
||||
else
|
||||
echo "Reusing bin/wasm.o"
|
||||
fi
|
||||
@@ -77,18 +87,19 @@ fi
|
||||
# (its only view of the wasm object) — but not the wasm .cpp sources.
|
||||
if needs_rebuild bin/main.o src/linux_fastcgi.cpp src/lib src/fastcgi src/wasm/backend.h; then
|
||||
echo "Compiling main..."
|
||||
time -p $COMPILER -c src/linux_fastcgi.cpp $SRCFLAGS $FLAGS -o bin/main.o 2>&1 || exit 1
|
||||
tmp="bin/main.o.tmp.$$"
|
||||
build_tmp_files+=("$tmp")
|
||||
time -p $COMPILER -c src/linux_fastcgi.cpp $SRCFLAGS $FLAGS -o "$tmp" 2>&1
|
||||
mv "$tmp" bin/main.o
|
||||
else
|
||||
echo "Reusing bin/main.o"
|
||||
fi
|
||||
|
||||
echo "Linking..."
|
||||
$COMPILER -rdynamic bin/main.o bin/wasm.o bin/sqlite3.o $FLAGS $LIBS -o bin/$GF.linux.bin 2>&1
|
||||
|
||||
if [ $? -eq 0 ]
|
||||
then
|
||||
ls -lh bin/ | grep $GF
|
||||
exit 0
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
binary_tmp="bin/$GF.linux.bin.tmp.$$"
|
||||
build_tmp_files+=("$binary_tmp")
|
||||
$COMPILER -rdynamic bin/main.o bin/wasm.o bin/sqlite3.o $FLAGS $LIBS -o "$binary_tmp" 2>&1
|
||||
test -s "$binary_tmp"
|
||||
chmod 0755 "$binary_tmp"
|
||||
mv "$binary_tmp" "bin/$GF.linux.bin"
|
||||
ls -lh "bin/$GF.linux.bin"
|
||||
|
||||
Executable
+75
@@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract a compact address-to-source table before a unit's DWARF is stripped."""
|
||||
|
||||
import argparse
|
||||
import ast
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
|
||||
|
||||
DIRECTORY = re.compile(r'^include_directories\[\s*(\d+)\] = (".*")$')
|
||||
FILE = re.compile(r'^file_names\[\s*(\d+)\]:$')
|
||||
ROW = re.compile(r'^0x([0-9a-fA-F]+)\s+(\d+)\s+(\d+)\s+(\d+)\s+')
|
||||
|
||||
|
||||
def quoted(value: str) -> str:
|
||||
return ast.literal_eval(value)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--dwarfdump", required=True)
|
||||
parser.add_argument("--wasm", required=True)
|
||||
parser.add_argument("--module", required=True)
|
||||
parser.add_argument("--output", required=True)
|
||||
args = parser.parse_args()
|
||||
|
||||
text = subprocess.run(
|
||||
[args.dwarfdump, "--debug-line", args.wasm],
|
||||
check=True,
|
||||
stdout=subprocess.PIPE,
|
||||
text=True,
|
||||
).stdout
|
||||
directories: dict[int, str] = {}
|
||||
files: dict[int, tuple[str, int]] = {}
|
||||
rows: list[tuple[int, int, int, int]] = []
|
||||
current_file = 0
|
||||
for line in text.splitlines():
|
||||
if match := DIRECTORY.match(line):
|
||||
directories[int(match.group(1))] = quoted(match.group(2))
|
||||
continue
|
||||
if match := FILE.match(line):
|
||||
current_file = int(match.group(1))
|
||||
continue
|
||||
stripped = line.strip()
|
||||
if current_file and stripped.startswith("name: "):
|
||||
files[current_file] = (quoted(stripped[6:]), 0)
|
||||
continue
|
||||
if current_file and stripped.startswith("dir_index: "):
|
||||
name, _ = files[current_file]
|
||||
files[current_file] = (name, int(stripped[11:]))
|
||||
continue
|
||||
if match := ROW.match(line):
|
||||
rows.append(tuple(int(value, 16 if index == 0 else 10) for index, value in enumerate(match.groups())))
|
||||
|
||||
paths: dict[int, str] = {}
|
||||
for file_id, (name, directory_id) in files.items():
|
||||
directory = directories.get(directory_id, "")
|
||||
path = name if os.path.isabs(name) else os.path.normpath(os.path.join(directory, name))
|
||||
if not os.path.isabs(path):
|
||||
path = os.path.abspath(path)
|
||||
if "\t" in path or "\n" in path or "\r" in path:
|
||||
raise ValueError(f"source-map path contains a control character: {path!r}")
|
||||
paths[file_id] = path
|
||||
|
||||
with open(args.output, "w", encoding="utf-8") as output:
|
||||
output.write(f"UCE_SOURCE_MAP_V1\t{args.module}\n")
|
||||
for file_id, path in sorted(paths.items()):
|
||||
output.write(f"F\t{file_id}\t{path}\n")
|
||||
for address, line, column, file_id in rows:
|
||||
output.write(f"L\t{address:x}\t{file_id}\t{line}\t{column}\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -160,7 +160,13 @@ def defined_symbols(path: Path, llvm_nm: str) -> list[str]:
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("wasm", type=Path)
|
||||
ap.add_argument("--abi-version", default="6")
|
||||
default_abi = "7"
|
||||
abi_header = Path(__file__).resolve().parents[1] / "src" / "wasm" / "abi.h"
|
||||
for line in abi_header.read_text().splitlines():
|
||||
if line.startswith("#define UCE_WASM_CORE_ABI_VERSION "):
|
||||
default_abi = line.rsplit(" ", 1)[1]
|
||||
break
|
||||
ap.add_argument("--abi-version", default=default_abi)
|
||||
ap.add_argument("--llvm-nm", default=None)
|
||||
ap.add_argument("--verbose", action="store_true")
|
||||
args = ap.parse_args()
|
||||
@@ -182,6 +188,9 @@ def main() -> int:
|
||||
for needle in required:
|
||||
if needle not in abi_text:
|
||||
errors.append(f"uce.abi missing {needle!r}")
|
||||
module_payloads = customs.get("uce.module", [])
|
||||
if not module_payloads or not module_payloads[-1]:
|
||||
errors.append("missing uce.module custom section")
|
||||
export_names = {name for name, _ in exports}
|
||||
forbidden_exports = {"uce_alloc", "uce_free"}
|
||||
for name in sorted(export_names & forbidden_exports):
|
||||
|
||||
@@ -9,18 +9,21 @@ DEST_DIR="$2"
|
||||
SRC_FN="$3"
|
||||
PP_FN="$4"
|
||||
WASM_FN="$5"
|
||||
UNIT_CACHE_DIR="${6:-$DEST_DIR}"
|
||||
|
||||
SDK=${WASI_SDK:-/opt/wasi-sdk}
|
||||
ABI_VERSION=${UCE_UNIT_ABI_VERSION:-6}
|
||||
ABI_VERSION=${UCE_UNIT_ABI_VERSION:-$(awk '/^#define UCE_WASM_CORE_ABI_VERSION / {print $3; exit}' src/wasm/abi.h)}
|
||||
ROOT=$(pwd)
|
||||
OBJ_FN="$DEST_DIR/$PP_FN.wasm.o"
|
||||
ABI_TMP="$DEST_DIR/$PP_FN.uce-abi.txt"
|
||||
MODULE_TMP="$DEST_DIR/$PP_FN.uce-module.txt"
|
||||
WASM_TMP="$DEST_DIR/$WASM_FN.tmp.$$"
|
||||
MAP_TMP="$DEST_DIR/$WASM_FN.source-map.tmp.$$"
|
||||
PCH_ENABLED=${UCE_WASM_UNIT_PCH:-1}
|
||||
PCH_DIR=${UCE_WASM_PCH_DIR:-/tmp/uce/wasm-w2/pch}
|
||||
PCH_DIR=${UCE_WASM_PCH_DIR:-$UNIT_CACHE_DIR/.pch}
|
||||
COMMON_FLAGS=(
|
||||
--target=wasm32-wasip1
|
||||
-fPIC -fvisibility=default -fvisibility-inlines-hidden
|
||||
-fPIC -fvisibility=hidden -fvisibility-inlines-hidden
|
||||
-O1 -g -std=c++20
|
||||
# The server captures this script's output and treats any non-empty result as
|
||||
# a compile failure (then drops the .wasm), so a successful build must be
|
||||
@@ -33,7 +36,7 @@ COMMON_FLAGS=(
|
||||
-DPLATFORM_NAME=\"wasm32-wasip1\"
|
||||
)
|
||||
|
||||
if [ ! -x "$SDK/bin/clang++" ] || [ ! -x "$SDK/bin/wasm-ld" ] || [ ! -x "$SDK/bin/llvm-objcopy" ]; then
|
||||
if [ ! -x "$SDK/bin/clang++" ] || [ ! -x "$SDK/bin/wasm-ld" ] || [ ! -x "$SDK/bin/llvm-objcopy" ] || [ ! -x "$SDK/bin/llvm-dwarfdump" ]; then
|
||||
echo "wasi-sdk tools not found; set WASI_SDK" >&2
|
||||
exit 1
|
||||
fi
|
||||
@@ -43,22 +46,31 @@ HEADER_HASH=$(find src/lib -maxdepth 1 -name '*.h' -type f -print0 | sort -z | x
|
||||
FLAGS_HASH=$(printf '%s\0' "${COMMON_FLAGS[@]}" -Isrc/lib | sha1sum | cut -c1-16)
|
||||
PCH_KEY=$(printf '%s\n%s\n%s\n%s\n' "$ABI_VERSION" "$TOOLCHAIN_ID" "$HEADER_HASH" "$FLAGS_HASH" | sha1sum | cut -c1-16)
|
||||
PCH_FN="$PCH_DIR/uce_lib-wasm-unit-$PCH_KEY.pch"
|
||||
PCH_TMP="$PCH_FN.tmp.$$"
|
||||
|
||||
mkdir -p "$DEST_DIR" >/dev/null 2>&1
|
||||
trap 'rm -f "$OBJ_FN" "$ABI_TMP" "$WASM_TMP"' EXIT
|
||||
trap 'rm -f "$OBJ_FN" "$ABI_TMP" "$MODULE_TMP" "$WASM_TMP" "$MAP_TMP" "$PCH_TMP"' EXIT
|
||||
|
||||
build_pch_if_needed() {
|
||||
if [ "$PCH_ENABLED" = "0" ]; then
|
||||
return 0
|
||||
fi
|
||||
mkdir -p "$PCH_DIR"
|
||||
if ! { exec 9>"$PCH_FN.lock"; } 2>/dev/null; then
|
||||
exec 9<"$PCH_FN.lock"
|
||||
fi
|
||||
flock 9
|
||||
if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then
|
||||
flock -u 9
|
||||
exec 9>&-
|
||||
return 0
|
||||
fi
|
||||
"$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \
|
||||
-Isrc/lib \
|
||||
-x c++-header src/lib/uce_lib.h -o "$PCH_FN.tmp"
|
||||
mv "$PCH_FN.tmp" "$PCH_FN"
|
||||
-x c++-header src/lib/uce_lib.h -o "$PCH_TMP"
|
||||
mv "$PCH_TMP" "$PCH_FN"
|
||||
flock -u 9
|
||||
exec 9>&-
|
||||
}
|
||||
|
||||
cat > "$ABI_TMP" <<EOF
|
||||
@@ -67,6 +79,7 @@ unit_abi_version=$ABI_VERSION
|
||||
toolchain=$TOOLCHAIN_ID
|
||||
source=$SRC_FN
|
||||
EOF
|
||||
printf '%s' "$(basename "$WASM_TMP")" > "$MODULE_TMP"
|
||||
|
||||
build_pch_if_needed
|
||||
PCH_FLAGS=()
|
||||
@@ -80,6 +93,7 @@ fi
|
||||
-c "$DEST_DIR/$PP_FN" -o "$OBJ_FN"
|
||||
|
||||
"$SDK/bin/wasm-ld" -shared --experimental-pic \
|
||||
--gc-sections \
|
||||
--unresolved-symbols=import-dynamic \
|
||||
--Bsymbolic \
|
||||
"$OBJ_FN" -o "$WASM_TMP" \
|
||||
@@ -92,9 +106,19 @@ fi
|
||||
--export-if-defined=__uce_once \
|
||||
--export-if-defined=__uce_init
|
||||
|
||||
"$SDK/bin/llvm-objcopy" --add-section=uce.abi="$ABI_TMP" "$WASM_TMP"
|
||||
python3 scripts/build_unit_source_map.py \
|
||||
--dwarfdump "$SDK/bin/llvm-dwarfdump" \
|
||||
--wasm "$WASM_TMP" \
|
||||
--module "$(basename "$WASM_TMP")" \
|
||||
--output "$MAP_TMP"
|
||||
|
||||
"$SDK/bin/llvm-objcopy" --strip-debug \
|
||||
--add-section=uce.abi="$ABI_TMP" \
|
||||
--add-section=uce.module="$MODULE_TMP" \
|
||||
"$WASM_TMP"
|
||||
|
||||
python3 scripts/check_unit_wasm.py "$WASM_TMP" --abi-version "$ABI_VERSION" --llvm-nm "$SDK/bin/llvm-nm"
|
||||
mv "$MAP_TMP" "$DEST_DIR/$WASM_FN.source-map"
|
||||
mv "$WASM_TMP" "$DEST_DIR/$WASM_FN"
|
||||
|
||||
rm -f "$OBJ_FN" "$ABI_TMP"
|
||||
rm -f "$OBJ_FN" "$ABI_TMP" "$MODULE_TMP"
|
||||
|
||||
@@ -13,6 +13,7 @@ StateDirectory=uce
|
||||
CacheDirectory=uce
|
||||
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
|
||||
ExecStart=/usr/lib/uce/bin/uce_fastcgi.linux.bin
|
||||
ExecStartPost=/usr/bin/bash /usr/lib/uce/scripts/systemd/wait-ready.sh
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
TimeoutStopSec=30
|
||||
|
||||
@@ -53,7 +53,7 @@ require_command() {
|
||||
|
||||
verify_tree() {
|
||||
local root="$1"
|
||||
for tool in clang++ wasm-ld llvm-objcopy llvm-nm; do
|
||||
for tool in clang++ wasm-ld llvm-objcopy llvm-nm llvm-dwarfdump; do
|
||||
if [[ ! -x "$root/bin/$tool" ]]; then
|
||||
echo "Missing WASI SDK tool: $root/bin/$tool" >&2
|
||||
return 1
|
||||
|
||||
@@ -58,7 +58,7 @@ if [[ "$action" == "list" ]]; then
|
||||
fi
|
||||
|
||||
if [[ "$action" == "run" ]]; then
|
||||
groups=(demo http site doc-gate security task-lifetime starter tcp)
|
||||
groups=(demo http site doc-gate-{1..30} security task-lifetime pool-isolation starter tcp)
|
||||
if [[ "$include_kill" == "1" ]]; then
|
||||
groups+=(wasm-kill)
|
||||
fi
|
||||
@@ -67,11 +67,32 @@ if [[ "$action" == "run" ]]; then
|
||||
curl -sS --max-time "$curl_timeout" --fail-with-body --unix-socket "$socket_path" "${base_url}&group=${group}"
|
||||
done
|
||||
scripts/test_dependency_invalidation.sh
|
||||
scripts/test_compiler_lock_directory.sh
|
||||
scripts/test_abi_generation_rollout.sh
|
||||
scripts/test_parallel_precompile.sh
|
||||
timeout --signal=TERM --kill-after=5s 175s scripts/test_parallel_proactive_compile.sh
|
||||
scripts/test_cold_component_deadline.sh
|
||||
scripts/test_wasm_invocation_timeout.sh
|
||||
timeout --signal=TERM --kill-after=5s 130s scripts/test_wasm_compile_timeout.sh
|
||||
scripts/test_nested_component_props.sh
|
||||
scripts/test_component_once_prefetch.sh
|
||||
scripts/test_relative_component_cache.sh
|
||||
scripts/test_password_hashing.sh
|
||||
scripts/test_crypto_operation_native.sh
|
||||
scripts/test_hardened_http_native.sh
|
||||
scripts/test_mysql_epoch_refresh.sh
|
||||
scripts/test_mysql_persistent_pool.sh
|
||||
scripts/test_mysql_persistent_pool_idle.sh
|
||||
scripts/test_log_timeliness.sh
|
||||
scripts/test_raw_http_request_log.sh
|
||||
scripts/test_component_resolution_ttl.sh
|
||||
timeout --signal=TERM --kill-after=5s 120s scripts/test_entry_freshness_ttl.sh
|
||||
scripts/test_unit_export_surface.sh
|
||||
timeout --signal=TERM --kill-after=5s 120s scripts/test_wasm_metadata_buffer.sh
|
||||
timeout --signal=TERM --kill-after=5s 150s scripts/test_wasm_metadata_deadline.sh
|
||||
timeout --signal=TERM --kill-after=5s 240s scripts/test_dynamic_compile_failures.sh
|
||||
scripts/test_wasm_source_locations.sh
|
||||
scripts/test_server_arguments.sh
|
||||
scripts/test_cli_worker_isolation.sh
|
||||
scripts/test_socket_activation.sh
|
||||
fi
|
||||
|
||||
@@ -18,6 +18,7 @@ CONFIG_SOURCE="$REPO_ROOT/etc/uce/settings.cfg"
|
||||
CONFIG_DEST="/etc/uce/settings.cfg"
|
||||
|
||||
action="${1:-setup}"
|
||||
precompile_timeout="${UCE_PRECOMPILE_TIMEOUT:-900s}"
|
||||
|
||||
render_runtime_file() {
|
||||
local source="$1"
|
||||
@@ -52,7 +53,22 @@ case "$action" in
|
||||
install_unit
|
||||
systemctl enable "$UNIT_NAME"
|
||||
;;
|
||||
start|stop|restart|status)
|
||||
restart)
|
||||
if [[ "$REPO_ROOT" != "/usr/lib/uce" && "$REPO_ROOT" != "/opt/uce" ]]; then
|
||||
"$REPO_ROOT/scripts/build_linux.sh"
|
||||
fi
|
||||
(
|
||||
cd "$REPO_ROOT"
|
||||
service_user=$(systemctl show "$UNIT_NAME" -p User --value)
|
||||
if [[ -n "$service_user" && "$(id -u)" == "0" ]]; then
|
||||
runuser -u "$service_user" -- timeout --signal=TERM --kill-after=10s "$precompile_timeout" nice -n 10 "$REPO_ROOT/bin/uce_fastcgi.linux.bin" --precompile
|
||||
else
|
||||
timeout --signal=TERM --kill-after=10s "$precompile_timeout" nice -n 10 "$REPO_ROOT/bin/uce_fastcgi.linux.bin" --precompile
|
||||
fi
|
||||
)
|
||||
systemctl restart "$UNIT_NAME"
|
||||
;;
|
||||
start|stop|status)
|
||||
systemctl "$action" "$UNIT_NAME"
|
||||
;;
|
||||
logs)
|
||||
|
||||
@@ -12,8 +12,8 @@ RuntimeDirectoryPreserve=yes
|
||||
StateDirectory=uce
|
||||
CacheDirectory=uce
|
||||
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
|
||||
ExecStartPre=/usr/bin/bash /Code/uce.openfu.com/uce/scripts/build_linux.sh
|
||||
ExecStart=/Code/uce.openfu.com/uce/bin/uce_fastcgi.linux.bin
|
||||
ExecStartPost=/usr/bin/bash /Code/uce.openfu.com/uce/scripts/systemd/wait-ready.sh
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
TimeoutStopSec=30
|
||||
|
||||
Executable
+20
@@ -0,0 +1,20 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
socket_path="/run/uce/cli.sock"
|
||||
if [[ -r /etc/uce/settings.cfg ]]; then
|
||||
configured_socket=$(awk -F= '/^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
if [[ -n "${configured_socket:-}" ]]; then
|
||||
socket_path="$configured_socket"
|
||||
fi
|
||||
fi
|
||||
|
||||
for ((attempt = 0; attempt < 200; attempt++)); do
|
||||
if [[ -S "$socket_path" ]] && [[ "$(curl -sS --max-time 0.2 --unix-socket "$socket_path" http://localhost/ping 2>/dev/null || true)" == "uce-cli: ok" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
sleep 0.05
|
||||
done
|
||||
|
||||
echo "UCE CLI socket did not become ready: $socket_path" >&2
|
||||
exit 1
|
||||
Executable
+101
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="abi-generation-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
[[ -z "${configured_site_directory:-}" ]] || site_directory="$configured_site_directory"
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
|
||||
body_file="/tmp/uce-abi-generation-body-$$"
|
||||
ready_file="/tmp/uce-abi-generation-ready-$$"
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
rm -f "$body_file" "$ready_file"
|
||||
[[ -z "${artifact_dir:-}" ]] || rm -rf "$artifact_dir"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
|
||||
printf '%s\n' \
|
||||
'RENDER(Request& context) { print("entry-current"); }' \
|
||||
'CLI(Request& context) {' \
|
||||
' if(context.get["artifact"] == "entry") print(unit_info(context.params["SCRIPT_FILENAME"])["wasm_name"].to_string());' \
|
||||
' else print("entry-current");' \
|
||||
'}' >"$source_dir/entry.uce"
|
||||
printf '%s\n' \
|
||||
'COMPONENT(Request& context) { print("component-current"); }' >"$source_dir/child.uce"
|
||||
printf '%s\n' \
|
||||
'RENDER(Request& context) { DValue props; print(component("child", props, context)); }' \
|
||||
'CLI(Request& context) {' \
|
||||
' DValue props; component("child", props, context);' \
|
||||
' print(unit_info("child.uce")["wasm_name"].to_string());' \
|
||||
'}' >"$source_dir/component-parent.uce"
|
||||
|
||||
entry_url="http://127.0.0.1/$test_name/entry.uce"
|
||||
component_url="http://127.0.0.1/$test_name/component-parent.uce"
|
||||
[[ "$(curl -fsS -H "Host: $http_host" "$entry_url")" == "entry-current" ]]
|
||||
entry_wasm=$(scripts/uce-cli --get "/$test_name/entry.uce" artifact=entry)
|
||||
component_wasm=$(scripts/uce-cli "/$test_name/component-parent.uce")
|
||||
artifact_dir=$(dirname "$entry_wasm")
|
||||
generation_name=$(basename "$(scripts/unit_cache_directory /tmp/uce-generation-probe)")
|
||||
if [[ "$entry_wasm" != *"/$generation_name/"* || "$component_wasm" != *"/$generation_name/"* ]]; then
|
||||
echo "unit artifacts are not isolated by compiler/core ABI generation: $entry_wasm $component_wasm" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
publish_incompatible_artifact_while_locked() {
|
||||
local source_file="$1"
|
||||
local wasm_file="$2"
|
||||
local marker="$3"
|
||||
local destination
|
||||
destination=$(dirname "$wasm_file")
|
||||
(
|
||||
exec 8>"$wasm_file.lock"
|
||||
flock 8
|
||||
UCE_UNIT_ABI_VERSION=6 scripts/compile_wasm_unit \
|
||||
"$(dirname "$source_file")" "$destination" "$source_file" \
|
||||
"$(basename "$source_file").cpp" "$(basename "$wasm_file")"
|
||||
sed -i \
|
||||
-e 's/^unit_abi_version=.*/unit_abi_version=0/' \
|
||||
-e 's/^wasm_core_abi_version=.*/wasm_core_abi_version=0/' \
|
||||
"${wasm_file%.wasm}.meta.txt"
|
||||
rm -f "${wasm_file%.wasm}.cwasm"
|
||||
printf '%s\n' "$marker" >"$ready_file"
|
||||
sleep 2
|
||||
) &
|
||||
lock_pid=$!
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ ! -e "$ready_file" && $SECONDS -lt $deadline ]]; do sleep 0.05; done
|
||||
if [[ ! -e "$ready_file" || "$(cat "$ready_file")" != "$marker" ]]; then
|
||||
echo "incompatible artifact was not published under lock: $marker" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
publish_incompatible_artifact_while_locked "$source_dir/entry.uce" "$entry_wasm" entry
|
||||
started_at=$(date +%s%N)
|
||||
entry_status=$(curl -sS --max-time 30 -o "$body_file" -w '%{http_code}' -H "Host: $http_host" "$entry_url")
|
||||
entry_elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
|
||||
wait "$lock_pid"
|
||||
if [[ "$entry_status" != "200" || "$(cat "$body_file")" != "entry-current" || $entry_elapsed_ms -lt 1500 ]]; then
|
||||
echo "ABI-incompatible entry artifact was served instead of joining its rebuild: status=$entry_status elapsed=${entry_elapsed_ms}ms body=$(cat "$body_file")" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rm -f "$ready_file"
|
||||
publish_incompatible_artifact_while_locked "$source_dir/child.uce" "$component_wasm" component
|
||||
started_at=$(date +%s%N)
|
||||
component_status=$(curl -sS --max-time 30 -o "$body_file" -w '%{http_code}' -H "Host: $http_host" "$component_url")
|
||||
component_elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
|
||||
wait "$lock_pid"
|
||||
if [[ "$component_status" != "200" || "$(cat "$body_file")" != "component-current" || $component_elapsed_ms -lt 1500 ]]; then
|
||||
echo "ABI-incompatible component artifact was served instead of joining its rebuild: status=$component_status elapsed=${component_elapsed_ms}ms body=$(cat "$body_file")" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "ABI generation rollout passed"
|
||||
Executable
+153
@@ -0,0 +1,153 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" != "--inside" ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
|
||||
root="/tmp/uce-cli-worker-isolation-$$"
|
||||
site="$root/site"
|
||||
work="$root/work"
|
||||
settings="$root/settings.cfg"
|
||||
log="$root/service.log"
|
||||
cli_socket="$root/run/cli.sock"
|
||||
fastcgi_socket="$root/run/fastcgi.sock"
|
||||
server_pid=""
|
||||
|
||||
cleanup() {
|
||||
status=$?
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
deadline=$((SECONDS + 10))
|
||||
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
if kill -0 "$server_pid" 2>/dev/null; then kill -KILL "$server_pid" 2>/dev/null || true; fi
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
|
||||
rm -rf "$root"
|
||||
return "$status"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
|
||||
cp /etc/uce/settings.cfg "$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$fastcgi_socket
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$cli_socket
|
||||
CLI_WORKER_COUNT=2
|
||||
CLI_WORKER_MAX_REQUESTS=3
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||
WORKER_COUNT=2
|
||||
PROACTIVE_COMPILE_ENABLED=0
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
|
||||
cat >"$site/isolation.uce" <<'UCE'
|
||||
RENDER(Request& context)
|
||||
{
|
||||
print(request_perf()["worker_pid"].to_string());
|
||||
}
|
||||
|
||||
CLI(Request& context)
|
||||
{
|
||||
print(request_perf()["worker_pid"].to_string());
|
||||
}
|
||||
UCE
|
||||
|
||||
cat >"$site/nested.uce" <<UCE
|
||||
CLI(Request& context)
|
||||
{
|
||||
print(shell_exec("cd $(pwd) && timeout 10s scripts/uce-cli --socket $cli_socket /isolation.uce"));
|
||||
}
|
||||
UCE
|
||||
|
||||
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ ! -S "$cli_socket" ]] && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ -S "$cli_socket" ]] || { echo "private UCE CLI socket was not ready" >&2; exit 1; }
|
||||
timeout --signal=TERM --kill-after=1s 30s scripts/uce-cli --socket "$cli_socket" /isolation.uce >/dev/null
|
||||
timeout --signal=TERM --kill-after=1s 30s scripts/uce-cli --socket "$cli_socket" /nested.uce | grep -Eq '^[0-9]+([.]0+)?$'
|
||||
|
||||
fastcgi_request() {
|
||||
SCRIPT_FILENAME="$site/isolation.uce" SCRIPT_NAME=/isolation.uce REQUEST_URI=/isolation.uce REQUEST_METHOD=GET DOCUMENT_ROOT="$site" \
|
||||
timeout --signal=TERM --kill-after=1s 10s cgi-fcgi -bind -connect "$fastcgi_socket" | tr -d '\r' | awk 'NF{last=$0} END{print last}'
|
||||
}
|
||||
fastcgi_request >/dev/null
|
||||
|
||||
cli_request() {
|
||||
local output
|
||||
local deadline=$((SECONDS + 20))
|
||||
while (( SECONDS < deadline )); do
|
||||
if output=$(timeout --signal=TERM --kill-after=1s 10s scripts/uce-cli --socket "$cli_socket" /isolation.uce 2>/dev/null); then
|
||||
printf '%s\n' "$output"
|
||||
return 0
|
||||
fi
|
||||
sleep 0.05
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
cli_pids=()
|
||||
http_pids=()
|
||||
|
||||
# Concurrent completions must count individually and retire both initial CLI
|
||||
# workers despite neither connection pool necessarily becoming idle first.
|
||||
concurrent_jobs=()
|
||||
for n in $(seq 1 12); do
|
||||
cli_request >"$root/concurrent-$n" & concurrent_jobs+=("$!")
|
||||
done
|
||||
for job in "${concurrent_jobs[@]}"; do wait "$job"; done
|
||||
mapfile -t concurrent_cli < <(cat "$root"/concurrent-* | sort -u)
|
||||
cli_pids+=("${concurrent_cli[@]}")
|
||||
|
||||
for _ in $(seq 1 12); do
|
||||
cli_pids+=("$(cli_request)")
|
||||
http_pids+=("$(fastcgi_request)")
|
||||
done
|
||||
mapfile -t unique_cli < <(printf '%s\n' "${cli_pids[@]}" | sort -u)
|
||||
mapfile -t unique_http < <(printf '%s\n' "${http_pids[@]}" | sort -u)
|
||||
[[ ${#unique_cli[@]} -ge 4 ]] || { echo "CLI workers did not recycle after three requests: ${unique_cli[*]}" >&2; exit 1; }
|
||||
[[ ${#unique_http[@]} -ge 1 ]] || { echo "HTTP requests reached no public worker" >&2; exit 1; }
|
||||
for cli_pid in "${unique_cli[@]}"; do
|
||||
if printf '%s\n' "${unique_http[@]}" | grep -Fxq "$cli_pid"; then
|
||||
echo "CLI worker also served public HTTP: $cli_pid" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
[[ $(grep -c 'wasm FastCGI worker ready' "$log") -eq 2 ]]
|
||||
[[ $(grep -c 'wasm CLI worker ready' "$log") -ge 4 ]]
|
||||
|
||||
echo "CLI worker isolation and recycling passed: CLI ${unique_cli[*]}, public ${unique_http[*]}"
|
||||
|
||||
# A CLI-only runtime must not allocate a listener-less public Wasmtime worker.
|
||||
kill -TERM "$server_pid"
|
||||
wait "$server_pid"
|
||||
server_pid=""
|
||||
rm -f "$cli_socket" "$fastcgi_socket"
|
||||
cat >>"$settings" <<CFG
|
||||
FCGI_SOCKET_PATH=
|
||||
CLI_WORKER_COUNT=1
|
||||
CLI_WORKER_MAX_REQUESTS=0
|
||||
CFG
|
||||
: >"$log"
|
||||
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ ! -S "$cli_socket" ]] && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ -S "$cli_socket" ]] || { echo "CLI-only socket was not ready" >&2; exit 1; }
|
||||
cli_request >/dev/null
|
||||
[[ $(grep -c 'wasm CLI worker ready' "$log") -eq 1 ]]
|
||||
[[ $(grep -c 'wasm FastCGI worker ready' "$log") -eq 0 ]]
|
||||
|
||||
echo "CLI-only worker allocation passed"
|
||||
@@ -26,7 +26,7 @@ cleanup() {
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$bin_directory$(realpath "$source_dir")"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'CLI(Request& context) { DValue props; print(component("child", props, context)); }' >"$source_dir/parent.uce"
|
||||
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" != "--inside" ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 90s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
|
||||
root="/tmp/uce-compiler-lock-directory-$$"
|
||||
site="$root/site"
|
||||
work="$root/missing/parents/work"
|
||||
settings="$root/settings.cfg"
|
||||
log="$root/service.log"
|
||||
server_pid=""
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
rm -rf "$root"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$site" "$root/run" "$root/session" "$root/upload"
|
||||
printf '%s\n' 'CLI(Request& context) { print("recursive-lock-directory-ok"); }' >"$site/test.uce"
|
||||
cp /etc/uce/settings.cfg "$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$root/run/cli.sock
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||
WORKER_COUNT=1
|
||||
PROACTIVE_COMPILE_ENABLED=0
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
|
||||
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ ! -S "$root/run/cli.sock" ]] && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ -S "$root/run/cli.sock" ]] || { cat "$log" >&2; exit 1; }
|
||||
|
||||
rm -rf "$work"
|
||||
response=$(curl -sS --max-time 45 --fail-with-body --unix-socket "$root/run/cli.sock" http://localhost/test.uce) || { cat "$log" >&2; exit 1; }
|
||||
[[ "$response" == *"recursive-lock-directory-ok"* ]] || { printf '%s\n' "$response" >&2; cat "$log" >&2; exit 1; }
|
||||
find "$work" -type f -name '*.lock' -print -quit | grep -q . || { echo "compiler did not recreate a nested lock path" >&2; cat "$log" >&2; exit 1; }
|
||||
echo "compiler recursive lock directory passed"
|
||||
Executable
+49
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="component-once-prefetch-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
if [[ -n "${configured_site_directory:-}" ]]; then
|
||||
site_directory="$configured_site_directory"
|
||||
fi
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
bin_directory="${BIN_DIRECTORY:-}"
|
||||
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
|
||||
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
fi
|
||||
bin_directory="${bin_directory:-/tmp/uce/work}"
|
||||
cache_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$cache_dir" ]]; then
|
||||
rm -rf "$cache_dir"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'CLI(Request& context) {' \
|
||||
' String first = component("child", context);' \
|
||||
' String second = component("child", context);' \
|
||||
' DValue perf = request_perf();' \
|
||||
' print(first, ",", second, ",", perf["component_resolve_count"].to_u64(), ",", perf["component_loaded_reuse_count"].to_u64());' \
|
||||
'}' >"$source_dir/entry.uce"
|
||||
|
||||
printf '%s\n' \
|
||||
'ONCE(Request& context) { context.call["once"] = context.call["once"].to_u64() + 1; }' \
|
||||
'COMPONENT(Request& context) { print(context.call["once"].to_u64()); }' >"$source_dir/child.uce"
|
||||
|
||||
output=$(scripts/uce-cli "/$test_name/entry.uce")
|
||||
if [[ "$output" != "1,1,1,0" ]]; then
|
||||
echo "component ONCE slot was not returned with the primary handler: $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "component ONCE prefetch passed"
|
||||
Executable
+111
@@ -0,0 +1,111 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="component-resolution-ttl-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
worker_count="${UCE_TEST_WORKER_COUNT:-4}"
|
||||
if [[ -r /etc/uce/settings.cfg ]]; then
|
||||
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
site_directory="${configured_site_directory:-$site_directory}"
|
||||
fi
|
||||
if [[ -z "${UCE_TEST_WORKER_COUNT:-}" ]]; then
|
||||
configured_worker_count=$(awk -F= '/^[[:space:]]*WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
worker_count="${configured_worker_count:-$worker_count}"
|
||||
fi
|
||||
fi
|
||||
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
|
||||
source_dir="$site_directory/$test_name"
|
||||
bin_directory="${BIN_DIRECTORY:-}"
|
||||
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
|
||||
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
fi
|
||||
bin_directory="${bin_directory:-/tmp/uce/work}"
|
||||
cache_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$cache_dir" ]]; then
|
||||
rm -rf "$cache_dir"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' 'RENDER(Request& context) { print("parent:", request_perf()["worker_pid"].to_string(), ":", component("chosen", context)); }' >"$source_dir/parent.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("a"); }' >"$source_dir/a.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("b"); }' >"$source_dir/b.uce"
|
||||
ln -s a.uce "$source_dir/chosen.uce"
|
||||
|
||||
declare -A seen
|
||||
collect_workers() {
|
||||
local method="$1"
|
||||
local marker="$2"
|
||||
local path="${3:-parent.uce}"
|
||||
local deadline=$((SECONDS + 30))
|
||||
seen=()
|
||||
while (( SECONDS < deadline && ${#seen[@]} < worker_count )); do
|
||||
local response status body pid
|
||||
response=$(curl -sS --max-time 10 -X "$method" -H "Host: $http_host" -w $'\n%{http_code}' "http://127.0.0.1/$test_name/$path")
|
||||
status=${response##*$'\n'}
|
||||
body=${response%$'\n'*}
|
||||
if [[ "$status" == "503" ]]; then
|
||||
sleep 0.1
|
||||
continue
|
||||
fi
|
||||
if [[ "$status" != "200" || "$body" != parent:*":$marker" ]]; then
|
||||
echo "component resolution TTL probe failed: method=$method status=$status expected=$marker body=$body" >&2
|
||||
exit 1
|
||||
fi
|
||||
pid=${body#parent:}
|
||||
pid=${pid%%:*}
|
||||
seen["$pid"]=1
|
||||
done
|
||||
if (( ${#seen[@]} != worker_count )); then
|
||||
echo "component resolution TTL probe reached ${#seen[@]}/$worker_count workers" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
collect_missing_workers() {
|
||||
local deadline=$((SECONDS + 30))
|
||||
seen=()
|
||||
while (( SECONDS < deadline && ${#seen[@]} < worker_count )); do
|
||||
local response status body pid
|
||||
response=$(curl -sS --max-time 10 -H "Host: $http_host" -w $'\n%{http_code}' "http://127.0.0.1/$test_name/missing.uce")
|
||||
status=${response##*$'\n'}
|
||||
body=${response%$'\n'*}
|
||||
if [[ "$status" == "503" ]]; then
|
||||
sleep 0.1
|
||||
continue
|
||||
fi
|
||||
if [[ "$status" != "500" || "$body" != parent:* || "$body" != *"component not found: later"* ]]; then
|
||||
echo "missing component TTL probe failed: status=$status body=$body" >&2
|
||||
exit 1
|
||||
fi
|
||||
pid=${body#parent:}
|
||||
pid=${pid%%:*}
|
||||
seen["$pid"]=1
|
||||
done
|
||||
if (( ${#seen[@]} != worker_count )); then
|
||||
echo "missing component TTL probe reached ${#seen[@]}/$worker_count workers" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
collect_workers GET a
|
||||
ln -sfn b.uce "$source_dir/chosen.uce"
|
||||
collect_workers GET a
|
||||
collect_workers POST b
|
||||
collect_workers GET a
|
||||
sleep 10.2
|
||||
collect_workers GET b
|
||||
|
||||
printf '%s\n' 'RENDER(Request& context) { print("parent:", request_perf()["worker_pid"].to_string(), ":", component("later", context)); }' >"$source_dir/missing.uce"
|
||||
collect_missing_workers
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("created"); }' >"$source_dir/later.uce"
|
||||
collect_workers GET created missing.uce
|
||||
|
||||
echo "component resolution TTL passed"
|
||||
Executable
+176
@@ -0,0 +1,176 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_source="/tmp/uce-oauth-es256-native-$$.cpp"
|
||||
test_binary="/tmp/uce-oauth-es256-native-$$"
|
||||
cleanup() { rm -f "$test_source" "$test_binary"; }
|
||||
trap cleanup EXIT
|
||||
|
||||
cat >"$test_source" <<'EOF'
|
||||
#include "src/lib/types.cpp"
|
||||
#include "src/lib/dvalue.cpp"
|
||||
#include "src/lib/functionlib.cpp"
|
||||
#include <openssl/bn.h>
|
||||
#include <openssl/core_names.h>
|
||||
#include <openssl/ecdsa.h>
|
||||
#include <openssl/evp.h>
|
||||
#include <openssl/params.h>
|
||||
#include <memory>
|
||||
|
||||
String base64_encode(String raw)
|
||||
{
|
||||
if(raw.empty()) return("");
|
||||
String out(4 * ((raw.size() + 2) / 3), 0);
|
||||
int size = EVP_EncodeBlock((unsigned char*)out.data(), (const unsigned char*)raw.data(), (int)raw.size());
|
||||
out.resize(size > 0 ? (size_t)size : 0);
|
||||
return(out);
|
||||
}
|
||||
|
||||
String base64_decode(String raw, bool& ok)
|
||||
{
|
||||
ok = false;
|
||||
if(raw.empty() || raw.size() % 4) return("");
|
||||
String out(3 * raw.size() / 4, 0);
|
||||
int size = EVP_DecodeBlock((unsigned char*)out.data(), (const unsigned char*)raw.data(), (int)raw.size());
|
||||
if(size < 0) return("");
|
||||
while(!raw.empty() && raw.back() == '=') { size--; raw.pop_back(); }
|
||||
out.resize((size_t)size); ok = true; return(out);
|
||||
}
|
||||
|
||||
#include "src/lib/hash.cpp"
|
||||
|
||||
static String b64url_encode(String text)
|
||||
{
|
||||
String out=replace(replace(base64_encode(text),"+","-"),"/","_"); while(!out.empty()&&out.back()=='=') out.pop_back(); return out;
|
||||
}
|
||||
|
||||
static String b64url_decode(String text)
|
||||
{
|
||||
text = replace(replace(text, "-", "+"), "_", "/");
|
||||
while(text.size() % 4) text += "=";
|
||||
bool ok = false;
|
||||
String result = base64_decode(text, ok);
|
||||
return(ok ? result : String(""));
|
||||
}
|
||||
|
||||
static bool verify(DValue public_jwk, String jwt)
|
||||
{
|
||||
StringList parts = split(jwt, ".");
|
||||
if(parts.size() != 3) return(false);
|
||||
String x = b64url_decode(public_jwk["x"].to_string());
|
||||
String y = b64url_decode(public_jwk["y"].to_string());
|
||||
String raw = b64url_decode(parts[2]);
|
||||
if(x.size() != 32 || y.size() != 32 || raw.size() != 64) return(false);
|
||||
unsigned char point[65] = {4}; memcpy(point + 1, x.data(), 32); memcpy(point + 33, y.data(), 32);
|
||||
OSSL_PARAM params[] = { OSSL_PARAM_construct_utf8_string(OSSL_PKEY_PARAM_GROUP_NAME, (char*)"prime256v1", 0), OSSL_PARAM_construct_octet_string(OSSL_PKEY_PARAM_PUB_KEY, point, sizeof(point)), OSSL_PARAM_construct_end() };
|
||||
EVP_PKEY_CTX* build = EVP_PKEY_CTX_new_from_name(0, "EC", 0); EVP_PKEY* key = 0;
|
||||
if(!build || EVP_PKEY_fromdata_init(build) <= 0 || EVP_PKEY_fromdata(build, &key, EVP_PKEY_PUBLIC_KEY, params) <= 0) { EVP_PKEY_CTX_free(build); return(false); }
|
||||
EVP_PKEY_CTX_free(build);
|
||||
BIGNUM* r = BN_bin2bn((const unsigned char*)raw.data(), 32, 0); BIGNUM* s = BN_bin2bn((const unsigned char*)raw.data() + 32, 32, 0);
|
||||
ECDSA_SIG* sig = ECDSA_SIG_new(); int der_size = r && s && sig && ECDSA_SIG_set0(sig, r, s) ? i2d_ECDSA_SIG(sig, 0) : 0; r = s = 0;
|
||||
String der(der_size > 0 ? (size_t)der_size : 0, 0); unsigned char* out = (unsigned char*)der.data();
|
||||
bool ok = der_size > 0 && i2d_ECDSA_SIG(sig, &out) == der_size;
|
||||
EVP_MD_CTX* verify_ctx = EVP_MD_CTX_new();
|
||||
String signing_input = parts[0] + "." + parts[1];
|
||||
ok = ok && verify_ctx && EVP_DigestVerifyInit(verify_ctx, 0, EVP_sha256(), 0, key) > 0 && EVP_DigestVerify(verify_ctx, (const unsigned char*)der.data(), der.size(), (const unsigned char*)signing_input.data(), signing_input.size()) == 1;
|
||||
EVP_MD_CTX_free(verify_ctx); ECDSA_SIG_free(sig); EVP_PKEY_free(key); return(ok);
|
||||
}
|
||||
|
||||
int main()
|
||||
{
|
||||
DValue key_request; key_request["operation"] = "key_generate"; key_request["algorithm"] = "ES256";
|
||||
DValue key = crypto_operation_native(key_request);
|
||||
DValue header; header["alg"] = "none"; header["kid"] = key["kid"]; DValue claims; claims["iss"] = "https://client.example";
|
||||
auto sign = [&](DValue private_jwk) { DValue request; request["operation"] = "jwt_sign"; request["algorithm"] = "ES256"; request["private_jwk"] = private_jwk; request["protected_header"] = header; request["claims"] = claims; return(crypto_operation_native(request)); };
|
||||
DValue signed_result = sign(key["private_jwk"]); String jwt = signed_result["jwt"].to_string();
|
||||
DValue wrong_curve = key["private_jwk"]; wrong_curve["crv"] = "P-384";
|
||||
DValue malformed = key["private_jwk"]; malformed["x"] = "bad=";
|
||||
DValue mismatch = key["private_jwk"]; String d = mismatch["d"].to_string(); d[0] = d[0] == 'A' ? 'B' : 'A'; mismatch["d"] = d;
|
||||
const String cose="pQECAyYgASFYIGsX0fLhLEJH-Lzm5WOkQPJ3A32BLeszoPShOUXYmMKWIlggT-NC4v4af5uO5-tKfA-eFivOM1drMV7Oy7ZAaDe_UfU", message="d2ViYXV0aG4gZml4ZWQgbWVzc2FnZQ", signature="MEUCIQCkatZK1VVsjk17uvyzyhjdAkMNWXPjxSOMqWcjmM_8XAIgDaSk3Qufyd0_6r9Dm9A8RQbFco-FdTBulq7bvRGoBC4";
|
||||
auto op = [&](String operation) { DValue r; r["operation"]=operation; r["algorithm"]="ES256"; return r; };
|
||||
auto parse = [&](String key) { DValue r=op("cose_es256_parse"); r["cose_key_base64url"]=key; return crypto_operation_native(r); };
|
||||
auto verify_cose = [&](String key,String msg,String sig) { DValue r=op("es256_verify"); r["cose_key_base64url"]=key; r["message_base64url"]=msg; r["signature_der_base64url"]=sig; return crypto_operation_native(r); };
|
||||
auto decode_cbor = [&](String bytes) { DValue r=op("cbor_decode"); r["cbor_base64url"]=b64url_encode(bytes); return crypto_operation_native(r); };
|
||||
DValue unsupported; unsupported["operation"] = "encrypt"; unsupported["algorithm"] = "ES256";
|
||||
DValue unknown_algorithm; unknown_algorithm["operation"] = "key_generate"; unknown_algorithm["algorithm"] = "none";
|
||||
DValue untyped_algorithm = key_request; untyped_algorithm["algorithm"] = (f64)256;
|
||||
DValue list_header; list_header.set_array(); DValue list_item; list_item = "not-an-object"; list_header.push(list_item); DValue list_request; list_request["operation"] = "jwt_sign"; list_request["algorithm"] = "ES256"; list_request["private_jwk"] = key["private_jwk"]; list_request["protected_header"] = list_header; list_request["claims"] = claims;
|
||||
DValue control_request = list_request; control_request["protected_header"] = header; control_request["claims"] = claims; control_request["claims"]["bad"] = String("control\nbyte");
|
||||
DValue oversized = key_request; oversized["ignored"] = String(33000, 'x');
|
||||
DValue nonfinite = key_request; nonfinite["ignored"] = std::numeric_limits<f64>::quiet_NaN();
|
||||
String tampered = jwt; if(tampered.size() > 2) tampered[tampered.size() - 2] = tampered[tampered.size() - 2] == 'A' ? 'B' : 'A';
|
||||
bool kid_ok = key["ok"].to_bool() && key["kid"].to_string() == key["thumbprint"].to_string();
|
||||
bool signed_ok = signed_result["ok"].to_bool() && jwt != "" && verify(key["public_jwk"], jwt);
|
||||
bool tamper_ok = !verify(key["public_jwk"], tampered);
|
||||
auto cbor_error = [&](String bytes) { return(decode_cbor(bytes)["error"].to_string() == "invalid_cbor"); };
|
||||
auto cose_error = [&](String encoded) { return(parse(encoded)["error"].to_string() == "invalid_cose_key"); };
|
||||
String cose_raw = b64url_decode(cose);
|
||||
String wrong_kty = cose_raw; wrong_kty[2] = 1;
|
||||
String wrong_alg = cose_raw; wrong_alg[4] = 0x27;
|
||||
String wrong_cose_curve = cose_raw; wrong_cose_curve[6] = 2;
|
||||
String missing_label = cose_raw; missing_label[0] = 0xa4; missing_label.resize(missing_label.size() - 35);
|
||||
String short_x = String("\xa5\x01\x02\x03\x26\x20\x01\x21\x58\x1f",10) + String(31, 'x') + String("\x22\x58\x20", 3) + String(32, 'y');
|
||||
String duplicate_label = cose_raw; duplicate_label[0] = 0xa6; duplicate_label += String("\x01\x02", 2);
|
||||
String invalid_point = String("\xa5\x01\x02\x03\x26\x20\x01\x21\x58\x20", 10) + String(32, 'x') + String("\x22\x58\x20", 3) + String(32, 'y');
|
||||
String der = b64url_decode(signature);
|
||||
String tampered_der = der; tampered_der[10] ^= 1;
|
||||
String trailing_der = der + String("\x00", 1);
|
||||
String noncanonical_der = der; noncanonical_der[1]++; noncanonical_der[3]++; noncanonical_der.insert(4, 1, '\0');
|
||||
DValue valid_verify = verify_cose(cose, message, signature);
|
||||
DValue tampered_signature = verify_cose(cose, message, b64url_encode(tampered_der));
|
||||
DValue tampered_message = verify_cose(cose, b64url_encode("tampered"), signature);
|
||||
DValue malformed_der = verify_cose(cose, message, b64url_encode(String("\x30\x00", 2)));
|
||||
DValue noncanonical_der_result = verify_cose(cose, message, b64url_encode(noncanonical_der));
|
||||
DValue trailing_der_result = verify_cose(cose, message, b64url_encode(trailing_der));
|
||||
DValue invalid_point_result = verify_cose(b64url_encode(invalid_point), message, signature);
|
||||
String large_cbor = String("\x58\x81", 2) + String(129, 'x');
|
||||
String oversized_cbor = String("\x5a\x00\x00\x40\x00", 5) + String(16384, 'x');
|
||||
String duplicate_compound = String("\xa2\x82\x41[\x41]\x01\x82\x41[\x41]\x02", 13);
|
||||
String distinct_compound = String("\xa2\x82\x41[\x41]\x01\x82\x41[\x41[\x02", 13);
|
||||
String node_overflow = String("\x99\x01\x00", 3) + String(256, 0);
|
||||
String decoded;
|
||||
bool cbor_valid = decode_cbor(String("\x82\x01\x62ok", 5))["ok"].to_bool();
|
||||
bool cbor_large = decode_cbor(large_cbor)["ok"].to_bool();
|
||||
bool cbor_control_text = decode_cbor(String("\x61\x01", 2))["ok"].to_bool();
|
||||
bool cbor_duplicate = cbor_error(String("\xa2\x01\x02\x01\x03", 5));
|
||||
bool cbor_compound_duplicate = cbor_error(duplicate_compound);
|
||||
bool cbor_compound_distinct = decode_cbor(distinct_compound)["ok"].to_bool();
|
||||
bool cbor_invalid_utf8 = cbor_error(String("\x61\x80", 2));
|
||||
bool cbor_truncated = cbor_error(String("\xa1", 1));
|
||||
bool cbor_trailing = cbor_error(String("\x01\x02", 2));
|
||||
bool cbor_depth = cbor_error(String(17, '\x81') + "\x00");
|
||||
bool cbor_nodes = cbor_error(node_overflow);
|
||||
bool cbor_size = cbor_error(oversized_cbor);
|
||||
bool cbor_indefinite = cbor_error(String("\x9f\x01\xff", 3));
|
||||
bool cbor_integer_overflow = cbor_error(String("\x5b\xff\xff\xff\xff\xff\xff\xff\xff", 9));
|
||||
bool cbor_nonminimal = cbor_error(String("\x18\x17", 2));
|
||||
bool b64_padding = !uce_base64url_decode(cose + "=", decoded, UCE_CBOR_MAX_BASE64URL);
|
||||
bool b64_truncated = !uce_base64url_decode("A", decoded, UCE_CBOR_MAX_BASE64URL);
|
||||
bool b64_trailing_bits = !uce_base64url_decode("AB", decoded, UCE_CBOR_MAX_BASE64URL);
|
||||
bool cose_valid = parse(cose)["ok"].to_bool();
|
||||
bool cose_wrong_kty = cose_error(b64url_encode(wrong_kty));
|
||||
bool cose_wrong_alg = cose_error(b64url_encode(wrong_alg));
|
||||
bool cose_wrong_curve = cose_error(b64url_encode(wrong_cose_curve));
|
||||
bool cose_short_coordinate = cose_error(b64url_encode(short_x));
|
||||
bool cose_missing_label = cose_error(b64url_encode(missing_label));
|
||||
bool cose_duplicate_label = cose_error(b64url_encode(duplicate_label));
|
||||
bool cose_invalid_point = invalid_point_result["error"].to_string() == "invalid_key_or_payload";
|
||||
bool verify_valid = valid_verify["ok"].to_bool() && valid_verify["valid"].to_bool();
|
||||
bool verify_tampered_signature = tampered_signature["ok"].to_bool() && !tampered_signature["valid"].to_bool();
|
||||
bool verify_tampered_message = tampered_message["ok"].to_bool() && !tampered_message["valid"].to_bool();
|
||||
bool verify_malformed_der = malformed_der["error"].to_string() == "invalid_signature";
|
||||
bool verify_noncanonical_der = noncanonical_der_result["error"].to_string() == "invalid_signature";
|
||||
bool verify_trailing_der = trailing_der_result["error"].to_string() == "invalid_signature";
|
||||
bool cbor_ok = cbor_valid && cbor_large && cbor_control_text && cbor_duplicate && cbor_compound_duplicate && cbor_compound_distinct && cbor_invalid_utf8 && cbor_truncated && cbor_trailing && cbor_depth && cbor_nodes && cbor_size && cbor_indefinite && cbor_integer_overflow && cbor_nonminimal;
|
||||
bool cose_ok = cose_valid && cose_wrong_kty && cose_wrong_alg && cose_wrong_curve && cose_short_coordinate && cose_missing_label && cose_duplicate_label && cose_invalid_point;
|
||||
bool cose_negative = verify_valid && verify_tampered_signature && verify_tampered_message && verify_malformed_der && verify_noncanonical_der && verify_trailing_der;
|
||||
bool b64_ok = b64_padding && b64_truncated && b64_trailing_bits;
|
||||
bool negatives_ok = !sign(wrong_curve)["ok"].to_bool() && !sign(malformed)["ok"].to_bool() && !sign(mismatch)["ok"].to_bool() && crypto_operation_native(unsupported)["error"].to_string() == "unsupported_operation" && crypto_operation_native(unknown_algorithm)["error"].to_string() == "unsupported_algorithm" && crypto_operation_native(untyped_algorithm)["error"].to_string() == "invalid_request" && crypto_operation_native(list_request)["error"].to_string() == "invalid_key_or_payload" && crypto_operation_native(control_request)["error"].to_string() == "invalid_request" && crypto_operation_native(oversized)["error"].to_string() == "invalid_request" && crypto_operation_native(nonfinite)["error"].to_string() == "invalid_request";
|
||||
if(!(kid_ok && signed_ok && tamper_ok && negatives_ok && cose_ok && cbor_ok && cose_negative && b64_ok)) std::cerr << "kid=" << kid_ok << " signed=" << signed_ok << " tamper=" << tamper_ok << " negatives=" << negatives_ok << " cbor=" << cbor_ok << " cose=" << cose_ok << " verify=" << cose_negative << " b64=" << b64_ok << " large=" << cbor_large << " compound=" << cbor_compound_duplicate << "/" << cbor_compound_distinct << " der=" << verify_malformed_der << "/" << verify_noncanonical_der << "/" << verify_trailing_der << " cborparts=" << cbor_valid << cbor_control_text << cbor_duplicate << cbor_invalid_utf8 << cbor_truncated << cbor_trailing << cbor_depth << cbor_nodes << cbor_size << cbor_indefinite << cbor_integer_overflow << cbor_nonminimal << "\\n";
|
||||
return(kid_ok && signed_ok && tamper_ok && negatives_ok && cose_ok && cbor_ok && cose_negative && b64_ok ? 0 : 1);
|
||||
}
|
||||
EOF
|
||||
"${CXX:-c++}" -std=c++20 -fpermissive -I. "$test_source" -lpcre2-8 -lcrypto -o "$test_binary"
|
||||
"$test_binary"
|
||||
echo "native structured crypto operation passed"
|
||||
@@ -18,6 +18,10 @@ fi
|
||||
bin_directory="${bin_directory:-/tmp/uce/work}"
|
||||
cache_dir=""
|
||||
mutation_file="/tmp/uce-dependency-mutation-$$"
|
||||
component_mutation_file="/tmp/uce-component-mutation-$$"
|
||||
component_post_body="/tmp/uce-component-post-body-$$"
|
||||
component_post_headers="/tmp/uce-component-post-headers-$$"
|
||||
component_lock_ready_file="/tmp/uce-component-lock-ready-$$"
|
||||
post_body="/tmp/uce-dependency-post-body-$$"
|
||||
post_headers="/tmp/uce-dependency-post-headers-$$"
|
||||
lock_ready_file="/tmp/uce-dependency-lock-ready-$$"
|
||||
@@ -32,11 +36,11 @@ cleanup() {
|
||||
if [[ -n "$cache_dir" ]]; then
|
||||
rm -rf "$cache_dir"
|
||||
fi
|
||||
rm -f "$mutation_file" "$post_body" "$post_headers" "$lock_ready_file" "$nested_lock_ready_file" "$race_output" "$race_error" "$race_status"
|
||||
rm -f "$mutation_file" "$component_mutation_file" "$component_post_body" "$component_post_headers" "$component_lock_ready_file" "$post_body" "$post_headers" "$lock_ready_file" "$nested_lock_ready_file" "$race_output" "$race_error" "$race_status"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$bin_directory$(realpath "$source_dir")"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'#ifndef UCE_DEPENDENCY_CACHE_CHILD' \
|
||||
@@ -69,6 +73,141 @@ if [[ "$(http_marker)" != *"dependency-marker-a"* ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Compiler source canonicalization must retain symlink-target identity while
|
||||
# avoiding realpath's repeated per-segment probes on ordinary source graphs.
|
||||
printf '%s\n' 'String symlink_dependency_marker() { return("symlink-marker-a"); }' >"$source_dir/symlink-target-a.uce"
|
||||
printf '%s\n' 'String symlink_dependency_marker() { return("symlink-marker-b"); }' >"$source_dir/symlink-target-b.uce"
|
||||
ln -s "symlink-target-a.uce" "$source_dir/symlink-child.uce"
|
||||
printf '%s\n' '#load "symlink-child.uce"' 'CLI(Request& context) { print(symlink_dependency_marker()); }' >"$source_dir/symlink-parent.uce"
|
||||
assert_marker symlink-parent symlink-marker-a
|
||||
ln -sfn "symlink-target-b.uce" "$source_dir/symlink-child.uce"
|
||||
assert_marker symlink-parent symlink-marker-b
|
||||
|
||||
# A repeated exact transitive load is signature-deduplicated before path
|
||||
# canonicalization, but the shared dependency must still invalidate the parent.
|
||||
printf '%s\n' '#ifndef UCE_DIAMOND_COMMON' '#define UCE_DIAMOND_COMMON' 'String diamond_marker() { return("diamond-a"); }' '#endif' >"$source_dir/diamond-common.uce"
|
||||
printf '%s\n' '#load "diamond-common.uce"' >"$source_dir/diamond-left.uce"
|
||||
printf '%s\n' '#load "diamond-common.uce"' >"$source_dir/diamond-right.uce"
|
||||
printf '%s\n' '#load "diamond-left.uce"' '#load "diamond-right.uce"' 'CLI(Request& context) { print(diamond_marker()); }' >"$source_dir/diamond-parent.uce"
|
||||
assert_marker diamond-parent diamond-a
|
||||
sed -i 's/diamond-a/diamond-b/' "$source_dir/diamond-common.uce"
|
||||
assert_marker diamond-parent diamond-b
|
||||
|
||||
# HTTP entry units can resolve route/components dynamically. A changed dynamic
|
||||
# component must enter the demand-priority queue just like a changed entry unit;
|
||||
# otherwise a common dependency rebuild can leave the requested page stale for
|
||||
# the duration of the full proactive queue.
|
||||
printf '%s\n' "COMPONENT(Request& context) { if(context.params[\"REQUEST_METHOD\"] == \"POST\") file_put_contents(\"$component_mutation_file\", \"executed\"); print(\"http-component-marker-a\"); }" >"$source_dir/http-component-child.uce"
|
||||
printf '%s\n' 'RENDER(Request& context) { DValue props; print(component("http-component-child", props, context)); }' >"$source_dir/http-component-parent.uce"
|
||||
http_component_marker() {
|
||||
curl -fsS -H "Host: $http_host" "http://127.0.0.1/$test_name/http-component-parent.uce"
|
||||
}
|
||||
if [[ "$(http_component_marker)" != *"http-component-marker-a"* ]]; then
|
||||
echo "HTTP component warm-up did not return marker-a" >&2
|
||||
exit 1
|
||||
fi
|
||||
sed -i 's/http-component-marker-a/http-component-marker-b/' "$source_dir/http-component-child.uce"
|
||||
started_at=$(date +%s%N)
|
||||
http_component_output=$(http_component_marker)
|
||||
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
|
||||
if [[ "$http_component_output" != *"http-component-marker-a"* && "$http_component_output" != *"http-component-marker-b"* ]]; then
|
||||
echo "stale HTTP component request returned neither complete artifact: $http_component_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
if (( elapsed_ms >= 2000 )); then
|
||||
echo "HTTP component request spent ${elapsed_ms}ms rebuilding a stale artifact" >&2
|
||||
exit 1
|
||||
fi
|
||||
deadline=$((SECONDS + 15))
|
||||
while [[ "$http_component_output" != *"http-component-marker-b"* && $SECONDS -lt $deadline ]]; do
|
||||
sleep 0.2
|
||||
http_component_output=$(http_component_marker)
|
||||
done
|
||||
if [[ "$http_component_output" != *"http-component-marker-b"* ]]; then
|
||||
echo "requested stale HTTP component did not receive a demand-priority rebuild" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf '%s\n' 'String http_component_dependency_marker() { return("http-component-dependency-a"); }' >"$source_dir/http-component-dependency.uce"
|
||||
printf '%s\n' '#load "http-component-dependency.uce"' 'COMPONENT(Request& context) { print(http_component_dependency_marker()); }' >"$source_dir/http-transitive-child.uce"
|
||||
printf '%s\n' 'RENDER(Request& context) { DValue props; print(component("http-transitive-child", props, context)); }' >"$source_dir/http-transitive-parent.uce"
|
||||
http_transitive_marker() {
|
||||
curl -fsS -H "Host: $http_host" "http://127.0.0.1/$test_name/http-transitive-parent.uce"
|
||||
}
|
||||
if [[ "$(http_transitive_marker)" != *"http-component-dependency-a"* ]]; then
|
||||
echo "HTTP transitive component warm-up did not return dependency-a" >&2
|
||||
exit 1
|
||||
fi
|
||||
sed -i 's/http-component-dependency-a/http-component-dependency-b/' "$source_dir/http-component-dependency.uce"
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ "$(http_transitive_marker)" != *"http-component-dependency-b"* && $SECONDS -lt $deadline ]]; do sleep 0.2; done
|
||||
if [[ "$(http_transitive_marker)" != *"http-component-dependency-b"* ]]; then
|
||||
echo "HTTP dynamic component did not converge after its loaded dependency changed" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Mutations must never execute a stale dynamically resolved component, even
|
||||
# inside the short read-only freshness memo window.
|
||||
http_component_wasm="$cache_dir/http-component-child.uce.wasm"
|
||||
(
|
||||
exec 8>"$http_component_wasm.lock"
|
||||
flock 8
|
||||
: >"$component_lock_ready_file"
|
||||
sleep 2
|
||||
) &
|
||||
component_lock_pid=$!
|
||||
deadline=$((SECONDS + 5))
|
||||
while [[ ! -e "$component_lock_ready_file" && $SECONDS -lt $deadline ]]; do sleep 0.05; done
|
||||
if [[ ! -e "$component_lock_ready_file" ]]; then
|
||||
echo "HTTP component mutation lock was not acquired before deadline" >&2
|
||||
exit 1
|
||||
fi
|
||||
sed -i 's/http-component-marker-b/http-component-marker-c/' "$source_dir/http-component-child.uce"
|
||||
rm -f "$component_mutation_file"
|
||||
component_post_status=$(curl -sS -o "$component_post_body" -D "$component_post_headers" -w '%{http_code}' -X POST -H "Host: $http_host" "http://127.0.0.1/$test_name/http-component-parent.uce")
|
||||
if [[ "$component_post_status" != "503" ]]; then
|
||||
echo "stale HTTP component mutation did not fail closed: status=$component_post_status body=$(cat "$component_post_body")" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -qi '^Retry-After: 1' "$component_post_headers"; then
|
||||
echo "stale HTTP component mutation omitted Retry-After" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -e "$component_mutation_file" ]]; then
|
||||
echo "stale HTTP component mutation executed application code" >&2
|
||||
exit 1
|
||||
fi
|
||||
wait "$component_lock_pid"
|
||||
deadline=$((SECONDS + 15))
|
||||
while [[ "$(http_component_marker)" != *"http-component-marker-c"* && $SECONDS -lt $deadline ]]; do sleep 0.2; done
|
||||
if [[ "$(http_component_marker)" != *"http-component-marker-c"* ]]; then
|
||||
echo "HTTP component did not recover after the guarded mutation rebuild" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# A failed parent build must remember the exact dependency graph that failed.
|
||||
# If a dependency is restored byte-for-byte to the last working source, the
|
||||
# old successful metadata must not make that transient failure permanent.
|
||||
printf '%s\n' \
|
||||
'#ifndef UCE_DEPENDENCY_CACHE_CHILD' \
|
||||
'#define UCE_DEPENDENCY_CACHE_CHILD' \
|
||||
'String dependency_cache_marker() { return(deliberate_dependency_compile_failure); }' \
|
||||
'#endif' >"$source_dir/child.uce"
|
||||
if restored_failure=$(scripts/uce-cli --get "/$test_name/parent.uce" __uce_expected_compile_failure=1 2>&1); then
|
||||
echo "invalid dependency unexpectedly compiled: $restored_failure" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$restored_failure" != *"deliberate_dependency_compile_failure"* ]]; then
|
||||
echo "invalid dependency did not report the expected compiler failure: $restored_failure" >&2
|
||||
exit 1
|
||||
fi
|
||||
printf '%s\n' \
|
||||
'#ifndef UCE_DEPENDENCY_CACHE_CHILD' \
|
||||
'#define UCE_DEPENDENCY_CACHE_CHILD' \
|
||||
'String dependency_cache_marker() { return("dependency-marker-a"); }' \
|
||||
'#endif' >"$source_dir/child.uce"
|
||||
assert_marker parent dependency-marker-a
|
||||
|
||||
# The load-graph signature is content-addressed. A byte-identical dependency
|
||||
# touch must invalidate stat caches without recompiling its parent artifact.
|
||||
parent_wasm="$cache_dir/parent.uce.wasm"
|
||||
@@ -259,7 +398,7 @@ rm -f "$cache_dir/parent.uce.cwasm"
|
||||
for _ in {1..16}; do assert_marker parent dependency-marker-c; done
|
||||
|
||||
race_unit="$source_dir/source-race.uce"
|
||||
race_cpp="$cache_dir/source-race.uce.cpp"
|
||||
race_cpp=""
|
||||
{
|
||||
printf '%s\n' 'String source_race_marker() { return("source-race-marker-a"); }'
|
||||
for i in $(seq 1 3500); do
|
||||
@@ -275,16 +414,17 @@ rm -f "$cache_dir/source-race.uce.cpp" "$cache_dir/source-race.uce.wasm" "$cache
|
||||
) &
|
||||
race_pid=$!
|
||||
deadline=$((SECONDS + 10))
|
||||
while [[ ! -e "$race_cpp" && $SECONDS -lt $deadline ]]; do
|
||||
while [[ -z "$race_cpp" && $SECONDS -lt $deadline ]]; do
|
||||
race_cpp=$(find "$cache_dir" -maxdepth 1 -name 'source-race.uce.invocation-*.cpp' -print -quit 2>/dev/null || true)
|
||||
if ! kill -0 "$race_pid" 2>/dev/null; then
|
||||
echo "source-race compile finished before publishing generated C++" >&2
|
||||
echo "source-race compile finished before staging generated C++" >&2
|
||||
cat "$race_output" "$race_error" >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
sleep 0.02
|
||||
done
|
||||
if [[ ! -e "$race_cpp" ]]; then
|
||||
echo "source-race compile did not publish generated C++ before deadline" >&2
|
||||
if [[ -z "$race_cpp" || ! -e "$race_cpp" ]]; then
|
||||
echo "source-race compile did not stage generated C++ before deadline" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -q 'source-race-marker-a' "$race_cpp"; then
|
||||
@@ -310,14 +450,23 @@ fi
|
||||
|
||||
worker_count=$(awk -F= '/^[[:space:]]*WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg 2>/dev/null || true)
|
||||
worker_count="${worker_count:-4}"
|
||||
cli_worker_count=$(awk -F= '/^[[:space:]]*CLI_WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); value=$2} END{print value}' /etc/uce/settings.cfg 2>/dev/null || true)
|
||||
cli_worker_count="${cli_worker_count:-0}"
|
||||
cli_worker_max_requests=$(awk -F= '/^[[:space:]]*CLI_WORKER_MAX_REQUESTS[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); value=$2} END{print value}' /etc/uce/settings.cfg 2>/dev/null || true)
|
||||
cli_worker_max_requests="${cli_worker_max_requests:-8}"
|
||||
worker_pids=""
|
||||
for _ in {1..48}; do
|
||||
output=$(scripts/uce-cli "/$test_name/parent.uce")
|
||||
worker_pids+="${output##*:}"$'\n'
|
||||
done
|
||||
unique_workers=$(printf '%s' "$worker_pids" | sed '/^$/d' | sort -u | wc -l)
|
||||
if (( unique_workers > worker_count )); then
|
||||
echo "worker pool recycled during 48 requests: $unique_workers PIDs for $worker_count workers" >&2
|
||||
if (( cli_worker_count > 0 && cli_worker_max_requests > 0 )); then
|
||||
if (( unique_workers <= cli_worker_count )); then
|
||||
echo "dedicated CLI pool did not recycle during 48 requests: $unique_workers PIDs for $cli_worker_count workers" >&2
|
||||
exit 1
|
||||
fi
|
||||
elif (( unique_workers > worker_count )); then
|
||||
echo "shared worker pool recycled during 48 requests: $unique_workers PIDs for $worker_count workers" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
Executable
+167
@@ -0,0 +1,167 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" != "--inside" ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 240s unshare --mount --net --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
ip link set lo up
|
||||
root="/tmp/uce-dynamic-compile-failures-$$"
|
||||
site="$root/site"
|
||||
work="$root/work"
|
||||
settings="$root/settings.cfg"
|
||||
log="$root/service.log"
|
||||
port=18080
|
||||
server_pid=""
|
||||
cleanup() {
|
||||
status=$?
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
|
||||
while umount /etc/uce/settings.cfg 2>/dev/null; do :; done
|
||||
rm -rf "$root"
|
||||
return "$status"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
|
||||
cat >"$site/driver.uce" <<'UCE'
|
||||
RENDER(Request& context) {
|
||||
print("driver:");
|
||||
if(context.get["mode"] == "render") unit_render("broken-render", context);
|
||||
else if(context.get["mode"] == "call") unit_call("broken-call", "COMPONENT");
|
||||
else print(component("child", context));
|
||||
}
|
||||
UCE
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("good-v1"); }' >"$site/child.uce"
|
||||
printf '%s\n' 'RENDER(Request& context) { print("render-ok"); }' >"$site/broken-render.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("call-ok"); }' >"$site/broken-call.uce"
|
||||
|
||||
write_settings() {
|
||||
local show_errors="$1" serve_good="$2" proactive="$3"
|
||||
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|PROACTIVE_COMPILE_ENABLED|PROACTIVE_COMPILE_CHECK_INTERVAL|PROACTIVE_COMPILE_JOBS|WORKER_COUNT|SHOW_DYNAMIC_COMPILE_ERRORS|SERVE_LAST_KNOWN_GOOD)[[:space:]]*=/d' \
|
||||
/etc/uce/settings.cfg >"$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$root/run/cli.sock
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=$port
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||
PROACTIVE_COMPILE_ENABLED=$proactive
|
||||
PROACTIVE_COMPILE_CHECK_INTERVAL=1
|
||||
PROACTIVE_COMPILE_JOBS=1
|
||||
WORKER_COUNT=1
|
||||
SHOW_DYNAMIC_COMPILE_ERRORS=$show_errors
|
||||
SERVE_LAST_KNOWN_GOOD=$serve_good
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
}
|
||||
start_server() {
|
||||
: >"$log"
|
||||
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
for _ in $(seq 1 500); do
|
||||
curl -sS --max-time 1 "http://127.0.0.1:$port/driver.uce" >/dev/null 2>&1 && return
|
||||
kill -0 "$server_pid" 2>/dev/null || break
|
||||
sleep 0.02
|
||||
done
|
||||
cat "$log" >&2
|
||||
return 1
|
||||
}
|
||||
stop_server() {
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
server_pid=""
|
||||
}
|
||||
request() {
|
||||
local method="${1:-GET}" query="${2:-}"
|
||||
if [[ "$method" == POST ]]; then
|
||||
curl -sS --max-time 30 -w $'\n%{http_code}' -X POST --data '' "http://127.0.0.1:$port/driver.uce?$query"
|
||||
else
|
||||
curl -sS --max-time 30 -w $'\n%{http_code}' "http://127.0.0.1:$port/driver.uce?$query"
|
||||
fi
|
||||
}
|
||||
body_of() { printf '%s' "${1%$'\n'*}"; }
|
||||
status_of() { printf '%s' "${1##*$'\n'}"; }
|
||||
|
||||
write_settings 1 0 0
|
||||
start_server
|
||||
printf '%s\n' 'COMPONENT(Request& context) { UCE_DYNAMIC_COMPILE_MARKER }' >"$site/child.uce"
|
||||
response=$(request GET)
|
||||
body=$(body_of "$response")
|
||||
[[ "$(status_of "$response")" == 500 ]]
|
||||
[[ "$body" == *UCE_DYNAMIC_COMPILE_MARKER* ]]
|
||||
[[ "$body" != *"component not found"* ]]
|
||||
printf '%s\n' 'RENDER(Request& context) { UCE_RENDER_COMPILE_MARKER }' >"$site/broken-render.uce"
|
||||
response=$(request GET mode=render)
|
||||
[[ "$(status_of "$response")" == 500 && "$(body_of "$response")" == *UCE_RENDER_COMPILE_MARKER* ]]
|
||||
printf '%s\n' 'COMPONENT(Request& context) { UCE_CALL_COMPILE_MARKER }' >"$site/broken-call.uce"
|
||||
response=$(request GET mode=call)
|
||||
[[ "$(body_of "$response")" == *UCE_CALL_COMPILE_MARKER* ]]
|
||||
[[ "$(body_of "$response")" != *"function 'COMPONENT' not found"* ]]
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("recovered"); }' >"$site/child.uce"
|
||||
response=$(request GET)
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:recovered" ]]
|
||||
stop_server
|
||||
|
||||
write_settings 0 0 0
|
||||
start_server
|
||||
printf '%s\n' 'COMPONENT(Request& context) { UCE_HIDDEN_COMPILE_MARKER }' >"$site/child.uce"
|
||||
response=$(request GET)
|
||||
body=$(body_of "$response")
|
||||
[[ "$(status_of "$response")" == 500 ]]
|
||||
[[ "$body" == *"component not found: child"* ]]
|
||||
[[ "$body" != *UCE_HIDDEN_COMPILE_MARKER* ]]
|
||||
stop_server
|
||||
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("good-v1"); }' >"$site/child.uce"
|
||||
write_settings 1 1 1
|
||||
start_server
|
||||
for _ in $(seq 1 100); do
|
||||
response=$(request GET)
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]] && break
|
||||
sleep 0.1
|
||||
done
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]]
|
||||
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
|
||||
for artifact in "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map"; do [[ -s "$artifact" ]]; done
|
||||
before_hashes=$(sha256sum "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map")
|
||||
printf '%s\n' 'COMPONENT(Request& context) { UCE_STALE_COMPILE_MARKER }' >"$site/child.uce"
|
||||
for _ in $(seq 1 30); do
|
||||
response=$(request GET)
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]]
|
||||
sleep 0.1
|
||||
done
|
||||
[[ -s "$cache/child.uce.compile.txt" ]]
|
||||
[[ "$(sha256sum "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map")" == "$before_hashes" ]]
|
||||
stop_server
|
||||
start_server
|
||||
response=$(request GET)
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]] || { echo "last-known-good artifact did not survive worker restart: $response" >&2; exit 1; }
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("permission-repair"); }' >"$site/child.uce"
|
||||
chmod 000 "$site/child.uce"
|
||||
for _ in $(seq 1 100); do
|
||||
grep -q "source file is not readable" "$cache/child.uce.compile.txt" 2>/dev/null && break
|
||||
sleep 0.05
|
||||
done
|
||||
[[ "$(sha256sum "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map")" == "$before_hashes" ]]
|
||||
response=$(request GET)
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]] || { echo "unreadable source removed last-known-good service: $response" >&2; exit 1; }
|
||||
chmod 644 "$site/child.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("good-v2"); }' >"$site/child.uce"
|
||||
for _ in $(seq 1 200); do
|
||||
response=$(request GET)
|
||||
if [[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v2" ]]; then break; fi
|
||||
sleep 0.1
|
||||
done
|
||||
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v2" ]] || { echo "repaired unit was not published: $response" >&2; exit 1; }
|
||||
|
||||
echo "dynamic compile failure handling passed"
|
||||
Executable
+175
@@ -0,0 +1,175 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="entry-freshness-ttl-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
worker_count="${UCE_TEST_WORKER_COUNT:-4}"
|
||||
bin_directory="${BIN_DIRECTORY:-/tmp/uce/work}"
|
||||
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
|
||||
if [[ -r /etc/uce/settings.cfg ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
configured_worker_count=$(awk -F= '/^[[:space:]]*WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
[[ -n "${UCE_TEST_SITE_DIRECTORY:-}" || -z "$configured_site_directory" ]] || site_directory="$configured_site_directory"
|
||||
[[ -n "${UCE_TEST_WORKER_COUNT:-}" || -z "$configured_worker_count" ]] || worker_count="$configured_worker_count"
|
||||
[[ -n "${BIN_DIRECTORY:-}" || -z "$configured_bin_directory" ]] || bin_directory="$configured_bin_directory"
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
cache_dir=""
|
||||
mutation_file="/tmp/$test_name-mutation"
|
||||
post_body="/tmp/$test_name-post-body"
|
||||
post_headers="/tmp/$test_name-post-headers"
|
||||
generation_file=""
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "$generation_file" ]]; then
|
||||
exec 7>"$generation_file.lock"
|
||||
flock 7
|
||||
printf 'test-cleanup:%s\n' "$(date +%s%N)" >"$generation_file"
|
||||
flock -u 7
|
||||
exec 7>&-
|
||||
fi
|
||||
rm -rf "$source_dir"
|
||||
[[ -z "$cache_dir" ]] || rm -rf "$cache_dir"
|
||||
rm -f "$mutation_file" "$post_body" "$post_headers"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
generation_file="$(scripts/unit_cache_directory "$bin_directory")/source-generation.txt"
|
||||
|
||||
printf '%s\n' 'String entry_freshness_marker() { return("marker-a"); }' >"$source_dir/child.uce"
|
||||
printf '%s\n' \
|
||||
'#load "child.uce"' \
|
||||
"void entry_freshness_output(Request& context) { DValue perf = request_perf(); if(context.params[\"REQUEST_METHOD\"] == \"POST\") file_put_contents(\"$mutation_file\", \"executed\"); print(entry_freshness_marker(), \":\", perf[\"worker_pid\"].to_string(), \":\", perf[\"ready_freshness_cache_hit_count\"].to_string(), \":\", perf[\"ready_freshness_full_check_us\"].to_string()); }" \
|
||||
'RENDER(Request& context) { entry_freshness_output(context); }' \
|
||||
'CLI(Request& context) { entry_freshness_output(context); }' >"$source_dir/parent.uce"
|
||||
|
||||
http_probe() {
|
||||
curl -fsS --max-time 5 -H "Host: $http_host" "http://127.0.0.1/$test_name/parent.uce"
|
||||
}
|
||||
|
||||
declare -A seen
|
||||
collect_all_workers() {
|
||||
local require_hit="$1"
|
||||
local expected_marker="$2"
|
||||
local deadline=$((SECONDS + 35))
|
||||
seen=()
|
||||
while (( SECONDS < deadline && ${#seen[@]} < worker_count )); do
|
||||
local output marker pid hit full
|
||||
output=$(http_probe)
|
||||
IFS=: read -r marker pid hit full <<<"$output"
|
||||
pid=${pid%%.*}; hit=${hit%%.*}; full=${full%%.*}
|
||||
if [[ "$marker" != "$expected_marker" || -z "$pid" ]]; then
|
||||
echo "entry freshness probe returned unexpected output: $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$require_hit" == "1" && ( "$hit" == "0" || "$full" != "0" ) ]]; then
|
||||
continue
|
||||
fi
|
||||
seen["$pid"]=1
|
||||
done
|
||||
if (( ${#seen[@]} != worker_count )); then
|
||||
echo "entry freshness probe reached ${#seen[@]}/$worker_count workers (hit=$require_hit)" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
collect_all_workers 0 marker-a
|
||||
collect_all_workers 1 marker-a
|
||||
|
||||
parent_wasm="$cache_dir/parent.uce.wasm"
|
||||
exec 8>"$parent_wasm.lock"
|
||||
flock 8
|
||||
sed -i 's/marker-a/marker-b/' "$source_dir/child.uce"
|
||||
rm -f "$mutation_file"
|
||||
post_status=$(curl -sS --max-time 5 -o "$post_body" -D "$post_headers" -w '%{http_code}' -X POST -H "Host: $http_host" "http://127.0.0.1/$test_name/parent.uce")
|
||||
if [[ "$post_status" != "503" || -e "$mutation_file" ]] || ! grep -qi '^Retry-After: 1' "$post_headers"; then
|
||||
echo "stale mutation did not fail closed: status=$post_status body=$(cat "$post_body")" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
sleep 10.2
|
||||
expired_output=$(http_probe)
|
||||
IFS=: read -r expired_marker expired_pid expired_hit expired_full <<<"$expired_output"
|
||||
expired_hit=${expired_hit%%.*}; expired_full=${expired_full%%.*}
|
||||
if [[ "$expired_marker" != "marker-a" || "$expired_hit" != "0" || "$expired_full" == "0" ]]; then
|
||||
echo "expired positive entry did not perform a full safe check: $expired_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
flock -u 8
|
||||
exec 8>&-
|
||||
|
||||
deadline=$((SECONDS + 25))
|
||||
output=""
|
||||
while (( SECONDS < deadline )); do
|
||||
output=$(http_probe)
|
||||
[[ "$output" == marker-b:* ]] && break
|
||||
sleep 0.2
|
||||
done
|
||||
if [[ "$output" != marker-b:* ]]; then
|
||||
echo "entry did not converge after dependency rebuild: $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
collect_all_workers 1 marker-b
|
||||
|
||||
exec 9>"$generation_file.lock"
|
||||
flock 9
|
||||
printf 'test:%s\n' "$(date +%s%N)" >"$generation_file"
|
||||
flock -u 9
|
||||
exec 9>&-
|
||||
generation_output=$(http_probe)
|
||||
IFS=: read -r generation_marker generation_pid generation_hit generation_full <<<"$generation_output"
|
||||
generation_hit=${generation_hit%%.*}; generation_full=${generation_full%%.*}
|
||||
if [[ "$generation_marker" != "marker-b" || "$generation_hit" != "0" || "$generation_full" == "0" ]]; then
|
||||
echo "source generation change did not invalidate immediately: $generation_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
collect_all_workers 1 marker-b
|
||||
touch "$parent_wasm"
|
||||
identity_output=$(http_probe)
|
||||
IFS=: read -r identity_marker identity_pid identity_hit identity_full <<<"$identity_output"
|
||||
identity_hit=${identity_hit%%.*}; identity_full=${identity_full%%.*}
|
||||
if [[ "$identity_marker" != "marker-b" || "$identity_hit" != "0" || "$identity_full" == "0" ]]; then
|
||||
echo "wasm identity change did not invalidate immediately: $identity_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
collect_all_workers 1 marker-b
|
||||
touch "$cache_dir/parent.uce.meta.txt"
|
||||
metadata_output=$(http_probe)
|
||||
IFS=: read -r metadata_marker metadata_pid metadata_hit metadata_full <<<"$metadata_output"
|
||||
metadata_hit=${metadata_hit%%.*}; metadata_full=${metadata_full%%.*}
|
||||
if [[ "$metadata_marker" != "marker-b" || "$metadata_hit" != "0" || "$metadata_full" == "0" ]]; then
|
||||
echo "metadata identity change did not invalidate immediately: $metadata_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
collect_all_workers 1 marker-b
|
||||
exec 9>"$generation_file.lock"
|
||||
flock 9
|
||||
rm -f "$generation_file"
|
||||
flock -u 9
|
||||
missing_generation_output=$(http_probe)
|
||||
IFS=: read -r missing_generation_marker missing_generation_pid missing_generation_hit missing_generation_full <<<"$missing_generation_output"
|
||||
missing_generation_hit=${missing_generation_hit%%.*}; missing_generation_full=${missing_generation_full%%.*}
|
||||
if [[ "$missing_generation_marker" != "marker-b" || "$missing_generation_hit" != "0" || "$missing_generation_full" == "0" ]]; then
|
||||
echo "missing generation token did not force a full check: $missing_generation_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
flock 9
|
||||
printf 'test:%s\n' "$(date +%s%N)" >"$generation_file"
|
||||
flock -u 9
|
||||
exec 9>&-
|
||||
|
||||
cli_output=$(timeout --signal=TERM --kill-after=3s 20s scripts/uce-cli "/$test_name/parent.uce")
|
||||
IFS=: read -r cli_marker cli_pid cli_hit cli_full <<<"$cli_output"
|
||||
cli_hit=${cli_hit%%.*}; cli_full=${cli_full%%.*}
|
||||
if [[ "$cli_marker" != "marker-b" || "$cli_hit" != "0" || "$cli_full" == "0" ]]; then
|
||||
echo "CLI unexpectedly reused the HTTP freshness cache: $cli_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "entry freshness TTL passed"
|
||||
@@ -0,0 +1,37 @@
|
||||
#include "src/lib/types.cpp"
|
||||
#include "src/lib/dvalue.cpp"
|
||||
#include "src/lib/functionlib.cpp"
|
||||
#include "src/wasm/hardened_http_internal.h"
|
||||
#include <iostream>
|
||||
struct Fake { std::vector<String> answers, argv, env; String input; HardenedHttpExecResult result; };
|
||||
static DValue request() { DValue r; r["method"]="POST"; r["url"]="https://api.example.test/token"; r["headers"]["Accept"]="application/json"; r["body"]="client_secret=SECRET"; DValue& s=r["security"]; s["https_only"].set_bool(true); s["public_dns_only"].set_bool(true); s["pin_dns"].set_bool(true); s["isolated_curl"].set_bool(true); s["no_redirects"].set_bool(true); return r; }
|
||||
static DValue run(Fake& f,DValue r) { HardenedHttpHooks h; h.resolve=[&](String){return f.answers;}; h.execute=[&](std::vector<String> a,String i,std::vector<String> e,u64,size_t){f.argv=a;f.input=i;f.env=e;return f.result;}; return hardened_http_request_internal(r,9000,h); }
|
||||
static String headers(int status=200) { return "HTTP/1.1 "+std::to_string(status)+" OK\r\nContent-Type: application/json\r\n\r\n"; }
|
||||
static bool exists(String path) { return access(path.c_str(),F_OK)==0; }
|
||||
static void wait_for(String path) { for(int n=0;n<500&&!exists(path);n++) usleep(1000); }
|
||||
static bool process_live(pid_t pid) { char state=0; String path="/proc/"+std::to_string((long long)pid)+"/stat"; int fd=open(path.c_str(),O_RDONLY); if(fd<0) return false; char text[256]{}; ssize_t n=read(fd,text,sizeof(text)-1); close(fd); if(n<=0) return false; char* close_paren=strrchr(text,')'); return close_paren&&close_paren[2]!='Z'; }
|
||||
int main(int argc,char** argv) {
|
||||
if(argc==3&&String(argv[1])=="--child-timeout") { pid_t child=fork(); if(child==0) { usleep(100000); int fd=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"leaked",6); close(fd); } _exit(0); } for(;;) pause(); }
|
||||
if(argc==2&&String(argv[1])=="--child-output") { String x(4096,'x'); for(int n=0;n<64;n++) write(1,x.data(),x.size()); return 0; }
|
||||
if(argc==3&&String(argv[1])=="--child-fds") { int inherited=atoi(argv[2]); bool closed=fcntl(inherited,F_GETFD)==-1&&errno==EBADF; bool header=fcntl(3,F_GETFD)!=-1; write(1,closed&&header?"closed":"open",closed&&header?6:4); return closed&&header?0:1; }
|
||||
if(argc==5&&String(argv[1])=="--child-job") { int ready=open(argv[3],O_WRONLY|O_CREAT|O_TRUNC,0600); if(ready>=0) { write(ready,"ready",5); close(ready); } pid_t child=fork(); if(child==0) { int pidfile=open(argv[4],O_WRONLY|O_CREAT|O_TRUNC,0600); if(pidfile>=0) { String pid=std::to_string((long long)getpid()); write(pidfile,pid.data(),pid.size()); close(pidfile); } usleep(150000); int marker=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(marker>=0) { write(marker,"leaked",6); close(marker); } _exit(0); } for(;;) pause(); }
|
||||
if(argc==3&&String(argv[1])=="--child-async-timeout") { setsid(); HardenedHttpExecResult result=hardened_http_exec_argv_capture({"/bin/sleep","1"},"",20,4096,false,false); if(result.timed_out&&result.exit_code==137) { int fd=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"typed",5); close(fd); } } return 0; }
|
||||
if(argc==3&&String(argv[1])=="--child-async-output") { setsid(); HardenedHttpExecResult result=hardened_http_exec_argv_capture({"/proc/self/exe","--child-output"},"",1000,1024,false,false); if(result.output_limited&&result.exit_code==137) { int fd=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"typed",5); close(fd); } } return 0; }
|
||||
bool ok=true; auto need=[&](bool x,const char* n){if(!x){std::cerr<<n<<"\n";ok=false;}};
|
||||
Fake f; f.answers={"8.8.8.8"}; f.result.exit_code=0; f.result.stderr_text="SECRET"; f.result.headers_text=headers(); f.result.body_text="one\r\n\r\ntwo"; DValue out=run(f,request());
|
||||
need(out["error"].to_string()==""&&out["error"].to_string().find("SECRET")==String::npos&&out["body"].to_string()=="one\r\n\r\ntwo","body framing / non-secret errors"); need(f.argv.size()>2&&f.argv[0]=="/usr/bin/curl"&&f.argv[1]=="--disable","absolute curl disable"); need(f.env.size()==1&&f.env[0]=="PATH=/usr/bin:/bin","clean env"); need(std::find(f.argv.begin(),f.argv.end(),"SECRET")==f.argv.end()&&f.input.find("SECRET")!=String::npos,"secret stdin only"); need(std::find(f.argv.begin(),f.argv.end(),"--resolve")!=f.argv.end(),"pinned dns");
|
||||
Fake interim; interim.answers={"8.8.8.8"}; interim.result.exit_code=0; interim.result.headers_text="HTTP/1.1 100 Continue\r\n\r\n"+headers(201); interim.result.body_text="{}"; need(run(interim,request())["status"].to_u64()==201,"interim headers");
|
||||
for(String bad:{"127.0.0.1","10.0.0.1","169.254.1.1","192.168.1.1","::1","2001:db8::1"}) { Fake x; x.answers={"8.8.8.8",bad}; need(run(x,request())["error"].to_string()=="unsafe_dns","dns matrix"); }
|
||||
for(auto pair:std::vector<std::pair<int,String>>{{302,headers(302)},{500,headers(500)},{200,"bad\r\n\r\n"}}) { Fake x; x.answers={"8.8.8.8"};x.result.exit_code=0;x.result.headers_text=pair.second;need(run(x,request())["error"].to_string()==(pair.first==302?"redirect_not_allowed":pair.first==500?"http_status":"malformed_output"),"status errors"); }
|
||||
DValue bad=request(); bad["method"]="TRACE"; Fake x; need(run(x,bad)["error"].to_string()=="invalid_request"&&x.argv.empty(),"method validation"); bad=request();bad["headers"]["Host"]="evil";need(run(x,bad)["error"].to_string()=="invalid_request","header validation"); bad=request();bad["headers"]["Accept"]="ok\r\nInjected: x";need(run(x,bad)["error"].to_string()=="invalid_request","header CRLF validation");bad=request();bad["url"]="https://127.0.0.1/";need(run(x,bad)["error"].to_string()=="invalid_request","url validation");bad=request();bad["follow_redirects"].set_bool(true);need(run(x,bad)["error"].to_string()=="invalid_request","redirect composition");
|
||||
for(String key:{"https_only","public_dns_only","pin_dns","isolated_curl","no_redirects"}) { bad=request(); bad["security"].remove(key); need(run(x,bad)["error"].to_string()=="invalid_request","partial hardening fails closed"); bad=request(); bad["security"][key].set_bool(false); need(run(x,bad)["error"].to_string()=="invalid_request","false hardening fails closed"); bad=request(); bad["security"][key]="true"; need(run(x,bad)["error"].to_string()=="invalid_request","non-boolean hardening fails closed"); }
|
||||
DValue legacy=request(); legacy["security"].clear(); legacy["security"]["unrelated"]="value"; need(!hardened_http_security_requested(legacy.key("security")),"unknown security object remains legacy");
|
||||
Fake large;large.answers={"8.8.8.8"};large.result.exit_code=0;large.result.headers_text=headers();large.result.body_text=String(65537,'x');need(run(large,request())["error"].to_string()=="response_too_large","body cap");
|
||||
String base="/tmp/hardened-http-"+std::to_string((long long)getpid()), marker=base+"-marker"; unlink(marker.c_str()); HardenedHttpExecResult timeout=hardened_http_exec_argv_capture({"/proc/self/exe","--child-timeout",marker},"",30,4096,false); usleep(150000); need(timeout.timed_out&&timeout.exit_code==137&&!exists(marker),"timeout kills descendants"); unlink(marker.c_str());
|
||||
HardenedHttpExecResult overflow=hardened_http_exec_argv_capture({"/proc/self/exe","--child-output"},"",1000,1024,false);need(overflow.output_limited&&overflow.body_text.size()<=1024,"output cap/reap");
|
||||
int inherited=open("/dev/null",O_RDONLY), high_inherited=fcntl(inherited,F_DUPFD,10); close(inherited); HardenedHttpExecResult fds=hardened_http_exec_argv_capture({"/proc/self/exe","--child-fds",std::to_string(high_inherited)},"",1000,4096,false); close(high_inherited); need(fds.exit_code==0&&fds.body_text=="closed","only stdio and header fd inherited");
|
||||
String ready=base+"-ready", descendant=base+"-descendant", sentinel=base+"-sentinel"; unlink(marker.c_str()); unlink(ready.c_str()); unlink(descendant.c_str()); unlink(sentinel.c_str()); pid_t worker=fork(); if(worker==0) { setsid(); hardened_http_exec_argv_capture({"/proc/self/exe","--child-job",marker,ready,descendant},"",5000,4096,false,false); _exit(0); } wait_for(ready); wait_for(descendant); pid_t unrelated=fork(); if(unrelated==0) { setsid(); usleep(150000); int fd=open(sentinel.c_str(),O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"alive",5); close(fd); } _exit(0); } pid_t descendant_pid=exists(descendant)?(pid_t)strtol([](String path){ int fd=open(path.c_str(),O_RDONLY); char text[32]{}; ssize_t n=fd<0?-1:read(fd,text,sizeof(text)-1); if(fd>=0) close(fd); return String(text,n>0?(size_t)n:0); }(descendant).c_str(),0,10):0; need(exists(ready)&&descendant_pid>0,"async worker and descendant started"); kill(-worker,SIGKILL); waitpid(worker,0,0); usleep(250000); waitpid(unrelated,0,0); need(!process_live(descendant_pid)&&!exists(marker)&&exists(sentinel),"async cancellation leaves no live descendant and kills only its worker group"); unlink(marker.c_str()); unlink(ready.c_str()); unlink(descendant.c_str()); unlink(sentinel.c_str());
|
||||
String async_timeout=base+"-async-timeout"; unlink(async_timeout.c_str()); pid_t timeout_worker=fork(); if(timeout_worker==0) { execl("/proc/self/exe","test_hardened_http_native","--child-async-timeout",async_timeout.c_str(),(char*)0); _exit(127); } int timeout_status=0; waitpid(timeout_worker,&timeout_status,0); need(WIFEXITED(timeout_status)&&WEXITSTATUS(timeout_status)==0&&exists(async_timeout),"async timeout records a typed terminal result before worker exit"); unlink(async_timeout.c_str());
|
||||
String async_output=base+"-async-output"; unlink(async_output.c_str()); pid_t output_worker=fork(); if(output_worker==0) { execl("/proc/self/exe","test_hardened_http_native","--child-async-output",async_output.c_str(),(char*)0); _exit(127); } int output_status=0; waitpid(output_worker,&output_status,0); need(WIFEXITED(output_status)&&WEXITSTATUS(output_status)==0&&exists(async_output),"async output cap records a typed terminal result before worker exit"); unlink(async_output.c_str());
|
||||
return ok?0:1;
|
||||
}
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
repo=$(cd "$(dirname "$0")/.." && pwd)
|
||||
bin="${TMPDIR:-/tmp}/test_hardened_http_native.$$"
|
||||
trap 'rm -f "$bin"' EXIT
|
||||
compiler=$(command -v clang++ || command -v g++)
|
||||
"$compiler" -std=c++20 -fpermissive -I"$repo" "$repo/scripts/test_hardened_http_native.cpp" -lpcre2-8 -o "$bin"
|
||||
timeout --signal=TERM --kill-after=2s 15s "$bin"
|
||||
@@ -26,7 +26,7 @@ cleanup() {
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$bin_directory$(realpath "$source_dir")"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' 'CLI(Request& context) { deliberate_log_timeliness_compile_failure }' >"$source_dir/probe.uce"
|
||||
rm -rf "$cache_dir"
|
||||
|
||||
@@ -38,7 +38,7 @@ cleanup() {
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$bin_directory$(realpath "$source_dir")"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
mariadb -e "DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'"
|
||||
test_user_created=true
|
||||
|
||||
|
||||
@@ -18,10 +18,11 @@ cache_dir=""
|
||||
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
|
||||
test_user="uce_pool_$$"
|
||||
test_database="uce_pool_$$"
|
||||
test_database_other="uce_pool_other_$$"
|
||||
test_password=$(printf '%s' "$test_name-$(date +%s%N)" | sha256sum | cut -c1-32)
|
||||
|
||||
cleanup() {
|
||||
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'" >/dev/null 2>&1 || true
|
||||
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP DATABASE IF EXISTS \`$test_database_other\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'" >/dev/null 2>&1 || true
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$cache_dir" ]]; then
|
||||
rm -rf "$cache_dir"
|
||||
@@ -29,15 +30,22 @@ cleanup() {
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$bin_directory$(realpath "$source_dir")"
|
||||
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE DATABASE \`$test_database\`; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'; GRANT ALL ON \`$test_database\`.* TO '$test_user'@'127.0.0.1'"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP DATABASE IF EXISTS \`$test_database_other\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE DATABASE \`$test_database\`; CREATE DATABASE \`$test_database_other\`; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'; GRANT ALL ON \`$test_database\`.* TO '$test_user'@'127.0.0.1'; GRANT ALL ON \`$test_database_other\`.* TO '$test_user'@'127.0.0.1'; CREATE TABLE \`$test_database\`.pool_identity (label VARCHAR(16) NOT NULL); INSERT INTO \`$test_database\`.pool_identity VALUES ('primary'); CREATE TABLE \`$test_database_other\`.pool_identity (label VARCHAR(16) NOT NULL); INSERT INTO \`$test_database_other\`.pool_identity VALUES ('other')"
|
||||
|
||||
printf '%s\n' \
|
||||
'RENDER(Request& context)' \
|
||||
'{' \
|
||||
" MySQL* db = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\");" \
|
||||
' if(db == 0 || mysql_error(db) != "") { context.set_status(500, "MySQL connect failed"); print("connect-failed"); return; }' \
|
||||
" mysql_query(db, \"USE \\\`$test_database\\\`\");" \
|
||||
" MySQL* db = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\", \"$test_database\");" \
|
||||
" MySQL* other = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\", \"$test_database_other\");" \
|
||||
" MySQL* unselected = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\");" \
|
||||
' if(db == 0 || other == 0 || unselected == 0 || !mysql_connected(db) || !mysql_connected(other) || !mysql_connected(unselected)) { context.set_status(500, "MySQL connect failed"); print("connect-failed"); return; }' \
|
||||
' String primary_database, primary_label, other_database, other_label;' \
|
||||
' mysql_query(db, "SELECT DATABASE() AS db, label FROM pool_identity").each([&](DValue row, String key) { primary_database = row["db"].to_string(); primary_label = row["label"].to_string(); });' \
|
||||
' mysql_query(other, "SELECT DATABASE() AS db, label FROM pool_identity").each([&](DValue row, String key) { other_database = row["db"].to_string(); other_label = row["label"].to_string(); });' \
|
||||
' String unselected_database;' \
|
||||
' mysql_query(unselected, "SELECT DATABASE() AS db").each([&](DValue row, String key) { unselected_database = row["db"].to_string(); });' \
|
||||
" bool database_clean = primary_database == \"$test_database\" && primary_label == \"primary\" && other_database == \"$test_database_other\" && other_label == \"other\" && unselected_database == \"\";" \
|
||||
' DValue marker_rows = mysql_query(db, "SELECT @uce_pool_marker AS marker");' \
|
||||
' String marker; marker_rows.each([&](DValue row, String key) { marker = row["marker"].to_string(); });' \
|
||||
' bool marker_clean = marker == "";' \
|
||||
@@ -45,19 +53,39 @@ printf '%s\n' \
|
||||
' bool temp_clean = mysql_error(db) != "";' \
|
||||
" mysql_query(db, \"SET @uce_pool_marker='dirty'\");" \
|
||||
' mysql_query(db, "CREATE TEMPORARY TABLE uce_persistent_temp (id INT PRIMARY KEY)");' \
|
||||
" mysql_query(db, \"USE \\\`$test_database_other\\\`\");" \
|
||||
" mysql_query(other, \"USE \\\`$test_database\\\`\");" \
|
||||
" mysql_query(unselected, \"USE \\\`$test_database\\\`\");" \
|
||||
' DValue perf = request_perf();' \
|
||||
' String source;' \
|
||||
' perf["mysql_operations"].each([&](DValue operation, String key) { if(operation["op"].to_string() == "connect") source = operation["source"].to_string(); });' \
|
||||
' print(perf["worker_pid"].to_string(), "|", source, "|", marker_clean ? "clean" : "dirty", "|", temp_clean ? "clean" : "dirty");' \
|
||||
' perf["mysql_operations"].each([&](DValue operation, String key) { if(operation["op"].to_string() == "connect" && operation["source"].to_string() == "worker") source = "worker"; });' \
|
||||
' print(perf["worker_pid"].to_string(), "|", source, "|", marker_clean ? "clean" : "dirty", "|", temp_clean ? "clean" : "dirty", "|", database_clean ? "clean" : "dirty");' \
|
||||
' mysql_disconnect(unselected);' \
|
||||
' mysql_disconnect(other);' \
|
||||
' mysql_disconnect(db);' \
|
||||
'}' >"$source_dir/test.uce"
|
||||
|
||||
printf '%s\n' \
|
||||
'RENDER(Request& context)' \
|
||||
'{' \
|
||||
" MySQL* db = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\", \"${test_database}_missing\");" \
|
||||
' String error = mysql_error(db);' \
|
||||
' print(!mysql_connected(db) && error != "" ? "database-selection-failed" : "database-selection-was-ignored");' \
|
||||
' mysql_disconnect(db);' \
|
||||
'}' >"$source_dir/failure.uce"
|
||||
|
||||
failure=$(curl -fsS --max-time 10 -H "Host: $http_host" "http://127.0.0.1/$test_name/failure.uce")
|
||||
if [[ "$failure" != "database-selection-failed" ]]; then
|
||||
echo "Unknown initial database did not surface a connection failure: $failure" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
reused=0
|
||||
for _ in $(seq 1 160); do
|
||||
output=$(curl -fsS --max-time 10 -H "Host: $http_host" "http://127.0.0.1/$test_name/test.uce")
|
||||
if [[ "$output" == *"|worker|"* ]]; then
|
||||
reused=$((reused + 1))
|
||||
if [[ "$output" != *"|worker|clean|clean" ]]; then
|
||||
if [[ "$output" != *"|worker|clean|clean|clean" ]]; then
|
||||
echo "Persistent MySQL reuse leaked cross-request state: $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
Executable
+145
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" != "--inside" ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 150s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
|
||||
name="mysql-pool-idle-test-$$"
|
||||
root="/tmp/$name"
|
||||
site="$root/site"
|
||||
work="$root/work"
|
||||
settings="$root/settings.cfg"
|
||||
log="$root/service.log"
|
||||
socket="$root/run/cli.sock"
|
||||
test_user="uce_pool_idle_$$"
|
||||
test_database="uce_pool_idle_$$"
|
||||
test_password=$(printf '%s' "$name-$(date +%s%N)" | sha256sum | cut -c1-32)
|
||||
server_pid=""
|
||||
|
||||
cleanup() {
|
||||
status=$?
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
deadline=$((SECONDS + 10))
|
||||
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
if kill -0 "$server_pid" 2>/dev/null; then kill -KILL "$server_pid" 2>/dev/null || true; fi
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'" >/dev/null 2>&1 || true
|
||||
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
|
||||
rm -rf "$root"
|
||||
return "$status"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
|
||||
cp /etc/uce/settings.cfg "$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$socket
|
||||
CLI_WORKER_COUNT=1
|
||||
CLI_WORKER_MAX_REQUESTS=0
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||
WORKER_COUNT=1
|
||||
PROACTIVE_COMPILE_ENABLED=0
|
||||
MYSQL_PERSISTENT_POOL_SIZE=8
|
||||
MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS=2
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
|
||||
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE DATABASE \`$test_database\`; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'; GRANT ALL ON \`$test_database\`.* TO '$test_user'@'127.0.0.1'"
|
||||
|
||||
cat >"$site/database.uce" <<UCE
|
||||
CLI(Request& context)
|
||||
{
|
||||
MySQL* db = mysql_connect("127.0.0.1", "$test_user", "$test_password", "$test_database");
|
||||
if(!mysql_connected(db)) { print("connect-failed"); return; }
|
||||
String connection_id;
|
||||
mysql_query(db, "SELECT CONNECTION_ID() AS id").each([&](DValue row, String key) { connection_id = row["id"].to_string(); });
|
||||
String source;
|
||||
DValue perf = request_perf();
|
||||
perf["mysql_operations"].each([&](DValue operation, String key) {
|
||||
if(operation["op"].to_string() == "connect") source = operation["source"].to_string();
|
||||
});
|
||||
print(perf["worker_pid"].to_string(), "|", connection_id, "|", source);
|
||||
mysql_disconnect(db);
|
||||
}
|
||||
UCE
|
||||
printf '%s\n' 'CLI(Request& context) { print(request_perf()["worker_pid"].to_string(), "|plain"); }' >"$site/plain.uce"
|
||||
|
||||
timeout --signal=TERM --kill-after=5s 120s bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ ! -S "$socket" ]] && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ -S "$socket" ]] || { echo "private UCE CLI socket was not ready" >&2; exit 1; }
|
||||
|
||||
request() {
|
||||
timeout --signal=TERM --kill-after=1s 30s scripts/uce-cli --socket "$socket" "$1"
|
||||
}
|
||||
|
||||
first=$(request /database.uce)
|
||||
second=$(request /database.uce)
|
||||
IFS='|' read -r first_pid first_id first_source <<<"$first"
|
||||
IFS='|' read -r second_pid second_id second_source <<<"$second"
|
||||
[[ "$first_source" == "new" && "$second_source" == "worker" ]]
|
||||
[[ "$first_pid" == "$second_pid" && "$first_id" == "$second_id" ]]
|
||||
|
||||
sleep 3
|
||||
plain=$(request /plain.uce)
|
||||
[[ "$plain" == "$first_pid|plain" ]]
|
||||
deadline=$((SECONDS + 5))
|
||||
while (( SECONDS < deadline )); do
|
||||
connections=$(mariadb --batch --skip-column-names -e "SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE USER='$test_user'")
|
||||
[[ "$connections" == "0" ]] && break
|
||||
sleep 0.05
|
||||
done
|
||||
[[ "$connections" == "0" ]] || { echo "expired pooled connection remained after a request boundary" >&2; exit 1; }
|
||||
|
||||
third=$(request /database.uce)
|
||||
IFS='|' read -r third_pid third_id third_source <<<"$third"
|
||||
[[ "$third_pid" == "$first_pid" && "$third_source" == "new" && "$third_id" != "$first_id" ]]
|
||||
|
||||
sleep 3
|
||||
plain=$(request /plain.uce)
|
||||
[[ "$plain" == "$first_pid|plain" ]]
|
||||
deadline=$((SECONDS + 5))
|
||||
while (( SECONDS < deadline )); do
|
||||
connections=$(mariadb --batch --skip-column-names -e "SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE USER='$test_user'")
|
||||
[[ "$connections" == "0" ]] && break
|
||||
sleep 0.05
|
||||
done
|
||||
[[ "$connections" == "0" ]] || { echo "final pooled fixture connection remained after idle eviction" >&2; exit 1; }
|
||||
|
||||
retained=$(request /database.uce)
|
||||
IFS='|' read -r retained_pid retained_id retained_source <<<"$retained"
|
||||
[[ "$retained_pid" == "$first_pid" && "$retained_source" == "new" ]]
|
||||
connections=$(mariadb --batch --skip-column-names -e "SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE USER='$test_user'")
|
||||
[[ "$connections" == "1" ]] || { echo "shutdown fixture connection was not retained in the worker pool" >&2; exit 1; }
|
||||
|
||||
journal_cursor=$(journalctl -u mariadb.service -n 0 --show-cursor --no-pager | sed -n 's/^-- cursor: //p')
|
||||
[[ -n "$journal_cursor" ]] || { echo "could not capture MariaDB journal cursor" >&2; exit 1; }
|
||||
kill -TERM "$server_pid"
|
||||
deadline=$((SECONDS + 10))
|
||||
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ ! -e "/proc/$server_pid" ]] || { echo "private UCE server did not stop cleanly" >&2; exit 1; }
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
server_pid=""
|
||||
sleep 1
|
||||
shutdown_log=$(journalctl -u mariadb.service --after-cursor "$journal_cursor" --no-pager)
|
||||
if grep -F "$test_user" <<<"$shutdown_log" | grep -Fq "Aborted connection"; then
|
||||
echo "clean UCE shutdown aborted a retained MySQL connection" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Persistent MySQL idle eviction preserved hot reuse, retired expired connections, and closed a retained connection on clean shutdown"
|
||||
@@ -12,12 +12,22 @@ if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
threshold_ms="${UCE_NESTED_COMPONENT_PROPS_MAX_MS:-1500}"
|
||||
bin_directory="${BIN_DIRECTORY:-}"
|
||||
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
|
||||
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
fi
|
||||
bin_directory="${bin_directory:-/tmp/uce/work}"
|
||||
cache_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$cache_dir" ]]; then
|
||||
rm -rf "$cache_dir"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'CLI(Request& context) {' \
|
||||
|
||||
Executable
+179
@@ -0,0 +1,179 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="parallel-precompile-test-$$"
|
||||
test_root="/tmp/$test_name"
|
||||
source_dir="$test_root/site"
|
||||
bin_directory="$test_root/work"
|
||||
absolute_source_dir=""
|
||||
artifact_dir=""
|
||||
generation_log=""
|
||||
generation_pid=""
|
||||
precompile_timeout="${UCE_TEST_PRECOMPILE_TIMEOUT:-120s}"
|
||||
|
||||
precompile() {
|
||||
timeout "$precompile_timeout" env UCE_PRECOMPILE_FILES_IN="$source_dir" UCE_PRECOMPILE_BIN_DIRECTORY="$bin_directory" "$@" bin/uce_fastcgi.linux.bin --precompile
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "$generation_pid" ]] && kill -0 "$generation_pid" 2>/dev/null; then
|
||||
kill "$generation_pid" 2>/dev/null || true
|
||||
cleanup_deadline=$((SECONDS + 10))
|
||||
while kill -0 "$generation_pid" 2>/dev/null && (( SECONDS < cleanup_deadline )); do
|
||||
sleep 0.05
|
||||
done
|
||||
if kill -0 "$generation_pid" 2>/dev/null; then
|
||||
kill -KILL "$generation_pid" 2>/dev/null || true
|
||||
fi
|
||||
wait "$generation_pid" 2>/dev/null || true
|
||||
fi
|
||||
rm -rf "$test_root"
|
||||
if [[ -n "$generation_log" ]]; then
|
||||
rm -f "$generation_log"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
absolute_source_dir=$(realpath "$source_dir")
|
||||
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$absolute_source_dir"
|
||||
|
||||
write_units() {
|
||||
local version="$1"
|
||||
for unit in 0 1 2 3; do
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-%s-%s"); }\n' "$version" "$unit" >"$source_dir/unit-$unit.uce"
|
||||
done
|
||||
}
|
||||
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-warmup"); }\n' >"$source_dir/warmup.uce"
|
||||
precompile UCE_PRECOMPILE_JOBS=1 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" >/dev/null
|
||||
rm "$source_dir/warmup.uce"
|
||||
|
||||
write_units serial
|
||||
serial_start=$(date +%s%N)
|
||||
serial_output=$(precompile UCE_PRECOMPILE_JOBS=1 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared")
|
||||
serial_ns=$(( $(date +%s%N) - serial_start ))
|
||||
if ! grep -Eq 'with 1 job: .* 4 compiled, 0 failed, worker status ok' <<<"$serial_output"; then
|
||||
echo "serial precompile did not compile the four controlled units" >&2
|
||||
echo "$serial_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
write_units parallel
|
||||
parallel_start=$(date +%s%N)
|
||||
parallel_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared")
|
||||
parallel_ns=$(( $(date +%s%N) - parallel_start ))
|
||||
if [[ $(grep -Ec '^Precompile worker [12]/2:' <<<"$parallel_output") -ne 2 ]]; then
|
||||
echo "parallel precompile did not report both workers" >&2
|
||||
echo "$parallel_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! grep -Eq 'with 2 jobs: .* 4 compiled, 0 failed, worker status ok' <<<"$parallel_output"; then
|
||||
echo "parallel precompile did not compile the four controlled units" >&2
|
||||
echo "$parallel_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
for unit in 0 1 2 3; do
|
||||
if [[ ! -s "$artifact_dir/unit-$unit.uce.wasm" || ! -s "$artifact_dir/unit-$unit.uce.cwasm" ]]; then
|
||||
echo "parallel precompile did not publish wasm and serialized artifacts for unit $unit" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-race-0"); }\n' >"$source_dir/race-0.uce"
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-race-1"); }\n' >"$source_dir/race-1.uce"
|
||||
race_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-race")
|
||||
if ! grep -Eq 'with 2 jobs: .* 2 compiled, 0 failed, worker status ok' <<<"$race_output" || \
|
||||
[[ $(find "$artifact_dir/pch-race" -maxdepth 1 -type f -name '*.pch' | wc -l) -ne 1 ]] || \
|
||||
find "$artifact_dir/pch-race" -maxdepth 1 -type f -name '*.tmp.*' -print -quit | grep -q .; then
|
||||
echo "parallel precompile did not publish exactly one clean shared PCH" >&2
|
||||
echo "$race_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm "$source_dir/race-0.uce" "$source_dir/race-1.uce"
|
||||
|
||||
for unit in $(seq -w 0 9); do
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-generation-%s"); }\n' "$unit" >"$source_dir/generation-$unit.uce"
|
||||
done
|
||||
generation_log="/tmp/$test_name-generation.log"
|
||||
precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" >"$generation_log" 2>&1 &
|
||||
generation_pid=$!
|
||||
generation_deadline=$((SECONDS + 60))
|
||||
while [[ ! -s "$artifact_dir/generation-0.uce.meta.txt" ]]; do
|
||||
if ! kill -0 "$generation_pid" 2>/dev/null; then
|
||||
echo "parallel precompile exited before the controlled source mutation" >&2
|
||||
wait "$generation_pid" || true
|
||||
cat "$generation_log" >&2
|
||||
exit 1
|
||||
fi
|
||||
if (( SECONDS >= generation_deadline )); then
|
||||
echo "timed out waiting for the controlled precompile mutation boundary" >&2
|
||||
exit 1
|
||||
fi
|
||||
sleep 0.05
|
||||
done
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-generation-mutated"); }\n' >"$source_dir/generation-0.uce"
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-generation-added"); }\n' >"$source_dir/generation-new.uce"
|
||||
if ! kill -0 "$generation_pid" 2>/dev/null; then
|
||||
echo "parallel precompile exited at the controlled source mutation boundary" >&2
|
||||
exit 1
|
||||
fi
|
||||
set +e
|
||||
wait "$generation_pid"
|
||||
generation_rc=$?
|
||||
set -e
|
||||
generation_pid=""
|
||||
if [[ $generation_rc -eq 0 ]] || ! grep -q 'source generation changed during precompile; candidate generation rejected' "$generation_log"; then
|
||||
echo "parallel precompile accepted a source generation that changed during compilation" >&2
|
||||
cat "$generation_log" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ -e "$artifact_dir/generation-new.uce.wasm" || -e "$artifact_dir/generation-new.uce.cwasm" ]]; then
|
||||
echo "parallel precompile published an added unit outside the rejected candidate source set" >&2
|
||||
exit 1
|
||||
fi
|
||||
generation_retry_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared")
|
||||
if ! grep -Eq 'with 2 jobs: .* 2 compiled, 0 failed, worker status ok' <<<"$generation_retry_output"; then
|
||||
echo "parallel precompile did not reconcile the changed and added units on retry" >&2
|
||||
echo "$generation_retry_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f "$generation_log"
|
||||
|
||||
printf 'CLI(Request& context) { print(deliberate_parallel_precompile_failure); }\n' >"$source_dir/broken.uce"
|
||||
set +e
|
||||
failure_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" 2>&1)
|
||||
failure_rc=$?
|
||||
set -e
|
||||
if [[ $failure_rc -eq 0 ]] || ! grep -Eq 'with 2 jobs: .* 1 failed, worker status failed' <<<"$failure_output"; then
|
||||
echo "parallel precompile did not aggregate a controlled worker failure" >&2
|
||||
echo "$failure_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
set +e
|
||||
repeat_failure_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" 2>&1)
|
||||
repeat_failure_rc=$?
|
||||
set -e
|
||||
if [[ $repeat_failure_rc -eq 0 ]] || ! grep -Eq 'with 2 jobs: .* 1 failed, worker status failed' <<<"$repeat_failure_output"; then
|
||||
echo "parallel precompile accepted a persisted current compile failure" >&2
|
||||
echo "$repeat_failure_output" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm "$source_dir/broken.uce"
|
||||
|
||||
rm -f "$source_dir"/unit-*.uce
|
||||
for unit in $(seq -w 0 4); do
|
||||
printf 'CLI(Request& context) { print("parallel-precompile-bounds-%s"); }\n' "$unit" >"$source_dir/bounds-$unit.uce"
|
||||
done
|
||||
negative_jobs_output=$(precompile UCE_PRECOMPILE_JOBS=-1)
|
||||
malformed_jobs_output=$(precompile UCE_PRECOMPILE_JOBS=invalid)
|
||||
oversized_jobs_output=$(precompile UCE_PRECOMPILE_JOBS=17)
|
||||
if ! grep -q 'with 1 job: 16 units,' <<<"$negative_jobs_output" || ! grep -q 'with 2 jobs: 16 units,' <<<"$malformed_jobs_output" || ! grep -q 'with 16 jobs: 16 units,' <<<"$oversized_jobs_output"; then
|
||||
echo "precompile job bounds did not map negative/malformed/oversized input to 1/2/16" >&2
|
||||
exit 1
|
||||
fi
|
||||
rm -f "$source_dir"/bounds-*.uce "$source_dir"/generation-*.uce
|
||||
|
||||
printf 'parallel precompile passed: serial %.3fs, parallel %.3fs\n' \
|
||||
"$(awk -v ns="$serial_ns" 'BEGIN { print ns / 1000000000 }')" \
|
||||
"$(awk -v ns="$parallel_ns" 'BEGIN { print ns / 1000000000 }')"
|
||||
Executable
+218
@@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" != "--inside" ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 150s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
|
||||
name="parallel-proactive-test-$$"
|
||||
root="/tmp/$name"
|
||||
site="$root/site"
|
||||
work="$root/work"
|
||||
settings="$root/settings.cfg"
|
||||
log="$root/service.log"
|
||||
shim_log="$root/compile.tsv"
|
||||
server_pid=""
|
||||
|
||||
cleanup() {
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
deadline=$((SECONDS + 10))
|
||||
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
if kill -0 "$server_pid" 2>/dev/null; then kill -KILL "$server_pid" 2>/dev/null || true; fi
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
rm -rf "$root"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
|
||||
cp /etc/uce/settings.cfg "$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$root/run/cli.sock
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||
WASM_COMPILE_SCRIPT=$root/compile
|
||||
WORKER_COUNT=1
|
||||
PROACTIVE_COMPILE_ENABLED=1
|
||||
PROACTIVE_COMPILE_JOBS=2
|
||||
PROACTIVE_COMPILE_CHECK_INTERVAL=1
|
||||
COMPILE_FAILURE_RETRY_SECONDS=2
|
||||
SERVE_LAST_KNOWN_GOOD=0
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
|
||||
cat >"$root/compile" <<SHIM
|
||||
#!/usr/bin/env bash
|
||||
set -uo pipefail
|
||||
root='$root'
|
||||
real='$(pwd)/scripts/compile_wasm_unit'
|
||||
source_file="\$3"
|
||||
exec 9>>"\$root/counter.lock"
|
||||
flock 9
|
||||
active=0
|
||||
if [[ -r "\$root/active" ]]; then read -r active <"\$root/active"; fi
|
||||
active=\$((active + 1))
|
||||
printf '%s\n' "\$active" >"\$root/active"
|
||||
maximum=0
|
||||
if [[ -r "\$root/maximum" ]]; then read -r maximum <"\$root/maximum"; fi
|
||||
if (( active > maximum )); then printf '%s\n' "\$active" >"\$root/maximum"; fi
|
||||
nice_value=\$(timeout 2s ps -o ni= -p "\$PPID" | tr -d ' ')
|
||||
owner_pid=\$PPID
|
||||
for _ in 1 2 3 4; do
|
||||
owner_command=\$(timeout 2s ps -o comm= -p "\$owner_pid" | tr -d ' ')
|
||||
[[ "\$owner_command" == uce_fastcgi.* ]] && break
|
||||
owner_pid=\$(timeout 2s ps -o ppid= -p "\$owner_pid" | tr -d ' ')
|
||||
[[ -n "\$owner_pid" && "\$owner_pid" != "1" ]] || break
|
||||
done
|
||||
printf '%s\t%s\t%s\t%s\n' "\$(date +%s%N)" "\$nice_value" "\$source_file" "\$owner_pid" >>"\$root/compile.tsv"
|
||||
flock -u 9
|
||||
if [[ -r "\$root/hold-paths" ]] && grep -Fxq "\$source_file" "\$root/hold-paths"; then
|
||||
: >"\$root/held-\$owner_pid"
|
||||
deadline=\$((SECONDS + 20))
|
||||
while [[ ! -e "\$root/release" && \$SECONDS -lt \$deadline ]]; do sleep 0.05; done
|
||||
[[ -e "\$root/release" ]] || exit 98
|
||||
fi
|
||||
sleep 0.75
|
||||
"\$real" "\$@"
|
||||
rc=\$?
|
||||
flock 9
|
||||
active=1
|
||||
if [[ -r "\$root/active" ]]; then read -r active <"\$root/active"; fi
|
||||
printf '%s\n' "\$((active - 1))" >"\$root/active"
|
||||
flock -u 9
|
||||
exit "\$rc"
|
||||
SHIM
|
||||
chmod +x "$root/compile"
|
||||
|
||||
printf '%s\n' 'String common_marker() { return("common-a"); }' >"$site/common.uce"
|
||||
for unit in 0 1 2 3; do
|
||||
printf '%s\n' '#load "common.uce"' "CLI(Request& context) { print(common_marker(), \"-$unit\"); }" >"$site/unit-$unit.uce"
|
||||
done
|
||||
printf '%s\n' 'CLI(Request& context) { print("removed"); }' >"$site/removed.uce"
|
||||
printf '%s\n' 'CLI(Request& context) { print("baseline"); }' >"$site/broken.uce"
|
||||
|
||||
timeout --signal=TERM --kill-after=5s 120s bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
deadline=$((SECONDS + 20))
|
||||
while [[ ! -S "$root/run/cli.sock" ]] && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ -S "$root/run/cli.sock" ]] || { echo "private UCE CLI socket was not ready" >&2; cat "$log" >&2; exit 1; }
|
||||
|
||||
generation="$(scripts/unit_cache_directory "$work")"
|
||||
artifacts="$generation$(realpath "$site")"
|
||||
deadline=$((SECONDS + 60))
|
||||
while (( SECONDS < deadline )); do
|
||||
ready=1
|
||||
for unit in 0 1 2 3; do
|
||||
[[ -s "$artifacts/unit-$unit.uce.wasm" && -s "$artifacts/unit-$unit.uce.cwasm" ]] || ready=0
|
||||
done
|
||||
(( ready == 1 )) && break
|
||||
sleep 0.1
|
||||
done
|
||||
(( ready == 1 )) || { echo "parallel scanners did not publish all valid units" >&2; cat "$log" >&2; exit 1; }
|
||||
[[ "$(<"$root/maximum")" -ge 2 ]] || { echo "parallel scanners never overlapped" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
grep -q 'proactive compiler worker 1/2 ready' "$log"
|
||||
grep -q 'proactive compiler worker 2/2 ready' "$log"
|
||||
for unit in 0 1 2 3; do
|
||||
[[ "$(grep -c "$site/unit-$unit.uce" "$shim_log")" -eq 1 ]] || { echo "unit-$unit was not compiled exactly once" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
done
|
||||
deadline=$((SECONDS + 15))
|
||||
while { [[ ! -s "$artifacts/broken.uce.cwasm" || ! -s "$artifacts/removed.uce.cwasm" ]] || [[ "$(<"$root/active")" != "0" ]]; } && (( SECONDS < deadline )); do sleep 0.1; done
|
||||
[[ -s "$artifacts/broken.uce.cwasm" && -s "$artifacts/removed.uce.cwasm" && "$(<"$root/active")" == "0" ]] || { echo "phase fixtures did not become idle" >&2; cat "$log" >&2; exit 1; }
|
||||
exec 7>>"$root/counter.lock"
|
||||
flock 7
|
||||
printf '0\n' >"$root/maximum"
|
||||
flock -u 7
|
||||
mapfile -t scanner_pids < <(awk -F '\t' -v site="$site/" '$3 ~ ("^" site "unit-[0-9]+[.]uce$") { count[$4]++ } END { for(pid in count) if(count[pid] >= 2) print pid }' "$shim_log" | sort -n)
|
||||
[[ "${#scanner_pids[@]}" -eq 2 ]] || { echo "controlled units were not split between both scanners" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
assert_scanner_memory_bounded() {
|
||||
for scanner_pid in "${scanner_pids[@]}"; do
|
||||
retained_kb=$(awk '/^(VmRSS|VmSwap):/{total += $2} END{print total + 0}' "/proc/$scanner_pid/status")
|
||||
# Scanner bookkeeping is small; 64 MiB leaves ample headroom while still
|
||||
# catching the measured old 12-serialization retention (~73 MiB).
|
||||
(( retained_kb < 65536 )) || { echo "proactive scanner retained serialized-module arenas: pid=$scanner_pid retained_kb=$retained_kb" >&2; exit 1; }
|
||||
done
|
||||
}
|
||||
assert_scanner_memory_bounded
|
||||
blockers=()
|
||||
victims=()
|
||||
for scanner_pid in "${scanner_pids[@]}"; do
|
||||
mapfile -t scanner_units < <(awk -F '\t' -v pid="$scanner_pid" -v site="$site/" '$4 == pid && $3 ~ ("^" site "unit-[0-9]+[.]uce$") { print $3 }' "$shim_log")
|
||||
blockers+=("${scanner_units[0]}")
|
||||
victims+=("${scanner_units[1]}")
|
||||
done
|
||||
printf '%s\n' "${blockers[@]}" >"$root/hold-paths"
|
||||
rm -f "$root"/held-* "$root/release"
|
||||
sed -i 's/common-a/common-b/' "$site/common.uce"
|
||||
deadline=$((SECONDS + 20))
|
||||
while { [[ ! -e "$root/held-${scanner_pids[0]}" ]] || [[ ! -e "$root/held-${scanner_pids[1]}" ]]; } && (( SECONDS < deadline )); do sleep 0.05; done
|
||||
[[ -e "$root/held-${scanner_pids[0]}" && -e "$root/held-${scanner_pids[1]}" ]] || { echo "both scanners did not reach their controlled queue barriers" >&2; cat "$log" >&2; exit 1; }
|
||||
for victim in "${victims[@]}"; do
|
||||
victim_marker=$(UCE_CLI_SOCKET="$root/run/cli.sock" timeout 20s scripts/uce-cli "/$(basename "$victim")")
|
||||
[[ "$victim_marker" == *"common-b-"* ]] || { echo "request worker did not publish queued victim: $victim_marker" >&2; exit 1; }
|
||||
done
|
||||
touch "$root/release"
|
||||
deadline=$((SECONDS + 30))
|
||||
while (( SECONDS < deadline )); do
|
||||
rebuilt=1
|
||||
for unit in 0 1 2 3; do
|
||||
[[ "$(grep -c "$site/unit-$unit.uce" "$shim_log" || true)" -ge 2 ]] || rebuilt=0
|
||||
done
|
||||
(( rebuilt == 1 )) && break
|
||||
sleep 0.1
|
||||
done
|
||||
(( rebuilt == 1 )) || { echo "common dependency edit did not rebuild all scanner shards" >&2; cat "$log" >&2; exit 1; }
|
||||
deadline=$((SECONDS + 5))
|
||||
while [[ "$(<"$root/active")" != "0" ]] && (( SECONDS < deadline )); do sleep 0.1; done
|
||||
sleep 1.25
|
||||
for unit in 0 1 2 3; do
|
||||
[[ "$(grep -c "$site/unit-$unit.uce" "$shim_log")" -eq 2 ]] || { echo "unit-$unit common-dependency rebuild was not exact" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
done
|
||||
rm -f "$root/hold-paths" "$root"/held-* "$root/release"
|
||||
[[ "$(<"$root/maximum")" -ge 2 ]] || { echo "common dependency fanout did not overlap across scanner owners" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
marker=$(UCE_CLI_SOCKET="$root/run/cli.sock" timeout 20s scripts/uce-cli /unit-0.uce)
|
||||
[[ "$marker" == *"common-b-0"* ]] || { echo "rebuilt parent did not execute the changed dependency: $marker" >&2; exit 1; }
|
||||
rm "$site/removed.uce"
|
||||
registry="$generation/known-uce-files.txt"
|
||||
deadline=$((SECONDS + 5))
|
||||
while grep -q "$site/removed.uce" "$registry" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.1; done
|
||||
if grep -q "$site/removed.uce" "$registry" 2>/dev/null; then echo "removed unit remained tracked" >&2; cat "$log" >&2; exit 1; fi
|
||||
|
||||
printf '%s\n' 'CLI(Request& context) { print(deliberate_parallel_scanner_failure); }' >"$site/broken.uce"
|
||||
deadline=$((SECONDS + 30))
|
||||
while [[ "$(grep -c "$site/broken.uce" "$shim_log" || true)" -lt 3 ]] && (( SECONDS < deadline )); do sleep 0.1; done
|
||||
[[ "$(grep -c "$site/broken.uce" "$shim_log" || true)" -ge 3 ]] || { echo "persisted scanner failure was not retried" >&2; cat "$log" >&2; exit 1; }
|
||||
mapfile -t broken_times < <(awk -F '\t' -v path="$site/broken.uce" '$3 == path { print $1 }' "$shim_log")
|
||||
(( broken_times[2] - broken_times[1] >= 2000000000 )) || { echo "scanner failure retry ignored its backoff" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
[[ ! -e "$artifacts/broken.uce.wasm" ]] || { echo "failed scanner build published wasm" >&2; exit 1; }
|
||||
printf '%s\n' 'CLI(Request& context) { print("recovered"); }' >"$site/broken.uce"
|
||||
deadline=$((SECONDS + 8))
|
||||
while [[ ! -s "$artifacts/broken.uce.cwasm" ]] && (( SECONDS < deadline )); do sleep 0.1; done
|
||||
[[ -s "$artifacts/broken.uce.cwasm" ]] || { echo "changed failed source waited for the unchanged-failure backoff" >&2; cat "$log" >&2; exit 1; }
|
||||
|
||||
printf '%s\n' 'CLI(Request& context) { print("priority"); }' >"$site/priority.uce"
|
||||
priority_file="$generation/proactive-priority.txt"
|
||||
exec 8>"$priority_file.lock"
|
||||
flock 8
|
||||
printf '%s\n' "$(realpath "$site/priority.uce")" >>"$priority_file"
|
||||
flock -u 8
|
||||
deadline=$((SECONDS + 15))
|
||||
while [[ ! -s "$artifacts/priority.uce.cwasm" ]] && (( SECONDS < deadline )); do sleep 0.1; done
|
||||
[[ -s "$artifacts/priority.uce.cwasm" ]] || { echo "priority compiler did not publish requested unit" >&2; cat "$log" >&2; exit 1; }
|
||||
priority_nice=$(awk -F '\t' -v path="$site/priority.uce" '$3 == path { print $2; exit }' "$shim_log")
|
||||
[[ "$priority_nice" == "5" ]] || { echo "priority queue was not owned by the nice-5 compiler: $priority_nice" >&2; cat "$shim_log" >&2; exit 1; }
|
||||
|
||||
# Recheck after dependency fanout, failure/retry, recovery, and priority work;
|
||||
# the historical leak accumulated in the long-lived scanners across phases.
|
||||
assert_scanner_memory_bounded
|
||||
|
||||
printf '%s\n' 'parallel proactive compile passed'
|
||||
@@ -26,7 +26,7 @@ cleanup() {
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$bin_directory$(realpath "$source_dir")"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'CLI(Request& context) {' \
|
||||
|
||||
Executable
+59
@@ -0,0 +1,59 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
http_port="${UCE_RAW_HTTP_TEST_PORT:-}"
|
||||
if [[ -z "$http_port" && -r /etc/uce/settings.cfg ]]; then
|
||||
http_port=$(awk -F= '/^[[:space:]]*HTTP_PORT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
fi
|
||||
http_port="${http_port:-8080}"
|
||||
request_path="${UCE_RAW_HTTP_TEST_PATH:-/info/index.uce}"
|
||||
marker="raw-http-log-$$-$(date +%s%N)"
|
||||
separator="?"
|
||||
if [[ "$request_path" == *\?* ]]; then
|
||||
separator="&"
|
||||
fi
|
||||
started_at=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
|
||||
response=$(curl -fsS --max-time 15 "http://127.0.0.1:${http_port}${request_path}${separator}__uce_log_probe=${marker}")
|
||||
if [[ -z "$response" ]]; then
|
||||
echo "raw HTTP probe returned an empty response" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
request_logs=""
|
||||
for _ in $(seq 1 30); do
|
||||
request_logs=$(journalctl -u uce --since "$started_at" --no-pager | grep '(r)' | grep "$marker" || true)
|
||||
if [[ $(printf '%s\n' "$request_logs" | grep -c '(r)' || true) -ge 2 ]]; then
|
||||
break
|
||||
fi
|
||||
sleep 0.1
|
||||
done
|
||||
|
||||
if [[ $(printf '%s\n' "$request_logs" | grep -c '(r)' || true) -ne 2 ]]; then
|
||||
echo "expected exactly two raw HTTP request-stage records" >&2
|
||||
printf '%s\n' "$request_logs" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ $(printf '%s\n' "$request_logs" | grep -c 'transport:http' || true) -ne 1 ||
|
||||
$(printf '%s\n' "$request_logs" | grep -c 'transport:fastcgi' || true) -ne 1 ]]; then
|
||||
echo "raw HTTP request-stage records did not identify HTTP and FastCGI transports" >&2
|
||||
printf '%s\n' "$request_logs" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
mapfile -t durations < <(printf '%s\n' "$request_logs" | sed -n 's/.*[[:space:]]\([0-9][0-9.]*\)s[[:space:]]*fps:.*/\1/p')
|
||||
if [[ ${#durations[@]} -ne 2 ]]; then
|
||||
echo "could not parse both raw HTTP request-stage durations" >&2
|
||||
printf '%s\n' "$request_logs" >&2
|
||||
exit 1
|
||||
fi
|
||||
for duration in "${durations[@]}"; do
|
||||
awk -v duration="$duration" 'BEGIN { exit !(duration > 0 && duration < 60) }' || {
|
||||
echo "raw HTTP request-stage duration is outside (0, 60) seconds: ${duration}s" >&2
|
||||
printf '%s\n' "$request_logs" >&2
|
||||
exit 1
|
||||
}
|
||||
done
|
||||
|
||||
echo "raw HTTP request log passed"
|
||||
Executable
+48
@@ -0,0 +1,48 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="relative-component-cache-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
if [[ -n "${configured_site_directory:-}" ]]; then
|
||||
site_directory="$configured_site_directory"
|
||||
fi
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
bin_directory="${BIN_DIRECTORY:-}"
|
||||
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
|
||||
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
fi
|
||||
bin_directory="${bin_directory:-/tmp/uce/work}"
|
||||
cache_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$cache_dir" ]]; then
|
||||
rm -rf "$cache_dir"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir/a" "$source_dir/b"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'CLI(Request& context) {' \
|
||||
' print(component("a/parent", context));' \
|
||||
' print("/");' \
|
||||
' print(component("b/parent", context));' \
|
||||
'}' >"$source_dir/entry.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print(component("child", context)); }' >"$source_dir/a/parent.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("relative-a"); }' >"$source_dir/a/child.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print(component("child", context)); }' >"$source_dir/b/parent.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("relative-b"); }' >"$source_dir/b/child.uce"
|
||||
|
||||
output=$(scripts/uce-cli "/$test_name/entry.uce")
|
||||
if [[ "$output" != "relative-a/relative-b" ]]; then
|
||||
echo "relative component cache crossed caller boundaries: $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "relative component cache passed"
|
||||
Executable
+101
@@ -0,0 +1,101 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
binary="$PWD/bin/uce_fastcgi.linux.bin"
|
||||
root=$(mktemp -d /tmp/uce-server-arguments.XXXXXX)
|
||||
listener_pid=""
|
||||
cleanup() {
|
||||
[[ -n "$listener_pid" ]] && kill "$listener_pid" 2>/dev/null || true
|
||||
rm -rf "$root"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
cfg="$root/settings.cfg"
|
||||
socket_path="$root/sentinel.sock"
|
||||
cp /etc/uce/settings.cfg "$cfg"
|
||||
sed -E -i \
|
||||
-e "s|^[[:space:]]*FCGI_SOCKET_PATH[[:space:]]*=.*|FCGI_SOCKET_PATH=|" \
|
||||
-e "s|^[[:space:]]*FCGI_PORT[[:space:]]*=.*|FCGI_PORT=|" \
|
||||
-e "s|^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=.*|CLI_SOCKET_PATH=$socket_path|" \
|
||||
-e "s|^[[:space:]]*HTTP_PORT[[:space:]]*=.*|HTTP_PORT=|" \
|
||||
-e "s|^[[:space:]]*PROACTIVE_COMPILE_ENABLED[[:space:]]*=.*|PROACTIVE_COMPILE_ENABLED=false|" \
|
||||
-e "s|^[[:space:]]*WORKER_COUNT[[:space:]]*=.*|WORKER_COUNT=1|" \
|
||||
-e "s|^[[:space:]]*BIN_DIRECTORY[[:space:]]*=.*|BIN_DIRECTORY=$root/bin|" \
|
||||
-e "s|^[[:space:]]*TMP_UPLOAD_PATH[[:space:]]*=.*|TMP_UPLOAD_PATH=$root/upload|" \
|
||||
-e "s|^[[:space:]]*SESSION_PATH[[:space:]]*=.*|SESSION_PATH=$root/session|" \
|
||||
"$cfg"
|
||||
|
||||
python3 -c 'import socket,sys,time; s=socket.socket(socket.AF_UNIX); s.bind(sys.argv[1]); s.listen(); time.sleep(30)' "$socket_path" &
|
||||
listener_pid=$!
|
||||
for _ in $(seq 1 50); do
|
||||
[[ -S "$socket_path" ]] && break
|
||||
sleep 0.02
|
||||
done
|
||||
[[ -S "$socket_path" ]]
|
||||
socket_inode=$(stat -c %i "$socket_path")
|
||||
|
||||
invoke() {
|
||||
local output="$1"
|
||||
shift
|
||||
local timeout_seconds="${INVOKE_TIMEOUT_SECONDS:-2}"
|
||||
timeout --signal=TERM --kill-after=1s "$timeout_seconds" unshare --mount --fork \
|
||||
bash -c 'mount --bind "$1" /etc/uce/settings.cfg; exec "$2" "${@:3}"' \
|
||||
_ "$cfg" "$binary" "$@" >"$output.stdout" 2>"$output.stderr"
|
||||
}
|
||||
|
||||
for option in --help -h; do
|
||||
invoke "$root/help" "$option"
|
||||
grep -q '^Usage: uce_fastcgi' "$root/help.stdout"
|
||||
grep -q -- '--precompile' "$root/help.stdout"
|
||||
grep -q -- '--serialize-module' "$root/help.stdout"
|
||||
[[ ! -s "$root/help.stderr" ]]
|
||||
done
|
||||
|
||||
for arguments in '--unknown' '--precompile extra' '--serialize-module' '--help extra'; do
|
||||
read -r -a argv <<<"$arguments"
|
||||
set +e
|
||||
invoke "$root/invalid" "${argv[@]}"
|
||||
rc=$?
|
||||
set -e
|
||||
[[ $rc -eq 2 ]]
|
||||
grep -q '^invalid arguments$' "$root/invalid.stderr"
|
||||
grep -q '^Usage: uce_fastcgi' "$root/invalid.stderr"
|
||||
[[ ! -s "$root/invalid.stdout" ]]
|
||||
done
|
||||
|
||||
cp bin/wasm/core.wasm "$root/serialize.wasm"
|
||||
INVOKE_TIMEOUT_SECONDS=30 invoke "$root/serialize" --serialize-module "$root/serialize.wasm"
|
||||
[[ -s "$root/serialize.cwasm" ]]
|
||||
[[ ! -s "$root/serialize.stderr" ]]
|
||||
|
||||
# Serialization shares the unit compile lock. If the wasm path is replaced
|
||||
# while the serializer is waiting, it must read the replacement and must not
|
||||
# publish a stale native artifact for the old inode.
|
||||
cp bin/wasm/core.wasm "$root/race.wasm"
|
||||
exec 8>"$root/race.wasm.lock"
|
||||
flock 8
|
||||
INVOKE_TIMEOUT_SECONDS=30 invoke "$root/race" --serialize-module "$root/race.wasm" &
|
||||
race_pid=$!
|
||||
sleep 0.1
|
||||
kill -0 "$race_pid"
|
||||
printf 'not wasm\n' >"$root/race.wasm.next"
|
||||
mv "$root/race.wasm.next" "$root/race.wasm"
|
||||
flock -u 8
|
||||
set +e
|
||||
wait "$race_pid"
|
||||
race_rc=$?
|
||||
set -e
|
||||
[[ $race_rc -eq 1 ]]
|
||||
grep -Eqi 'wasm|magic|WebAssembly|compile' "$root/race.stderr"
|
||||
[[ ! -e "$root/race.cwasm" ]]
|
||||
if compgen -G "$root/race.cwasm.*.tmp" >/dev/null; then
|
||||
echo "failed serialization left a temporary artifact" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
[[ -S "$socket_path" ]]
|
||||
[[ "$(stat -c %i "$socket_path")" == "$socket_inode" ]]
|
||||
kill -0 "$listener_pid"
|
||||
|
||||
echo 'server argument handling passed: help and invalid arguments exited before listener setup'
|
||||
@@ -24,6 +24,9 @@ inode_before=$(stat -c %i "$socket_path")
|
||||
load_pid=$!
|
||||
sleep 0.05
|
||||
systemctl restart "$service_name"
|
||||
systemctl is-active --quiet "$socket_name"
|
||||
inode_during=$(stat -c %i "$socket_path")
|
||||
[[ "$inode_during" = "$inode_before" ]]
|
||||
wait "$load_pid"
|
||||
|
||||
inode_after=$(stat -c %i "$socket_path")
|
||||
|
||||
Executable
+92
@@ -0,0 +1,92 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="unit-export-surface-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
bin_directory="${UCE_TEST_BIN_DIRECTORY:-/tmp/uce/work}"
|
||||
if [[ -r /etc/uce/settings.cfg ]]; then
|
||||
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*HTTP_DOCUMENT_ROOT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
site_directory="${configured_site_directory:-$site_directory}"
|
||||
fi
|
||||
if [[ -z "${UCE_TEST_BIN_DIRECTORY:-}" ]]; then
|
||||
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
bin_directory="${configured_bin_directory:-$bin_directory}"
|
||||
fi
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
artifact_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$artifact_dir" ]]; then
|
||||
rm -rf "$artifact_dir"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
absolute_source_dir=$(realpath "$source_dir")
|
||||
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$absolute_source_dir"
|
||||
|
||||
printf '%s\n' \
|
||||
'String visibility_used() { return("private-used"); }' \
|
||||
'String visibility_unused() { return("uce-private-unused-marker-8f61d2"); }' \
|
||||
'EXPORT String visibility_shared() { return("shared"); }' \
|
||||
'CLI(Request& context) { print(visibility_shared(), ":", visibility_used(), ":", component("named:NAMED", context)); }' \
|
||||
>"$source_dir/entry.uce"
|
||||
printf '%s\n' \
|
||||
'String named_used() { return("private-named-used"); }' \
|
||||
'String named_unused() { return("uce-named-unused-marker-4ae973"); }' \
|
||||
'EXPORT String named_shared() { return("named-export"); }' \
|
||||
'COMPONENT:NAMED(Request& context) { print(named_shared(), ":", named_used()); }' \
|
||||
>"$source_dir/named.uce"
|
||||
|
||||
output=$(scripts/uce-cli "/$test_name/entry.uce")
|
||||
if [[ "$output" != "shared:private-used:named-export:private-named-used" ]]; then
|
||||
echo "unit export surface runtime failed: $output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
entry_wasm="$artifact_dir/entry.uce.wasm"
|
||||
named_wasm="$artifact_dir/named.uce.wasm"
|
||||
python3 - "$entry_wasm" "$named_wasm" "$absolute_source_dir" <<'PY'
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from scripts.check_unit_wasm import collect
|
||||
|
||||
cases = [
|
||||
(Path(sys.argv[1]), {"__wasm_call_ctors", "__uce_set_current_request", "__uce_cli", "visibility_shared"}, b"uce-private-unused-marker-8f61d2"),
|
||||
(Path(sys.argv[2]), {"__wasm_call_ctors", "__uce_set_current_request", "__uce_component_NAMED", "named_shared"}, b"uce-named-unused-marker-4ae973"),
|
||||
]
|
||||
source_dir = Path(sys.argv[3])
|
||||
for path, allowed, unused_marker in cases:
|
||||
data = path.read_bytes()
|
||||
customs, imports, exports = collect(path)
|
||||
names = {name for name, _ in exports}
|
||||
unexpected = names - allowed
|
||||
missing = allowed - names
|
||||
if unexpected or missing:
|
||||
raise SystemExit(f"{path}: unexpected exports={sorted(unexpected)} missing={sorted(missing)} all={sorted(names)}")
|
||||
if unused_marker in data:
|
||||
raise SystemExit(f"{path}: retained unused private code marker")
|
||||
if len(imports) >= 40:
|
||||
raise SystemExit(f"{path}: retained {len(imports)} imports (expected fewer than 40)")
|
||||
if path.stat().st_size >= 1024 * 1024:
|
||||
raise SystemExit(f"{path}: artifact is {path.stat().st_size} bytes (expected under 1 MiB)")
|
||||
source_map = Path(str(path) + ".source-map")
|
||||
if not source_map.is_file() or source_map.stat().st_size >= 256 * 1024:
|
||||
raise SystemExit(f"{path}: missing or oversized source map")
|
||||
module = customs["uce.module"][-1].decode()
|
||||
lines = source_map.read_text().splitlines()
|
||||
if not lines or lines[0] != f"UCE_SOURCE_MAP_V1\t{module}":
|
||||
raise SystemExit(f"{path}: source map does not match wasm module identity")
|
||||
expected_source = str(source_dir / path.name.removesuffix(".wasm"))
|
||||
if not any(line.startswith("F\t") and line.endswith("\t" + expected_source) for line in lines):
|
||||
raise SystemExit(f"{path}: source map does not identify {expected_source}")
|
||||
if not any(line.startswith("L\t") for line in lines):
|
||||
raise SystemExit(f"{path}: source map has no address rows")
|
||||
PY
|
||||
|
||||
echo "unit export surface passed"
|
||||
Executable
+252
@@ -0,0 +1,252 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
error_page_path=$(sed -n '/^bool render_wasm_error_page(/,/^}/p' src/linux_fastcgi.cpp)
|
||||
[[ "$error_page_path" == *'get_shared_unit_bounded('* && "$error_page_path" == *'wasm_invocation_remaining_ms(invocation_deadline)'* ]]
|
||||
[[ "$error_page_path" != *'get_shared_unit(&request'* ]]
|
||||
|
||||
if [[ "${1:-}" != --inside ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
|
||||
root="/tmp/uce-compile-timeout-$$"
|
||||
site="$root/site"
|
||||
work="$root/work"
|
||||
settings="$root/settings.cfg"
|
||||
log="$root/service.log"
|
||||
server_pid=""
|
||||
|
||||
cleanup() {
|
||||
status=$?
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
for _ in $(seq 1 100); do kill -0 "$server_pid" 2>/dev/null || break; sleep 0.02; done
|
||||
kill -KILL "$server_pid" 2>/dev/null || true
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
|
||||
rm -rf "$root"
|
||||
return "$status"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$site/components" "$work" "$root/run" "$root/session" "$root/upload"
|
||||
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|CLI_WORKER_COUNT|CLI_WORKER_MAX_REQUESTS|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|WASM_COMPILE_SCRIPT|WASM_INVOCATION_TIMEOUT_MS|WASM_EPOCH_PERIOD_MS|PROACTIVE_COMPILE_ENABLED|WORKER_COUNT|COMPILE_FAILURE_RETRY_SECONDS)[[:space:]]*=/d' \
|
||||
/etc/uce/settings.cfg >"$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$root/run/cli.sock
|
||||
CLI_WORKER_COUNT=1
|
||||
CLI_WORKER_MAX_REQUESTS=0
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd -P)/bin/wasm/core.wasm
|
||||
WASM_COMPILE_SCRIPT=$root/compile
|
||||
WASM_INVOCATION_TIMEOUT_MS=2000
|
||||
WASM_EPOCH_PERIOD_MS=20
|
||||
PROACTIVE_COMPILE_ENABLED=0
|
||||
WORKER_COUNT=1
|
||||
COMPILE_FAILURE_RETRY_SECONDS=60
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
|
||||
mkdir -p "$root/compiler/src/wasm" "$root/compiler/scripts"
|
||||
cp -a src/lib "$root/compiler/src/lib"
|
||||
cp -a src/wasm/abi.h "$root/compiler/src/wasm/abi.h"
|
||||
cp -a scripts/compile_wasm_unit scripts/build_unit_source_map.py scripts/check_unit_wasm.py "$root/compiler/scripts/"
|
||||
chmod -R a+rX "$root/compiler"
|
||||
|
||||
cat >"$root/compile" <<SHIM
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
source_file="\$3"
|
||||
if [[ -r '$root/stage-timeout' ]] && grep -Fxq "\$source_file" '$root/stage-timeout'; then
|
||||
printf '%s\n' "\$\$" >'$root/compiler-pid'
|
||||
(sleep 3; touch '$root/late-descendant') &
|
||||
cp "\$2/\$(basename "\$source_file").wasm" "\$2/\$5"
|
||||
cp "\$2/\$(basename "\$source_file").wasm.source-map" "\$2/\$5.source-map"
|
||||
sleep 60
|
||||
fi
|
||||
if [[ -r '$root/copy-delay' ]] && grep -Fxq "\$source_file" '$root/copy-delay'; then
|
||||
cp "\$2/\$(basename "\$source_file").wasm" "\$2/\$5"
|
||||
cp "\$2/\$(basename "\$source_file").wasm.source-map" "\$2/\$5.source-map"
|
||||
sleep 1.7
|
||||
touch '$root/copy-delay-complete'
|
||||
exit 0
|
||||
fi
|
||||
if [[ -r '$root/hold' ]] && grep -Fxq "\$source_file" '$root/hold'; then
|
||||
printf '%s\n' "\$\$" >'$root/compiler-pid'
|
||||
(sleep 3; touch '$root/late-descendant') &
|
||||
sleep 60
|
||||
fi
|
||||
if [[ -r '$root/silent-fail' ]] && grep -Fxq "\$source_file" '$root/silent-fail'; then exit 23; fi
|
||||
if [[ -r '$root/empty-success' ]] && grep -Fxq "\$source_file" '$root/empty-success'; then exit 0; fi
|
||||
if [[ -r '$root/delay' ]] && grep -Fxq "\$source_file" '$root/delay'; then sleep 1; fi
|
||||
exec '$root/compiler/scripts/compile_wasm_unit' "\$@"
|
||||
SHIM
|
||||
chmod +x "$root/compile"
|
||||
|
||||
cat >"$site/driver.uce" <<'UCE'
|
||||
CLI(Request& context) {
|
||||
if(context.get["health"] == "1") { print(request_perf()["worker_pid"].to_u64(), "|health"); return; }
|
||||
if(context.get["dynamic"] == "1") { print(component("slow-component.uce")); return; }
|
||||
String target = first(context.get["target"], "slow.uce");
|
||||
print(unit_compile(target) ? "compiled" : "failed");
|
||||
}
|
||||
UCE
|
||||
printf '%s\n' 'CLI(Request& context) { print("entry"); }' >"$site/cold-entry.uce"
|
||||
printf '%s\n' 'CLI(Request& context) { print("slow"); }' >"$site/slow.uce"
|
||||
printf '%s\n' 'CLI(Request& context) { print("silent"); }' >"$site/silent.uce"
|
||||
printf '%s\n' 'CLI(Request& context) { print("empty"); }' >"$site/empty.uce"
|
||||
printf '%s\n' '#load "transitive-child.uce"' 'CLI(Request& context) { print("parent"); }' >"$site/transitive.uce"
|
||||
printf '%s\n' 'String transitive_value() { return("a"); }' >"$site/transitive-child.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("component"); }' >"$site/components/slow-component.uce"
|
||||
|
||||
timeout 40s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >"$root/precompile.log" 2>&1 || { cat "$root/precompile.log" >&2; exit 1; }
|
||||
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
|
||||
generation=$(scripts/unit_cache_directory "$work")
|
||||
service_user=${UCE_TEST_SERVICE_USER:-www-data}
|
||||
service_uid=$(id -u "$service_user")
|
||||
service_gid=$(id -g "$service_user")
|
||||
generation_before=$(<"$generation/source-generation.txt")
|
||||
[[ $(sysctl -n fs.protected_hardlinks) == 1 ]]
|
||||
[[ $(stat -c %u "$cache/slow.uce.wasm") == 0 && $(stat -c %u "$generation/source-generation.txt") == 0 ]]
|
||||
[[ $(stat -c %u "$cache/slow.uce.wasm.lock") == 0 && $(stat -c %u "$generation/source-generation.txt.lock") == 0 ]]
|
||||
find "$work" -type d -exec chown "$service_uid:$service_gid" {} +
|
||||
chown -R "$service_uid:$service_gid" "$root/run" "$root/session" "$root/upload"
|
||||
chown "$service_uid:$service_gid" "$root"
|
||||
rm -f "$cache/cold-entry.uce."* "$cache/components/slow-component.uce."*
|
||||
|
||||
timeout --signal=TERM --kill-after=5s 90s setpriv --reuid="$service_uid" --regid="$service_gid" --clear-groups bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
for _ in $(seq 1 400); do [[ -S "$root/run/cli.sock" ]] && break; sleep 0.02; done
|
||||
[[ -S "$root/run/cli.sock" ]] || { cat "$log" >&2; exit 1; }
|
||||
|
||||
request() {
|
||||
local spec="$1" path="${1%%\?*}"
|
||||
if [[ "$spec" == *\?* ]]; then
|
||||
local query="${spec#*\?}"
|
||||
IFS='&' read -r -a params <<<"$query"
|
||||
timeout 5s scripts/uce-cli --get "$path" "${params[@]}"
|
||||
else
|
||||
timeout 5s scripts/uce-cli "$path"
|
||||
fi
|
||||
}
|
||||
driver=""
|
||||
for _ in $(seq 1 100); do
|
||||
set +e
|
||||
driver=$(request '/driver.uce?health=1' 2>/dev/null)
|
||||
rc=$?
|
||||
set -e
|
||||
(( rc == 0 )) && [[ "$driver" == *'|health' ]] && break
|
||||
sleep 0.05
|
||||
done
|
||||
[[ "$driver" == *'|health' ]] || { echo "driver did not become ready: $driver" >&2; exit 1; }
|
||||
worker_pid=${driver%%|*}
|
||||
[[ "$driver" == "$worker_pid|health" ]]
|
||||
[[ "$(request /driver.uce)" == compiled ]]
|
||||
[[ $(stat -c %u "$cache/slow.uce.wasm") == "$service_uid" ]]
|
||||
[[ $(stat -c %u "$generation/source-generation.txt") == "$service_uid" ]]
|
||||
[[ $(<"$generation/source-generation.txt") != "$generation_before" ]]
|
||||
! grep -q 'Could not write.*source-generation.txt' "$log"
|
||||
before=$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")
|
||||
|
||||
truncate -s 68719476736 "$cache/slow.uce.cwasm"
|
||||
printf '%s\n' "$site/slow.uce" >"$root/copy-delay"
|
||||
rm -f "$root/copy-delay-complete"
|
||||
started=$(date +%s%N)
|
||||
set +e
|
||||
copy_timeout=$(request /driver.uce 2>&1)
|
||||
set -e
|
||||
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||
rm -f "$root/copy-delay"
|
||||
[[ "$copy_timeout" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "foreign rollback copy lacked timeout: $copy_timeout" >&2; exit 1; }
|
||||
(( elapsed >= 1700 && elapsed <= 3500 )) || { echo "foreign rollback copy took ${elapsed}ms" >&2; exit 1; }
|
||||
[[ -e "$root/copy-delay-complete" ]] || { echo "foreign rollback compiler did not finish before timeout" >&2; exit 1; }
|
||||
[[ $(stat -c '%u:%s' "$cache/slow.uce.cwasm") == '0:68719476736' ]]
|
||||
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
|
||||
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
|
||||
rm -f "$cache/slow.uce.cwasm"
|
||||
|
||||
assert_timeout() {
|
||||
local url="$1" source="$2" started elapsed output
|
||||
printf '%s\n' "$source" >"$root/hold"
|
||||
rm -f "$root/compiler-pid" "$root/late-descendant"
|
||||
started=$(date +%s%N)
|
||||
set +e
|
||||
output=$(request "$url" 2>&1)
|
||||
set -e
|
||||
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||
[[ "$output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "$url lacked timeout: $output" >&2; exit 1; }
|
||||
(( elapsed >= 1700 && elapsed <= 3500 )) || { echo "$url took ${elapsed}ms" >&2; exit 1; }
|
||||
[[ "$(request '/driver.uce?health=1')" == "$worker_pid|health" ]]
|
||||
[[ -s "$root/compiler-pid" ]]
|
||||
compiler_pid=$(<"$root/compiler-pid")
|
||||
! kill -0 "$compiler_pid" 2>/dev/null || { echo "compiler $compiler_pid survived" >&2; exit 1; }
|
||||
sleep 0.1
|
||||
[[ ! -e "$root/late-descendant" ]]
|
||||
rm -f "$root/hold"
|
||||
}
|
||||
|
||||
assert_timeout /driver.uce "$site/slow.uce"
|
||||
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
|
||||
[[ ! -e "$cache/slow.uce.compile.txt" && ! -e "$cache/slow.uce.wasm-check.txt" ]]
|
||||
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
|
||||
[[ "$(request /driver.uce)" == compiled ]]
|
||||
before=$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")
|
||||
|
||||
printf '%s\n' "$site/silent.uce" >"$root/silent-fail"
|
||||
[[ "$(request '/driver.uce?target=silent.uce')" == failed ]]
|
||||
grep -q 'status 23' "$cache/silent.uce.compile.txt"
|
||||
rm -f "$root/silent-fail"
|
||||
printf '%s\n' "$site/empty.uce" >"$root/empty-success"
|
||||
[[ "$(request '/driver.uce?target=empty.uce')" == failed ]]
|
||||
grep -q 'without creating' "$cache/empty.uce.compile.txt"
|
||||
rm -f "$root/empty-success"
|
||||
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
|
||||
|
||||
printf '%s\n' "$site/slow.uce" >"$root/stage-timeout"
|
||||
assert_timeout /driver.uce "$site/slow.uce"
|
||||
rm -f "$root/stage-timeout"
|
||||
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
|
||||
|
||||
transitive_before=$(sha256sum "$cache/transitive.uce.wasm" "$cache/transitive.uce.wasm.source-map" "$cache/transitive.uce.cpp" "$cache/transitive.uce.exports.txt" "$cache/transitive.uce.meta.txt")
|
||||
printf '%s\n' 'String transitive_value() { return("b"); }' >"$site/transitive-child.uce"
|
||||
assert_timeout '/driver.uce?target=transitive.uce' "$site/transitive-child.uce"
|
||||
[[ "$transitive_before" == "$(sha256sum "$cache/transitive.uce.wasm" "$cache/transitive.uce.wasm.source-map" "$cache/transitive.uce.cpp" "$cache/transitive.uce.exports.txt" "$cache/transitive.uce.meta.txt")" ]]
|
||||
|
||||
assert_timeout '/driver.uce?dynamic=1' "$site/components/slow-component.uce"
|
||||
assert_timeout /cold-entry.uce "$site/cold-entry.uce"
|
||||
|
||||
printf '%s\n' "$site/slow.uce" >"$root/delay"
|
||||
timeout 25s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >/dev/null
|
||||
rm -f "$root/delay"
|
||||
|
||||
exec 8>"$generation/known-uce-files.txt.lock"
|
||||
flock 8
|
||||
started=$(date +%s%N)
|
||||
set +e
|
||||
registry_locked=$(request /driver.uce 2>&1)
|
||||
set -e
|
||||
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||
flock -u 8
|
||||
[[ "$registry_locked" == *UCE_INVOCATION_TIMEOUT:* && "$elapsed" -le 3500 ]]
|
||||
|
||||
exec 9>"$cache/slow.uce.wasm.lock"
|
||||
flock 9
|
||||
started=$(date +%s%N)
|
||||
set +e
|
||||
locked=$(request /driver.uce 2>&1)
|
||||
set -e
|
||||
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||
flock -u 9
|
||||
[[ "$locked" == *UCE_INVOCATION_TIMEOUT:* && "$elapsed" -le 3500 ]]
|
||||
|
||||
echo "wasm compile timeout passed (entry, explicit, dynamic, unit/registry locks, error page, process group, staging, native precompile, foreign-owned artifacts)"
|
||||
Executable
+141
@@ -0,0 +1,141 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
raw_calls=$(grep -En -- '(->|\.)call\(' src/wasm/worker.cpp)
|
||||
if [[ $(wc -l <<<"$raw_calls") -ne 1 || "$raw_calls" != *'return(func.call(context, args));'* ]]; then
|
||||
echo "raw Wasmtime call bypasses call_guest(): $raw_calls" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "${1:-}" == "--static-only" ]]; then
|
||||
echo "wasm invocation timeout static call gate passed"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
settings_file="${UCE_SETTINGS_FILE:-/etc/uce/settings.cfg}"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
socket_path="${UCE_CLI_SOCKET:-/run/uce/cli.sock}"
|
||||
bin_directory="${BIN_DIRECTORY:-/tmp/uce/work}"
|
||||
if [[ -r "$settings_file" ]]; then
|
||||
[[ -n "${UCE_TEST_SITE_DIRECTORY:-}" ]] || site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
|
||||
[[ -n "${UCE_CLI_SOCKET:-}" ]] || socket_path=$(awk -F= '/^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
|
||||
[[ -n "${BIN_DIRECTORY:-}" ]] || bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
|
||||
invocation_ms=$(awk -F= '/^[[:space:]]*WASM_INVOCATION_TIMEOUT_MS[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
|
||||
cli_worker_count=$(awk -F= '/^[[:space:]]*CLI_WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); value=$2} END{print value}' "$settings_file")
|
||||
fi
|
||||
site_directory="${site_directory:-site}"
|
||||
socket_path="${socket_path:-/run/uce/cli.sock}"
|
||||
bin_directory="${bin_directory:-/tmp/uce/work}"
|
||||
invocation_ms="${invocation_ms:-30000}"
|
||||
cli_worker_count="${cli_worker_count:-0}"
|
||||
curl_transport=(--unix-socket "$socket_path")
|
||||
test_base_url="http://localhost"
|
||||
if (( cli_worker_count > 0 )); then
|
||||
test_http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
|
||||
curl_transport=(--resolve "$test_http_host:80:127.0.0.1")
|
||||
test_base_url="http://$test_http_host"
|
||||
fi
|
||||
test_name="invocation-timeout-test-$$"
|
||||
source_dir="$site_directory/$test_name"
|
||||
pid_file="/tmp/uce-$test_name-worker"
|
||||
cache_dir=""
|
||||
body_file="/tmp/uce-$test_name-body"
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
[[ -z "$cache_dir" ]] || rm -rf "$cache_dir"
|
||||
rm -f "$pid_file" "$body_file"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'void timeout_test_run(Request& context) {' \
|
||||
' if(context.get["warm"] == "1") { print("warm"); return; }' \
|
||||
" file_put_contents(\"$pid_file\", std::to_string(request_perf()[\"worker_pid\"].to_u64()));" \
|
||||
' while(true) time_precise();' \
|
||||
'}' \
|
||||
'CLI(Request& context) { timeout_test_run(context); }' \
|
||||
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/hostcall-loop.uce"
|
||||
printf '%s\n' \
|
||||
'void timeout_test_run(Request& context) {' \
|
||||
' if(context.get["warm"] == "1") { print("warm"); return; }' \
|
||||
' if(context.get["quick"] == "1") { print(shell_exec("printf quick")); return; }' \
|
||||
' if(context.get["status"] == "1") { DValue spec; spec["cmd"] = "exit 7"; spec["timeout_ms"] = (f64)500; print(shell_exec(spec)["exit_code"].to_u64()); return; }' \
|
||||
' if(context.get["zero"] == "1") { DValue spec; spec["cmd"] = "printf zero"; spec["timeout_ms"] = (f64)0; print(shell_exec(spec)["stdout"].to_string()); return; }' \
|
||||
' if(context.get["job"] == "1") { DValue spec; spec["cmd"] = "sleep 2"; spec["timeout_ms"] = (f64)5000; u64 job = shell_spawn(spec); f64 started = time_precise(); job_await(job, 300); u64 elapsed = (u64)((time_precise() - started) * 1000); job_cancel(job); print(elapsed); return; }' \
|
||||
" file_put_contents(\"$pid_file\", std::to_string(request_perf()[\"worker_pid\"].to_u64()));" \
|
||||
' print(shell_exec("printf shell-start; sleep 60 & printf shell-end"));' \
|
||||
'}' \
|
||||
'CLI(Request& context) { timeout_test_run(context); }' \
|
||||
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/legacy-shell.uce"
|
||||
printf '%s\n' \
|
||||
'void timeout_test_run(Request& context) {' \
|
||||
' if(context.get["warm"] == "1") { print("warm"); return; }' \
|
||||
" file_put_contents(\"$pid_file\", std::to_string(request_perf()[\"worker_pid\"].to_u64()));" \
|
||||
' while(true) sleep(60);' \
|
||||
' print("sleep-end");' \
|
||||
'}' \
|
||||
'CLI(Request& context) { timeout_test_run(context); }' \
|
||||
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/sleep.uce"
|
||||
printf '%s\n' \
|
||||
'void timeout_test_run(Request& context) { print(request_perf()["worker_pid"].to_u64(), "|health"); }' \
|
||||
'CLI(Request& context) { timeout_test_run(context); }' \
|
||||
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/health.uce"
|
||||
|
||||
max_seconds=$(( (invocation_ms + 15000) / 1000 ))
|
||||
(( max_seconds >= 10 )) || max_seconds=10
|
||||
|
||||
request() {
|
||||
local unit="$1"
|
||||
curl -sS --max-time "$max_seconds" -o "$body_file" -w '%{http_code}' "${curl_transport[@]}" "$test_base_url/$test_name/$unit"
|
||||
}
|
||||
|
||||
same_worker_health() {
|
||||
local expected_pid="$1"
|
||||
kill -0 "$expected_pid" 2>/dev/null || return 1
|
||||
local health
|
||||
health=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/health.uce")
|
||||
[[ "$health" =~ ^[0-9]+\|health$ ]]
|
||||
}
|
||||
|
||||
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/hostcall-loop.uce?warm=1" >/dev/null
|
||||
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?warm=1" >/dev/null
|
||||
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/sleep.uce?warm=1" >/dev/null
|
||||
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/health.uce" >/dev/null
|
||||
quick=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?quick=1")
|
||||
[[ "$quick" == quick ]] || { echo "quick legacy shell returned: $quick" >&2; exit 1; }
|
||||
exit_status=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?status=1")
|
||||
[[ "$exit_status" == 7 ]] || { echo "structured shell lost exit status: $exit_status" >&2; exit 1; }
|
||||
zero_timeout=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?zero=1")
|
||||
[[ "$zero_timeout" == zero ]] || { echo "structured shell zero-timeout default changed: $zero_timeout" >&2; exit 1; }
|
||||
job_elapsed=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?job=1")
|
||||
[[ "$job_elapsed" =~ ^[0-9]+$ && "$job_elapsed" -ge 200 && "$job_elapsed" -lt 500 ]] || { echo "job_await two-call duration was ${job_elapsed}ms" >&2; exit 1; }
|
||||
|
||||
for unit in hostcall-loop.uce legacy-shell.uce sleep.uce; do
|
||||
rm -f "$pid_file"
|
||||
started_ns=$(date +%s%N)
|
||||
status=$(request "$unit")
|
||||
elapsed_ms=$(( ($(date +%s%N) - started_ns) / 1000000 ))
|
||||
[[ "$status" == "500" ]] || { echo "$unit returned HTTP $status" >&2; exit 1; }
|
||||
if [[ "$unit" == legacy-shell.uce ]]; then
|
||||
if (( invocation_ms <= 5000 )); then
|
||||
grep -q 'UCE_INVOCATION_TIMEOUT:' "$body_file" || { echo "$unit lacked invocation-timeout classification" >&2; exit 1; }
|
||||
expected_ms=$invocation_ms
|
||||
else
|
||||
grep -q 'UCE_HOSTCALL_TIMEOUT:' "$body_file" || { echo "$unit lacked hostcall-timeout classification" >&2; exit 1; }
|
||||
expected_ms=5000
|
||||
fi
|
||||
else
|
||||
grep -q 'UCE_INVOCATION_TIMEOUT:' "$body_file" || { echo "$unit lacked invocation-timeout classification: $(cat "$body_file")" >&2; exit 1; }
|
||||
expected_ms=$invocation_ms
|
||||
fi
|
||||
(( elapsed_ms >= expected_ms - 1000 && elapsed_ms <= expected_ms + 3000 )) || { echo "$unit completed in ${elapsed_ms}ms; expected about ${expected_ms}ms" >&2; exit 1; }
|
||||
grep -q "$unit" "$body_file" || { echo "$unit lacked a source-mapped trace" >&2; exit 1; }
|
||||
[[ -s "$pid_file" ]] || { echo "$unit did not record its worker" >&2; exit 1; }
|
||||
worker_pid=$(<"$pid_file")
|
||||
same_worker_health "$worker_pid" || { echo "worker $worker_pid did not survive $unit" >&2; exit 1; }
|
||||
done
|
||||
|
||||
echo "wasm invocation timeout passed (absolute ${invocation_ms}ms; sleep and process hostcalls bounded)"
|
||||
Executable
+87
@@ -0,0 +1,87 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="wasm-metadata-buffer-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
bin_directory="${UCE_TEST_BIN_DIRECTORY:-/tmp/uce/work}"
|
||||
if [[ -r /etc/uce/settings.cfg ]]; then
|
||||
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*HTTP_DOCUMENT_ROOT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
site_directory="${configured_site_directory:-$site_directory}"
|
||||
fi
|
||||
if [[ -z "${UCE_TEST_BIN_DIRECTORY:-}" ]]; then
|
||||
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
bin_directory="${configured_bin_directory:-$bin_directory}"
|
||||
fi
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
artifact_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
if [[ -n "$artifact_dir" ]]; then
|
||||
rm -rf "$artifact_dir"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
|
||||
|
||||
printf '%s\n' \
|
||||
'CLI(Request& context) {' \
|
||||
' String target = context.get["unit"];' \
|
||||
' String rendered = component(target, context);' \
|
||||
' DValue perf = request_perf();' \
|
||||
' DValue selected;' \
|
||||
' perf["unit_module_operations"].each([&](DValue operation, String key) {' \
|
||||
' if(operation["kind"].to_string() == "component" && contains(operation["unit"].to_string(), target + ".uce")) selected = operation;' \
|
||||
' });' \
|
||||
' print(rendered, "\t", selected["source"].to_string(), "\t", selected["read_count"].to_string(), "\t", selected["read_bytes"].to_string(), "\t", selected["read_us"].to_string());' \
|
||||
'}' >"$source_dir/parent.uce"
|
||||
|
||||
payload=$(head -c 131072 /dev/zero | tr '\0' x)
|
||||
for unit in $(seq 0 7); do
|
||||
printf 'String metadata_payload_%s() { return("%s"); }\nCOMPONENT(Request& context) { String payload = metadata_payload_%s(); u64 offset = std::atoi(context.get["offset"].c_str()) %% payload.size(); print(std::to_string(payload.size()), ":", payload.substr(offset, 1)); }\n' \
|
||||
"$unit" "$payload" "$unit" >"$source_dir/component-$unit.uce"
|
||||
done
|
||||
unset payload
|
||||
|
||||
declare -a read_times=()
|
||||
for unit in $(seq 0 7); do
|
||||
found=0
|
||||
for attempt in $(seq 1 20); do
|
||||
output=$(timeout --signal=TERM --kill-after=2s 30s scripts/uce-cli "/$test_name/parent.uce?unit=component-$unit&offset=$attempt")
|
||||
IFS=$'\t' read -r marker source read_count read_bytes read_us <<<"$output"
|
||||
if [[ "$marker" != "131072:x" ]]; then
|
||||
echo "metadata buffer component failed: unit=$unit output=$output" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$source" != "serialized" ]]; then
|
||||
continue
|
||||
fi
|
||||
wasm_size=$(stat -c %s "$artifact_dir/component-$unit.uce.wasm")
|
||||
if ! awk -v count="$read_count" -v bytes="$read_bytes" -v size="$wasm_size" \
|
||||
'BEGIN { exit !(count > 0 && count <= 64 && bytes > 0 && bytes <= count * 4096 && bytes < size) }'; then
|
||||
echo "metadata buffer read was not bounded: unit=$unit reads=$read_count bytes=$read_bytes wasm=$wasm_size us=$read_us" >&2
|
||||
exit 1
|
||||
fi
|
||||
read_times+=("$read_us")
|
||||
found=1
|
||||
break
|
||||
done
|
||||
if (( found == 0 )); then
|
||||
echo "metadata buffer did not observe a serialized cold-worker load for component-$unit" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
mapfile -t sorted < <(printf '%s\n' "${read_times[@]}" | sort -n)
|
||||
median=${sorted[$(( ${#sorted[@]} / 2 ))]}
|
||||
maximum=${sorted[$(( ${#sorted[@]} - 1 ))]}
|
||||
if ! awk -v median="$median" 'BEGIN { exit !(median < 3000) }'; then
|
||||
echo "metadata buffer median exceeded 3ms: median=${median}us maximum=${maximum}us samples=${read_times[*]}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "wasm metadata buffer passed: ${#read_times[@]} cold serialized loads, median ${median}us, maximum ${maximum}us"
|
||||
Executable
+474
@@ -0,0 +1,474 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
if [[ "${1:-}" != "--inside" ]]; then
|
||||
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||
fi
|
||||
|
||||
root="/tmp/uce-metadata-deadline-$$"
|
||||
site="$root/site"
|
||||
work="$root/work"
|
||||
settings="$root/settings.cfg"
|
||||
socket="$root/run/cli.sock"
|
||||
log="$root/service.log"
|
||||
control="$root/pread.control"
|
||||
server_pid=""
|
||||
|
||||
cleanup() {
|
||||
status=$?
|
||||
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||
kill -TERM "$server_pid" 2>/dev/null || true
|
||||
for _ in $(seq 1 100); do kill -0 "$server_pid" 2>/dev/null || break; sleep 0.02; done
|
||||
kill -KILL "$server_pid" 2>/dev/null || true
|
||||
wait "$server_pid" 2>/dev/null || true
|
||||
fi
|
||||
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
|
||||
rm -rf "$root"
|
||||
return "$status"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$site/components" "$work" "$root/run" "$root/session" "$root/upload"
|
||||
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|CLI_WORKER_COUNT|CLI_WORKER_MAX_REQUESTS|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|WASM_INVOCATION_TIMEOUT_MS|WASM_EPOCH_PERIOD_MS|PROACTIVE_COMPILE_ENABLED|SERVE_LAST_KNOWN_GOOD|SHOW_DYNAMIC_COMPILE_ERRORS|WORKER_COUNT)[[:space:]]*=/d' \
|
||||
/etc/uce/settings.cfg >"$settings"
|
||||
cat >>"$settings" <<CFG
|
||||
BIN_DIRECTORY=$work
|
||||
PRECOMPILE_FILES_IN=$site
|
||||
SITE_DIRECTORY=$site
|
||||
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||
FCGI_PORT=
|
||||
CLI_SOCKET_PATH=$socket
|
||||
CLI_WORKER_COUNT=1
|
||||
CLI_WORKER_MAX_REQUESTS=0
|
||||
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||
HTTP_PORT=
|
||||
HTTP_DOCUMENT_ROOT=$site
|
||||
SESSION_PATH=$root/session
|
||||
TMP_UPLOAD_PATH=$root/upload
|
||||
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||
WASM_INVOCATION_TIMEOUT_MS=1000
|
||||
WASM_EPOCH_PERIOD_MS=10
|
||||
PROACTIVE_COMPILE_ENABLED=0
|
||||
SERVE_LAST_KNOWN_GOOD=0
|
||||
SHOW_DYNAMIC_COMPILE_ERRORS=1
|
||||
WORKER_COUNT=1
|
||||
CFG
|
||||
mount --bind "$settings" /etc/uce/settings.cfg
|
||||
|
||||
cat >"$site/driver.uce" <<'UCE'
|
||||
CLI(Request& context) {
|
||||
if(context.get["health"] == "1") { print(request_perf()["worker_pid"].to_u64(), "|health"); return; }
|
||||
print(component(context.get["target"], context));
|
||||
}
|
||||
UCE
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("deadline-component"); }' >"$site/components/deadline.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("truncate-component"); }' >"$site/components/truncate.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("mutation-component"); }' >"$site/components/mutation.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("uleb-component"); }' >"$site/components/uleb.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("duplicate-component"); }' >"$site/components/duplicate.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("full-component"); }' >"$site/components/full.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("full-uleb-component"); }' >"$site/components/full-uleb.uce"
|
||||
printf '%s\n' 'COMPONENT(Request& context) { print("full-duplicate-component"); }' >"$site/components/full-duplicate.uce"
|
||||
|
||||
timeout --signal=TERM --kill-after=5s 60s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" \
|
||||
UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >"$root/precompile.log" 2>&1
|
||||
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
|
||||
for unit in deadline truncate mutation uleb duplicate full full-uleb full-duplicate; do
|
||||
[[ -s "$cache/components/$unit.uce.wasm" && -s "$cache/components/$unit.uce.cwasm" ]]
|
||||
cp "$cache/components/$unit.uce.wasm" "$root/$unit.original.wasm"
|
||||
done
|
||||
|
||||
cat >"$root/interpose.c" <<'C'
|
||||
#define _GNU_SOURCE
|
||||
#define _LARGEFILE64_SOURCE
|
||||
#include <dlfcn.h>
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <limits.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <sys/stat.h>
|
||||
#include <time.h>
|
||||
#include <unistd.h>
|
||||
|
||||
struct control {
|
||||
char target[PATH_MAX];
|
||||
char mode[16];
|
||||
int delay_ms;
|
||||
char ready[PATH_MAX];
|
||||
char release[PATH_MAX];
|
||||
char counter[PATH_MAX];
|
||||
};
|
||||
|
||||
static int read_control(int fd, struct control* control)
|
||||
{
|
||||
const char* path = getenv("UCE_TEST_PREAD_CONTROL");
|
||||
if(!path || !*path) return 0;
|
||||
int input = open(path, O_RDONLY | O_CLOEXEC);
|
||||
if(input < 0) return 0;
|
||||
char text[PATH_MAX * 4];
|
||||
ssize_t count = read(input, text, sizeof(text) - 1);
|
||||
close(input);
|
||||
if(count <= 0) return 0;
|
||||
text[count] = 0;
|
||||
if(sscanf(text, "%4095s %15s %d %4095s %4095s %4095s", control->target, control->mode,
|
||||
&control->delay_ms, control->ready, control->release, control->counter) != 6) return 0;
|
||||
char link[64], resolved[PATH_MAX];
|
||||
snprintf(link, sizeof(link), "/proc/self/fd/%d", fd);
|
||||
ssize_t length = readlink(link, resolved, sizeof(resolved) - 1);
|
||||
if(length <= 0) return 0;
|
||||
resolved[length] = 0;
|
||||
return strcmp(resolved, control->target) == 0;
|
||||
}
|
||||
|
||||
static void mark(const char* path)
|
||||
{
|
||||
if(strcmp(path, "-") == 0) return;
|
||||
int fd = open(path, O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0600);
|
||||
if(fd >= 0) { ssize_t ignored = write(fd, "1\n", 2); (void)ignored; close(fd); }
|
||||
}
|
||||
|
||||
static void wait_release(const struct control* control)
|
||||
{
|
||||
if(strcmp(control->release, "-") == 0) return;
|
||||
struct timespec started, now, pause = { 0, 10000000 };
|
||||
clock_gettime(CLOCK_MONOTONIC, &started);
|
||||
while(access(control->release, F_OK) != 0)
|
||||
{
|
||||
nanosleep(&pause, 0);
|
||||
clock_gettime(CLOCK_MONOTONIC, &now);
|
||||
if(now.tv_sec - started.tv_sec >= 2) break;
|
||||
}
|
||||
}
|
||||
|
||||
static void delay_ms(int milliseconds)
|
||||
{
|
||||
if(milliseconds <= 0) return;
|
||||
struct timespec pause = { milliseconds / 1000, (milliseconds % 1000) * 1000000L };
|
||||
while(nanosleep(&pause, &pause) != 0 && errno == EINTR) {}
|
||||
}
|
||||
|
||||
static void before_read(const struct control* control)
|
||||
{
|
||||
mark(control->counter);
|
||||
if(strcmp(control->mode, "before") == 0 && access(control->ready, F_OK) != 0)
|
||||
{
|
||||
mark(control->ready);
|
||||
wait_release(control);
|
||||
}
|
||||
if(strcmp(control->mode, "delay") == 0) delay_ms(control->delay_ms);
|
||||
}
|
||||
|
||||
static void after_read(const struct control* control)
|
||||
{
|
||||
if(strcmp(control->mode, "after") == 0 && access(control->ready, F_OK) != 0)
|
||||
{
|
||||
mark(control->ready);
|
||||
wait_release(control);
|
||||
}
|
||||
}
|
||||
|
||||
ssize_t pread(int fd, void* buffer, size_t size, off_t offset)
|
||||
{
|
||||
static ssize_t (*real_pread)(int, void*, size_t, off_t);
|
||||
if(!real_pread) real_pread = dlsym(RTLD_NEXT, "pread");
|
||||
struct control control;
|
||||
int targeted = read_control(fd, &control);
|
||||
if(targeted) before_read(&control);
|
||||
ssize_t result = real_pread(fd, buffer, size, offset);
|
||||
if(targeted) after_read(&control);
|
||||
return result;
|
||||
}
|
||||
|
||||
ssize_t pread64(int fd, void* buffer, size_t size, off64_t offset)
|
||||
{
|
||||
static ssize_t (*real_pread64)(int, void*, size_t, off64_t);
|
||||
if(!real_pread64) real_pread64 = dlsym(RTLD_NEXT, "pread64");
|
||||
struct control control;
|
||||
int targeted = read_control(fd, &control);
|
||||
if(targeted) before_read(&control);
|
||||
ssize_t result = real_pread64(fd, buffer, size, offset);
|
||||
if(targeted) after_read(&control);
|
||||
return result;
|
||||
}
|
||||
C
|
||||
timeout 15s cc -shared -fPIC -O2 -Wall -Wextra -Werror -o "$root/interpose.so" "$root/interpose.c" -ldl
|
||||
|
||||
timeout --signal=TERM --kill-after=5s 90s env LD_PRELOAD="$root/interpose.so" UCE_TEST_PREAD_CONTROL="$control" \
|
||||
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||
server_pid=$!
|
||||
for _ in $(seq 1 400); do [[ -S "$socket" ]] && break; sleep 0.02; done
|
||||
[[ -S "$socket" ]] || { cat "$log" >&2; exit 1; }
|
||||
|
||||
request() {
|
||||
timeout --signal=TERM --kill-after=1s 5s scripts/uce-cli --socket "$socket" --get /driver.uce "$@"
|
||||
}
|
||||
|
||||
health=$(request health=1)
|
||||
worker_pid=${health%%|*}
|
||||
[[ "$health" == "$worker_pid|health" ]]
|
||||
|
||||
restore() {
|
||||
local unit="$1" wasm="$cache/components/$1.uce.wasm" cwasm="$cache/components/$1.uce.cwasm"
|
||||
cp "$root/$unit.original.wasm" "$wasm"
|
||||
python3 - "$wasm" "$cwasm" <<'PY'
|
||||
import os, pathlib, sys
|
||||
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
|
||||
stamp = cwasm.stat().st_mtime_ns - 1_000_000
|
||||
os.utime(wasm, ns=(stamp, stamp))
|
||||
PY
|
||||
}
|
||||
|
||||
prepare_deadline() {
|
||||
local wasm="$cache/components/deadline.uce.wasm" cwasm="$cache/components/deadline.uce.cwasm"
|
||||
python3 - "$wasm" "$cwasm" <<'PY'
|
||||
import os, pathlib, sys
|
||||
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
|
||||
with wasm.open("ab") as output:
|
||||
output.write(b"\0\1\0" * 7000)
|
||||
stamp = cwasm.stat().st_mtime_ns - 1_000_000
|
||||
os.utime(wasm, ns=(stamp, stamp))
|
||||
PY
|
||||
}
|
||||
|
||||
prepare_mutation() {
|
||||
local wasm="$cache/components/mutation.uce.wasm" cwasm="$cache/components/mutation.uce.cwasm"
|
||||
python3 - "$wasm" "$cwasm" "$root/mutation-offset" <<'PY'
|
||||
import os, pathlib, sys
|
||||
wasm, cwasm, offset_file = map(pathlib.Path, sys.argv[1:])
|
||||
payload = bytes([7]) + b"padding" + bytes(8192)
|
||||
value = len(payload)
|
||||
encoded = bytearray()
|
||||
while True:
|
||||
byte = value & 0x7f
|
||||
value >>= 7
|
||||
encoded.append(byte | (0x80 if value else 0))
|
||||
if not value:
|
||||
break
|
||||
with wasm.open("ab") as output:
|
||||
output.write(bytes([0]) + encoded + payload)
|
||||
offset_file.write_text(str(wasm.stat().st_size - 4096))
|
||||
stamp = cwasm.stat().st_mtime_ns - 1_000_000
|
||||
os.utime(wasm, ns=(stamp, stamp))
|
||||
PY
|
||||
}
|
||||
|
||||
wait_ready() {
|
||||
local ready="$1" request_pid="$2" deadline=$((SECONDS + 3))
|
||||
while [[ ! -e "$ready" ]] && kill -0 "$request_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.01; done
|
||||
[[ -e "$ready" ]] || { echo "metadata read gate did not activate" >&2; return 1; }
|
||||
}
|
||||
|
||||
prepare_deadline
|
||||
deadline_wasm="$cache/components/deadline.uce.wasm"
|
||||
deadline_counter="$root/deadline.counter"
|
||||
printf '%s delay 1100 - - %s\n' "$deadline_wasm" "$deadline_counter" >"$control"
|
||||
started=$(date +%s%N)
|
||||
set +e
|
||||
deadline_output=$(request target=components/deadline.uce 2>&1)
|
||||
set -e
|
||||
elapsed_ms=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||
[[ "$deadline_output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "metadata scan lacked canonical timeout: $deadline_output" >&2; exit 1; }
|
||||
(( elapsed_ms >= 1000 && elapsed_ms < 3000 )) || { echo "metadata timeout took ${elapsed_ms}ms" >&2; exit 1; }
|
||||
[[ $(wc -l <"$deadline_counter") -eq 1 ]] || { echo "metadata scanner read past the first delayed refill" >&2; exit 1; }
|
||||
rm -f "$control"
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
restore deadline
|
||||
[[ "$(request target=components/deadline.uce)" == deadline-component ]]
|
||||
|
||||
rm -f "$cache/components/deadline.uce.cwasm"
|
||||
touch "$deadline_wasm"
|
||||
printf '%s delay 1100 - - %s\n' "$deadline_wasm" "$root/full-deadline.counter" >"$control"
|
||||
started=$(date +%s%N)
|
||||
set +e
|
||||
full_deadline_output=$(request target=components/deadline.uce 2>&1)
|
||||
set -e
|
||||
elapsed_ms=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||
[[ "$full_deadline_output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "full artifact read lacked canonical timeout: $full_deadline_output" >&2; exit 1; }
|
||||
(( elapsed_ms >= 1000 && elapsed_ms < 3000 )) || { echo "full artifact timeout took ${elapsed_ms}ms" >&2; exit 1; }
|
||||
[[ $(wc -l <"$root/full-deadline.counter") -eq 1 ]] || { echo "full artifact reader continued after the first delayed chunk" >&2; exit 1; }
|
||||
rm -f "$control"
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
[[ "$(request target=components/deadline.uce)" == deadline-component ]]
|
||||
[[ -s "$cache/components/deadline.uce.cwasm" ]]
|
||||
|
||||
truncate_wasm="$cache/components/truncate.uce.wasm"
|
||||
truncate_ready="$root/truncate.ready"
|
||||
truncate_release="$root/truncate.release"
|
||||
printf '%s before 0 %s %s %s\n' "$truncate_wasm" "$truncate_ready" "$truncate_release" "$root/truncate.counter" >"$control"
|
||||
( set +e; request target=components/truncate.uce >"$root/truncate.output" 2>&1; printf '%s\n' "$?" >"$root/truncate.status" ) &
|
||||
request_pid=$!
|
||||
wait_ready "$truncate_ready" "$request_pid"
|
||||
truncate -s 4 "$truncate_wasm"
|
||||
touch "$truncate_release"
|
||||
wait "$request_pid"
|
||||
[[ $(<"$root/truncate.status") -ne 0 ]]
|
||||
grep -q '\[wasm\] component load failed: not a supported wasm module' "$log"
|
||||
! grep -q 'truncate-component' "$root/truncate.output"
|
||||
rm -f "$control"
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
restore truncate
|
||||
[[ "$(request target=components/truncate.uce)" == truncate-component ]]
|
||||
|
||||
prepare_mutation
|
||||
mutation_wasm="$cache/components/mutation.uce.wasm"
|
||||
mutation_ready="$root/mutation.ready"
|
||||
mutation_release="$root/mutation.release"
|
||||
printf '%s after 0 %s %s %s\n' "$mutation_wasm" "$mutation_ready" "$mutation_release" "$root/mutation.counter" >"$control"
|
||||
( set +e; request target=components/mutation.uce >"$root/mutation.output" 2>&1; printf '%s\n' "$?" >"$root/mutation.status" ) &
|
||||
request_pid=$!
|
||||
wait_ready "$mutation_ready" "$request_pid"
|
||||
python3 - "$mutation_wasm" "$(<"$root/mutation-offset")" <<'PY'
|
||||
import os, pathlib, sys
|
||||
path = pathlib.Path(sys.argv[1])
|
||||
offset = int(sys.argv[2])
|
||||
with path.open("r+b", buffering=0) as artifact:
|
||||
artifact.seek(offset)
|
||||
byte = artifact.read(1)
|
||||
artifact.seek(offset)
|
||||
artifact.write(bytes([byte[0] ^ 1]))
|
||||
os.fsync(artifact.fileno())
|
||||
PY
|
||||
touch "$mutation_release"
|
||||
wait "$request_pid"
|
||||
[[ $(<"$root/mutation.status") -ne 0 ]]
|
||||
grep -q '\[wasm\] component load failed: wasm artifact changed while loading metadata' "$log"
|
||||
! grep -q 'mutation-component' "$root/mutation.output"
|
||||
rm -f "$control"
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
restore mutation
|
||||
[[ "$(request target=components/mutation.uce)" == mutation-component ]]
|
||||
|
||||
uleb_wasm="$cache/components/uleb.uce.wasm"
|
||||
uleb_cwasm="$cache/components/uleb.uce.cwasm"
|
||||
python3 - "$uleb_wasm" "$uleb_cwasm" <<'PY'
|
||||
import os, pathlib, sys
|
||||
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
|
||||
with wasm.open("r+b", buffering=0) as artifact:
|
||||
artifact.seek(9)
|
||||
artifact.write(bytes([0x80]) * 9 + bytes([0x02]))
|
||||
os.fsync(artifact.fileno())
|
||||
stamp = cwasm.stat().st_mtime_ns - 1_000_000
|
||||
os.utime(wasm, ns=(stamp, stamp))
|
||||
PY
|
||||
set +e
|
||||
uleb_output=$(request target=components/uleb.uce 2>&1)
|
||||
set -e
|
||||
grep -q '\[wasm\] component load failed: malformed wasm section header' "$log"
|
||||
[[ "$uleb_output" != *uleb-component* ]]
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
restore uleb
|
||||
[[ "$(request target=components/uleb.uce)" == uleb-component ]]
|
||||
|
||||
duplicate_wasm="$cache/components/duplicate.uce.wasm"
|
||||
duplicate_cwasm="$cache/components/duplicate.uce.cwasm"
|
||||
python3 - "$duplicate_wasm" "$duplicate_cwasm" <<'PY'
|
||||
import os, pathlib, sys
|
||||
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
|
||||
payload = bytes([10]) + b"uce.module" + b"duplicate"
|
||||
value = len(payload)
|
||||
encoded = bytearray()
|
||||
while True:
|
||||
byte = value & 0x7f
|
||||
value >>= 7
|
||||
encoded.append(byte | (0x80 if value else 0))
|
||||
if not value:
|
||||
break
|
||||
with wasm.open("ab") as artifact:
|
||||
artifact.write(bytes([0]) + encoded + payload)
|
||||
stamp = cwasm.stat().st_mtime_ns - 1_000_000
|
||||
os.utime(wasm, ns=(stamp, stamp))
|
||||
PY
|
||||
set +e
|
||||
duplicate_output=$(request target=components/duplicate.uce 2>&1)
|
||||
set -e
|
||||
grep -q '\[wasm\] component load failed: duplicate uce.module metadata section' "$log"
|
||||
[[ "$duplicate_output" != *duplicate-component* ]]
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
restore duplicate
|
||||
[[ "$(request target=components/duplicate.uce)" == duplicate-component ]]
|
||||
|
||||
full_wasm="$cache/components/full.uce.wasm"
|
||||
full_cwasm="$cache/components/full.uce.cwasm"
|
||||
python3 - "$full_wasm" <<'PY'
|
||||
import pathlib, sys
|
||||
wasm = pathlib.Path(sys.argv[1])
|
||||
payload = bytes([7]) + b"uce.abi" + bytes(1024 * 1024)
|
||||
value = len(payload)
|
||||
encoded = bytearray()
|
||||
while True:
|
||||
byte = value & 0x7f
|
||||
value >>= 7
|
||||
encoded.append(byte | (0x80 if value else 0))
|
||||
if not value:
|
||||
break
|
||||
with wasm.open("ab") as artifact:
|
||||
artifact.write(bytes([0]) + encoded + payload)
|
||||
PY
|
||||
rm -f "$full_cwasm"
|
||||
set +e
|
||||
full_output=$(request target=components/full.uce 2>&1)
|
||||
set -e
|
||||
grep -Eq '\[wasm\] component load failed: .*oversized wasm metadata section' "$log"
|
||||
[[ "$full_output" != *full-component* ]]
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
cp "$root/full.original.wasm" "$full_wasm"
|
||||
[[ "$(request target=components/full.uce)" == full-component ]]
|
||||
[[ -s "$full_cwasm" ]]
|
||||
|
||||
full_uleb_wasm="$cache/components/full-uleb.uce.wasm"
|
||||
full_uleb_cwasm="$cache/components/full-uleb.uce.cwasm"
|
||||
python3 - "$full_uleb_wasm" <<'PY'
|
||||
import os, pathlib, sys
|
||||
wasm = pathlib.Path(sys.argv[1])
|
||||
with wasm.open("r+b", buffering=0) as artifact:
|
||||
artifact.seek(9)
|
||||
artifact.write(bytes([0x80]) * 9 + bytes([0x02]))
|
||||
os.fsync(artifact.fileno())
|
||||
PY
|
||||
rm -f "$full_uleb_cwasm"
|
||||
malformed_before=$(grep -c 'malformed wasm section header' "$log" || true)
|
||||
set +e
|
||||
full_uleb_output=$(request target=components/full-uleb.uce 2>&1)
|
||||
set -e
|
||||
malformed_after=$(grep -c 'malformed wasm section header' "$log" || true)
|
||||
(( malformed_after == malformed_before + 1 ))
|
||||
[[ "$full_uleb_output" != *full-uleb-component* ]]
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
cp "$root/full-uleb.original.wasm" "$full_uleb_wasm"
|
||||
[[ "$(request target=components/full-uleb.uce)" == full-uleb-component ]]
|
||||
[[ -s "$full_uleb_cwasm" ]]
|
||||
|
||||
full_duplicate_wasm="$cache/components/full-duplicate.uce.wasm"
|
||||
full_duplicate_cwasm="$cache/components/full-duplicate.uce.cwasm"
|
||||
python3 - "$full_duplicate_wasm" <<'PY'
|
||||
import pathlib, sys
|
||||
wasm = pathlib.Path(sys.argv[1])
|
||||
payload = bytes([10]) + b"uce.module" + b"duplicate"
|
||||
value = len(payload)
|
||||
encoded = bytearray()
|
||||
while True:
|
||||
byte = value & 0x7f
|
||||
value >>= 7
|
||||
encoded.append(byte | (0x80 if value else 0))
|
||||
if not value:
|
||||
break
|
||||
with wasm.open("ab") as artifact:
|
||||
artifact.write(bytes([0]) + encoded + payload)
|
||||
PY
|
||||
rm -f "$full_duplicate_cwasm"
|
||||
duplicate_before=$(grep -c 'duplicate uce.module metadata section' "$log" || true)
|
||||
set +e
|
||||
full_duplicate_output=$(request target=components/full-duplicate.uce 2>&1)
|
||||
set -e
|
||||
duplicate_after=$(grep -c 'duplicate uce.module metadata section' "$log" || true)
|
||||
(( duplicate_after == duplicate_before + 1 ))
|
||||
[[ "$full_duplicate_output" != *full-duplicate-component* ]]
|
||||
[[ "$(request health=1)" == "$worker_pid|health" ]]
|
||||
cp "$root/full-duplicate.original.wasm" "$full_duplicate_wasm"
|
||||
[[ "$(request target=components/full-duplicate.uce)" == full-duplicate-component ]]
|
||||
[[ -s "$full_duplicate_cwasm" ]]
|
||||
|
||||
! grep -Eiq 'panic|segfault|permission denied' "$log"
|
||||
echo "wasm metadata deadline passed: streamed and full-read timeouts, mutation guards, and parser bounds passed; same worker recovered"
|
||||
Executable
+54
@@ -0,0 +1,54 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
test_name="wasm-source-location-test-$$"
|
||||
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
|
||||
bin_directory="${UCE_TEST_BIN_DIRECTORY:-/tmp/uce/work}"
|
||||
if [[ -r /etc/uce/settings.cfg ]]; then
|
||||
configured_site_directory=$(awk -F= '/^[[:space:]]*HTTP_DOCUMENT_ROOT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
|
||||
[[ -n "${UCE_TEST_SITE_DIRECTORY:-}" ]] || site_directory="${configured_site_directory:-$site_directory}"
|
||||
[[ -n "${UCE_TEST_BIN_DIRECTORY:-}" ]] || bin_directory="${configured_bin_directory:-$bin_directory}"
|
||||
fi
|
||||
source_dir="$site_directory/$test_name"
|
||||
absolute_source_dir=""
|
||||
artifact_dir=""
|
||||
|
||||
cleanup() {
|
||||
rm -rf "$source_dir"
|
||||
[[ -z "$artifact_dir" ]] || rm -rf "$artifact_dir"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
mkdir -p "$source_dir"
|
||||
absolute_source_dir=$(realpath "$source_dir")
|
||||
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$absolute_source_dir"
|
||||
printf '%s\n' 'CLI(Request& context) { __builtin_trap(); }' >"$source_dir/entry.uce"
|
||||
|
||||
set +e
|
||||
output=$(scripts/uce-cli "/$test_name/entry.uce" 2>&1)
|
||||
status=$?
|
||||
set -e
|
||||
if [[ $status -eq 0 ]]; then
|
||||
echo "trapping wasm unit unexpectedly succeeded" >&2
|
||||
exit 1
|
||||
fi
|
||||
expected="$absolute_source_dir/entry.uce:1:25"
|
||||
if [[ "$output" != *'wasm `unreachable` instruction executed'* || "$output" != *"source locations:"* || "$output" != *"$expected"* ]]; then
|
||||
echo "wasm trap omitted its source location:" >&2
|
||||
echo "$output" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
rm "$artifact_dir/entry.uce.wasm.source-map"
|
||||
set +e
|
||||
without_map=$(scripts/uce-cli "/$test_name/entry.uce" 2>&1)
|
||||
status=$?
|
||||
set -e
|
||||
if [[ $status -eq 0 || "$without_map" != *'wasm `unreachable` instruction executed'* || "$without_map" == *"source locations:"* ]]; then
|
||||
echo "missing-map trap fallback failed:" >&2
|
||||
echo "$without_map" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "wasm source locations passed"
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
base="${1:-/tmp/uce/work}"
|
||||
compiler_abi=$(awk '/^#define UCE_COMPILER_UNIT_ABI_VERSION / {print $3; exit}' src/wasm/abi.h)
|
||||
core_abi=$(awk '/^#define UCE_WASM_CORE_ABI_VERSION / {print $3; exit}' src/wasm/abi.h)
|
||||
printf '%s/units-c%s-w%s\n' "${base%/}" "$compiler_abi" "$core_abi"
|
||||
@@ -5,3 +5,8 @@ EXPORT DValue* test_func(DValue* call_param)
|
||||
print("HELLO FROM TEST FUNCTION");
|
||||
return(0);
|
||||
}
|
||||
|
||||
RENDER(Request& context)
|
||||
{
|
||||
print("HELLO FROM UNIT RENDER");
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
MySQL Functions
|
||||
|
||||
mysql_connect
|
||||
mysql_connected
|
||||
mysql_disconnect
|
||||
mysql_error
|
||||
mysql_escape
|
||||
|
||||
@@ -14,6 +14,7 @@ hmac_sha256
|
||||
hmac_sha256_hex
|
||||
random_bytes
|
||||
crypto_equal
|
||||
crypto_operation
|
||||
password_hash
|
||||
password_verify
|
||||
password_needs_rehash
|
||||
|
||||
@@ -14,6 +14,8 @@ Renders another `.uce` file as a component and returns the captured output as a
|
||||
|
||||
Component props are passed in `context.props`.
|
||||
|
||||
Calling a missing component or named component handler is a server error and sets HTTP status 500. Probe intentionally optional components with `component_exists()` before calling `component()`.
|
||||
|
||||
Because `<?= ... ?>` HTML-escapes its value, embed component markup with `<?: component(...) ?>`, `print(component(...))`, or use `component_render(...)` for direct output.
|
||||
|
||||
## Named Components
|
||||
|
||||
@@ -17,6 +17,8 @@ Component props are passed through `context.props`, and `name:COMPONENTFUNC` may
|
||||
|
||||
When `name` starts with `:`, the runtime resolves that named handler against the current `.uce` file.
|
||||
|
||||
Calling a missing component or named component handler is a server error and sets HTTP status 500. Probe intentionally optional components with `component_exists()` before calling `component_render()`.
|
||||
|
||||
If the target file defines `ONCE(Request& context)`, that hook runs once per request before the file's first component or render entrypoint.
|
||||
|
||||
Use `component_render()` when you want to write component output directly from C++ code instead of capturing it as a `String`.
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
:sig
|
||||
DValue crypto_operation(DValue request)
|
||||
|
||||
:params
|
||||
request : structured operation, algorithm, and operation-specific fields
|
||||
return value : map with ok, bounded error code, and operation-specific output
|
||||
|
||||
:content
|
||||
Runs one explicitly supported structured asymmetric cryptographic operation. The initial allowlist is `key_generate` with `ES256` and `jwt_sign` with `ES256`. Unknown operations and algorithms fail closed.
|
||||
|
||||
ES256 signing validates that `x`, `y`, and `d` form one P-256 key, forces the protected `alg` to `ES256`, and emits a compact JWT with a 64-byte JOSE signature. Requests are capped at 32 KiB; `cbor_decode` accepts at most 16 KiB decoded CBOR (21,846 canonical base64url characters), 256 nodes, and depth 16. CBOR is definite-length only, validates UTF-8 text, preserves typed map keys, rejects structurally duplicate keys and trailing bytes, and reports malformed CBOR as `invalid_cbor`. Header and claims roots must be JSON objects containing valid UTF-8 without raw control bytes.
|
||||
|
||||
This function does not replace typed digest, HMAC, password, randomness, or constant-time comparison APIs. It exposes no raw signing, arbitrary curve/digest selection, encryption, or generic OpenSSL access. Keep returned private JWKs secret.
|
||||
|
||||
:example
|
||||
DValue key_request;
|
||||
key_request["operation"] = "key_generate";
|
||||
key_request["algorithm"] = "ES256";
|
||||
DValue key = crypto_operation(key_request);
|
||||
|
||||
DValue sign_request;
|
||||
sign_request["operation"] = "jwt_sign";
|
||||
sign_request["algorithm"] = "ES256";
|
||||
sign_request["private_jwk"] = key["private_jwk"];
|
||||
sign_request["protected_header"]["typ"] = "JWT";
|
||||
sign_request["claims"]["iss"] = "https://client.example";
|
||||
DValue signed_jwt = crypto_operation(sign_request);
|
||||
print(signed_jwt["ok"].to_bool() ? "signed" : "failed", "\n");
|
||||
@@ -1,8 +1,18 @@
|
||||
# http_request
|
||||
|
||||
Performs a bounded outbound HTTP(S) request using the runtime `curl` binary. Request fields: `method`, `url`, `headers`, `body`, `timeout_ms`, `follow_redirects`.
|
||||
Performs an outbound HTTP(S) request using the runtime `curl` binary. Existing request fields remain backward-compatible: `method`, `url`, `headers`, `body`, `timeout_ms`, and `follow_redirects`.
|
||||
|
||||
Returns `{ status, headers, body, error }`. `headers` is a name/value map. A missing `curl` binary returns a clear `error` string.
|
||||
An absent `security` map, or a map containing none of these recognized keys, keeps the legacy request behavior. If a `security` map contains any recognized key, it selects hardening and **every** recognized field below must be explicitly boolean `true`; missing, partial, non-true, or `false` fields fail closed with `invalid_request`.
|
||||
|
||||
- `https_only` rejects non-HTTPS URLs, IP-literal hosts, and URL userinfo.
|
||||
- `public_dns_only` validates **every** DNS answer against the public IPv4 policy. IPv6 answers currently fail closed.
|
||||
- `pin_dns` pins curl to one validated answer with `--resolve`, retaining the URL hostname for TLS/SNI.
|
||||
- `isolated_curl` uses absolute `/usr/bin/curl`, `--disable` as argv[1], cleared environment, no proxy/config/netrc/HSTS/Alt-Svc inheritance, no redirects, a three-second connect bound, and a ten-second total bound.
|
||||
- `no_redirects` makes redirect following and `follow_redirects=true` invalid composition.
|
||||
|
||||
Hardened requests bound body input/output to 64 KiB and response headers to 8 KiB. Async hardened requests keep curl and its descendants in the job worker process group, so cancelling that job kills that group only. Methods are limited to `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, and `OPTIONS`; header names/values are validated and caller-controlled `Host`, framing, connection, and expectation headers are rejected. Sensitive request bodies go to curl stdin, never argv. Errors are typed non-secret values such as `invalid_request`, `unsafe_dns`, `timeout`, `response_too_large`, `redirect_not_allowed`, `http_status`, and `network_failure`.
|
||||
|
||||
Returns `{ status, headers, body, error }`.
|
||||
|
||||
:see
|
||||
>socket
|
||||
@@ -12,3 +22,10 @@ DValue req; req["method"] = "GET"; req["url"] = "http://127.0.0.1/doc/index.uce"
|
||||
req["headers"]["Host"] = "uce.openfu.com"; req["timeout_ms"] = (f64)2000;
|
||||
DValue resp = http_request(req);
|
||||
print("HTTP ", resp["status"].to_u64(), ", ", resp["body"].to_string().length(), " bytes returned\n");
|
||||
|
||||
:example
|
||||
// Hardened requests reject IP-literal hosts before they start curl.
|
||||
DValue hardened; hardened["method"]="GET"; hardened["url"]="https://127.0.0.1/";
|
||||
for(String key:{"https_only","public_dns_only","pin_dns","isolated_curl","no_redirects"}) hardened["security"][key].set_bool(true);
|
||||
DValue rejected=http_request(hardened);
|
||||
print(rejected["error"].to_string(), "\n");
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# http_request_async
|
||||
|
||||
Starts the same bounded curl-backed request as `http_request()` in the file-backed async job registry and returns a job id. Use `job_await()` or `job_result()` to retrieve the HTTP result.
|
||||
Starts the same request shape as `http_request()`, including its opt-in `security` hardening fields, in the file-backed async job registry and returns a job id. An unknown `security` object remains legacy; any recognized hardening key requires all five keys to be explicit `true` or the request fails closed. Hardened curl remains in this job's worker process group, so `job_cancel()` kills curl and its descendants without signalling other jobs. Use `job_await()` or `job_result()` to retrieve the HTTP result.
|
||||
|
||||
:see
|
||||
>socket
|
||||
|
||||
@@ -45,6 +45,7 @@ Options:
|
||||
|
||||
- `options["gfm"]` defaults to `true` and turns on GitHub-style extras such as tables, task lists, autolinks, and strikethrough.
|
||||
- `options["allow_html"]` defaults to `false`. When true, raw HTML blocks and inline tags may pass through as `raw_html` nodes instead of being escaped as plain text.
|
||||
- `options["heading_offset"]` shifts rendered heading levels while leaving AST `level` values unchanged. The default is `0`; final levels clamp to `h1` through `h6`. Use `1` when Markdown is nested below a component-owned page title.
|
||||
- `options["components"]` declares renderer extension points using normal UCE components.
|
||||
|
||||
Directive hook example:
|
||||
|
||||
@@ -13,6 +13,7 @@ return value : string containing the Memcache server's response
|
||||
Executes a raw command on an open Memcache connection and returns the server response as a string.
|
||||
|
||||
This is the low-level escape hatch for Memcache operations that are not covered by the dedicated helpers.
|
||||
UCE reads the complete framed ASCII response, including multi-value retrievals and `stats`, rather than returning a socket chunk. The complete write and read share a one-second deadline and responses are limited to 8 MiB. A timeout, malformed frame, or oversized response returns an empty string and makes that connection unusable so unread bytes cannot affect a later command. Commands ending in `noreply` and `quit` return immediately with an empty string.
|
||||
|
||||
:example
|
||||
u64 conn = memcache_connect();
|
||||
|
||||
@@ -1,23 +1,24 @@
|
||||
:sig
|
||||
MySQL* mysql_connect(String host = "localhost", String username = "root", String password = "")
|
||||
MySQL* mysql_connect(String host = "localhost", String username = "root", String password = "", String database = "")
|
||||
|
||||
:params
|
||||
host : host name of the MySQL server
|
||||
username : user name
|
||||
password : password
|
||||
database : optional initial database name
|
||||
return value : pointer to the MySQL connection struct
|
||||
|
||||
:see
|
||||
>mysql
|
||||
|
||||
:content
|
||||
Establishes a connection to a MySQL server and returns a pointer to the connection struct.
|
||||
Establishes a connection to a MySQL server and returns a request-owned pointer to the connection struct. When `database` is non-empty, the server selects it as part of connection establishment and restores it after every cross-request connection reset. A failed connection also returns a wrapper so that `mysql_error()` can report the failure; use `mysql_connected()` before issuing optional work that requires a live server handle.
|
||||
|
||||
This connection handle is then used with helpers such as `mysql_query()`, `mysql_error()`, and `mysql_disconnect()`.
|
||||
|
||||
MySQL handles are request-scoped framework resources. Repeated `mysql_connect()` calls with the same host and credentials reuse one server connection within the current request. Each call creates a lease and `mysql_disconnect()` releases that lease. At request cleanup UCE returns the server connection to the current worker's persistent pool, then resets it before cross-request reuse. The pool holds up to `MYSQL_PERSISTENT_POOL_SIZE` connections per worker (default 8); set the size to 0 to disable cross-request reuse. Never store a `MySQL*` in globals, sessions, or other state that can outlive the current request.
|
||||
MySQL handles are request-scoped framework resources. Repeated `mysql_connect()` calls with the same host, credentials, and database reuse one server connection within the current request. Database identity is part of both request-local and worker-persistent pool keys. Each call creates a lease and `mysql_disconnect()` releases that lease. At request cleanup UCE returns the server connection to the current worker's persistent pool, then resets it before cross-request reuse. The pool holds up to `MYSQL_PERSISTENT_POOL_SIZE` connections per worker (default 8); set the size to 0 to disable cross-request reuse. `MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS` retires idle entries at the first later request boundary (default 300, 0 keeps unlimited idle retention). A completely idle worker has no boundary and does not proactively close sockets. Never store a `MySQL*` in globals, sessions, or other state that can outlive the current request.
|
||||
|
||||
:example
|
||||
MySQL* db = mysql_connect();
|
||||
print(db != 0 ? "connected to MySQL" : "no MySQL server reachable with these credentials", "\n");
|
||||
print(mysql_connected(db) ? "connected to MySQL" : "no MySQL server reachable with these credentials", "\n");
|
||||
if(db != 0) mysql_disconnect(db);
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
:sig
|
||||
bool mysql_connected(MySQL* m)
|
||||
|
||||
:params
|
||||
m : pointer returned by mysql_connect()
|
||||
return value : true only when the wrapper owns a live server connection
|
||||
|
||||
:see
|
||||
>mysql
|
||||
|
||||
:content
|
||||
Tests whether a MySQL wrapper has a usable server connection. `mysql_connect()` retains a failed wrapper so that callers can retrieve its diagnostic through `mysql_error()`; this helper distinguishes that state from a live handle without consuming the diagnostic.
|
||||
|
||||
:example
|
||||
MySQL* db = mysql_connect();
|
||||
if(!mysql_connected(db))
|
||||
{
|
||||
print("database unavailable: ", mysql_error(db), "\n");
|
||||
}
|
||||
else print("database connected\n");
|
||||
if(db != 0) mysql_disconnect(db);
|
||||
@@ -16,6 +16,8 @@ Executes a MySQL query and returns the resulting data, if any.
|
||||
|
||||
`params` provides the query parameter values used by the statement. Use named `:name` placeholders only; positional `?` placeholders are rejected.
|
||||
|
||||
Ordinary placeholders are escaped and quoted as SQL string values. For grammar positions that require an unquoted non-negative integer, such as `LIMIT` and `OFFSET`, append `!` to the placeholder (`:limit!`). Unsigned placeholders fail before query execution unless their value is a non-empty sequence of decimal digits. They never accept signs, whitespace, expressions, identifiers, or other SQL fragments.
|
||||
|
||||
The result is returned as a `DValue`, which makes it easy to iterate through rows and read fields with the usual `DValue` accessors.
|
||||
|
||||
After an insert, update, or delete, use `mysql_affected_rows()` to inspect how many rows changed.
|
||||
@@ -24,7 +26,8 @@ After an insert, update, or delete, use `mysql_affected_rows()` to inspect how m
|
||||
MySQL* db = mysql_connect();
|
||||
if(db != 0)
|
||||
{
|
||||
DValue rows = mysql_query(db, "select 'ada@example.test' as email, 1 + 1 as total");
|
||||
StringMap params; params["limit"] = "1";
|
||||
DValue rows = mysql_query(db, "select 'ada@example.test' as email, 1 + 1 as total limit :limit!", params);
|
||||
String email = "none"; String total = "?";
|
||||
rows.each([&](DValue r, String key) { email = r["email"].to_string(); total = r["total"].to_string(); });
|
||||
print(email, " / total=", total, "\n");
|
||||
|
||||
@@ -11,17 +11,23 @@ return value : performance snapshot for the active request/workspace
|
||||
:content
|
||||
Returns a DValue with timing and process metadata such as worker pid, parent pid, request count, request start times, native dispatch, workspace setup and birth, context application, guest execution, and hostcall timing. `accept_us` is divided into `transport_params_us` (FastCGI begin through the end of parameters), `transport_input_us` (parameters through the end of input), and `handler_queue_us` (input close through handler entry). This distinguishes upstream request delivery from work inside the UCE handler.
|
||||
|
||||
Hostcall totals include component resolution; MySQL, memcache, and component resolution also expose their own count and microsecond fields. Workspace setup, birth, and context application expose matching wall and thread-CPU microseconds. `execution_cpu_us` is the remaining workspace thread CPU through the snapshot after those three phases. `mysql_operations` is an ordered, query-text-free list of up to 64 logical MySQL operations and their microsecond durations. A connect operation also identifies its source as `new`, cross-request `worker`, or same-request `request`; the corresponding `mysql_connection_open_count`, `mysql_connection_reuse_count`, and `mysql_request_pool_hit_count` fields provide totals. `mysql_operations_dropped` reports any overflow. The profiling hostcall itself is excluded from those totals so repeated snapshots remain comparable.
|
||||
Hostcall totals include component resolution. When `WASM_PROFILE_HOSTCALL_CPU=1`, `hostcall_cpu_us` separates thread CPU from hostcall wall time and each bounded `hostcall_operations` item includes the same `cpu_us` attribution. This diagnostic is off by default because sampling thread CPU around every hostcall adds hot-path cost; `hostcall_cpu_profiled` identifies whether the current request used it, and disabled counters remain zero. `WASM_PROFILE_THREAD_RUNTIME=1` adds one bounded kernel thread snapshot per request: user/system CPU, voluntary/involuntary context switches, minor/major faults, and start/end logical CPU plus migration state. `thread_runtime_profiled` identifies the opt-in diagnostic; its counters are absent when disabled. MySQL, memcache, and component resolution also expose their own count and microsecond fields. Workspace setup, birth, and context application expose matching wall and thread-CPU microseconds. `execution_cpu_us` is the remaining workspace thread CPU through the snapshot after those three phases. `unit_module_operations` is a source-root-relative list of up to 32 unit loads. Each item attributes module lookup/read/parse/build/classification plus allocation, import construction, symbol resolution, instantiation, and initialization; no caller-supplied paths are exposed. `mysql_operations` is an ordered, query-text-free list of up to 64 logical MySQL operations and their microsecond durations. A connect operation also identifies its source as `new`, cross-request `worker`, or same-request `request`; the corresponding `mysql_connection_open_count`, `mysql_connection_reuse_count`, and `mysql_request_pool_hit_count` fields provide totals. `mysql_operations_dropped` reports any overflow. The profiling hostcall itself is excluded from those totals so repeated snapshots remain comparable.
|
||||
|
||||
Component resolution is divided into `component_path_us`, `component_artifact_us`, `component_load_us`, and `component_link_us`. These aggregate path resolution, artifact readiness/freshness, Wasmtime side-module loading, and exported-handler lookup/table placement without exposing source paths.
|
||||
|
||||
Successful first loads within the request are counted by `unit_load_count` and divided into `unit_module_us`, `unit_allocate_us`, `unit_import_us`, `unit_instantiate_us`, and `unit_initialize_us`. These cover compiled-module lookup, guest memory/table allocation, import construction, Wasmtime instantiation, and relocations/constructors/request binding. Repeated handlers from an already loaded unit are excluded.
|
||||
Entry readiness reports inclusive `ready_freshness_us` plus `ready_source_generation_us`, `ready_freshness_full_check_us`, and `ready_freshness_cache_hit_count`. Read-only HTTP requests can reuse a positive result for at most ten seconds only while source generation and exact artifact identities remain unchanged. CLI, mutation, miss, expiry, and uncertain states perform the complete dependency check.
|
||||
|
||||
`unit_module_cache_hit_count` and `unit_module_cache_miss_count` divide module loads by the worker's compiled-module cache. A miss is further identified by `unit_module_serialized_cache_hit_count` when Wasmtime deserializes the current `.cwasm`; `unit_module_compile_count` means it fell back to compiling the `.wasm`. `unit_module_lookup_us`, `unit_module_read_us`, `unit_module_read_bytes`, `unit_module_parse_us`, `unit_module_compile_us`, and `unit_module_classify_us` divide `unit_module_us` into artifact stat/cache lookup, wasm metadata/full-artifact read volume, custom-section parse, deserialize-or-compile, and immutable import classification. A current serialized-module hit scans only section headers and the `dylink.0`/`uce.abi` payloads; compilation fallback reads the complete wasm. The phase sum can be below the total because allocation and cache publication overhead remain in the aggregate.
|
||||
Successful first loads within the request are counted by `unit_load_count` and divided into `unit_module_us`, `unit_allocate_us`, `unit_import_us`, `unit_instantiate_us`, and `unit_initialize_us`. These cover compiled-module lookup, guest memory/table allocation, import construction, Wasmtime instantiation, and relocations/constructors/request binding. `entry_unit_load_count` and `entry_unit_materialize_us` isolate the initial page/CLI unit; `dynamic_include_load_count` and `dynamic_include_materialize_us` isolate side units first requested through `component()`. The per-unit bounded list adds `kind` (`entry` or `component`) and `materialize_us`, the inclusive host-side time from module acquisition through request binding. Repeated handlers from an already loaded unit are excluded.
|
||||
|
||||
`unit_module_cache_hit_count` and `unit_module_cache_miss_count` divide module loads by the worker's compiled-module cache. A miss is further identified by `unit_module_serialized_cache_hit_count` when Wasmtime deserializes the current `.cwasm`; `unit_module_compile_count` means it fell back to compiling the `.wasm`. `unit_module_lookup_us`, `unit_module_read_us`, `unit_module_read_bytes`, `unit_module_read_count`, `unit_module_parse_us`, `unit_module_compile_us`, and `unit_module_classify_us` divide `unit_module_us` into artifact stat/cache lookup, wasm metadata/full-artifact read volume, positional reads, custom-section parse, deserialize-or-compile, and immutable import classification. Per-unit `unit_module_operations` expose the same `read_count`. On a serialized-module hit, `read_bytes` is the physical bounded read-ahead volume rather than only the logical selected bytes. The scanner skips code/data bodies and retains one bounded `dylink.0`, `uce.abi`, and `uce.module` section. It checks the absolute invocation deadline between sections and bounded 4 KiB positional reads, and verifies descriptor identity before and after scanning. Compilation fallback reads the complete wasm in deadline-checked 64 KiB positional chunks under the same identity guard. The phase sum can be below the total because allocation and cache publication overhead remain in the aggregate.
|
||||
|
||||
`unit_symbol_resolve_count` and `unit_symbol_resolve_us` isolate function and data symbol lookup within `unit_import_us`. The remainder of import time covers import-vector construction, Wasmtime Globals, GOT function table placement, and related bindings.
|
||||
|
||||
Wasm FastCGI workers retain up to `MYSQL_PERSISTENT_POOL_SIZE` credential-keyed MySQL connections (default `8`; set `0` to disable). UCE calls the client library's connection-reset operation before another request receives a cached connection, clearing transactions, temporary tables, session variables, and selected databases while avoiding a new authentication handshake. Same-request leases continue to share state until request cleanup.
|
||||
The request log's `wasm-ready`, `wasm`, `workspace`, `invoke`, `collect`, and `post` fields complete the timing boundary after this in-page snapshot: compiled-artifact readiness before backend entry, native backend wall time, complete workspace wall time, entry invocation, guest-output/meta collection, and native response assembly after the backend. They do not expose request data. This distinction is useful when a completed FastCGI request takes longer than the page's mid-render `request_perf()` snapshot.
|
||||
|
||||
Workspace birth is subdivided into `birth_policy_us`, `birth_import_us`, `birth_instantiate_us`, `birth_exports_us`, and `birth_initialize_us`. Request-context transfer reports `context_bytes`, `context_encode_us`, `context_allocate_us`, `context_write_us`, `context_guest_apply_us`, and `context_free_us`. These bounded aggregate fields expose sizes and timing only, never request values.
|
||||
|
||||
Wasm FastCGI workers retain up to `MYSQL_PERSISTENT_POOL_SIZE` credential-keyed MySQL connections (default `8`; set `0` to disable). `MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS` retires an idle entry at the first later request boundary (default `300`; `0` disables age eviction). UCE calls the client library's connection-reset operation before another request receives a cached connection, clearing transactions, temporary tables, session variables, and selected databases while avoiding a new authentication handshake. Same-request leases continue to share state until request cleanup.
|
||||
|
||||
:example
|
||||
DValue perf = request_perf();
|
||||
|
||||
@@ -21,7 +21,7 @@ Starts a repeating background worker process.
|
||||
|
||||
If a process with the same `key` is already running anywhere in the runtime instance, `task_repeat()` does not start a second worker and instead returns the PID of the existing one. Coordination is through the same shared task state used by `task()`.
|
||||
|
||||
`timeout` bounds the lifetime of the repeating worker. The default is ten minutes. Pass `0` only for workers that have another shutdown path.
|
||||
`timeout` bounds the lifetime of the repeating worker. The default is ten minutes. Pass `0` only for workers that have another shutdown path. In Wasm, each individual callback invocation remains bounded by the runtime invocation timeout even when the repeating process lifetime is unbounded.
|
||||
|
||||
:example
|
||||
task_repeat("doc-demo-repeat", 60.0, []() { usleep(10000); });
|
||||
|
||||
@@ -14,7 +14,8 @@ to_bool
|
||||
int_val
|
||||
|
||||
:content
|
||||
Parses a trimmed string as a base-10 unsigned integer.
|
||||
Parses a trimmed string as a base-10 unsigned integer. An optional leading `+`
|
||||
is accepted. Negative and overflowing values return the fallback.
|
||||
|
||||
Unlike `int_val()`, the whole trimmed string must be consumed by the parser. Empty strings and partially parsed values such as `"42px"` return the fallback.
|
||||
|
||||
|
||||
@@ -18,6 +18,8 @@ Calls another UCE file and executes its `RENDER(Request& context)` function.
|
||||
|
||||
If `context` is omitted, the current active request context is used.
|
||||
|
||||
Calling a missing unit or render handler is a server error and sets HTTP status 500. If a unit is intentionally optional, inspect it with `unit_info()` before calling `unit_render()`.
|
||||
|
||||
Examples:
|
||||
|
||||
|
||||
|
||||
@@ -33,3 +33,9 @@ The example uses query-string routing in the same style as the PHP starter, but
|
||||
Direct requests to `/examples/uce-starter/index.uce` still work, but self-links are canonicalized back to `/examples/uce-starter/`.
|
||||
|
||||
The demo account pages use a small file-backed user store under `/tmp/uce-starter-data/` with session-based login state.
|
||||
|
||||
|
||||
## Optional Datastar Assets
|
||||
|
||||
This starter vendors Datastar v1.0.2 as `js/datastar.js` for future server-rendered interactive islands. It is not loaded by the default page shell. `lib/datastar.uce` contains small Datastar SSE formatting helpers, and `views/datastar-example.uce` is an unlinked opt-in example route.
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,38 @@
|
||||
String datastar_data_lines(String field, String value)
|
||||
{
|
||||
String out = "";
|
||||
for(String line : split(value, "\n"))
|
||||
out += "data: " + field + " " + line + "\n";
|
||||
return(out);
|
||||
}
|
||||
|
||||
void datastar_sse_headers(Request& context)
|
||||
{
|
||||
context.call["app"]["page_type"] = "blank";
|
||||
context.header["Content-Type"] = "text/event-stream; charset=utf-8";
|
||||
context.header["Cache-Control"] = "no-cache, no-store, must-revalidate";
|
||||
context.header["X-Accel-Buffering"] = "no";
|
||||
}
|
||||
|
||||
String datastar_event(String event_name, StringMap fields)
|
||||
{
|
||||
String out = "event: " + event_name + "\n";
|
||||
for(const auto& field : fields)
|
||||
out += datastar_data_lines(field.first, field.second);
|
||||
out += "\n";
|
||||
return(out);
|
||||
}
|
||||
|
||||
String datastar_patch_elements(String html)
|
||||
{
|
||||
StringMap fields;
|
||||
fields["elements"] = html;
|
||||
return(datastar_event("datastar-patch-elements", fields));
|
||||
}
|
||||
|
||||
String datastar_patch_signals(String signals_json)
|
||||
{
|
||||
StringMap fields;
|
||||
fields["signals"] = signals_json;
|
||||
return(datastar_event("datastar-patch-signals", fields));
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
#load "../lib/app.uce"
|
||||
#load "../lib/datastar.uce"
|
||||
|
||||
COMPONENT(Request& context)
|
||||
{
|
||||
if(context.get["stream"] == "1")
|
||||
{
|
||||
datastar_sse_headers(context);
|
||||
print(datastar_patch_elements("<div id=\"datastar-example-result\">Server-rendered UCE Datastar fragment</div>"));
|
||||
return;
|
||||
}
|
||||
|
||||
<>
|
||||
<h1>Datastar optional example</h1>
|
||||
<p>Datastar is vendored beside the starter client libraries for future server-rendered islands. It is not loaded globally.</p>
|
||||
<script type="module" src="<?= app_asset_url("js/datastar.js", context) ?>"></script>
|
||||
<button class="btn" data-on:click="@get('<?= app_link("datastar-example", context) ?>&stream=1')">Patch a fragment</button>
|
||||
<div id="datastar-example-result">Waiting for a server patch.</div>
|
||||
</>;
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -20,6 +20,37 @@ CLI(Request& context)
|
||||
print(cli_arg(context, "missing", "fallback"), "\n");
|
||||
return;
|
||||
}
|
||||
if(action == "task_repeat_unbounded")
|
||||
{
|
||||
String marker = "/tmp/uce-task-repeat-unbounded.txt";
|
||||
file_unlink(marker);
|
||||
pid_t pid = task_repeat("uce-task-repeat-unbounded", 0.05, [marker]() { file_put_contents(marker, "ran"); }, 0);
|
||||
usleep(200000);
|
||||
bool ran = file_get_contents(marker) == "ran";
|
||||
if(pid > 0) task_kill(pid, 15);
|
||||
file_unlink(marker);
|
||||
print(pid > 0 && ran ? "task repeat unbounded ok\n" : "task repeat unbounded failed\n");
|
||||
return;
|
||||
}
|
||||
if(action == "mysql_unsigned_params")
|
||||
{
|
||||
MySQL db;
|
||||
StringMap params; params["limit"] = "1";
|
||||
String parsed = db.parse_query_parameters("SELECT 7 LIMIT :limit!", params);
|
||||
StringMap comparison; comparison["left"] = "7"; comparison["right"] = "8";
|
||||
String inequality = db.parse_query_parameters("SELECT :left!=:right", comparison);
|
||||
StringMap empty_name; empty_name[""] = "1";
|
||||
db.parameter_error = false; db.statement_info = "";
|
||||
db.parse_query_parameters("SELECT 1 LIMIT :!", empty_name);
|
||||
bool empty_rejected = db.parameter_error;
|
||||
StringMap invalid; invalid["limit"] = "1;SELECT 9";
|
||||
db.parameter_error = false;
|
||||
db.statement_info = "";
|
||||
db.parse_query_parameters("SELECT 1 LIMIT :limit!", invalid);
|
||||
bool ok = parsed.find("LIMIT 1") != String::npos && inequality.find("'7'!='8'") != String::npos && empty_rejected && db.parameter_error && db.statement_info.find("must contain only decimal digits") != String::npos;
|
||||
print(ok ? "mysql unsigned params ok\n" : "mysql unsigned params failed: parsed=" + parsed + ", parameter_error=" + (db.parameter_error ? "1" : "0") + ", statement=" + db.statement_info + "\n");
|
||||
return;
|
||||
}
|
||||
context.set_status(400, "CLI Error");
|
||||
print("unknown cli action: ", action, "\n");
|
||||
}
|
||||
|
||||
@@ -149,6 +149,7 @@ StringList cli_demo_error_markers()
|
||||
markers.push_back(String("uce compile ") + "error");
|
||||
markers.push_back(String("fatal signal during ") + "request");
|
||||
markers.push_back(String("uncaught exception during ") + "request");
|
||||
markers.push_back("component not found");
|
||||
return(markers);
|
||||
}
|
||||
|
||||
@@ -196,7 +197,7 @@ bool cli_doc_example_is_placeholder(String source)
|
||||
return(false);
|
||||
}
|
||||
|
||||
void cli_run_doc_pages_gate()
|
||||
void cli_run_doc_pages_gate(u64 shard = 0, u64 shard_count = 1)
|
||||
{
|
||||
StringList error_markers;
|
||||
error_markers.push_back("doc example error");
|
||||
@@ -206,18 +207,23 @@ void cli_run_doc_pages_gate()
|
||||
error_markers.push_back(String("uncaught exception during ") + "request");
|
||||
error_markers.push_back("timed out acquiring compile lock");
|
||||
|
||||
for(String file_name : ls("../doc/pages/"))
|
||||
u64 page_index = 0;
|
||||
for(String file_name : ls("../doc/pages/").sort())
|
||||
{
|
||||
String source = file_get_contents("../doc/pages/" + file_name);
|
||||
String page = nibble(file_name, ".");
|
||||
if(page == "")
|
||||
continue;
|
||||
if(page_index++ % shard_count != shard)
|
||||
continue;
|
||||
bool has_example = cli_doc_source_has_example(source);
|
||||
String path = "/doc/index.uce?p=" + uri_encode(page);
|
||||
CliHttpResponse res;
|
||||
String summary;
|
||||
bool ok = false;
|
||||
for(u64 attempt = 0; attempt < 3; attempt++)
|
||||
// Frontend GETs may serve the last complete nested component while its
|
||||
// prioritized replacement compiles; cover a normal cold WASI compile.
|
||||
for(u64 attempt = 0; attempt < 12; attempt++)
|
||||
{
|
||||
if(attempt > 0)
|
||||
usleep(500000);
|
||||
@@ -326,11 +332,31 @@ void cli_run_site_suite(bool skip_local_service_pages = false)
|
||||
|
||||
void cli_run_security_smoke()
|
||||
{
|
||||
CliHttpResponse missing_unit = cli_frontend("/tests/unit_failure.uce");
|
||||
cli_test_case("uce_unit_failure:unit_render missing target is HTTP 500",
|
||||
missing_unit.status == 500 && cli_contains(missing_unit.body, "unit not found: does-not-exist.uce"),
|
||||
"status=" + std::to_string(missing_unit.status) + " body=" + cli_truncate(missing_unit.body));
|
||||
|
||||
CliHttpResponse missing_component = cli_frontend("/tests/component_failure.uce");
|
||||
cli_test_case("uce_component_failure:component_render missing target is HTTP 500",
|
||||
missing_component.status == 500 && cli_contains(missing_component.body, "component not found: components/does-not-exist"),
|
||||
"status=" + std::to_string(missing_component.status) + " body=" + cli_truncate(missing_component.body));
|
||||
|
||||
CliHttpResponse missing_captured_component = cli_frontend("/tests/component_failure.uce?mode=capture");
|
||||
cli_test_case("uce_component_failure:component missing target is HTTP 500",
|
||||
missing_captured_component.status == 500 && cli_contains(missing_captured_component.body, "component not found: components/does-not-exist"),
|
||||
"status=" + std::to_string(missing_captured_component.status) + " body=" + cli_truncate(missing_captured_component.body));
|
||||
|
||||
CliHttpResponse optional_component = cli_frontend("/tests/component_failure.uce?mode=optional");
|
||||
cli_test_case("uce_component_failure:component_exists keeps optional lookup non-fatal",
|
||||
optional_component.status == 200 && optional_component.body == "optional component absent",
|
||||
"status=" + std::to_string(optional_component.status) + " body=" + cli_truncate(optional_component.body));
|
||||
|
||||
CliHttpResponse dotdot = cli_direct_http("/../site/demo/hello.uce");
|
||||
cli_test_case("uce_security_smoke:direct HTTP rejects dot-dot script traversal", dotdot.status != 200 && !cli_contains(dotdot.body, "hello world"), "direct HTTP dot-dot traversal returned HTTP " + std::to_string(dotdot.status));
|
||||
cli_test_case("uce_security_smoke:direct HTTP rejects dot-dot script traversal", dotdot.status == 404 && !cli_contains(dotdot.body, "hello world"), "direct HTTP dot-dot traversal returned HTTP " + std::to_string(dotdot.status));
|
||||
|
||||
CliHttpResponse spoof = cli_direct_http("/no-such-script.uce", "Script-Filename: /Code/uce.openfu.com/uce/site/demo/hello.uce\r\n");
|
||||
cli_test_case("uce_security_smoke:direct HTTP ignores Script-Filename header", spoof.status != 200 && !cli_contains(spoof.body, "hello world"), "direct HTTP Script-Filename override returned HTTP " + std::to_string(spoof.status));
|
||||
cli_test_case("uce_security_smoke:direct HTTP ignores Script-Filename header", spoof.status == 404 && !cli_contains(spoof.body, "hello world"), "direct HTTP Script-Filename override returned HTTP " + std::to_string(spoof.status));
|
||||
|
||||
CliHttpResponse sanitizer = cli_frontend("/tests/security_headers.uce");
|
||||
String headers_lower = to_lower(sanitizer.headers);
|
||||
@@ -381,6 +407,33 @@ void cli_run_task_lifetime_smoke()
|
||||
cli_test_case("uce_task_lifetime:delayed callback after request return", res.status == 200 && cli_contains(res.body, "late task scheduled") && cli_contains(observed, "late callback ran"), "status=" + std::to_string(res.status) + " observed=" + cli_truncate(observed));
|
||||
}
|
||||
|
||||
void cli_run_pool_isolation()
|
||||
{
|
||||
DValue worker_counts;
|
||||
bool clean = true;
|
||||
String detail;
|
||||
for(u64 i = 0; i < 64; i++)
|
||||
{
|
||||
String token = "request-" + std::to_string(i);
|
||||
CliHttpResponse res = cli_frontend("/tests/pool-isolation.uce?token=" + token);
|
||||
StringList parts = split(trim(res.body), "|");
|
||||
bool current = res.status == 200 && parts.size() == 3 &&
|
||||
parts[0] == "fresh" && parts[1] == token && int_val(parts[2]) > 0;
|
||||
if(!current && detail == "")
|
||||
detail = "request=" + std::to_string(i) + " status=" + std::to_string(res.status) + " body=" + cli_truncate(res.body);
|
||||
clean = clean && current;
|
||||
if(parts.size() == 3)
|
||||
worker_counts[parts[2]] = worker_counts[parts[2]].to_u64() + 1;
|
||||
}
|
||||
bool reused = false;
|
||||
worker_counts.each([&](DValue count, String) {
|
||||
if(count.to_u64() > 1)
|
||||
reused = true;
|
||||
});
|
||||
cli_test_case("uce_pool_isolation:recycled slots reset request state", clean && reused,
|
||||
clean && reused ? "64 requests retained fresh globals, constructors, ONCE, context, and worker identity" : first(detail, "no worker served a repeated request"));
|
||||
}
|
||||
|
||||
void cli_run_tcp_smoke()
|
||||
{
|
||||
u64 fd80 = socket_connect("127.0.0.1", 80);
|
||||
@@ -391,6 +444,11 @@ void cli_run_tcp_smoke()
|
||||
cli_test_case("uce_tcp_smoke:http websocket port 8080", fd8080 != 0, "TCP connect " + String(fd8080 != 0 ? "succeeded" : "failed") + " on port 8080");
|
||||
if(fd8080 != 0)
|
||||
socket_close(fd8080);
|
||||
CliHttpResponse direct = cli_direct_http("/tests/pool-isolation.uce?token=raw-http-framing");
|
||||
StringList direct_parts = split(trim(direct.body), "|");
|
||||
bool one_response = direct.status == 200 && direct_parts.size() == 3 && direct_parts[0] == "fresh" && direct_parts[1] == "raw-http-framing" && direct.body.find("HTTP/1.1 ") == String::npos;
|
||||
cli_test_case("uce_tcp_smoke:raw HTTP dynamic response is framed once", one_response,
|
||||
one_response ? "single status/header block and exact dynamic body" : "status=" + std::to_string(direct.status) + " body=" + cli_truncate(direct.body));
|
||||
}
|
||||
|
||||
void cli_run_wasm_kill(bool include_kill)
|
||||
@@ -410,7 +468,7 @@ void cli_run_wasm_kill(bool include_kill)
|
||||
void cli_print_list(bool include_kill)
|
||||
{
|
||||
print("UCE CLI test groups:\n");
|
||||
print(" demo, http, site, doc-gate, security, task-lifetime, starter, tcp");
|
||||
print(" demo, http, site, doc-gate-1..30, security, task-lifetime, pool-isolation, starter, tcp");
|
||||
if(include_kill)
|
||||
print(", wasm-kill");
|
||||
print("\n");
|
||||
@@ -425,12 +483,22 @@ bool cli_run_group(String group, bool skip_local_service_pages, bool include_kil
|
||||
cli_run_http_smoke();
|
||||
else if(group == "site")
|
||||
cli_run_site_suite(skip_local_service_pages);
|
||||
else if(group == "doc-gate")
|
||||
cli_run_doc_pages_gate();
|
||||
else if(str_starts_with(group, "doc-gate-"))
|
||||
{
|
||||
u64 shard = int_val(group.substr(9));
|
||||
if(shard < 1 || shard > 30)
|
||||
{
|
||||
cli_test_case("uce_cli_runner:known doc gate shard", false, "unknown group: " + group);
|
||||
return(false);
|
||||
}
|
||||
cli_run_doc_pages_gate(shard - 1, 30);
|
||||
}
|
||||
else if(group == "security")
|
||||
cli_run_security_smoke();
|
||||
else if(group == "task-lifetime")
|
||||
cli_run_task_lifetime_smoke();
|
||||
else if(group == "pool-isolation")
|
||||
cli_run_pool_isolation();
|
||||
else if(group == "starter")
|
||||
cli_run_starter_parity();
|
||||
else if(group == "tcp")
|
||||
@@ -469,7 +537,11 @@ CLI(Request& context)
|
||||
cli_run_group(group, input["skip_local_service_pages"].to_bool(), include_kill);
|
||||
else
|
||||
{
|
||||
for(String item : { "demo", "http", "site", "doc-gate", "security", "task-lifetime", "starter", "tcp" })
|
||||
for(String item : { "demo", "http", "site" })
|
||||
cli_run_group(item, input["skip_local_service_pages"].to_bool(), include_kill);
|
||||
for(u64 shard = 1; shard <= 30; shard++)
|
||||
cli_run_group("doc-gate-" + std::to_string(shard), input["skip_local_service_pages"].to_bool(), include_kill);
|
||||
for(String item : { "security", "task-lifetime", "pool-isolation", "starter", "tcp" })
|
||||
cli_run_group(item, input["skip_local_service_pages"].to_bool(), include_kill);
|
||||
cli_run_group("wasm-kill", input["skip_local_service_pages"].to_bool(), include_kill);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
RENDER(Request& context)
|
||||
{
|
||||
if(context.get["mode"] == "optional")
|
||||
print(component_exists("components/does-not-exist") ? "unexpected" : "optional component absent");
|
||||
else if(context.get["mode"] == "capture")
|
||||
print(component("components/does-not-exist", context));
|
||||
else
|
||||
component_render("components/does-not-exist", context);
|
||||
}
|
||||
@@ -55,6 +55,13 @@ RENDER(Request& context)
|
||||
check("component_render() named handler", footer_markup.find("Named footer render") != String::npos, footer_markup);
|
||||
check("component(\"unit:NAME\") named handler", header_markup.find("Component Output") != String::npos && header_markup.find("This body comes from component()") == String::npos, header_markup);
|
||||
check("ob_start() / ob_get_close()", buffer_markup == "buffered-text", buffer_markup);
|
||||
DValue perf = request_perf();
|
||||
String unit_operations = json_encode(perf["unit_module_operations"]);
|
||||
check(
|
||||
"request_perf() separates entry code from dynamic component materialization",
|
||||
perf["entry_unit_load_count"].to_u64() == 1 && perf["entry_unit_materialize_us"].to_u64() > 0 && perf["dynamic_include_load_count"].to_u64() > 0 && perf["dynamic_include_materialize_us"].to_u64() > 0 && perf["component_loaded_reuse_count"].to_u64() > 0 && unit_operations.find("\"kind\": \"entry\"") != String::npos && unit_operations.find("\"kind\": \"component\"") != String::npos && unit_operations.find("\"materialize_us\"") != String::npos,
|
||||
unit_operations
|
||||
);
|
||||
|
||||
?><div class="tests-section">
|
||||
<h3>Rendered Component</h3>
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
COMPONENT(Request& context)
|
||||
{
|
||||
DValue& node = context.props["node"];
|
||||
<><div class="tests-heading" data-hook="heading" data-source-level="<?= std::to_string(node["level"].to_u64()) ?>"><?: context.props["default_html"].to_string() ?></div></>
|
||||
}
|
||||
+28
-3
@@ -31,7 +31,7 @@ RENDER(Request& context)
|
||||
StringMap colon_uri_headers = split_http_headers("GET /clock.uce?t=12:30 HTTP/1.1\r\nHost: colon.example\r\n");
|
||||
check("split_http_headers() request line with colon in URI", colon_uri_headers["REQUEST_METHOD"] == "GET" && colon_uri_headers["DOCUMENT_URI"] == "/clock.uce" && colon_uri_headers["QUERY_STRING"] == "t=12:30" && colon_uri_headers["HTTP_HOST"] == "colon.example", var_dump(colon_uri_headers));
|
||||
check("trim() / split_kv() / split_http_headers()", trim(" padded value ") == "padded value" && kv["alpha"] == "one" && kv["empty"] == "" && http_headers["REQUEST_METHOD"] == "GET" && http_headers["DOCUMENT_URI"] == "/demo.uce" && http_headers["QUERY_STRING"] == "x=1" && http_headers["HTTP_X_EMPTY"] == "" && leading_crlf_headers["REQUEST_METHOD"] == "GET" && leading_crlf_headers["DOCUMENT_URI"] == "/lead.uce" && leading_crlf_headers["HTTP_HOST"] == "lead.example" && header_only["REQUEST_METHOD"] == "" && header_only["HTTP_HOST"] == "example.test" && header_only["HTTP_X_TOKEN"] == "abc", trim(" padded value ") + " / " + var_dump(kv) + " / " + var_dump(http_headers) + " / " + var_dump(leading_crlf_headers) + " / " + var_dump(header_only));
|
||||
check("replace()", replace("alpha-beta-beta", "beta", "done") == "alpha-done-done", replace("alpha-beta-beta", "beta", "done"));
|
||||
check("replace()", replace("alpha-beta-beta", "beta", "done") == "alpha-done-done" && replace("hello", "", "X") == "hello", replace("alpha-beta-beta", "beta", "done"));
|
||||
check("html_escape() attribute-safe quotes", html_escape("<&>\"Don't") == "<&>"Don't", html_escape("<&>\"Don't"));
|
||||
check("regex_match()", regex_match("[A-Z][a-z]+", "Alice") && !regex_match("[A-Z][a-z]+", "Alice!"), "full-string validation");
|
||||
|
||||
@@ -51,6 +51,8 @@ RENDER(Request& context)
|
||||
check("str_ends_with()", str_ends_with("component.uce", ".uce"), "component.uce ends with .uce");
|
||||
check("to_lower() / to_upper()", to_lower("MiXeD") == "mixed" && to_upper("MiXeD") == "MIXED", to_lower("MiXeD") + " / " + to_upper("MiXeD"));
|
||||
check("request context params", context.params["SCRIPT_URL"] != "" && context.params["BASE_URL"] != "" && context.params["ROUTE_PATH"] != "" && context.params["ROUTE_PAGE"] != "" && context.params["ROUTE_VALID"] == "1", "script=" + context.params["SCRIPT_URL"] + " base=" + context.params["BASE_URL"] + " route=" + context.params["ROUTE_PATH"] + " page=" + context.params["ROUTE_PAGE"] + " valid=" + context.params["ROUTE_VALID"]);
|
||||
check("request transport stays out of context.call", !context.call.has("params") && !context.call.has("get") && !context.call.has("post") && !context.call.has("cookies") && !context.call.has("session") && !context.call.has("entry_unit") && !context.call.has("in"), json_encode(context.call));
|
||||
check("request server configuration", context.server && context.server->config["BIN_DIRECTORY"] != "" && context.server->config["SITE_DIRECTORY"] != "" && context.server->config["SESSION_PATH"] != "" && context.server->config["WASM_CORE_PATH"] != "", "bin=" + context.server->config["BIN_DIRECTORY"] + " site=" + context.server->config["SITE_DIRECTORY"] + " session=" + context.server->config["SESSION_PATH"] + " core=" + context.server->config["WASM_CORE_PATH"]);
|
||||
String saved_query_string = context.params["QUERY_STRING"];
|
||||
context.params["QUERY_STRING"] = "workspace/projects&theme=dark";
|
||||
check("request_query_path() delegates to request_query_route()", request_query_path(context) == request_query_route(context)["l_path"].to_string() && request_query_path(context) == "workspace/projects", request_query_path(context) + " / " + json_encode(request_query_route(context)));
|
||||
@@ -86,6 +88,16 @@ RENDER(Request& context)
|
||||
DValue nav_dash_summary = nav_dash.filter({"title"});
|
||||
check("DValue collection helpers", empty_pop.to_string() == "" && app_nav.is_list() && nav_titles["0"].to_string() == "Dashboard" && join(nav_dash_summary.keys(), ",") == "title" && nav_dash.values()["0"].to_string() == "app" && nav_dash.values()["1"].to_string() == "Dashboard", json_encode(nav_titles));
|
||||
|
||||
DValue copy_source; copy_source["nested"]["value"] = "original";
|
||||
DValue copied = copy_source;
|
||||
copied["nested"]["value"] = "copy";
|
||||
DValue move_source; move_source["nested"]["value"] = "moved";
|
||||
DValue moved = std::move(move_source);
|
||||
DValue reference_target; DValue reference; reference.set_reference(&reference_target);
|
||||
DValue reference_source; reference_source["nested"]["value"] = "target";
|
||||
reference = reference_source;
|
||||
check("DValue copy/move assignment preserves value and reference semantics", copy_source["nested"]["value"].to_string() == "original" && copied["nested"]["value"].to_string() == "copy" && moved["nested"]["value"].to_string() == "moved" && reference_target["nested"]["value"].to_string() == "target", json_encode(moved) + " / " + json_encode(reference_target));
|
||||
|
||||
DValue conv;
|
||||
conv["name"] = "ada";
|
||||
conv["count"] = "12";
|
||||
@@ -130,6 +142,11 @@ RENDER(Request& context)
|
||||
DValue uceb_decoded;
|
||||
bool uceb_ok = ucb_decode(uceb_encoded, uceb_decoded, &uceb_error);
|
||||
check("UCEB2 DValue codec round-trip preserves scalar types", uceb_ok && uceb_decoded["name"].to_string() == "phase1" && uceb_decoded["name"].get_type_name() == "String" && uceb_decoded["nested"]["answer"].to_string() == "42" && uceb_decoded["float"].get_type_name() == "f64" && uceb_decoded["float"].to_f64() > 0.00000009 && uceb_decoded["float"].to_f64() < 0.00000011 && uceb_decoded["bool"].get_type_name() == "bool" && uceb_decoded["bool"].to_bool() && uceb_decoded["items"].is_list() && uceb_decoded["items"]["1"].to_string() == "second" && uceb_decoded["empty_list"].is_list() && uceb_decoded["binary"].to_string().size() == uceb_binary.size() && uceb_decoded["binary"].to_string() == uceb_binary && uceb_decoded["pointer"].to_string() == "", "bytes=" + std::to_string((u64)uceb_encoded.size()) + " error=" + uceb_error + " float=" + uceb_decoded["float"].to_string() + " types=" + uceb_decoded["name"].get_type_name() + "/" + uceb_decoded["float"].get_type_name() + "/" + uceb_decoded["bool"].get_type_name());
|
||||
StringMap uceb_flat_source = {{"alpha", "one"}, {"binary", uceb_binary}, {"empty", ""}};
|
||||
DValue uceb_flat_decoded;
|
||||
String uceb_flat_error;
|
||||
bool uceb_flat_ok = ucb_decode(ucb_encode_flat_string_map(uceb_flat_source), uceb_flat_decoded, &uceb_flat_error);
|
||||
check("UCEB2 flat string-map codec", uceb_flat_ok && uceb_flat_decoded.to_stringmap() == uceb_flat_source, uceb_flat_error + " / " + json_encode(uceb_flat_decoded));
|
||||
|
||||
DValue uceb_invalid;
|
||||
String uceb_bad_magic = "NOPE";
|
||||
@@ -195,11 +212,19 @@ RENDER(Request& context)
|
||||
String decoded_base64 = base64_decode(encoded_base64, base64_ok);
|
||||
bool invalid_base64_ok = true;
|
||||
base64_decode("AA=A", invalid_base64_ok);
|
||||
check("base64_encode() / base64_decode() binary-safe", base64_ok && decoded_base64 == binary_payload && decoded_base64.size() == binary_payload.size() && !invalid_base64_ok, encoded_base64 + " bytes=" + std::to_string((u64)decoded_base64.size()));
|
||||
bool early_padding_base64_ok = true;
|
||||
base64_decode("A=AA", early_padding_base64_ok);
|
||||
check("base64_encode() / base64_decode() binary-safe", base64_ok && decoded_base64 == binary_payload && decoded_base64.size() == binary_payload.size() && !invalid_base64_ok && !early_padding_base64_ok, encoded_base64 + " bytes=" + std::to_string((u64)decoded_base64.size()));
|
||||
|
||||
String utf8_sample = "A\xC3\xA9";
|
||||
auto utf8_parts = split_utf8(utf8_sample);
|
||||
check("split_utf8()", utf8_parts.size() == 2, "count=" + std::to_string(utf8_parts.size()));
|
||||
String truncated_utf8 = "abc";
|
||||
truncated_utf8.push_back((char)0xE2);
|
||||
truncated_utf8.push_back((char)0x82);
|
||||
auto truncated_utf8_parts = split_utf8(truncated_utf8);
|
||||
String lone_utf8_lead(1, (char)0xF0);
|
||||
auto compound_lone_utf8_parts = split_utf8(lone_utf8_lead, true);
|
||||
check("split_utf8()", utf8_parts.size() == 2 && truncated_utf8_parts.size() == 4 && truncated_utf8_parts[3].size() == 2 && compound_lone_utf8_parts.size() == 1 && compound_lone_utf8_parts[0] == lone_utf8_lead, "count=" + std::to_string(utf8_parts.size()) + " truncated=" + std::to_string(truncated_utf8_parts.size()));
|
||||
|
||||
DValue payload;
|
||||
payload["name"] = "uce";
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
#include "testlib.h"
|
||||
|
||||
RENDER(Request& context)
|
||||
{
|
||||
u64 passed = 0;
|
||||
u64 failed = 0;
|
||||
u64 skipped = 0;
|
||||
auto check = [&](String name, bool ok, String detail)
|
||||
{
|
||||
site_tests_case(name, ok ? "pass" : "fail", detail);
|
||||
if(ok) passed++; else failed++;
|
||||
};
|
||||
site_tests_page_start("Structured crypto operations", "Algorithm-selected P-256 JWK creation and ES256 JWT signing.");
|
||||
DValue key_request;
|
||||
key_request["operation"] = "key_generate";
|
||||
key_request["algorithm"] = "ES256";
|
||||
DValue key = crypto_operation(key_request);
|
||||
DValue header;
|
||||
header["alg"] = "none";
|
||||
header["kid"] = key["kid"];
|
||||
DValue claims;
|
||||
claims["iss"] = "https://client.example";
|
||||
claims["sub"] = "client.example";
|
||||
DValue sign_request;
|
||||
sign_request["operation"] = "jwt_sign";
|
||||
sign_request["algorithm"] = "ES256";
|
||||
sign_request["private_jwk"] = key["private_jwk"];
|
||||
sign_request["protected_header"] = header;
|
||||
sign_request["claims"] = claims;
|
||||
DValue signed_result = crypto_operation(sign_request);
|
||||
String jwt = signed_result["jwt"].to_string();
|
||||
StringList parts = split(jwt, ".");
|
||||
String header_b64 = parts.size() == 3 ? parts[0] : "";
|
||||
while(header_b64.size() % 4) header_b64 += "=";
|
||||
String decoded_header = base64_decode(header_b64);
|
||||
DValue wrong_curve = key["private_jwk"];
|
||||
wrong_curve["crv"] = "P-384";
|
||||
DValue malformed = key["private_jwk"];
|
||||
malformed["x"] = "not_base64url=";
|
||||
DValue wrong_request = sign_request;
|
||||
wrong_request["private_jwk"] = wrong_curve;
|
||||
DValue malformed_request = sign_request;
|
||||
malformed_request["private_jwk"] = malformed;
|
||||
DValue cose_request;
|
||||
cose_request["operation"] = "cose_es256_parse";
|
||||
cose_request["algorithm"] = "ES256";
|
||||
cose_request["cose_key_base64url"] = "pQECAyYgASFYIGsX0fLhLEJH-Lzm5WOkQPJ3A32BLeszoPShOUXYmMKWIlggT-NC4v4af5uO5-tKfA-eFivOM1drMV7Oy7ZAaDe_UfU";
|
||||
DValue cose = crypto_operation(cose_request);
|
||||
DValue verify_request;
|
||||
verify_request["operation"] = "es256_verify";
|
||||
verify_request["algorithm"] = "ES256";
|
||||
verify_request["cose_key_base64url"] = cose_request["cose_key_base64url"];
|
||||
verify_request["message_base64url"] = "d2ViYXV0aG4gZml4ZWQgbWVzc2FnZQ";
|
||||
verify_request["signature_der_base64url"] = "MEUCIQCkatZK1VVsjk17uvyzyhjdAkMNWXPjxSOMqWcjmM_8XAIgDaSk3Qufyd0_6r9Dm9A8RQbFco-FdTBulq7bvRGoBC4";
|
||||
DValue unsupported;
|
||||
unsupported["operation"] = "encrypt";
|
||||
unsupported["algorithm"] = "ES256";
|
||||
DValue list_header;
|
||||
list_header.set_array();
|
||||
DValue list_item;
|
||||
list_item = "not-an-object";
|
||||
list_header.push(list_item);
|
||||
DValue list_request = sign_request;
|
||||
list_request["protected_header"] = list_header;
|
||||
DValue control_request = sign_request;
|
||||
control_request["claims"]["bad"] = String("control\nbyte");
|
||||
check("crypto_operation() generates an ES256 key", key["ok"].to_bool() && key["private_jwk"]["kty"].to_string() == "EC" && key["private_jwk"]["crv"].to_string() == "P-256" && key["public_jwk"]["d"].to_string() == "" && key["kid"].to_string() == key["thumbprint"].to_string(), key["kid"].to_string());
|
||||
check("crypto_operation() signs ES256 JWTs", signed_result["ok"].to_bool() && parts.size() == 3 && parts[2].size() == 86 && decoded_header.find("ES256") != String::npos, signed_result["error"].to_string());
|
||||
check("crypto_operation() rejects invalid signing requests", !crypto_operation(wrong_request)["ok"].to_bool() && !crypto_operation(malformed_request)["ok"].to_bool() && !crypto_operation(list_request)["ok"].to_bool() && crypto_operation(control_request)["error"].to_string() == "invalid_request", "signing validation");
|
||||
check("crypto_operation() parses an ES256 COSE key", cose["ok"].to_bool(), cose["error"].to_string());
|
||||
check("crypto_operation() verifies ES256 DER signatures", crypto_operation(verify_request)["ok"].to_bool() && crypto_operation(verify_request)["valid"].to_bool(), crypto_operation(verify_request)["error"].to_string());
|
||||
check("crypto_operation() rejects unsupported operations", crypto_operation(unsupported)["error"].to_string() == "unsupported_operation", crypto_operation(unsupported)["error"].to_string());
|
||||
site_tests_summary(passed, failed, skipped, "Structured crypto tests generate ephemeral P-256 keys and retain no key material.");
|
||||
site_tests_page_end();
|
||||
}
|
||||
+31
-7
@@ -132,7 +132,10 @@ RENDER(Request& context)
|
||||
String link_path = "/tmp/uce-site-tests-link";
|
||||
file_unlink(link_path);
|
||||
bool symlink_ok = file_symlink(tmp_created, link_path);
|
||||
check("file_temp() / file_chmod() / file_symlink() / file_fsync()", tmp_created != "" && file_exists(tmp_created) && chmod_ok && fsync_ok && symlink_ok, tmp_created + " -> " + link_path);
|
||||
bool temp_ready = tmp_created != "" && file_exists(tmp_created) && chmod_ok && fsync_ok && symlink_ok;
|
||||
file_unlink(link_path);
|
||||
file_unlink(tmp_created);
|
||||
check("file_temp() / file_chmod() / file_symlink() / file_fsync()", temp_ready && !file_exists(link_path) && !file_exists(tmp_created), tmp_created + " -> " + link_path);
|
||||
|
||||
String start_dir = process_start_directory();
|
||||
String old_cwd = cwd_get();
|
||||
@@ -149,18 +152,26 @@ RENDER(Request& context)
|
||||
cfg["yes"] = "yes";
|
||||
cfg["no"] = "off";
|
||||
check("to_u64() / to_s64() / to_f64() / to_bool()", to_u64(cfg["u64"], 7) == 42 && to_u64(cfg["bad"], 7) == 7 && to_s64(cfg["s64"], 5) == -12 && to_s64(cfg["bad"], 5) == 5 && to_f64(cfg["f64"], 1.0) > 3.49 && to_f64(cfg["bad"], 1.25) == 1.25 && to_bool(cfg["yes"], false) && !to_bool(cfg["no"], true) && !to_bool("unknown", false), var_dump(cfg));
|
||||
check("to_u64() rejects negative and overflowing text", to_u64("-1", 7) == 7 && to_u64("+42", 7) == 42 && to_u64("18446744073709551615", 7) == 18446744073709551615ULL && to_u64("18446744073709551616", 7) == 7, std::to_string(to_u64("-1", 7)) + "/" + std::to_string(to_u64("18446744073709551616", 7)));
|
||||
|
||||
DValue perf = request_perf();
|
||||
u64 module_misses = perf["unit_module_cache_miss_count"].to_u64();
|
||||
u64 serialized_hits = perf["unit_module_serialized_cache_hit_count"].to_u64();
|
||||
u64 module_read_bytes = perf["unit_module_read_bytes"].to_u64();
|
||||
bool serialized_reads_bounded = module_misses == 0 || serialized_hits != module_misses || (module_read_bytes > 0 && module_read_bytes < module_misses * 1024 * 1024);
|
||||
bool unit_module_profile_valid = serialized_reads_bounded && perf["unit_module_cache_hit_count"].to_u64() + module_misses == perf["unit_load_count"].to_u64() && serialized_hits + perf["unit_module_compile_count"].to_u64() == module_misses && perf["unit_module_lookup_us"].type != 'S' && perf["unit_module_read_us"].type != 'S' && perf["unit_module_read_bytes"].type != 'S' && perf["unit_module_parse_us"].type != 'S' && perf["unit_module_compile_us"].type != 'S' && perf["unit_module_classify_us"].type != 'S';
|
||||
bool unit_operation_profile_valid = true;
|
||||
perf["unit_module_operations"].each([&](DValue operation, String key) {
|
||||
unit_operation_profile_valid = unit_operation_profile_valid && operation["unit"].to_string() != "" && operation["read_count"].type != 'S' && operation["allocate_us"].type != 'S' && operation["import_us"].type != 'S' && operation["symbol_resolve_count"].type != 'S' && operation["symbol_resolve_us"].type != 'S' && operation["instantiate_us"].type != 'S' && operation["initialize_us"].type != 'S';
|
||||
});
|
||||
bool unit_module_profile_valid = serialized_reads_bounded && unit_operation_profile_valid && perf["unit_module_cache_hit_count"].to_u64() + module_misses == perf["unit_load_count"].to_u64() && serialized_hits + perf["unit_module_compile_count"].to_u64() == module_misses && perf["unit_module_lookup_us"].type != 'S' && perf["unit_module_read_us"].type != 'S' && perf["unit_module_read_bytes"].type != 'S' && perf["unit_module_read_count"].type != 'S' && perf["unit_module_parse_us"].type != 'S' && perf["unit_module_compile_us"].type != 'S' && perf["unit_module_classify_us"].type != 'S';
|
||||
bool transport_profile_valid = perf["transport_params_us"].type != 'S' && perf["transport_input_us"].type != 'S' && perf["handler_queue_us"].type != 'S' && perf["accept_us"].to_f64() + 2 >= perf["transport_params_us"].to_f64() + perf["transport_input_us"].to_f64() + perf["handler_queue_us"].to_f64();
|
||||
u64 phase_cpu_us = perf["workspace_setup_cpu_us"].to_u64() + perf["workspace_birth_cpu_us"].to_u64() + perf["context_apply_cpu_us"].to_u64();
|
||||
bool workspace_cpu_profile_valid = perf["workspace_wall_us"].to_u64() > 0 && perf["workspace_cpu_us"].to_u64() > 0 && perf["workspace_cpu_us"].to_u64() <= perf["workspace_wall_us"].to_u64() + 2 && (perf["workspace_cpu_us"].to_u64() >= perf["workspace_wall_us"].to_u64() ? perf["workspace_wait_us"].to_u64() == 0 : perf["workspace_wait_us"].to_u64() + perf["workspace_cpu_us"].to_u64() == perf["workspace_wall_us"].to_u64()) && perf["workspace_setup_cpu_us"].to_u64() <= perf["workspace_setup_us"].to_u64() + 2 && perf["workspace_birth_cpu_us"].to_u64() <= perf["workspace_birth_us"].to_u64() + 2 && perf["context_apply_cpu_us"].to_u64() <= perf["context_apply_us"].to_u64() + 2 && phase_cpu_us + perf["execution_cpu_us"].to_u64() == perf["workspace_cpu_us"].to_u64();
|
||||
u64 birth_profile_us = perf["birth_policy_us"].to_u64() + perf["birth_import_us"].to_u64() + perf["birth_instantiate_us"].to_u64() + perf["birth_exports_us"].to_u64() + perf["birth_initialize_us"].to_u64();
|
||||
u64 context_profile_us = perf["context_encode_us"].to_u64() + perf["context_allocate_us"].to_u64() + perf["context_write_us"].to_u64() + perf["context_guest_apply_us"].to_u64() + perf["context_free_us"].to_u64();
|
||||
bool workspace_cpu_profile_valid = perf["workspace_wall_us"].to_u64() > 0 && perf["workspace_cpu_us"].to_u64() > 0 && perf["workspace_cpu_us"].to_u64() <= perf["workspace_wall_us"].to_u64() + 2 && (perf["workspace_cpu_us"].to_u64() >= perf["workspace_wall_us"].to_u64() ? perf["workspace_wait_us"].to_u64() == 0 : perf["workspace_wait_us"].to_u64() + perf["workspace_cpu_us"].to_u64() == perf["workspace_wall_us"].to_u64()) && perf["workspace_setup_cpu_us"].to_u64() <= perf["workspace_setup_us"].to_u64() + 2 && perf["workspace_birth_cpu_us"].to_u64() <= perf["workspace_birth_us"].to_u64() + 2 && perf["context_apply_cpu_us"].to_u64() <= perf["context_apply_us"].to_u64() + 2 && birth_profile_us <= perf["workspace_birth_us"].to_u64() && perf["context_bytes"].to_u64() > 0 && context_profile_us <= perf["context_apply_us"].to_u64() && phase_cpu_us + perf["execution_cpu_us"].to_u64() == perf["workspace_cpu_us"].to_u64();
|
||||
u64 operation_hostcalls = 0;
|
||||
u64 operation_hostcall_us = 0;
|
||||
u64 operation_hostcall_cpu_us = 0;
|
||||
u64 operation_kinds = 0;
|
||||
bool hostcall_operation_profile_valid = true;
|
||||
perf["hostcall_operations"].each([&](DValue operation, String key) {
|
||||
@@ -168,14 +179,27 @@ RENDER(Request& context)
|
||||
operation_kinds++;
|
||||
operation_hostcalls += operation["count"].to_u64();
|
||||
operation_hostcall_us += operation["us"].to_u64();
|
||||
hostcall_operation_profile_valid = hostcall_operation_profile_valid && name != "" && !contains(name, "uce_host_") && !contains(name, "/") && operation["count"].to_u64() > 0;
|
||||
operation_hostcall_cpu_us += operation["cpu_us"].to_u64();
|
||||
hostcall_operation_profile_valid = hostcall_operation_profile_valid && name != "" && !contains(name, "uce_host_") && !contains(name, "/") && operation["count"].to_u64() > 0 && operation["cpu_us"].to_u64() <= operation["us"].to_u64() + 2;
|
||||
});
|
||||
u64 dropped_hostcall_operations = perf["hostcall_operations_dropped"].to_u64();
|
||||
hostcall_operation_profile_valid = hostcall_operation_profile_valid && operation_kinds > 0 && operation_kinds <= 32 && ((dropped_hostcall_operations == 0 && operation_hostcalls == perf["hostcall_count"].to_u64() && operation_hostcall_us == perf["hostcall_us"].to_u64()) || (dropped_hostcall_operations > 0 && operation_kinds == 32 && operation_hostcalls < perf["hostcall_count"].to_u64() && operation_hostcall_us <= perf["hostcall_us"].to_u64()));
|
||||
bool perf_stable = transport_profile_valid && workspace_cpu_profile_valid && unit_module_profile_valid && hostcall_operation_profile_valid && perf["worker_pid"].to_u64() > 0 && perf["running_us"].to_f64() > 0 && perf["dispatch_us"].type != 'S' && perf["workspace_setup_us"].type != 'S' && perf["workspace_birth_us"].type != 'S' && perf["context_apply_us"].type != 'S' && perf["hostcall_count"].to_u64() > 0 && perf["hostcall_us"].type != 'S' && perf["guest_us"].to_f64() > 0 && perf["component_resolve_count"].type != 'S' && perf["component_path_us"].type != 'S' && perf["component_artifact_us"].type != 'S' && perf["component_load_us"].type != 'S' && perf["component_link_us"].type != 'S' && perf["unit_load_count"].to_u64() > 0 && perf["unit_module_us"].type != 'S' && perf["unit_allocate_us"].type != 'S' && perf["unit_import_us"].type != 'S' && perf["unit_symbol_resolve_count"].type != 'S' && perf["unit_symbol_resolve_us"].type != 'S' && perf["unit_instantiate_us"].type != 'S' && perf["unit_initialize_us"].type != 'S';
|
||||
hostcall_operation_profile_valid = hostcall_operation_profile_valid && perf["hostcall_cpu_profiled"].type == 'B' && perf["hostcall_cpu_us"].to_u64() <= perf["hostcall_us"].to_u64() + 2 && operation_kinds > 0 && operation_kinds <= 32 && ((dropped_hostcall_operations == 0 && operation_hostcalls == perf["hostcall_count"].to_u64() && operation_hostcall_us == perf["hostcall_us"].to_u64() && operation_hostcall_cpu_us == perf["hostcall_cpu_us"].to_u64()) || (dropped_hostcall_operations > 0 && operation_kinds == 32 && operation_hostcalls < perf["hostcall_count"].to_u64() && operation_hostcall_us <= perf["hostcall_us"].to_u64() && operation_hostcall_cpu_us <= perf["hostcall_cpu_us"].to_u64()));
|
||||
bool thread_runtime_profile_valid = perf["thread_runtime_profiled"].type == 'B' && (!perf["thread_runtime_profiled"].to_bool() || (perf["thread_cpu_start"].to_f64() >= 0 && perf["thread_cpu_end"].to_f64() >= 0 && perf["thread_cpu_migrated"].type == 'B' && perf["thread_user_cpu_us"].to_u64() + perf["thread_system_cpu_us"].to_u64() <= perf["workspace_wall_us"].to_u64() + 5 && perf["thread_voluntary_context_switches"].type != 'S' && perf["thread_involuntary_context_switches"].type != 'S' && perf["thread_minor_faults"].type != 'S' && perf["thread_major_faults"].type != 'S'));
|
||||
bool perf_stable = transport_profile_valid && workspace_cpu_profile_valid && unit_module_profile_valid && hostcall_operation_profile_valid && thread_runtime_profile_valid && perf["worker_pid"].to_u64() > 0 && perf["running_us"].to_f64() > 0 && perf["ready_normalize_us"].type != 'S' && perf["ready_mutation_check_us"].type != 'S' && perf["ready_artifact_stat_us"].type != 'S' && perf["ready_freshness_us"].type != 'S' && perf["ready_source_generation_us"].type != 'S' && perf["ready_freshness_full_check_us"].type != 'S' && perf["ready_freshness_cache_hit_count"].type != 'S' && perf["ready_worker_us"].type != 'S' && perf["ready_check_count"].to_u64() > 0 && perf["dispatch_us"].type != 'S' && perf["workspace_setup_us"].type != 'S' && perf["workspace_birth_us"].type != 'S' && perf["context_apply_us"].type != 'S' && perf["hostcall_count"].to_u64() > 0 && perf["hostcall_us"].type != 'S' && perf["guest_us"].to_f64() > 0 && perf["component_resolve_count"].type != 'S' && perf["component_loaded_reuse_count"].type != 'S' && perf["component_path_us"].type != 'S' && perf["component_artifact_us"].type != 'S' && perf["component_load_us"].type != 'S' && perf["component_link_us"].type != 'S' && perf["unit_load_count"].to_u64() > 0 && perf["unit_module_us"].type != 'S' && perf["unit_allocate_us"].type != 'S' && perf["unit_import_us"].type != 'S' && perf["unit_symbol_resolve_count"].type != 'S' && perf["unit_symbol_resolve_us"].type != 'S' && perf["unit_instantiate_us"].type != 'S' && perf["unit_initialize_us"].type != 'S';
|
||||
u64 profiled_ready_normalize = perf["ready_normalize_us"].to_u64();
|
||||
u64 profiled_ready_mutation_check = perf["ready_mutation_check_us"].to_u64();
|
||||
u64 profiled_ready_artifact_stat = perf["ready_artifact_stat_us"].to_u64();
|
||||
u64 profiled_ready_freshness = perf["ready_freshness_us"].to_u64();
|
||||
u64 profiled_ready_source_generation = perf["ready_source_generation_us"].to_u64();
|
||||
u64 profiled_ready_freshness_full_check = perf["ready_freshness_full_check_us"].to_u64();
|
||||
u64 profiled_ready_freshness_cache_hit_count = perf["ready_freshness_cache_hit_count"].to_u64();
|
||||
u64 profiled_ready_worker = perf["ready_worker_us"].to_u64();
|
||||
u64 profiled_ready_checks = perf["ready_check_count"].to_u64();
|
||||
u64 profiled_hostcalls = perf["hostcall_count"].to_u64();
|
||||
f64 profiled_hostcall_us = perf["hostcall_us"].to_f64();
|
||||
u64 profiled_hostcall_cpu_us = perf["hostcall_cpu_us"].to_u64();
|
||||
u64 profiled_components = perf["component_resolve_count"].to_u64();
|
||||
u64 profiled_component_loaded_reuses = perf["component_loaded_reuse_count"].to_u64();
|
||||
u64 profiled_component_path = perf["component_path_us"].to_u64();
|
||||
u64 profiled_component_artifact = perf["component_artifact_us"].to_u64();
|
||||
u64 profiled_component_load = perf["component_load_us"].to_u64();
|
||||
@@ -193,7 +217,7 @@ RENDER(Request& context)
|
||||
for(u64 i = 0; i < 512 && perf_stable; i++)
|
||||
{
|
||||
perf = request_perf();
|
||||
perf_stable = perf["worker_pid"].to_u64() > 0 && perf["running_us"].to_f64() > 0 && perf["dispatch_us"].type != 'S' && perf["workspace_setup_us"].type != 'S' && perf["workspace_birth_us"].type != 'S' && perf["context_apply_us"].type != 'S' && perf["hostcall_count"].to_u64() == profiled_hostcalls && perf["hostcall_us"].to_f64() == profiled_hostcall_us && perf["guest_us"].to_f64() > 0 && perf["component_resolve_count"].to_u64() == profiled_components && perf["component_path_us"].to_u64() == profiled_component_path && perf["component_artifact_us"].to_u64() == profiled_component_artifact && perf["component_load_us"].to_u64() == profiled_component_load && perf["component_link_us"].to_u64() == profiled_component_link && perf["unit_load_count"].to_u64() == profiled_unit_loads && perf["unit_module_us"].to_u64() == profiled_unit_module && perf["unit_allocate_us"].to_u64() == profiled_unit_allocate && perf["unit_import_us"].to_u64() == profiled_unit_import && perf["unit_symbol_resolve_count"].to_u64() == profiled_unit_symbol_resolve_count && perf["unit_symbol_resolve_us"].to_u64() == profiled_unit_symbol_resolve && perf["unit_instantiate_us"].to_u64() == profiled_unit_instantiate && perf["unit_initialize_us"].to_u64() == profiled_unit_initialize && json_encode(perf["mysql_operations"]) == profiled_mysql_operations && json_encode(perf["hostcall_operations"]) == profiled_hostcall_operations;
|
||||
perf_stable = perf["worker_pid"].to_u64() > 0 && perf["running_us"].to_f64() > 0 && perf["ready_normalize_us"].to_u64() == profiled_ready_normalize && perf["ready_mutation_check_us"].to_u64() == profiled_ready_mutation_check && perf["ready_artifact_stat_us"].to_u64() == profiled_ready_artifact_stat && perf["ready_freshness_us"].to_u64() == profiled_ready_freshness && perf["ready_source_generation_us"].to_u64() == profiled_ready_source_generation && perf["ready_freshness_full_check_us"].to_u64() == profiled_ready_freshness_full_check && perf["ready_freshness_cache_hit_count"].to_u64() == profiled_ready_freshness_cache_hit_count && perf["ready_worker_us"].to_u64() == profiled_ready_worker && perf["ready_check_count"].to_u64() == profiled_ready_checks && perf["dispatch_us"].type != 'S' && perf["workspace_setup_us"].type != 'S' && perf["workspace_birth_us"].type != 'S' && perf["context_apply_us"].type != 'S' && perf["hostcall_count"].to_u64() == profiled_hostcalls && perf["hostcall_us"].to_f64() == profiled_hostcall_us && perf["hostcall_cpu_us"].to_u64() == profiled_hostcall_cpu_us && perf["guest_us"].to_f64() > 0 && perf["component_resolve_count"].to_u64() == profiled_components && perf["component_loaded_reuse_count"].to_u64() == profiled_component_loaded_reuses && perf["component_path_us"].to_u64() == profiled_component_path && perf["component_artifact_us"].to_u64() == profiled_component_artifact && perf["component_load_us"].to_u64() == profiled_component_load && perf["component_link_us"].to_u64() == profiled_component_link && perf["unit_load_count"].to_u64() == profiled_unit_loads && perf["unit_module_us"].to_u64() == profiled_unit_module && perf["unit_allocate_us"].to_u64() == profiled_unit_allocate && perf["unit_import_us"].to_u64() == profiled_unit_import && perf["unit_symbol_resolve_count"].to_u64() == profiled_unit_symbol_resolve_count && perf["unit_symbol_resolve_us"].to_u64() == profiled_unit_symbol_resolve && perf["unit_instantiate_us"].to_u64() == profiled_unit_instantiate && perf["unit_initialize_us"].to_u64() == profiled_unit_initialize && json_encode(perf["mysql_operations"]) == profiled_mysql_operations && json_encode(perf["hostcall_operations"]) == profiled_hostcall_operations;
|
||||
}
|
||||
check("request_perf() stages stable repeated snapshots", perf_stable, json_encode(perf));
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ markdown.uce|Markdown|Markdown parsing, rendering, and component hooks.|http sui
|
||||
units.uce|Units|unit_call(), lifecycle hooks, and unit metadata.|http suite uce public|Units|1|1
|
||||
websockets.ws.uce|WebSockets|Browser-driven WebSocket helper checks.|http suite uce public websocket|WebSockets|1|1
|
||||
io.uce|Filesystem|Filesystem helpers that are restricted outside trusted networks.|http suite uce internal|Filesystem|1|1
|
||||
crypto_operation.uce|Structured Crypto|Algorithm-selected key generation and JWT signing coverage.|http suite uce public crypto|Structured crypto operations|1|1
|
||||
sqlite.uce|SQLite|SQLite connector with prepared named parameters and DValue rows.|http suite uce internal sqlite|SQLite|1|1
|
||||
zip.uce|ZIP|Archive helpers that create and extract temporary server-side files.|http suite uce internal|ZIP|1|1
|
||||
services.uce|Sockets And Services|Network/service helpers that are restricted outside trusted networks.|http suite uce internal|Sockets And Services|1|1
|
||||
|
||||
+10
-1
@@ -23,12 +23,21 @@ RENDER(Request& context)
|
||||
DValue ast = markdown_to_ast(markdown_src, options);
|
||||
String ast_json = json_encode(ast);
|
||||
String html = markdown_to_html(markdown_src, options);
|
||||
DValue heading_options;
|
||||
heading_options["heading_offset"] = "1";
|
||||
heading_options["components"]["node.heading"] = "components/markdown/heading";
|
||||
String shifted_headings = markdown_to_html("# Page child\n\n###### Deep child", heading_options);
|
||||
DValue negative_heading_options;
|
||||
negative_heading_options["heading_offset"] = "-2";
|
||||
String clamped_headings = markdown_to_html("# Top\n\n### Third", negative_heading_options);
|
||||
String ast_excerpt = ast_json.substr(0, ast_json.length() > 220 ? 220 : ast_json.length());
|
||||
|
||||
site_tests_page_start("Markdown", "Markdown parser coverage with component-backed directive and node hooks.");
|
||||
|
||||
check("markdown_to_ast()", ast_json.find("heading") != String::npos || ast_json.find("code_block") != String::npos, ast_excerpt);
|
||||
check("markdown_to_html()", html.find("<h1") != String::npos && html.find("tests-warning") != String::npos, html);
|
||||
check("heading offset preserves source AST and shifts default hook HTML", shifted_headings.find("data-hook=\"heading\"") != String::npos && shifted_headings.find("data-source-level=\"1\"") != String::npos && shifted_headings.find("<h2>Page child</h2>") != String::npos && shifted_headings.find("<h6>Deep child</h6>") != String::npos && shifted_headings.find("<h1>") == String::npos, shifted_headings);
|
||||
check("negative heading offset clamps at h1", clamped_headings.find("<h1>Top</h1>") != String::npos && clamped_headings.find("<h1>Third</h1>") != String::npos, clamped_headings);
|
||||
check("component hook output", html.find("tests-code-block") != String::npos, html);
|
||||
|
||||
?><div class="tests-section">
|
||||
@@ -38,4 +47,4 @@ RENDER(Request& context)
|
||||
|
||||
site_tests_summary(passed, failed, skipped, "The warning directive and code-block renderer both run through local component fixtures under site/tests/components/markdown.");
|
||||
site_tests_page_end();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
u64 pool_request_global = 0;
|
||||
u64 pool_constructor_value = 0;
|
||||
|
||||
struct PoolIsolationConstructor
|
||||
{
|
||||
PoolIsolationConstructor()
|
||||
{
|
||||
pool_constructor_value = 17;
|
||||
}
|
||||
};
|
||||
|
||||
PoolIsolationConstructor pool_isolation_constructor;
|
||||
|
||||
ONCE(Request& context)
|
||||
{
|
||||
pool_request_global++;
|
||||
}
|
||||
|
||||
RENDER(Request& context)
|
||||
{
|
||||
String token = context.get["token"];
|
||||
bool clean = pool_request_global == 1 &&
|
||||
pool_constructor_value == 17 &&
|
||||
context.call["pool-leak"].to_string() == "" &&
|
||||
context.props["pool-leak"].to_string() == "";
|
||||
DValue perf = request_perf();
|
||||
print(clean ? "fresh|" : "leaked|", token, "|", (u64)perf["worker_pid"].to_u64());
|
||||
|
||||
pool_request_global = 999;
|
||||
pool_constructor_value = 999;
|
||||
context.call["pool-leak"] = token;
|
||||
context.props["pool-leak"] = token;
|
||||
context.get["token"] = "dirty";
|
||||
}
|
||||
@@ -31,6 +31,8 @@ RENDER(Request& context)
|
||||
};
|
||||
|
||||
String nested = preprocessor_nested_literal();
|
||||
String trailing_backslash = "C:\\Users\\";
|
||||
String escaped_quote = "before\"after";
|
||||
|
||||
site_tests_page_start("Preprocessor", "Regression coverage for literal output rewriting and parser edge cases.");
|
||||
?>
|
||||
@@ -43,6 +45,8 @@ RENDER(Request& context)
|
||||
check("raw string terminator in nested literal", contains(nested, "nested )\" marker"), nested);
|
||||
check("entrypoint @fragment attribute captures output", context.call["fragments"]["preprocessor-test"].to_string() == "fragment attr once", context.call["fragments"]["preprocessor-test"].to_string());
|
||||
check("inline code island after dangerous literal", true, "parser returned to C++ after rendering literal content containing )\"");
|
||||
check("quote scanner handles trailing escaped backslash", trailing_backslash == "C:\\Users\\", trailing_backslash);
|
||||
check("quote scanner retains escaped quote content", escaped_quote == "before\"after", escaped_quote);
|
||||
|
||||
site_tests_summary(passed, failed, skipped, "Literal content containing the C++ raw-string terminator sequence must compile and render unchanged.");
|
||||
site_tests_page_end();
|
||||
|
||||
+58
-6
@@ -28,14 +28,20 @@ RENDER(Request& context)
|
||||
u64 sockfd = socket_connect("127.0.0.1", 80);
|
||||
if(sockfd != 0)
|
||||
{
|
||||
u64 closed_handle = socket_connect("127.0.0.1", 80);
|
||||
bool opaque_handles = sockfd == 1 && closed_handle == 2;
|
||||
socket_close(closed_handle);
|
||||
socket_close(closed_handle);
|
||||
socket_close(999999);
|
||||
bool closed_write_rejected = !socket_write(closed_handle, "invalid");
|
||||
bool write_ok = socket_write(sockfd, "GET /tests/index.uce HTTP/1.0\r\nHost: uce.openfu.com\r\n\r\n");
|
||||
String response = socket_read(sockfd, 4096, 2);
|
||||
socket_close(sockfd);
|
||||
bool has_nul = response.find(String("\0", 1)) != String::npos;
|
||||
mark(
|
||||
"socket_connect() / socket_write() / socket_read()",
|
||||
(write_ok && response.find("200 OK") != String::npos && !has_nul) ? "pass" : "fail",
|
||||
response.substr(0, response.length() > 220 ? 220 : response.length()) + (has_nul ? " [unexpected NUL]" : "")
|
||||
"socket handles reject invalid and closed indices",
|
||||
(opaque_handles && closed_write_rejected && write_ok && response.find("200 OK") != String::npos && !has_nul) ? "pass" : "fail",
|
||||
"handles=" + std::to_string(sockfd) + "/" + std::to_string(closed_handle) + " " + response.substr(0, response.length() > 180 ? 180 : response.length()) + (has_nul ? " [unexpected NUL]" : "")
|
||||
);
|
||||
}
|
||||
else
|
||||
@@ -66,7 +72,7 @@ RENDER(Request& context)
|
||||
socket_close(stopped_sockfd);
|
||||
mark(
|
||||
"server_start_http() / SERVE_HTTP:named / server_stop()",
|
||||
(custom_http_pid != 0 && write_ok && response.find("custom-http-named") != String::npos && response.find("query=alpha=1") != String::npos && stopped && listener_closed) ? "pass" : "fail",
|
||||
(custom_http_pid != 0 && write_ok && response.find("custom-http-named") != String::npos && response.find("query=alpha=1") != String::npos && response.find("\r\nHTTP/1.1 ") == String::npos && stopped && listener_closed) ? "pass" : "fail",
|
||||
response.substr(0, response.length() > 260 ? 260 : response.length()) + " stopped=" + (stopped ? "true" : "false") + " listener_closed=" + (listener_closed ? "true" : "false")
|
||||
);
|
||||
}
|
||||
@@ -90,10 +96,25 @@ RENDER(Request& context)
|
||||
StringMap mem_multi = memcache_get_multiple(memfd, {"site-tests-key", "site-tests-key2"});
|
||||
memcache_delete(memfd, "site-tests-key");
|
||||
memcache_delete(memfd, "site-tests-key2");
|
||||
String large_mem_value(200 * 1024, 'x');
|
||||
large_mem_value.replace(1024, 7, "\r\nEND\r\n");
|
||||
bool large_mem_stored = memcache_set(memfd, "site-tests-large", large_mem_value);
|
||||
String large_mem_loaded = memcache_get(memfd, "site-tests-large");
|
||||
String second_large_mem_value(160 * 1024, 'y');
|
||||
bool second_large_mem_stored = memcache_set(memfd, "site-tests-large-2", second_large_mem_value);
|
||||
StringMap large_mem_multi = memcache_get_multiple(memfd, {"site-tests-large", "site-tests-large-2"});
|
||||
String no_reply = memcache_command(memfd, "set site-tests-noreply 0 60 5 noreply\r\nvalue");
|
||||
String no_reply_value = memcache_get(memfd, "site-tests-noreply");
|
||||
String stats_mem = memcache_command(memfd, "stats");
|
||||
String stats_reset_mem = memcache_command(memfd, "stats reset");
|
||||
String after_stats_reset = memcache_get(memfd, "site-tests-noreply");
|
||||
bool large_mem_deleted = memcache_delete(memfd, "site-tests-large");
|
||||
bool second_large_mem_deleted = memcache_delete(memfd, "site-tests-large-2");
|
||||
bool no_reply_deleted = memcache_delete(memfd, "site-tests-noreply");
|
||||
mark(
|
||||
"memcache_connect() / memcache_command() / memcache_get_multiple()",
|
||||
mem_value == "value-1" && contains(raw_mem, "VALUE site-tests-key") && mem_multi["site-tests-key"] == "value-1" && mem_multi["site-tests-key2"] == "value-2" ? "pass" : "fail",
|
||||
"memcache value=" + mem_value + " raw=" + raw_mem + " multi=" + var_dump(mem_multi)
|
||||
mem_value == "value-1" && contains(raw_mem, "VALUE site-tests-key") && mem_multi["site-tests-key"] == "value-1" && mem_multi["site-tests-key2"] == "value-2" && large_mem_stored && large_mem_loaded == large_mem_value && second_large_mem_stored && large_mem_multi["site-tests-large"] == large_mem_value && large_mem_multi["site-tests-large-2"] == second_large_mem_value && no_reply == "" && no_reply_value == "value" && contains(stats_mem, "STAT ") && trim(stats_reset_mem) == "RESET" && after_stats_reset == "value" && large_mem_deleted && second_large_mem_deleted && no_reply_deleted ? "pass" : "fail",
|
||||
"memcache value=" + mem_value + " raw=" + raw_mem + " multi=" + var_dump(mem_multi) + " large=" + std::to_string(large_mem_loaded.size()) + " multi-large=" + std::to_string(large_mem_multi["site-tests-large"].size() + large_mem_multi["site-tests-large-2"].size()) + " noreply=" + no_reply_value + " stats=" + std::to_string(stats_mem.size()) + "/" + trim(stats_reset_mem) + " after-reset=" + after_stats_reset + " deleted=" + (large_mem_deleted && second_large_mem_deleted && no_reply_deleted ? "true" : "false")
|
||||
);
|
||||
}
|
||||
|
||||
@@ -115,6 +136,29 @@ RENDER(Request& context)
|
||||
parsed_underscore.find("'Ada'") != String::npos && parsed_underscore.find("_name") == String::npos ? "pass" : "fail",
|
||||
parsed_underscore
|
||||
);
|
||||
StringMap quoted_params;
|
||||
quoted_params["name_looking_text"] = "SUBSTITUTED";
|
||||
String escaped_quote_query = "SELECT 'a\\':name_looking_text'";
|
||||
String parsed_escaped_quote = placeholder_guard.parse_query_parameters(escaped_quote_query, quoted_params);
|
||||
mark(
|
||||
"mysql named placeholders stay opaque after escaped quotes",
|
||||
parsed_escaped_quote.find(":name_looking_text") != String::npos && parsed_escaped_quote.find("SUBSTITUTED") == String::npos ? "pass" : "fail",
|
||||
parsed_escaped_quote
|
||||
);
|
||||
MySQL unavailable_mysql;
|
||||
unavailable_mysql.connect("127.0.0.1", "__uce_intentionally_missing__", "not-a-real-password");
|
||||
mark(
|
||||
"mysql_connected() distinguishes a failed connection wrapper",
|
||||
!mysql_connected(&unavailable_mysql) ? "pass" : "fail",
|
||||
mysql_connected(&unavailable_mysql) ? "unexpected live handle" : "no live handle"
|
||||
);
|
||||
unavailable_mysql.query("SELECT 1");
|
||||
String unavailable_mysql_error = unavailable_mysql.error();
|
||||
mark(
|
||||
"mysql query after failed connect retains the connection error",
|
||||
unavailable_mysql_error != "" ? "pass" : "fail",
|
||||
unavailable_mysql_error
|
||||
);
|
||||
|
||||
MySQL mysql;
|
||||
mysql.connect("127.0.0.1", "root", "");
|
||||
@@ -144,6 +188,14 @@ RENDER(Request& context)
|
||||
inserted == 2 && updated == 1 && selected == 0 ? "pass" : "fail",
|
||||
"inserted=" + std::to_string((u64)inserted) + " updated=" + std::to_string((u64)updated) + " selected=" + std::to_string((u64)selected)
|
||||
);
|
||||
mysql.query("INSERT INTO uce_affected_rows_test VALUES (1,30)");
|
||||
u32 duplicate_error_code = mysql._preload_next_error_code;
|
||||
String duplicate_error = mysql.error();
|
||||
mark(
|
||||
"mysql query failures retain server diagnostics",
|
||||
duplicate_error_code > 1 && duplicate_error != "" && duplicate_error != "Unknown server error" ? "pass" : "fail",
|
||||
"code=" + std::to_string((u64)duplicate_error_code) + " error=" + duplicate_error
|
||||
);
|
||||
}
|
||||
DValue mysql_perf = request_perf();
|
||||
String mysql_operations = json_encode(mysql_perf["mysql_operations"]);
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
RENDER(Request& context)
|
||||
{
|
||||
unit_render("does-not-exist.uce", context);
|
||||
}
|
||||
@@ -18,6 +18,16 @@ RENDER(Request& context)
|
||||
auto unit_paths = units_list();
|
||||
DValue info = unit_info("call_helpers.uce");
|
||||
DValue relative_info = unit_info("components/../relative-child.uce");
|
||||
String outside_unit = "/tmp/uce-site-tests-outside-unit.uce";
|
||||
file_put_contents(outside_unit, "RENDER(Request& context) {}");
|
||||
DValue outside_info = unit_info(outside_unit);
|
||||
file_unlink(outside_unit);
|
||||
String generated_dir = path_join(context.params["UCE_BIN_DIRECTORY"], "site-tests-generated");
|
||||
mkdir(generated_dir);
|
||||
String generated_unit = path_join(generated_dir, "generated.uce");
|
||||
file_put_contents(generated_unit, "RENDER(Request& context) {}");
|
||||
DValue generated_info = unit_info(generated_unit);
|
||||
file_unlink(generated_unit);
|
||||
|
||||
ob_start();
|
||||
unit_call("call_helpers.uce", "emit_marker");
|
||||
@@ -32,6 +42,8 @@ RENDER(Request& context)
|
||||
check("units_list()", unit_paths.size() > 0, "count=" + std::to_string(unit_paths.size()));
|
||||
check("unit_info()", info["path"].to_string() != "", json_encode(info));
|
||||
check("compiler canonicalizes relative unit paths", relative_info["path"].to_string().find("/../") == String::npos && str_ends_with(relative_info["path"].to_string(), "/site/tests/relative-child.uce"), json_encode(relative_info));
|
||||
check("compiler rejects units outside document root", outside_info["path"].to_string() == "", json_encode(outside_info));
|
||||
check("compiler accepts generated units in BIN_DIRECTORY", generated_info["path"].to_string() == generated_unit, json_encode(generated_info));
|
||||
check("unit_compile()", unit_compile("call_helpers.uce"), "call_helpers.uce");
|
||||
check("unit_call()", call_output.find("UNIT_CALL_EXPORT_OK") != String::npos, call_output);
|
||||
check("unit_render()", render_output.find("data-unit-render=\"ok\"") != String::npos, render_output);
|
||||
|
||||
@@ -362,6 +362,7 @@ FastCGIServer::listen(const std::string& local_path)
|
||||
close(server_socket);
|
||||
throw;
|
||||
}
|
||||
server_socket_types[server_socket] = 'F';
|
||||
return server_socket;
|
||||
}
|
||||
|
||||
@@ -400,6 +401,23 @@ FastCGIServer::close_http_listeners()
|
||||
}
|
||||
}
|
||||
|
||||
void
|
||||
FastCGIServer::close_listeners_except(char type)
|
||||
{
|
||||
for(std::vector<int>::iterator it = server_sockets.begin(); it != server_sockets.end();)
|
||||
{
|
||||
int socket_handle = *it;
|
||||
if(server_socket_types[socket_handle] != type)
|
||||
{
|
||||
close(socket_handle);
|
||||
server_socket_types.erase(socket_handle);
|
||||
it = server_sockets.erase(it);
|
||||
continue;
|
||||
}
|
||||
++it;
|
||||
}
|
||||
}
|
||||
|
||||
bool
|
||||
FastCGIServer::is_http_like_type(char type)
|
||||
{
|
||||
@@ -672,10 +690,12 @@ FastCGIServer::process(int timeout_ms)
|
||||
Connection* doomed_connection = it->second;
|
||||
client_sockets.erase(it++);
|
||||
delete doomed_connection;
|
||||
if(calls_until_termination != -1 && client_sockets.size() == 0)
|
||||
{
|
||||
if(calls_until_termination > 0)
|
||||
calls_until_termination -= 1;
|
||||
if(calls_until_termination <= 0)
|
||||
if(calls_until_termination == 0)
|
||||
{
|
||||
close_listeners_except(0);
|
||||
if(client_sockets.empty())
|
||||
exit(0);
|
||||
}
|
||||
}
|
||||
@@ -858,6 +878,8 @@ FastCGIServer::process_http_request(FastCGIRequest& request, String& data)
|
||||
}
|
||||
else
|
||||
{
|
||||
if(request.stats.time_start == 0)
|
||||
request.stats.time_start = time_precise();
|
||||
on_complete(request);
|
||||
|
||||
assemble_output_buffer(
|
||||
@@ -1331,15 +1353,30 @@ FastCGIServer::assemble_output_buffer(FastCGIRequest& request, Connection* conne
|
||||
request.flags.output_closed = true;
|
||||
request.stats.time_end = time_precise();
|
||||
if(request.flags.log_request)
|
||||
printf("(r) pid:%i\t%s\t%0.6fs\tfps:%0.0f\tout:%0.1fkB\tmem:%0.0f/%0.0fkB\n",
|
||||
{
|
||||
const char* transport = !connection ? "fastcgi" :
|
||||
connection->type == 'H' ? "http" :
|
||||
connection->type == 'C' ? "cli" : "unknown";
|
||||
auto elapsed_us = [&](f64 from, f64 to) -> f64 {
|
||||
return(from > 0 && to >= from ? (to - from) * 1000000.0 : 0.0);
|
||||
};
|
||||
printf("(r) pid:%i\t%s\t%0.6fs\tfps:%0.0f\tout:%0.1fkB\tmem:%0.0f/%0.0fkB\twasm-ready:%0.3fms\twasm:%0.3fms\tworkspace:%0.3fms\tinvoke:%0.3fms\tcollect:%0.3fms\tpost:%0.3fms\ttransport:%s\n",
|
||||
my_pid,
|
||||
request.params["REQUEST_URI"].c_str(),
|
||||
request.stats.time_end - request.stats.time_start,
|
||||
1.0 / (request.stats.time_end - request.stats.time_start),
|
||||
(f32)(request.out.length()/1024),
|
||||
(f32)(request.stats.mem_high/1024),
|
||||
(f32)(request.stats.mem_alloc/1024)
|
||||
(f32)(request.stats.mem_alloc/1024),
|
||||
elapsed_us(request.stats.time_start, request.stats.wasm_handler_ready) / 1000.0,
|
||||
elapsed_us(request.stats.wasm_backend_started, request.stats.wasm_backend_finished) / 1000.0,
|
||||
(f64)request.stats.wasm_workspace_complete_us / 1000.0,
|
||||
(f64)request.stats.wasm_entry_invoke_us / 1000.0,
|
||||
(f64)request.stats.wasm_output_collect_us / 1000.0,
|
||||
elapsed_us(request.stats.wasm_backend_finished, request.stats.time_end) / 1000.0,
|
||||
transport
|
||||
);
|
||||
}
|
||||
|
||||
if(connection)
|
||||
{
|
||||
@@ -1377,6 +1414,8 @@ FastCGIServer::request_write_fgci(Connection& connection, RequestID id,
|
||||
//switch_to_system_alloc();
|
||||
write_data_fcgi(connection.output_buffer, id, request.out, FCGI_STDOUT);
|
||||
write_data_fcgi(connection.output_buffer, id, request.err, FCGI_STDERR);
|
||||
request.out.clear();
|
||||
request.err.clear();
|
||||
|
||||
FCGI_EndRequestRecord complete;
|
||||
bzero(&complete, sizeof(complete));
|
||||
|
||||
@@ -96,6 +96,7 @@ public:
|
||||
std::map<int, Connection*> client_sockets;
|
||||
|
||||
void close_http_listeners();
|
||||
void close_listeners_except(char type);
|
||||
void read_fgci(Connection&);
|
||||
static bool is_http_like_type(char type);
|
||||
Connection* open_client_connection(int server_socket, int client_socket);
|
||||
|
||||
@@ -74,8 +74,14 @@ void compiler_code_state_consume(CompilerCodeState& state, String& buffer, const
|
||||
|
||||
if(state.inside_quote)
|
||||
{
|
||||
if(state.quote_char == c && (i == 0 || content[i-1] != '\\'))
|
||||
state.inside_quote = false;
|
||||
if(state.quote_char == c)
|
||||
{
|
||||
u32 backslashes = 0;
|
||||
for(u32 j = i; j > 0 && content[j - 1] == '\\'; j -= 1)
|
||||
backslashes += 1;
|
||||
if(backslashes % 2 == 0)
|
||||
state.inside_quote = false;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -497,8 +503,8 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
|
||||
"#include \"uce_lib.h\" \n"+
|
||||
file_get_contents(
|
||||
context->server->config["COMPILER_SYS_PATH"] + "/" + context->server->config["SETUP_TEMPLATE"]
|
||||
)+
|
||||
"#line 1\n";
|
||||
)+
|
||||
"#line 1 " + json_escape(su->file_name) + "\n";
|
||||
CompilerCodeState code_state;
|
||||
String current_line = "";
|
||||
String literal_buffer = "";
|
||||
@@ -567,7 +573,7 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
|
||||
String resolved_unit = unit_name;
|
||||
if(resolved_unit != "" && resolved_unit[0] != '/')
|
||||
resolved_unit = expand_path(resolved_unit, su->src_path);
|
||||
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit(context, resolved_unit));
|
||||
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit_for_preprocess(context, resolved_unit));
|
||||
if(sub_su)
|
||||
parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n");
|
||||
}
|
||||
|
||||
+798
-61
File diff suppressed because it is too large
Load Diff
+28
-9
@@ -1,33 +1,52 @@
|
||||
#pragma once
|
||||
|
||||
#define RENDER(X) extern "C" void __uce_render(X)
|
||||
#define COMPONENT(X) extern "C" void __uce_component(X)
|
||||
#define ONCE(X) extern "C" void __uce_once(X)
|
||||
#define INIT(X) extern "C" void __uce_init(X)
|
||||
#define WS(X) extern "C" void __uce_websocket(X)
|
||||
#define CLI(X) extern "C" void __uce_cli(X)
|
||||
#define SERVE_HTTP(X) extern "C" void __uce_serve_http(X)
|
||||
#define EXPORT extern "C"
|
||||
#if defined(__UCE_WASM_UNIT__)
|
||||
#define UCE_UNIT_EXPORT __attribute__((visibility("default")))
|
||||
#else
|
||||
#define UCE_UNIT_EXPORT
|
||||
#endif
|
||||
|
||||
#define RENDER(X) extern "C" UCE_UNIT_EXPORT void __uce_render(X)
|
||||
#define COMPONENT(X) extern "C" UCE_UNIT_EXPORT void __uce_component(X)
|
||||
#define ONCE(X) extern "C" UCE_UNIT_EXPORT void __uce_once(X)
|
||||
#define INIT(X) extern "C" UCE_UNIT_EXPORT void __uce_init(X)
|
||||
#define WS(X) extern "C" UCE_UNIT_EXPORT void __uce_websocket(X)
|
||||
#define CLI(X) extern "C" UCE_UNIT_EXPORT void __uce_cli(X)
|
||||
#define SERVE_HTTP(X) extern "C" UCE_UNIT_EXPORT void __uce_serve_http(X)
|
||||
#define EXPORT extern "C" UCE_UNIT_EXPORT
|
||||
|
||||
String preprocess_shared_unit(Request* context, SharedUnit* su);
|
||||
String compiler_generated_cpp_path(Request* context, String source_file);
|
||||
String compiler_generated_cpp_path(SharedUnit* su);
|
||||
#ifndef __UCE_WASM_UNIT__
|
||||
String compiler_unit_bin_directory(Request* context);
|
||||
String compiler_unit_wasm_path(Request* context, String source_file);
|
||||
#endif
|
||||
void setup_unit_paths(Request* context, SharedUnit* su, String file_name);
|
||||
void compile_shared_unit(Request* context, SharedUnit* su);
|
||||
SharedUnit* get_shared_unit(Request* context, String file_name);
|
||||
#ifndef __UCE_WASM_UNIT__
|
||||
SharedUnit* get_shared_unit_for_preprocess(Request* context, String file_name);
|
||||
SharedUnit* get_shared_unit_bounded(Request* context, String file_name, u64 timeout_ms, bool* timed_out);
|
||||
bool unit_compile_bounded(Request* context, String path, u64 timeout_ms, bool* timed_out, String* error = 0);
|
||||
#endif
|
||||
String compiler_error_page_unit(Request* context, String config_key);
|
||||
bool compiler_unit_compile_pending(Request* context, String file_name);
|
||||
bool compiler_unit_compile_in_progress(Request* context, String file_name);
|
||||
bool compiler_request_can_serve_stale_artifact(Request* context);
|
||||
bool compiler_preserve_last_known_good(Request* context, String file_name);
|
||||
bool compiler_unit_can_serve_stale_artifact(Request* context, String file_name);
|
||||
void compiler_prioritize_unit(Request* context, String file_name);
|
||||
StringList compiler_take_priority_units(Request* context);
|
||||
String compiler_source_generation(Request* context);
|
||||
void compiler_mark_source_generation(Request* context);
|
||||
String compiler_site_directory(Request* context);
|
||||
StringList compiler_scan_site_units(Request* context);
|
||||
StringList compiler_list_known_units(Request* context);
|
||||
void compiler_set_known_units(Request* context, StringList files);
|
||||
void compiler_track_known_unit(Request* context, String file_name);
|
||||
void compiler_untrack_known_unit(Request* context, String file_name);
|
||||
bool compiler_unit_needs_recompile(Request* context, String file_name, bool* source_missing = 0);
|
||||
bool compiler_unit_needs_recompile(Request* context, String file_name, bool* source_missing = 0, bool allow_recent_source_stat = false, bool path_is_normalized = false, bool retry_current_failure = false);
|
||||
DValue unit_info(String path = "");
|
||||
StringList units_list();
|
||||
bool unit_compile(String path = "");
|
||||
|
||||
+152
-6
@@ -652,7 +652,7 @@ void DValue::set_bool(bool b)
|
||||
_list_mode = false;
|
||||
}
|
||||
|
||||
void DValue::set(DValue source)
|
||||
void DValue::set(const DValue& source)
|
||||
{
|
||||
DValue* target = reference_target();
|
||||
if(target)
|
||||
@@ -691,6 +691,42 @@ void DValue::set(DValue source)
|
||||
}
|
||||
}
|
||||
|
||||
void DValue::set(DValue&& source)
|
||||
{
|
||||
DValue* target = reference_target();
|
||||
if(target)
|
||||
{
|
||||
target->set(std::move(source));
|
||||
return;
|
||||
}
|
||||
set_type(source.type);
|
||||
switch(type)
|
||||
{
|
||||
case('S'):
|
||||
_String = std::move(source._String);
|
||||
_list_mode = false;
|
||||
break;
|
||||
case('F'):
|
||||
_float = source._float;
|
||||
_list_mode = false;
|
||||
break;
|
||||
case('B'):
|
||||
_bool = source._bool;
|
||||
_list_mode = false;
|
||||
break;
|
||||
case('M'):
|
||||
_map = std::move(source._map);
|
||||
_array_index = source._array_index;
|
||||
_list_mode = source._list_mode;
|
||||
break;
|
||||
case('P'):
|
||||
case('R'):
|
||||
_ptr = source._ptr;
|
||||
_list_mode = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
void DValue::set(StringMap source)
|
||||
{
|
||||
DValue* target = reference_target();
|
||||
@@ -782,7 +818,7 @@ DValue& DValue::operator [] (String s) {
|
||||
void DValue::operator = (String v) { set(v); }
|
||||
void DValue::operator = (f64 v) { set(v); }
|
||||
void DValue::operator = (void* v) { set(v); }
|
||||
void DValue::operator = (DValue v) { set(v); }
|
||||
void DValue::operator = (DValue v) { set(std::move(v)); }
|
||||
void DValue::operator = (StringMap v) { set(v); }
|
||||
|
||||
void DValue::push(const DValue& child)
|
||||
@@ -885,11 +921,11 @@ bool ucb_append_varint(String& out, u64 value)
|
||||
return(true);
|
||||
}
|
||||
|
||||
bool ucb_read_varint(const String& src, size_t& offset, u64& value_out)
|
||||
bool ucb_read_varint(const char* src, size_t src_size, size_t& offset, u64& value_out)
|
||||
{
|
||||
value_out = 0;
|
||||
u32 shift = 0;
|
||||
while(offset < src.size() && shift <= 63)
|
||||
while(offset < src_size && shift <= 63)
|
||||
{
|
||||
u8 byte = (u8)src[offset++];
|
||||
value_out |= ((u64)(byte & 0x7f) << shift);
|
||||
@@ -900,6 +936,11 @@ bool ucb_read_varint(const String& src, size_t& offset, u64& value_out)
|
||||
return(false);
|
||||
}
|
||||
|
||||
bool ucb_read_varint(const String& src, size_t& offset, u64& value_out)
|
||||
{
|
||||
return(ucb_read_varint(src.data(), src.size(), offset, value_out));
|
||||
}
|
||||
|
||||
char ucb_node_type(const DValue& value)
|
||||
{
|
||||
const DValue& target = value.deref();
|
||||
@@ -1076,7 +1117,7 @@ bool ucb_decode_node(const String& src, size_t& offset, DValue& out, String& err
|
||||
DValue child;
|
||||
if(!ucb_decode_node(src, offset, child, error, depth + 1))
|
||||
return(false);
|
||||
out[key] = child;
|
||||
out[key] = std::move(child);
|
||||
}
|
||||
return(true);
|
||||
}
|
||||
@@ -1113,6 +1154,30 @@ String ucb_encode(const DValue& value)
|
||||
return(out);
|
||||
}
|
||||
|
||||
// Internal request/config transport uses the same UCEB2 map representation
|
||||
// without first building a duplicate DValue tree.
|
||||
String ucb_encode_flat_string_map(const StringMap& value)
|
||||
{
|
||||
String out;
|
||||
out.append(UCEB_MAGIC, 4);
|
||||
out.push_back((char)UCEB_VERSION);
|
||||
out.push_back(0);
|
||||
out.push_back('M');
|
||||
ucb_append_varint(out, 0);
|
||||
ucb_append_varint(out, value.size());
|
||||
for(auto& child : value)
|
||||
{
|
||||
ucb_append_varint(out, child.first.size());
|
||||
out.append(child.first.data(), child.first.size());
|
||||
out.push_back(0);
|
||||
out.push_back('S');
|
||||
ucb_append_varint(out, child.second.size());
|
||||
out.append(child.second.data(), child.second.size());
|
||||
ucb_append_varint(out, 0);
|
||||
}
|
||||
return(out);
|
||||
}
|
||||
|
||||
bool ucb_decode(const String& encoded, DValue& out, String* error_out)
|
||||
{
|
||||
String error;
|
||||
@@ -1126,7 +1191,7 @@ bool ucb_decode(const String& encoded, DValue& out, String* error_out)
|
||||
DValue decoded;
|
||||
if(ucb_decode_node(encoded, offset, decoded, error) && offset == encoded.size())
|
||||
{
|
||||
out = decoded;
|
||||
out = std::move(decoded);
|
||||
if(error_out)
|
||||
*error_out = "";
|
||||
return(true);
|
||||
@@ -1139,6 +1204,87 @@ bool ucb_decode(const String& encoded, DValue& out, String* error_out)
|
||||
return(false);
|
||||
}
|
||||
|
||||
#ifdef __UCE_WASM_CORE__
|
||||
static bool ucb_decode_flat_string_map(const char* encoded, size_t encoded_size, StringMap& out, String* error_out)
|
||||
{
|
||||
String error;
|
||||
StringMap decoded;
|
||||
size_t offset = 0;
|
||||
auto fail = [&](String message) {
|
||||
error = message;
|
||||
return(false);
|
||||
};
|
||||
if(encoded_size < 5 || memcmp(encoded, UCEB_MAGIC, 4) != 0)
|
||||
fail("missing UCEB magic header");
|
||||
else if((u8)encoded[4] != UCEB_VERSION)
|
||||
fail("unsupported UCEB version");
|
||||
else
|
||||
{
|
||||
offset = 5;
|
||||
if(encoded_size - offset < 2)
|
||||
fail("unexpected end of UCEB2 string map");
|
||||
else
|
||||
{
|
||||
u8 flags = (u8)encoded[offset++];
|
||||
char type = encoded[offset++];
|
||||
u64 scalar_len = 0, child_count = 0;
|
||||
if(flags != 0 || type != 'M')
|
||||
fail("UCEB2 string map root must be a map");
|
||||
else if(!ucb_read_varint(encoded, encoded_size, offset, scalar_len) || scalar_len != 0)
|
||||
fail("UCEB2 string map root must have no scalar");
|
||||
else if(!ucb_read_varint(encoded, encoded_size, offset, child_count))
|
||||
fail("invalid UCEB2 string map child count");
|
||||
else
|
||||
{
|
||||
for(u64 i = 0; error == "" && i < child_count; i++)
|
||||
{
|
||||
u64 key_len = 0, value_len = 0, value_children = 0;
|
||||
if(!ucb_read_varint(encoded, encoded_size, offset, key_len) || key_len > encoded_size - offset)
|
||||
{
|
||||
fail("invalid UCEB2 string map key");
|
||||
break;
|
||||
}
|
||||
String key(encoded + offset, (size_t)key_len);
|
||||
offset += (size_t)key_len;
|
||||
if(encoded_size - offset < 2)
|
||||
{
|
||||
fail("unexpected end of UCEB2 string map value");
|
||||
break;
|
||||
}
|
||||
u8 value_flags = (u8)encoded[offset++];
|
||||
char value_type = encoded[offset++];
|
||||
if(value_flags != 0 || value_type != 'S' || !ucb_read_varint(encoded, encoded_size, offset, value_len) || value_len > encoded_size - offset)
|
||||
{
|
||||
fail("UCEB2 string map value must be a string scalar");
|
||||
break;
|
||||
}
|
||||
String value(encoded + offset, (size_t)value_len);
|
||||
offset += (size_t)value_len;
|
||||
if(!ucb_read_varint(encoded, encoded_size, offset, value_children) || value_children != 0)
|
||||
{
|
||||
fail("UCEB2 string map value cannot have children");
|
||||
break;
|
||||
}
|
||||
decoded[std::move(key)] = std::move(value);
|
||||
}
|
||||
if(error == "" && offset != encoded_size)
|
||||
fail("trailing bytes after UCEB2 string map");
|
||||
}
|
||||
}
|
||||
}
|
||||
if(error != "")
|
||||
{
|
||||
if(error_out)
|
||||
*error_out = error;
|
||||
return(false);
|
||||
}
|
||||
out = std::move(decoded);
|
||||
if(error_out)
|
||||
*error_out = "";
|
||||
return(true);
|
||||
}
|
||||
#endif
|
||||
|
||||
DValue ucb_decode(const String& encoded)
|
||||
{
|
||||
DValue out;
|
||||
|
||||
+6
-1
@@ -8,6 +8,9 @@ String json_escape(String s, char quote_char = '"');
|
||||
// JSON-decoded values, and metadata trees can be consumed without repetitive
|
||||
// manual parsing at each call site.
|
||||
struct DValue {
|
||||
DValue() = default;
|
||||
DValue(const DValue&) = default;
|
||||
DValue(DValue&&) = default;
|
||||
|
||||
char type = 'S';
|
||||
|
||||
@@ -49,7 +52,8 @@ struct DValue {
|
||||
void set(void* p);
|
||||
void set(f64 f);
|
||||
void set_bool(bool b);
|
||||
void set(DValue source);
|
||||
void set(const DValue& source);
|
||||
void set(DValue&& source);
|
||||
void set(StringMap source);
|
||||
void set_array();
|
||||
void set_reference(DValue* target);
|
||||
@@ -74,6 +78,7 @@ String to_String(DValue t);
|
||||
String var_dump(const DValue& map, String prefix = "", String postfix = "\n");
|
||||
|
||||
String ucb_encode(const DValue& value);
|
||||
String ucb_encode_flat_string_map(const StringMap& value);
|
||||
DValue ucb_decode(const String& encoded);
|
||||
bool ucb_decode(const String& encoded, DValue& out, String* error_out = 0);
|
||||
|
||||
|
||||
+20
-9
@@ -140,6 +140,8 @@ bool contains(String haystack, String needle)
|
||||
|
||||
String replace(String s, String search, String replace_with)
|
||||
{
|
||||
if(search == "")
|
||||
return(s);
|
||||
s64 last_spos = 0;
|
||||
auto spos = s.find(search);
|
||||
if(spos == std::string::npos)
|
||||
@@ -791,7 +793,7 @@ DValue array_merge(DValue a, DValue b)
|
||||
StringList split_utf8(String s, bool compound_characters)
|
||||
{
|
||||
StringList result;
|
||||
auto len = s.size();
|
||||
s64 len = (s64)s.size();
|
||||
String codepoint = "";
|
||||
for(s64 i = 0; i < len; i++)
|
||||
{
|
||||
@@ -800,13 +802,13 @@ StringList split_utf8(String s, bool compound_characters)
|
||||
{
|
||||
codepoint = "";
|
||||
codepoint.append(1, c);
|
||||
if(is_bit_set(c, 6))
|
||||
if(is_bit_set(c, 6) && i + 1 < len)
|
||||
{
|
||||
codepoint.append(1, s[++i]);
|
||||
if(is_bit_set(c, 5))
|
||||
if(is_bit_set(c, 5) && i + 1 < len)
|
||||
{
|
||||
codepoint.append(1, s[++i]);
|
||||
if(is_bit_set(c, 4))
|
||||
if(is_bit_set(c, 4) && i + 1 < len)
|
||||
{
|
||||
codepoint.append(1, s[++i]);
|
||||
}
|
||||
@@ -837,7 +839,7 @@ StringList split_utf8(String s, bool compound_characters)
|
||||
join_next = true;
|
||||
last_was_regional = false;
|
||||
}
|
||||
else if(s[0] == '\xF0' && s[1] == '\x9F' && s[2] == '\x87' && s[3] >= '\xA6' && s[3] <= '\xBF') // Regional indicator letters
|
||||
else if(s.size() == 4 && s[0] == '\xF0' && s[1] == '\x9F' && s[2] == '\x87' && s[3] >= '\xA6' && s[3] <= '\xBF') // Regional indicator letters
|
||||
{
|
||||
if(last_was_regional)
|
||||
{
|
||||
@@ -850,7 +852,7 @@ StringList split_utf8(String s, bool compound_characters)
|
||||
last_was_regional = true;
|
||||
}
|
||||
}
|
||||
else if(s[0] == '\xEF' && s[1] == '\xB8' && s[2] >= '\x80' && s[2] <= '\x8F') // Variation selector
|
||||
else if(s.size() == 3 && s[0] == '\xEF' && s[1] == '\xB8' && s[2] >= '\x80' && s[2] <= '\x8F') // Variation selector
|
||||
{
|
||||
compound_result[compound_result.size()-1] += s;
|
||||
last_was_regional = false;
|
||||
@@ -929,9 +931,18 @@ u64 to_u64(String s, u64 fallback)
|
||||
String raw = trim(s);
|
||||
if(raw == "")
|
||||
return(fallback);
|
||||
char* end = 0;
|
||||
unsigned long long value = strtoull(raw.c_str(), &end, 10);
|
||||
return(end && *end == 0 ? (u64)value : fallback);
|
||||
u64 value = 0;
|
||||
u64 offset = raw[0] == '+' ? 1 : 0;
|
||||
if(offset == raw.size())
|
||||
return(fallback);
|
||||
for(; offset < raw.size(); offset++)
|
||||
{
|
||||
char c = raw[offset];
|
||||
if(c < '0' || c > '9' || value > (UINT64_MAX - (u64)(c - '0')) / 10)
|
||||
return(fallback);
|
||||
value = value * 10 + (u64)(c - '0');
|
||||
}
|
||||
return(value);
|
||||
}
|
||||
|
||||
s64 to_s64(String s, s64 fallback)
|
||||
|
||||
@@ -41,6 +41,7 @@ void SHA1Final(unsigned char digest[20], SHA1_CTX* context);
|
||||
#include <string.h>
|
||||
#include <sys/types.h> /* for u_int*_t */
|
||||
#include "hash.h"
|
||||
#include "uri.h"
|
||||
|
||||
#ifndef BYTE_ORDER
|
||||
#if (BSD >= 199103)
|
||||
@@ -422,8 +423,69 @@ bool crypto_equal_native(String a, String b)
|
||||
return(diff == 0);
|
||||
}
|
||||
|
||||
static bool uce_crypto_utf8_string(String value)
|
||||
{
|
||||
for(size_t i = 0; i < value.size();)
|
||||
{
|
||||
u8 c = (u8)value[i];
|
||||
if(c < 0x80) { i++; continue; }
|
||||
size_t need = c >= 0xC2 && c <= 0xDF ? 1 : (c >= 0xE0 && c <= 0xEF ? 2 : (c >= 0xF0 && c <= 0xF4 ? 3 : 0));
|
||||
if(need == 0 || i + need >= value.size()) return(false);
|
||||
for(size_t j = 1; j <= need; j++) if(((u8)value[i + j] & 0xC0) != 0x80) return(false);
|
||||
u8 second = (u8)value[i + 1];
|
||||
if((c == 0xE0 && second < 0xA0) || (c == 0xED && second >= 0xA0) || (c == 0xF0 && second < 0x90) || (c == 0xF4 && second >= 0x90)) return(false);
|
||||
i += need + 1;
|
||||
}
|
||||
return(true);
|
||||
}
|
||||
|
||||
static bool uce_crypto_utf8_json_string(String value)
|
||||
{
|
||||
if(!uce_crypto_utf8_string(value)) return(false);
|
||||
for(unsigned char c : value) if(c < 0x20) return(false);
|
||||
return(true);
|
||||
}
|
||||
|
||||
static bool uce_crypto_value_valid(const DValue& value, size_t depth, size_t& nodes, size_t& bytes)
|
||||
{
|
||||
if(depth > 16 || ++nodes > 256) return(false);
|
||||
const DValue& item = value.deref();
|
||||
if(item.type == 'S')
|
||||
{
|
||||
bytes += item._String.size();
|
||||
return(bytes <= 32768 && uce_crypto_utf8_json_string(item._String));
|
||||
}
|
||||
if(item.type == 'F') return(std::isfinite(item._float));
|
||||
if(item.type == 'B') return(true);
|
||||
if(item.type != 'M') return(false);
|
||||
bool list = item.is_list();
|
||||
for(const auto& child : item._map)
|
||||
{
|
||||
if(!list)
|
||||
{
|
||||
bytes += child.first.size();
|
||||
if(bytes > 32768 || !uce_crypto_utf8_json_string(child.first)) return(false);
|
||||
}
|
||||
if(!uce_crypto_value_valid(child.second, depth + 1, nodes, bytes)) return(false);
|
||||
}
|
||||
return(true);
|
||||
}
|
||||
|
||||
bool crypto_operation_request_valid(DValue request)
|
||||
{
|
||||
const DValue& root = request.deref();
|
||||
if(root.type != 'M' || root.is_list()) return(false);
|
||||
size_t nodes = 0, bytes = 0;
|
||||
return(uce_crypto_value_valid(root, 0, nodes, bytes));
|
||||
}
|
||||
|
||||
#ifndef __UCE_WASM_CORE__
|
||||
#include <openssl/bn.h>
|
||||
#include <openssl/core_names.h>
|
||||
#include <openssl/ecdsa.h>
|
||||
#include <openssl/evp.h>
|
||||
#include <openssl/params.h>
|
||||
#include <openssl/param_build.h>
|
||||
#include <openssl/rand.h>
|
||||
|
||||
namespace {
|
||||
@@ -525,4 +587,385 @@ bool password_needs_rehash_native(String encoded)
|
||||
String salt, digest;
|
||||
return(!uce_password_parts(encoded, n, r, p, salt, digest) || n != UCE_PASSWORD_SCRYPT_N || r != UCE_PASSWORD_SCRYPT_R || p != UCE_PASSWORD_SCRYPT_P);
|
||||
}
|
||||
|
||||
namespace {
|
||||
static constexpr size_t UCE_ES256_COORDINATE_BYTES = 32;
|
||||
static constexpr size_t UCE_ES256_SIGNATURE_BYTES = 64;
|
||||
static constexpr size_t UCE_ES256_JSON_MAX = 16 * 1024;
|
||||
static constexpr size_t UCE_ES256_VALUE_MAX = 256;
|
||||
static constexpr size_t UCE_ES256_DEPTH_MAX = 16;
|
||||
static constexpr size_t UCE_ES256_COORDINATE_BASE64URL_MAX = 43;
|
||||
static constexpr size_t UCE_CBOR_MAX_BYTES = 16 * 1024;
|
||||
static constexpr size_t UCE_CBOR_MAX_BASE64URL = 21846;
|
||||
static constexpr size_t UCE_ES256_DER_MAX_BYTES = 144;
|
||||
static constexpr size_t UCE_ES256_DER_BASE64URL_MAX = 192;
|
||||
|
||||
struct UcePkeyDeleter { void operator()(EVP_PKEY* value) const { EVP_PKEY_free(value); } };
|
||||
struct UcePkeyCtxDeleter { void operator()(EVP_PKEY_CTX* value) const { EVP_PKEY_CTX_free(value); } };
|
||||
struct UceEcdsaSigDeleter { void operator()(ECDSA_SIG* value) const { ECDSA_SIG_free(value); } };
|
||||
struct UceBnDeleter { void operator()(BIGNUM* value) const { BN_clear_free(value); } };
|
||||
struct UceParamBldDeleter { void operator()(OSSL_PARAM_BLD* value) const { OSSL_PARAM_BLD_free(value); } };
|
||||
struct UceParamsDeleter { void operator()(OSSL_PARAM* value) const { OSSL_PARAM_free(value); } };
|
||||
|
||||
static String uce_base64url_encode(const unsigned char* bytes, size_t size)
|
||||
{
|
||||
String encoded = base64_encode(String((const char*)bytes, size));
|
||||
encoded = replace(replace(encoded, "+", "-"), "/", "_");
|
||||
while(!encoded.empty() && encoded.back() == '=') encoded.pop_back();
|
||||
return(encoded);
|
||||
}
|
||||
|
||||
static bool uce_base64url_decode(String encoded, String& decoded, size_t max_encoded)
|
||||
{
|
||||
if(encoded.empty() || encoded.size() > max_encoded || encoded.find('=') != String::npos || encoded.size() % 4 == 1)
|
||||
return(false);
|
||||
for(char c : encoded)
|
||||
if(!(c >= 'A' && c <= 'Z') && !(c >= 'a' && c <= 'z') && !(c >= '0' && c <= '9') && c != '-' && c != '_')
|
||||
return(false);
|
||||
String padded = replace(replace(encoded, "-", "+"), "_", "/");
|
||||
while(padded.size() % 4) padded += "=";
|
||||
bool ok = false;
|
||||
decoded = base64_decode(padded, ok);
|
||||
return(ok && uce_base64url_encode((const unsigned char*)decoded.data(), decoded.size()) == encoded);
|
||||
}
|
||||
|
||||
static bool uce_es256_json_value(const DValue& value, size_t depth, size_t& values, size_t& bytes)
|
||||
{
|
||||
if(depth > UCE_ES256_DEPTH_MAX || ++values > UCE_ES256_VALUE_MAX)
|
||||
return(false);
|
||||
const DValue& item = value.deref();
|
||||
if(item.type == 'S')
|
||||
return((bytes += item._String.size()) <= UCE_ES256_JSON_MAX);
|
||||
if(item.type == 'F' || item.type == 'B')
|
||||
return(true);
|
||||
if(item.type != 'M')
|
||||
return(false);
|
||||
for(const auto& child : item._map)
|
||||
{
|
||||
if((bytes += child.first.size()) > UCE_ES256_JSON_MAX || !uce_es256_json_value(child.second, depth + 1, values, bytes))
|
||||
return(false);
|
||||
}
|
||||
return(true);
|
||||
}
|
||||
|
||||
static bool uce_es256_json_map(const DValue& value)
|
||||
{
|
||||
if(value.deref().type != 'M' || value.deref().is_list())
|
||||
return(false);
|
||||
size_t values = 0, bytes = 0;
|
||||
return(uce_es256_json_value(value, 0, values, bytes));
|
||||
}
|
||||
|
||||
static bool uce_es256_jwk_string(const DValue& jwk, const String& field, String& value)
|
||||
{
|
||||
const DValue* found = jwk.key(field);
|
||||
if(!found)
|
||||
return(false);
|
||||
const DValue& item = found->deref();
|
||||
if(item.type != 'S' || item._String.empty() || item._String.size() > 32768)
|
||||
return(false);
|
||||
value = item._String;
|
||||
return(true);
|
||||
}
|
||||
|
||||
static std::unique_ptr<EVP_PKEY, UcePkeyDeleter> uce_es256_key_from_jwk(const DValue& jwk)
|
||||
{
|
||||
if(jwk.deref().type != 'M')
|
||||
return(nullptr);
|
||||
String kty, crv, x64, y64, d64, x, y, d;
|
||||
if(!uce_es256_jwk_string(jwk, "kty", kty) || !uce_es256_jwk_string(jwk, "crv", crv) || !uce_es256_jwk_string(jwk, "x", x64) || !uce_es256_jwk_string(jwk, "y", y64) || !uce_es256_jwk_string(jwk, "d", d64) ||
|
||||
kty != "EC" || crv != "P-256" || !uce_base64url_decode(x64, x, UCE_ES256_COORDINATE_BASE64URL_MAX) || !uce_base64url_decode(y64, y, UCE_ES256_COORDINATE_BASE64URL_MAX) || !uce_base64url_decode(d64, d, UCE_ES256_COORDINATE_BASE64URL_MAX) ||
|
||||
x.size() != UCE_ES256_COORDINATE_BYTES || y.size() != UCE_ES256_COORDINATE_BYTES || d.size() != UCE_ES256_COORDINATE_BYTES)
|
||||
return(nullptr);
|
||||
unsigned char public_key[65];
|
||||
public_key[0] = 4;
|
||||
memcpy(public_key + 1, x.data(), x.size());
|
||||
memcpy(public_key + 33, y.data(), y.size());
|
||||
std::unique_ptr<BIGNUM, UceBnDeleter> private_bn(BN_bin2bn((const unsigned char*)d.data(), d.size(), 0));
|
||||
std::unique_ptr<OSSL_PARAM_BLD, UceParamBldDeleter> builder(OSSL_PARAM_BLD_new());
|
||||
if(!private_bn || !builder || OSSL_PARAM_BLD_push_utf8_string(builder.get(), OSSL_PKEY_PARAM_GROUP_NAME, "prime256v1", 0) <= 0 ||
|
||||
OSSL_PARAM_BLD_push_octet_string(builder.get(), OSSL_PKEY_PARAM_PUB_KEY, public_key, sizeof(public_key)) <= 0 ||
|
||||
OSSL_PARAM_BLD_push_BN(builder.get(), OSSL_PKEY_PARAM_PRIV_KEY, private_bn.get()) <= 0)
|
||||
return(nullptr);
|
||||
std::unique_ptr<OSSL_PARAM, UceParamsDeleter> params(OSSL_PARAM_BLD_to_param(builder.get()));
|
||||
std::unique_ptr<EVP_PKEY_CTX, UcePkeyCtxDeleter> ctx(EVP_PKEY_CTX_new_from_name(0, "EC", 0));
|
||||
EVP_PKEY* raw = 0;
|
||||
if(!params || !ctx || EVP_PKEY_fromdata_init(ctx.get()) <= 0 || EVP_PKEY_fromdata(ctx.get(), &raw, EVP_PKEY_KEYPAIR, params.get()) <= 0)
|
||||
return(nullptr);
|
||||
std::unique_ptr<EVP_PKEY, UcePkeyDeleter> key(raw);
|
||||
std::unique_ptr<EVP_PKEY_CTX, UcePkeyCtxDeleter> check(EVP_PKEY_CTX_new(key.get(), 0));
|
||||
return(check && EVP_PKEY_public_check(check.get()) > 0 && EVP_PKEY_private_check(check.get()) > 0 && EVP_PKEY_pairwise_check(check.get()) > 0 ? std::move(key) : nullptr);
|
||||
}
|
||||
|
||||
static bool uce_es256_key_coordinates(EVP_PKEY* key, String& x, String& y, String& d)
|
||||
{
|
||||
unsigned char public_key[65]; size_t public_key_size = sizeof(public_key);
|
||||
BIGNUM* private_bn = 0;
|
||||
if(EVP_PKEY_get_octet_string_param(key, OSSL_PKEY_PARAM_PUB_KEY, public_key, sizeof(public_key), &public_key_size) <= 0 || public_key_size != sizeof(public_key) || public_key[0] != 4 ||
|
||||
EVP_PKEY_get_bn_param(key, OSSL_PKEY_PARAM_PRIV_KEY, &private_bn) <= 0)
|
||||
return(false);
|
||||
std::unique_ptr<BIGNUM, UceBnDeleter> private_key(private_bn);
|
||||
if(BN_bn2binpad(private_key.get(), (unsigned char*)d.data(), UCE_ES256_COORDINATE_BYTES) != UCE_ES256_COORDINATE_BYTES)
|
||||
return(false);
|
||||
x.assign((const char*)public_key + 1, UCE_ES256_COORDINATE_BYTES);
|
||||
y.assign((const char*)public_key + 33, UCE_ES256_COORDINATE_BYTES);
|
||||
return(true);
|
||||
}
|
||||
|
||||
enum class UceCborKind { Unsigned, Negative, Bytes, Text, Array, Map };
|
||||
struct UceCbor { UceCborKind kind; u64 number = 0; String bytes; std::vector<UceCbor> items; };
|
||||
static constexpr size_t UCE_CBOR_MAX_NODES = 256, UCE_CBOR_MAX_DEPTH = 16;
|
||||
|
||||
static bool uce_cbor_uint(const String& in, size_t& p, u8 ai, u64& n)
|
||||
{
|
||||
if(ai < 24) { n = ai; return(true); }
|
||||
size_t count = ai == 24 ? 1 : ai == 25 ? 2 : ai == 26 ? 4 : ai == 27 ? 8 : 0;
|
||||
if(!count || p > in.size() || count > in.size() - p) return(false);
|
||||
n = 0;
|
||||
for(size_t i = 0; i < count; i++)
|
||||
{
|
||||
if(n > (UINT64_MAX - (u8)in[p + i]) / 256) return(false);
|
||||
n = n * 256 + (u8)in[p + i];
|
||||
}
|
||||
p += count;
|
||||
return((count == 1 && n >= 24) || (count == 2 && n > 0xff) || (count == 4 && n > 0xffff) || (count == 8 && n > 0xffffffff));
|
||||
}
|
||||
|
||||
static bool uce_cbor_equal(const UceCbor& left, const UceCbor& right)
|
||||
{
|
||||
if(left.kind != right.kind || left.number != right.number || left.bytes != right.bytes || left.items.size() != right.items.size()) return(false);
|
||||
for(size_t i = 0; i < left.items.size(); i++) if(!uce_cbor_equal(left.items[i], right.items[i])) return(false);
|
||||
return(true);
|
||||
}
|
||||
|
||||
static bool uce_cbor_read(const String& in, size_t& p, UceCbor& out, size_t depth, size_t& nodes)
|
||||
{
|
||||
if(p >= in.size() || depth > UCE_CBOR_MAX_DEPTH || ++nodes > UCE_CBOR_MAX_NODES) return(false);
|
||||
u8 header = (u8)in[p++], major = header >> 5, ai = header & 31;
|
||||
u64 n = 0;
|
||||
if(ai == 31 || !uce_cbor_uint(in, p, ai, n)) return(false);
|
||||
if(major == 0 || major == 1)
|
||||
{
|
||||
out.kind = major ? UceCborKind::Negative : UceCborKind::Unsigned;
|
||||
out.number = n;
|
||||
return(true);
|
||||
}
|
||||
if(major == 2 || major == 3)
|
||||
{
|
||||
if(n > UCE_CBOR_MAX_BYTES || n > in.size() - p || (major == 3 && !uce_crypto_utf8_string(String(in.data() + p, (size_t)n)))) return(false);
|
||||
out.kind = major == 2 ? UceCborKind::Bytes : UceCborKind::Text;
|
||||
out.bytes.assign(in.data() + p, (size_t)n);
|
||||
p += (size_t)n;
|
||||
return(true);
|
||||
}
|
||||
if((major != 4 && major != 5) || n > UCE_CBOR_MAX_NODES) return(false);
|
||||
size_t item_count = (size_t)n * (major == 5 ? 2 : 1);
|
||||
if(item_count > UCE_CBOR_MAX_NODES - nodes) return(false);
|
||||
out.kind = major == 4 ? UceCborKind::Array : UceCborKind::Map;
|
||||
out.items.reserve(item_count);
|
||||
for(size_t i = 0; i < item_count; i++)
|
||||
{
|
||||
UceCbor child;
|
||||
if(!uce_cbor_read(in, p, child, depth + 1, nodes)) return(false);
|
||||
if(major == 5 && !(i & 1))
|
||||
for(size_t previous = 0; previous < out.items.size(); previous += 2)
|
||||
if(uce_cbor_equal(out.items[previous], child)) return(false);
|
||||
out.items.push_back(std::move(child));
|
||||
}
|
||||
return(true);
|
||||
}
|
||||
static DValue uce_cbor_value(const UceCbor& value)
|
||||
{
|
||||
DValue out; switch(value.kind) { case UceCborKind::Unsigned: out["type"]="unsigned"; out["value"]=std::to_string(value.number); break; case UceCborKind::Negative: out["type"]="negative"; out["value"]=value.number==UINT64_MAX?"-18446744073709551616":"-"+std::to_string(value.number+1); break; case UceCborKind::Bytes: out["type"]="bytes"; out["base64url"]=uce_base64url_encode((const unsigned char*)value.bytes.data(),value.bytes.size()); break; case UceCborKind::Text: out["type"]="text"; out["value"]=value.bytes; break; case UceCborKind::Array: out["type"]="array"; out["items"].set_array(); for(const auto& x:value.items) out["items"].push(uce_cbor_value(x)); break; case UceCborKind::Map: out["type"]="map"; out["entries"].set_array(); for(size_t i=0;i<value.items.size();i+=2) { DValue entry; entry["key"]=uce_cbor_value(value.items[i]); entry["value"]=uce_cbor_value(value.items[i+1]); out["entries"].push(entry); } } return out;
|
||||
}
|
||||
static bool uce_cbor_es256(const UceCbor& cose,String& x,String& y)
|
||||
{
|
||||
if(cose.kind!=UceCborKind::Map) return false; const UceCbor *kty=0,*alg=0,*crv=0,*xx=0,*yy=0;
|
||||
for(size_t i=0;i<cose.items.size();i+=2) { const UceCbor& k=cose.items[i]; const UceCbor& v=cose.items[i+1]; if(k.kind!=UceCborKind::Unsigned&&k.kind!=UceCborKind::Negative) continue; bool neg=k.kind==UceCborKind::Negative; if(!neg&&k.number==1) kty=&v; else if(!neg&&k.number==3) alg=&v; else if(neg&&k.number==0) crv=&v; else if(neg&&k.number==1) xx=&v; else if(neg&&k.number==2) yy=&v; }
|
||||
return(kty&&alg&&crv&&xx&&yy&&kty->kind==UceCborKind::Unsigned&&kty->number==2&&alg->kind==UceCborKind::Negative&&alg->number==6&&crv->kind==UceCborKind::Unsigned&&crv->number==1&&xx->kind==UceCborKind::Bytes&&yy->kind==UceCborKind::Bytes&&xx->bytes.size()==32&&yy->bytes.size()==32&&(x=xx->bytes,true)&&(y=yy->bytes,true));
|
||||
}
|
||||
static std::unique_ptr<EVP_PKEY,UcePkeyDeleter> uce_es256_public_key(String x,String y)
|
||||
{
|
||||
if(x.size()!=32||y.size()!=32) return nullptr; unsigned char point[65]={4}; memcpy(point+1,x.data(),32); memcpy(point+33,y.data(),32); OSSL_PARAM p[]={OSSL_PARAM_construct_utf8_string(OSSL_PKEY_PARAM_GROUP_NAME,(char*)"prime256v1",0),OSSL_PARAM_construct_octet_string(OSSL_PKEY_PARAM_PUB_KEY,point,sizeof(point)),OSSL_PARAM_construct_end()}; std::unique_ptr<EVP_PKEY_CTX,UcePkeyCtxDeleter> ctx(EVP_PKEY_CTX_new_from_name(0,"EC",0)); EVP_PKEY* raw=0; if(!ctx||EVP_PKEY_fromdata_init(ctx.get())<=0||EVP_PKEY_fromdata(ctx.get(),&raw,EVP_PKEY_PUBLIC_KEY,p)<=0) return nullptr; std::unique_ptr<EVP_PKEY,UcePkeyDeleter> key(raw); std::unique_ptr<EVP_PKEY_CTX,UcePkeyCtxDeleter> check(EVP_PKEY_CTX_new(key.get(),0)); return check&&EVP_PKEY_public_check(check.get())>0?std::move(key):nullptr;
|
||||
}
|
||||
static bool uce_es256_cose(String encoded,String& x,String& y)
|
||||
{
|
||||
String raw; UceCbor cose; size_t p=0,nodes=0; return uce_base64url_decode(encoded, raw, UCE_CBOR_MAX_BASE64URL) && raw.size() <= UCE_CBOR_MAX_BYTES && uce_cbor_read(raw, p, cose, 0, nodes) && p == raw.size() && uce_cbor_es256(cose, x, y) && uce_es256_public_key(x, y);
|
||||
}
|
||||
|
||||
static DValue uce_es256_jwk(String x, String y, String d = "")
|
||||
{
|
||||
DValue jwk;
|
||||
jwk["kty"] = "EC";
|
||||
jwk["crv"] = "P-256";
|
||||
jwk["x"] = uce_base64url_encode((const unsigned char*)x.data(), x.size());
|
||||
jwk["y"] = uce_base64url_encode((const unsigned char*)y.data(), y.size());
|
||||
if(d != "") jwk["d"] = uce_base64url_encode((const unsigned char*)d.data(), d.size());
|
||||
return(jwk);
|
||||
}
|
||||
|
||||
static String uce_es256_thumbprint(const DValue& public_jwk)
|
||||
{
|
||||
const DValue* x = public_jwk.key("x");
|
||||
const DValue* y = public_jwk.key("y");
|
||||
if(!x || !y)
|
||||
return("");
|
||||
String canonical = "{\"crv\":\"P-256\",\"kty\":\"EC\",\"x\":\"" + x->to_string() + "\",\"y\":\"" + y->to_string() + "\"}";
|
||||
String digest = sha256_native(canonical);
|
||||
return(uce_base64url_encode((const unsigned char*)digest.data(), digest.size()));
|
||||
}
|
||||
}
|
||||
|
||||
static DValue uce_es256_key_create()
|
||||
{
|
||||
std::unique_ptr<EVP_PKEY_CTX, UcePkeyCtxDeleter> ctx(EVP_PKEY_CTX_new_from_name(0, "EC", 0));
|
||||
EVP_PKEY* raw = 0;
|
||||
OSSL_PARAM params[] = { OSSL_PARAM_construct_utf8_string(OSSL_PKEY_PARAM_GROUP_NAME, (char*)"prime256v1", 0), OSSL_PARAM_construct_end() };
|
||||
if(!ctx || EVP_PKEY_keygen_init(ctx.get()) <= 0 || EVP_PKEY_CTX_set_params(ctx.get(), params) <= 0 || EVP_PKEY_generate(ctx.get(), &raw) <= 0)
|
||||
return(DValue());
|
||||
std::unique_ptr<EVP_PKEY, UcePkeyDeleter> key(raw);
|
||||
String x(UCE_ES256_COORDINATE_BYTES, 0), y(UCE_ES256_COORDINATE_BYTES, 0), d(UCE_ES256_COORDINATE_BYTES, 0);
|
||||
if(!uce_es256_key_coordinates(key.get(), x, y, d))
|
||||
return(DValue());
|
||||
DValue result;
|
||||
result["public_jwk"] = uce_es256_jwk(x, y);
|
||||
String kid = uce_es256_thumbprint(result["public_jwk"]);
|
||||
result["public_jwk"]["kid"] = kid;
|
||||
result["private_jwk"] = uce_es256_jwk(x, y, d);
|
||||
result["private_jwk"]["kid"] = kid;
|
||||
result["kid"] = kid;
|
||||
result["thumbprint"] = kid;
|
||||
return(result);
|
||||
}
|
||||
|
||||
static String uce_es256_jwt(DValue private_jwk, DValue protected_header, DValue claims)
|
||||
{
|
||||
if(!uce_es256_json_map(protected_header) || !uce_es256_json_map(claims))
|
||||
return("");
|
||||
std::unique_ptr<EVP_PKEY, UcePkeyDeleter> key = uce_es256_key_from_jwk(private_jwk);
|
||||
if(!key)
|
||||
return("");
|
||||
protected_header["alg"] = "ES256";
|
||||
String header_json = json_encode(protected_header);
|
||||
String claims_json = json_encode(claims);
|
||||
if(header_json.size() > UCE_ES256_JSON_MAX || claims_json.size() > UCE_ES256_JSON_MAX)
|
||||
return("");
|
||||
String signing_input = uce_base64url_encode((const unsigned char*)header_json.data(), header_json.size()) + "." + uce_base64url_encode((const unsigned char*)claims_json.data(), claims_json.size());
|
||||
std::unique_ptr<EVP_MD_CTX, decltype(&EVP_MD_CTX_free)> ctx(EVP_MD_CTX_new(), EVP_MD_CTX_free);
|
||||
size_t der_size = 0;
|
||||
if(!ctx || EVP_DigestSignInit(ctx.get(), 0, EVP_sha256(), 0, key.get()) <= 0 || EVP_DigestSign(ctx.get(), 0, &der_size, (const unsigned char*)signing_input.data(), signing_input.size()) <= 0 || der_size == 0 || der_size > 256)
|
||||
return("");
|
||||
String der(der_size, 0);
|
||||
if(EVP_DigestSign(ctx.get(), (unsigned char*)der.data(), &der_size, (const unsigned char*)signing_input.data(), signing_input.size()) <= 0)
|
||||
return("");
|
||||
const unsigned char* cursor = (const unsigned char*)der.data();
|
||||
std::unique_ptr<ECDSA_SIG, UceEcdsaSigDeleter> signature(d2i_ECDSA_SIG(0, &cursor, der_size));
|
||||
const BIGNUM *r = 0, *s = 0;
|
||||
if(!signature || cursor != (const unsigned char*)der.data() + der_size)
|
||||
return("");
|
||||
ECDSA_SIG_get0(signature.get(), &r, &s);
|
||||
String jose(UCE_ES256_SIGNATURE_BYTES, 0);
|
||||
if(!r || !s || BN_bn2binpad(r, (unsigned char*)jose.data(), UCE_ES256_COORDINATE_BYTES) != UCE_ES256_COORDINATE_BYTES || BN_bn2binpad(s, (unsigned char*)jose.data() + UCE_ES256_COORDINATE_BYTES, UCE_ES256_COORDINATE_BYTES) != UCE_ES256_COORDINATE_BYTES)
|
||||
return("");
|
||||
return(signing_input + "." + uce_base64url_encode((const unsigned char*)jose.data(), jose.size()));
|
||||
}
|
||||
|
||||
DValue crypto_operation_native(DValue request)
|
||||
{
|
||||
DValue result;
|
||||
result["ok"].set_bool(false);
|
||||
if(!crypto_operation_request_valid(request))
|
||||
{
|
||||
result["error"] = "invalid_request";
|
||||
return(result);
|
||||
}
|
||||
String operation, algorithm;
|
||||
if(!uce_es256_jwk_string(request, "operation", operation) || !uce_es256_jwk_string(request, "algorithm", algorithm))
|
||||
{
|
||||
result["error"] = "invalid_request";
|
||||
return(result);
|
||||
}
|
||||
if(algorithm != "ES256")
|
||||
{
|
||||
result["error"] = "unsupported_algorithm";
|
||||
return(result);
|
||||
}
|
||||
if(operation == "key_generate")
|
||||
{
|
||||
DValue key = uce_es256_key_create();
|
||||
if(key["private_jwk"]["d"].to_string() == "")
|
||||
{
|
||||
result["error"] = "operation_failed";
|
||||
return(result);
|
||||
}
|
||||
key["ok"].set_bool(true);
|
||||
key["operation"] = operation;
|
||||
key["algorithm"] = algorithm;
|
||||
return(key);
|
||||
}
|
||||
if(operation == "cbor_decode")
|
||||
{
|
||||
String encoded, raw; UceCbor value; size_t p=0,nodes=0;
|
||||
if(!uce_es256_jwk_string(request, "cbor_base64url", encoded) || !uce_base64url_decode(encoded, raw, UCE_CBOR_MAX_BASE64URL) || raw.size() > UCE_CBOR_MAX_BYTES || !uce_cbor_read(raw, p, value, 0, nodes) || p != raw.size()) { result["error"] = "invalid_cbor"; return(result); }
|
||||
result["ok"].set_bool(true); result["operation"]=operation; result["value"]=uce_cbor_value(value); return result;
|
||||
}
|
||||
if(operation == "cose_es256_parse")
|
||||
{
|
||||
String encoded,x,y; if(!uce_es256_jwk_string(request,"cose_key_base64url",encoded)||!uce_es256_cose(encoded,x,y)) { result["error"]="invalid_cose_key"; return result; }
|
||||
result["ok"].set_bool(true); result["operation"]=operation; result["algorithm"]=algorithm; result["x_base64url"]=uce_base64url_encode((const unsigned char*)x.data(),x.size()); result["y_base64url"]=uce_base64url_encode((const unsigned char*)y.data(),y.size()); return result;
|
||||
}
|
||||
if(operation == "es256_verify")
|
||||
{
|
||||
String encoded, message64, signature64, x, y, message, der;
|
||||
if(!uce_es256_jwk_string(request, "cose_key_base64url", encoded) || !uce_es256_jwk_string(request, "message_base64url", message64) || !uce_es256_jwk_string(request, "signature_der_base64url", signature64) || !uce_es256_cose(encoded, x, y) || !uce_base64url_decode(message64, message, UCE_CBOR_MAX_BASE64URL) || !uce_base64url_decode(signature64, der, UCE_ES256_DER_BASE64URL_MAX) || message.size() > UCE_CBOR_MAX_BYTES || der.empty() || der.size() > UCE_ES256_DER_MAX_BYTES)
|
||||
{
|
||||
result["error"] = "invalid_key_or_payload";
|
||||
return(result);
|
||||
}
|
||||
const unsigned char* cursor = (const unsigned char*)der.data();
|
||||
std::unique_ptr<ECDSA_SIG, UceEcdsaSigDeleter> signature(d2i_ECDSA_SIG(0, &cursor, der.size()));
|
||||
int canonical = signature ? i2d_ECDSA_SIG(signature.get(), 0) : 0;
|
||||
String reencoded(canonical > 0 ? (size_t)canonical : 0, 0);
|
||||
unsigned char* dest = (unsigned char*)reencoded.data();
|
||||
if(!signature || cursor != (const unsigned char*)der.data() + der.size() || canonical <= 0 || i2d_ECDSA_SIG(signature.get(), &dest) != canonical || reencoded != der)
|
||||
{
|
||||
result["error"] = "invalid_signature";
|
||||
return(result);
|
||||
}
|
||||
std::unique_ptr<EVP_PKEY, UcePkeyDeleter> key = uce_es256_public_key(x, y);
|
||||
std::unique_ptr<EVP_MD_CTX, decltype(&EVP_MD_CTX_free)> ctx(EVP_MD_CTX_new(), EVP_MD_CTX_free);
|
||||
if(!key || !ctx || EVP_DigestVerifyInit(ctx.get(), 0, EVP_sha256(), 0, key.get()) <= 0)
|
||||
{
|
||||
result["error"] = "operation_failed";
|
||||
return(result);
|
||||
}
|
||||
int verified = EVP_DigestVerify(ctx.get(), (const unsigned char*)der.data(), der.size(), (const unsigned char*)message.data(), message.size());
|
||||
if(verified < 0)
|
||||
{
|
||||
result["error"] = "operation_failed";
|
||||
return(result);
|
||||
}
|
||||
result["ok"].set_bool(true);
|
||||
result["operation"] = operation;
|
||||
result["algorithm"] = algorithm;
|
||||
result["valid"].set_bool(verified == 1);
|
||||
return(result);
|
||||
}
|
||||
if(operation == "jwt_sign")
|
||||
{
|
||||
String jwt = uce_es256_jwt(request["private_jwk"], request["protected_header"], request["claims"]);
|
||||
if(jwt == "")
|
||||
{
|
||||
result["error"] = "invalid_key_or_payload";
|
||||
return(result);
|
||||
}
|
||||
result["ok"].set_bool(true);
|
||||
result["operation"] = operation;
|
||||
result["algorithm"] = algorithm;
|
||||
result["jwt"] = jwt;
|
||||
return(result);
|
||||
}
|
||||
result["error"] = "unsupported_operation";
|
||||
return(result);
|
||||
}
|
||||
#endif
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
#pragma once
|
||||
|
||||
struct DValue;
|
||||
|
||||
/* ================ sha1.h ================ */
|
||||
/*
|
||||
SHA-1 in C
|
||||
@@ -27,6 +29,8 @@ bool crypto_equal_native(String a, String b);
|
||||
String password_hash_native(String password);
|
||||
bool password_verify_native(String password, String encoded);
|
||||
bool password_needs_rehash_native(String encoded);
|
||||
bool crypto_operation_request_valid(DValue request);
|
||||
DValue crypto_operation_native(DValue request);
|
||||
String sha256(String data);
|
||||
String sha256_hex(String data);
|
||||
String hmac_sha256(String key, String data);
|
||||
|
||||
@@ -1218,6 +1218,10 @@ String markdown_render_node(DValue node, DValue& options)
|
||||
else if(type == "heading")
|
||||
{
|
||||
s64 level = int_val(markdown_get_string(node, "level", "1"));
|
||||
s64 heading_offset = int_val(markdown_get_string(options, "heading_offset", "0"));
|
||||
if(heading_offset < -5) heading_offset = -5;
|
||||
if(heading_offset > 5) heading_offset = 5;
|
||||
level += heading_offset;
|
||||
if(level < 1) level = 1;
|
||||
if(level > 6) level = 6;
|
||||
html = String("<h") + level + ">" + children_html + String("</h") + level + ">";
|
||||
|
||||
+70
-16
@@ -3,7 +3,7 @@
|
||||
#include <stdlib.h>
|
||||
#include "mysql-connector.h"
|
||||
|
||||
// Same-credential mysql_connect() calls lease one request-local connection.
|
||||
// Same-target mysql_connect() calls lease one request-local connection.
|
||||
// mysql_disconnect() releases a lease; cleanup_mysql_connections() owns the
|
||||
// actual close at request end, including exception/fatal recovery paths.
|
||||
static void mysql_register_request_connection(MySQL* db)
|
||||
@@ -15,8 +15,12 @@ static void mysql_register_request_connection(MySQL* db)
|
||||
connections.push_back(db);
|
||||
}
|
||||
|
||||
bool MySQL::connect(String host, String username, String password)
|
||||
bool MySQL::connect(String host, String username, String password, String database)
|
||||
{
|
||||
request_host = host;
|
||||
request_username = username;
|
||||
request_password = password;
|
||||
request_database = database;
|
||||
// Register regardless of outcome: tracking is about the wrapper's
|
||||
// lifetime, not the connection's. disconnect()/~MySQL() unregister.
|
||||
mysql_register_request_connection(this);
|
||||
@@ -32,7 +36,7 @@ bool MySQL::connect(String host, String username, String password)
|
||||
}
|
||||
|
||||
if (mysql_real_connect((MYSQL*)connection, host.c_str(), username.c_str(), password.c_str(),
|
||||
NULL, 0, NULL, 0) == NULL)
|
||||
database == "" ? NULL : database.c_str(), 0, NULL, 0) == NULL)
|
||||
{
|
||||
auto e = mysql_error((MYSQL*)connection);
|
||||
fprintf(stderr, "%s\n", e);
|
||||
@@ -59,7 +63,17 @@ bool MySQL::connect(String host, String username, String password)
|
||||
|
||||
bool MySQL::reset_connection()
|
||||
{
|
||||
if(!connection || mysql_reset_connection((MYSQL*)connection) != 0)
|
||||
if(!connection)
|
||||
return(false);
|
||||
MYSQL* mysql = (MYSQL*)connection;
|
||||
String selected_database = mysql->db == NULL ? "" : mysql->db;
|
||||
// RESET CONNECTION deliberately preserves the selected database. An
|
||||
// unqualified lease cannot safely inherit one selected by its prior request.
|
||||
if(request_database == "" && selected_database != "")
|
||||
return(false);
|
||||
if(mysql_reset_connection(mysql) != 0)
|
||||
return(false);
|
||||
if(request_database != "" && selected_database != request_database && mysql_select_db(mysql, request_database.c_str()) != 0)
|
||||
return(false);
|
||||
_preload_next_error_code = 0;
|
||||
affected_rows = 0;
|
||||
@@ -226,13 +240,16 @@ DValue MySQL::query(String q)
|
||||
}
|
||||
if(!connection)
|
||||
{
|
||||
_preload_next_error_code = CR_UNKNOWN_ERROR;
|
||||
statement_info = "mysql connection is not open";
|
||||
if(_preload_next_error_code == 0)
|
||||
_preload_next_error_code = CR_UNKNOWN_ERROR;
|
||||
if(statement_info == "")
|
||||
statement_info = "mysql connection is not open";
|
||||
return(DValue());
|
||||
}
|
||||
_preload_next_error_code = mysql_query((MYSQL*)connection, q.c_str());
|
||||
int query_status = mysql_query((MYSQL*)connection, q.c_str());
|
||||
_preload_next_error_code = query_status == 0 ? 0 : mysql_errno((MYSQL*)connection);
|
||||
DValue result;
|
||||
if(_preload_next_error_code == 0)
|
||||
if(query_status == 0)
|
||||
result = get_pending_result();
|
||||
return(result);
|
||||
}
|
||||
@@ -275,11 +292,16 @@ static bool mysql_has_unquoted_positional_placeholder(String query)
|
||||
|
||||
DValue MySQL::query(String q, StringMap params)
|
||||
{
|
||||
// Positional ? placeholders survive named substitution (values are always
|
||||
// quoted by escape()), so the check in query(String) covers this path too.
|
||||
return(query(
|
||||
parse_query_parameters(q, params).c_str()
|
||||
));
|
||||
// Positional ? placeholders survive named substitution (ordinary values
|
||||
// are quoted by escape()), so the check in query(String) covers this path.
|
||||
parameter_error = false;
|
||||
String parsed = parse_query_parameters(q, params);
|
||||
if(parameter_error)
|
||||
{
|
||||
_preload_next_error_code = CR_UNKNOWN_ERROR;
|
||||
return(DValue());
|
||||
}
|
||||
return(query(parsed));
|
||||
}
|
||||
|
||||
String MySQL::parse_query_parameters(String query, StringMap map)
|
||||
@@ -289,6 +311,7 @@ String MySQL::parse_query_parameters(String query, StringMap map)
|
||||
|
||||
u8 mode = 0;
|
||||
char quote;
|
||||
bool escaped = false;
|
||||
String identifier;
|
||||
for(u32 i = 0; i < query.length(); i++)
|
||||
{
|
||||
@@ -305,6 +328,7 @@ String MySQL::parse_query_parameters(String query, StringMap map)
|
||||
result.append(1, c);
|
||||
mode = 2;
|
||||
quote = c;
|
||||
escaped = false;
|
||||
}
|
||||
else
|
||||
{
|
||||
@@ -313,10 +337,25 @@ String MySQL::parse_query_parameters(String query, StringMap map)
|
||||
}
|
||||
else if(mode == 1) // identifier mode
|
||||
{
|
||||
if(isalnum(c) || c == '_')
|
||||
if(isalnum((unsigned char)c) || c == '_')
|
||||
{
|
||||
identifier.append(1, c);
|
||||
}
|
||||
else if(c == '!' && query[i + 1] != '=')
|
||||
{
|
||||
String value = map[identifier];
|
||||
bool valid = identifier != "" && value != "";
|
||||
for(char digit : value)
|
||||
if(!isdigit((unsigned char)digit)) valid = false;
|
||||
if(!valid)
|
||||
{
|
||||
parameter_error = true;
|
||||
statement_info = "mysql unsigned parameter :" + identifier + "! must contain only decimal digits";
|
||||
return("");
|
||||
}
|
||||
result.append(value);
|
||||
mode = 0;
|
||||
}
|
||||
else
|
||||
{
|
||||
result.append(escape(map[identifier]));
|
||||
@@ -326,9 +365,19 @@ String MySQL::parse_query_parameters(String query, StringMap map)
|
||||
}
|
||||
else if(mode == 2) // quoted mode
|
||||
{
|
||||
result.append(1, c);
|
||||
if(escaped)
|
||||
{
|
||||
escaped = false;
|
||||
continue;
|
||||
}
|
||||
if(c == '\\')
|
||||
{
|
||||
escaped = true;
|
||||
continue;
|
||||
}
|
||||
if(c == quote)
|
||||
mode = 0;
|
||||
result.append(1, c);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -374,10 +423,15 @@ String MySQL::error()
|
||||
case(CR_OUT_OF_MEMORY):
|
||||
p = "Out of memory";
|
||||
break;
|
||||
default:
|
||||
case(CR_UNKNOWN_ERROR):
|
||||
p = "Unknown server error";
|
||||
break;
|
||||
default:
|
||||
if(connection && mysql_error((MYSQL*)connection)[0] != '\0')
|
||||
p = mysql_error((MYSQL*)connection);
|
||||
else
|
||||
p = "Unknown server error";
|
||||
break;
|
||||
}
|
||||
_preload_next_error_code = 0;
|
||||
return(p);
|
||||
|
||||
@@ -20,6 +20,7 @@ struct MySQL {
|
||||
u32 row_count = 0;
|
||||
u64 insert_id = 0;
|
||||
String statement_info = ""; //
|
||||
bool parameter_error = false;
|
||||
bool request_cleanup_delete = false;
|
||||
bool request_pooled = false;
|
||||
bool worker_persistent = false;
|
||||
@@ -27,10 +28,11 @@ struct MySQL {
|
||||
String request_host;
|
||||
String request_username;
|
||||
String request_password;
|
||||
String request_database;
|
||||
|
||||
std::vector<MySQLFieldInfo> field_info;
|
||||
|
||||
bool connect(String host = "localhost", String username = "root", String password = "");
|
||||
bool connect(String host = "localhost", String username = "root", String password = "", String database = "");
|
||||
bool reset_connection();
|
||||
void disconnect();
|
||||
String error();
|
||||
@@ -46,14 +48,14 @@ struct MySQL {
|
||||
|
||||
};
|
||||
|
||||
inline MySQL* mysql_connect(String host = "localhost", String username = "root", String password = "")
|
||||
inline MySQL* mysql_connect(String host = "localhost", String username = "root", String password = "", String database = "")
|
||||
{
|
||||
if(context)
|
||||
{
|
||||
for(void* raw : context->resources.mysql_connections)
|
||||
{
|
||||
MySQL* db = (MySQL*)raw;
|
||||
if(db && db->request_pooled && db->connection && db->request_host == host && db->request_username == username && db->request_password == password)
|
||||
if(db && db->request_pooled && db->connection && db->request_host == host && db->request_username == username && db->request_password == password && db->request_database == database)
|
||||
{
|
||||
db->request_leases++;
|
||||
return(db);
|
||||
@@ -67,10 +69,19 @@ inline MySQL* mysql_connect(String host = "localhost", String username = "root",
|
||||
db->request_host = host;
|
||||
db->request_username = username;
|
||||
db->request_password = password;
|
||||
db->connect(host, username, password);
|
||||
db->request_database = database;
|
||||
db->connect(host, username, password, database);
|
||||
return(db);
|
||||
}
|
||||
|
||||
// A failed connection retains a request-owned wrapper so mysql_error() remains
|
||||
// available. Callers that need to decide whether to issue work must test the
|
||||
// connection state rather than only the wrapper pointer.
|
||||
inline bool mysql_connected(MySQL* m)
|
||||
{
|
||||
return(m != 0 && m->connection != 0);
|
||||
}
|
||||
|
||||
inline void mysql_disconnect(MySQL* m)
|
||||
{
|
||||
if(!m)
|
||||
|
||||
+468
-21
@@ -64,6 +64,7 @@ int uce_host_crypto_equal(const char* a, size_t a_len, const char* b, size_t b_l
|
||||
size_t uce_host_password_hash(const char* password, size_t password_len, char* out, size_t cap);
|
||||
int uce_host_password_verify(const char* password, size_t password_len, const char* encoded, size_t encoded_len);
|
||||
int uce_host_password_needs_rehash(const char* encoded, size_t encoded_len);
|
||||
size_t uce_host_crypto_operation(const char* in, size_t in_len, char* out, size_t cap);
|
||||
size_t uce_host_http_request(const char* in, size_t in_len, char* out, size_t cap);
|
||||
uint64_t uce_host_http_request_async(const char* in, size_t in_len);
|
||||
size_t uce_host_shell_exec_dv(const char* in, size_t in_len, char* out, size_t cap);
|
||||
@@ -284,6 +285,25 @@ String password_hash(String password)
|
||||
}
|
||||
bool password_verify(String password, String encoded) { return(uce_host_password_verify(password.data(), password.size(), encoded.data(), encoded.size()) != 0); }
|
||||
bool password_needs_rehash(String encoded) { return(uce_host_password_needs_rehash(encoded.data(), encoded.size()) != 0); }
|
||||
DValue crypto_operation(DValue request)
|
||||
{
|
||||
if(!crypto_operation_request_valid(request))
|
||||
{
|
||||
DValue result;
|
||||
result["ok"].set_bool(false);
|
||||
result["error"] = "invalid_request";
|
||||
return(result);
|
||||
}
|
||||
String encoded = ucb_encode(request);
|
||||
size_t required = uce_host_crypto_operation(encoded.data(), encoded.size(), 0, 0);
|
||||
if(required == 0 || required > 64 * 1024)
|
||||
return(DValue());
|
||||
String out(required, 0);
|
||||
size_t got = uce_host_crypto_operation(encoded.data(), encoded.size(), &out[0], required);
|
||||
DValue result;
|
||||
String error;
|
||||
return(got <= required && ucb_decode(String(out.data(), got), result, &error) ? result : DValue());
|
||||
}
|
||||
String base64_decode(String raw) { return(wasm_string_hostcall_1(uce_host_base64_decode, raw)); }
|
||||
String random_bytes(u64 n)
|
||||
{
|
||||
@@ -305,6 +325,7 @@ u64 http_request_async(DValue req)
|
||||
return((u64)uce_host_http_request_async(encoded.data(), encoded.size()));
|
||||
}
|
||||
|
||||
|
||||
DValue shell_exec(DValue spec)
|
||||
{
|
||||
String encoded = ucb_encode(spec);
|
||||
@@ -737,6 +758,7 @@ StringMap default_config()
|
||||
#include <fcntl.h>
|
||||
#include <limits.h>
|
||||
#include <sys/file.h>
|
||||
#include <poll.h>
|
||||
#include <errno.h>
|
||||
#include "sys.h"
|
||||
#include "hash.h"
|
||||
@@ -752,10 +774,12 @@ bool crypto_equal(String a, String b) { return(crypto_equal_native(a, b)); }
|
||||
String password_hash(String password) { return(password_hash_native(password)); }
|
||||
bool password_verify(String password, String encoded) { return(password_verify_native(password, encoded)); }
|
||||
bool password_needs_rehash(String encoded) { return(password_needs_rehash_native(encoded)); }
|
||||
DValue crypto_operation(DValue request) { return(crypto_operation_native(request)); }
|
||||
|
||||
// Single definitions for the native split build (declared extern in sys.h).
|
||||
pid_t parent_pid = 0;
|
||||
pid_t my_pid = 0;
|
||||
bool task_child_process = false;
|
||||
|
||||
namespace {
|
||||
|
||||
@@ -1291,6 +1315,185 @@ String socket_read(u64 sockfd, u32 max_length, u32 timeout)
|
||||
return("");
|
||||
}
|
||||
|
||||
static bool memcache_decimal(String value, u64& result)
|
||||
{
|
||||
if(value == "")
|
||||
return(false);
|
||||
result = 0;
|
||||
for(char c : value)
|
||||
{
|
||||
if(c < '0' || c > '9' || result > (UINT64_MAX - (u64)(c - '0')) / 10)
|
||||
return(false);
|
||||
result = result * 10 + (u64)(c - '0');
|
||||
}
|
||||
return(true);
|
||||
}
|
||||
|
||||
static String memcache_command_line(const String& command)
|
||||
{
|
||||
size_t end = command.find("\r\n");
|
||||
return(trim(command.substr(0, end)));
|
||||
}
|
||||
|
||||
bool uce_memcache_command_has_no_reply(const String& command)
|
||||
{
|
||||
StringList words = split(memcache_command_line(command), " ");
|
||||
if(words.size() == 0)
|
||||
return(false);
|
||||
String verb = to_lower(words[0]);
|
||||
return(verb == "quit" || to_lower(words.back()) == "noreply");
|
||||
}
|
||||
|
||||
MemcacheResponseState uce_memcache_response_advance(const String& command, const String& response, MemcacheResponseParser& parser)
|
||||
{
|
||||
while(true)
|
||||
{
|
||||
if(parser.mode == MemcacheResponseParser::Mode::FirstLine)
|
||||
{
|
||||
size_t line_end = response.find("\r\n");
|
||||
if(line_end == String::npos)
|
||||
return(MemcacheResponseState::Incomplete);
|
||||
String line = response.substr(0, line_end);
|
||||
if(line == "ERROR" || line.rfind("CLIENT_ERROR", 0) == 0 || line.rfind("SERVER_ERROR", 0) == 0 || line == "RESET" || line == "OK")
|
||||
return(MemcacheResponseState::Complete);
|
||||
if(line.rfind("VALUE ", 0) == 0)
|
||||
parser.mode = MemcacheResponseParser::Mode::RetrievalHeader;
|
||||
else if(line.rfind("VA ", 0) == 0)
|
||||
{
|
||||
StringList fields = split(line, " ");
|
||||
u64 length = 0;
|
||||
if(fields.size() < 2 || !memcache_decimal(fields[1], length) || length > SIZE_MAX - line_end - 4)
|
||||
return(MemcacheResponseState::Malformed);
|
||||
parser.data_end = line_end + 2 + (size_t)length;
|
||||
parser.mode = MemcacheResponseParser::Mode::MetaData;
|
||||
}
|
||||
else
|
||||
{
|
||||
StringList request_words = split(memcache_command_line(command), " ");
|
||||
String request_verb = request_words.size() ? to_lower(request_words[0]) : String("");
|
||||
bool end_terminated = line.rfind("STAT ", 0) == 0 || line.rfind("ITEM ", 0) == 0 ||
|
||||
request_verb == "stats" || (request_verb == "lru_crawler" && request_words.size() > 1 && to_lower(request_words[1]) == "metadump");
|
||||
if(!end_terminated)
|
||||
return(MemcacheResponseState::Complete);
|
||||
parser.mode = MemcacheResponseParser::Mode::Lines;
|
||||
}
|
||||
}
|
||||
|
||||
if(parser.mode == MemcacheResponseParser::Mode::RetrievalHeader)
|
||||
{
|
||||
size_t line_end = response.find("\r\n", parser.offset);
|
||||
if(line_end == String::npos)
|
||||
return(MemcacheResponseState::Incomplete);
|
||||
String line = response.substr(parser.offset, line_end - parser.offset);
|
||||
if(line == "END")
|
||||
return(MemcacheResponseState::Complete);
|
||||
StringList fields = split(line, " ");
|
||||
u64 length = 0;
|
||||
if(fields.size() < 4 || fields[0] != "VALUE" || !memcache_decimal(fields[3], length) || length > SIZE_MAX - line_end - 4)
|
||||
return(MemcacheResponseState::Malformed);
|
||||
parser.data_end = line_end + 2 + (size_t)length;
|
||||
parser.mode = MemcacheResponseParser::Mode::RetrievalData;
|
||||
}
|
||||
|
||||
if(parser.mode == MemcacheResponseParser::Mode::RetrievalData)
|
||||
{
|
||||
if(response.size() < parser.data_end + 2)
|
||||
return(MemcacheResponseState::Incomplete);
|
||||
if(response.compare(parser.data_end, 2, "\r\n") != 0)
|
||||
return(MemcacheResponseState::Malformed);
|
||||
parser.offset = parser.data_end + 2;
|
||||
parser.mode = MemcacheResponseParser::Mode::RetrievalHeader;
|
||||
}
|
||||
|
||||
if(parser.mode == MemcacheResponseParser::Mode::MetaData)
|
||||
{
|
||||
if(response.size() < parser.data_end + 2)
|
||||
return(MemcacheResponseState::Incomplete);
|
||||
return(response.compare(parser.data_end, 2, "\r\n") == 0 ? MemcacheResponseState::Complete : MemcacheResponseState::Malformed);
|
||||
}
|
||||
|
||||
if(parser.mode == MemcacheResponseParser::Mode::Lines)
|
||||
{
|
||||
size_t line_end = response.find("\r\n", parser.offset);
|
||||
if(line_end == String::npos)
|
||||
return(MemcacheResponseState::Incomplete);
|
||||
if(response.compare(parser.offset, line_end - parser.offset, "END") == 0)
|
||||
return(MemcacheResponseState::Complete);
|
||||
parser.offset = line_end + 2;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static bool native_socket_wait(int fd, short events, u64 deadline)
|
||||
{
|
||||
while(true)
|
||||
{
|
||||
u64 now = monotonic_ms();
|
||||
if(now >= deadline)
|
||||
return(false);
|
||||
pollfd item = {fd, events, 0};
|
||||
int timeout = (int)std::min<u64>(deadline - now, INT_MAX);
|
||||
int ready = poll(&item, 1, timeout);
|
||||
if(ready > 0)
|
||||
return((item.revents & events) != 0);
|
||||
if(ready == 0 || errno != EINTR)
|
||||
return(false);
|
||||
}
|
||||
}
|
||||
|
||||
static String native_memcache_exchange(u64 connection, String command)
|
||||
{
|
||||
static constexpr size_t max_response = 8 * 1024 * 1024;
|
||||
int fd = (int)connection;
|
||||
u64 deadline = monotonic_ms() + 1000;
|
||||
String request = command + "\r\n";
|
||||
size_t sent = 0;
|
||||
while(sent < request.size())
|
||||
{
|
||||
if(!native_socket_wait(fd, POLLOUT, deadline))
|
||||
break;
|
||||
ssize_t count = send(fd, request.data() + sent, request.size() - sent, MSG_DONTWAIT | MSG_NOSIGNAL);
|
||||
if(count > 0)
|
||||
sent += (size_t)count;
|
||||
else if(count < 0 && (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK))
|
||||
continue;
|
||||
else
|
||||
break;
|
||||
}
|
||||
if(sent != request.size())
|
||||
{
|
||||
shutdown(fd, SHUT_RDWR);
|
||||
return("");
|
||||
}
|
||||
if(uce_memcache_command_has_no_reply(command))
|
||||
return("");
|
||||
|
||||
String response;
|
||||
response.reserve(16 * 1024);
|
||||
MemcacheResponseParser parser;
|
||||
while(response.size() < max_response)
|
||||
{
|
||||
MemcacheResponseState state = uce_memcache_response_advance(command, response, parser);
|
||||
if(state == MemcacheResponseState::Complete)
|
||||
return(response);
|
||||
if(state == MemcacheResponseState::Malformed || !native_socket_wait(fd, POLLIN, deadline))
|
||||
break;
|
||||
char buffer[64 * 1024];
|
||||
size_t remaining = max_response - response.size();
|
||||
ssize_t count = recv(fd, buffer, std::min(remaining, sizeof(buffer)), MSG_DONTWAIT);
|
||||
if(count > 0)
|
||||
response.append(buffer, (size_t)count);
|
||||
else if(count < 0 && (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK))
|
||||
continue;
|
||||
else
|
||||
break;
|
||||
}
|
||||
if(uce_memcache_response_advance(command, response, parser) == MemcacheResponseState::Complete)
|
||||
return(response);
|
||||
shutdown(fd, SHUT_RDWR);
|
||||
return("");
|
||||
}
|
||||
|
||||
String memcache_escape_key(String key)
|
||||
{
|
||||
String result;
|
||||
@@ -1320,26 +1523,18 @@ u64 memcache_connect(String host, u16 port)
|
||||
|
||||
String memcache_command(u64 connection, String command)
|
||||
{
|
||||
socket_write(connection, command+"\r\n");
|
||||
return(socket_read(connection)); // FIXME: do multi-chunk until END line is received!
|
||||
return(native_memcache_exchange(connection, command));
|
||||
}
|
||||
|
||||
bool memcache_set(u64 connection, String key, String value, u64 expires_in)
|
||||
{
|
||||
socket_write(connection,
|
||||
// set KEY META_DATA EXPIRY_TIME LENGTH_IN_BYTES
|
||||
String("set ") + memcache_escape_key(key) + " 0 " + std::to_string(expires_in) + " " + std::to_string(value.length()) + "\r\n" +
|
||||
value + "\r\n");
|
||||
return("STORED" == trim(socket_read(connection)));
|
||||
return("STORED" == trim(memcache_command(connection,
|
||||
String("set ") + memcache_escape_key(key) + " 0 " + std::to_string(expires_in) + " " + std::to_string(value.length()) + "\r\n" + value)));
|
||||
}
|
||||
|
||||
bool memcache_delete(u64 connection, String key)
|
||||
{
|
||||
socket_write(connection,
|
||||
// set KEY META_DATA EXPIRY_TIME LENGTH_IN_BYTES
|
||||
String("delete ") + memcache_escape_key(key) + "\r\n"
|
||||
);
|
||||
return("DELETED" == trim(socket_read(connection)));
|
||||
return("DELETED" == trim(memcache_command(connection, String("delete ") + memcache_escape_key(key))));
|
||||
}
|
||||
|
||||
String memcache_get(u64 connection, String key, String default_value)
|
||||
@@ -1389,6 +1584,7 @@ void on_segfault(int sig)
|
||||
|
||||
struct Worker {
|
||||
pid_t pid;
|
||||
char listener_type = 0;
|
||||
};
|
||||
|
||||
std::map<pid_t, Worker> workers;
|
||||
@@ -1396,27 +1592,262 @@ std::map<pid_t, Worker> workers;
|
||||
#include <sys/resource.h>
|
||||
#include <sys/prctl.h>
|
||||
|
||||
pid_t spawn_subprocess(std::function<void()> exec_after_spawn)
|
||||
// The process-wide reaper may run on a runtime thread while a request thread
|
||||
// synchronously owns a child. Preserve unknown statuses until that owner takes
|
||||
// them instead of forcing it to guess success after waitpid reports ECHILD.
|
||||
static const size_t child_exit_status_capacity = 256;
|
||||
static volatile sig_atomic_t child_exit_status_cursor = 0;
|
||||
static volatile sig_atomic_t child_exit_status_pids[child_exit_status_capacity] = {0};
|
||||
static volatile sig_atomic_t child_exit_status_values[child_exit_status_capacity] = {0};
|
||||
static volatile sig_atomic_t child_exit_status_sequences[child_exit_status_capacity] = {0};
|
||||
|
||||
static void child_exit_status_publish(pid_t pid, int status)
|
||||
{
|
||||
sig_atomic_t cursor = __atomic_fetch_add(&child_exit_status_cursor, 1, __ATOMIC_RELAXED);
|
||||
size_t slot = (size_t)(unsigned int)cursor % child_exit_status_capacity;
|
||||
__atomic_store_n(&child_exit_status_pids[slot], 0, __ATOMIC_RELEASE);
|
||||
__atomic_store_n(&child_exit_status_values[slot], (sig_atomic_t)status, __ATOMIC_RELAXED);
|
||||
__atomic_store_n(&child_exit_status_sequences[slot], cursor + 1, __ATOMIC_RELAXED);
|
||||
__atomic_store_n(&child_exit_status_pids[slot], (sig_atomic_t)pid, __ATOMIC_RELEASE);
|
||||
}
|
||||
|
||||
unsigned int child_exit_status_snapshot()
|
||||
{
|
||||
return((unsigned int)__atomic_load_n(&child_exit_status_cursor, __ATOMIC_ACQUIRE));
|
||||
}
|
||||
|
||||
bool child_exit_status_take(pid_t pid, int& status, unsigned int since)
|
||||
{
|
||||
for(size_t slot = 0; slot < child_exit_status_capacity; slot++)
|
||||
{
|
||||
unsigned int sequence = (unsigned int)__atomic_load_n(&child_exit_status_sequences[slot], __ATOMIC_ACQUIRE);
|
||||
unsigned int distance = sequence - since;
|
||||
if(sequence == 0 || distance == 0 || distance >= 0x80000000u)
|
||||
continue;
|
||||
sig_atomic_t expected = (sig_atomic_t)pid;
|
||||
if(__atomic_compare_exchange_n(&child_exit_status_pids[slot], &expected, 0, false, __ATOMIC_ACQUIRE, __ATOMIC_RELAXED))
|
||||
{
|
||||
status = (int)__atomic_load_n(&child_exit_status_values[slot], __ATOMIC_RELAXED);
|
||||
return(true);
|
||||
}
|
||||
}
|
||||
return(false);
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
class ProcessSigchldBlock
|
||||
{
|
||||
sigset_t previous;
|
||||
bool blocked = false;
|
||||
public:
|
||||
ProcessSigchldBlock()
|
||||
{
|
||||
sigset_t mask;
|
||||
sigemptyset(&mask);
|
||||
sigaddset(&mask, SIGCHLD);
|
||||
blocked = pthread_sigmask(SIG_BLOCK, &mask, &previous) == 0;
|
||||
}
|
||||
void restore()
|
||||
{
|
||||
if(blocked)
|
||||
{
|
||||
pthread_sigmask(SIG_SETMASK, &previous, 0);
|
||||
blocked = false;
|
||||
}
|
||||
}
|
||||
~ProcessSigchldBlock() { restore(); }
|
||||
};
|
||||
|
||||
}
|
||||
|
||||
DValue process_exec(String cmd, String input, StringMap env, u64 timeout_ms, u64 output_limit)
|
||||
{
|
||||
DValue result;
|
||||
result["exit_code"] = (f64)-1;
|
||||
result["stdout"] = "";
|
||||
result["stderr"] = "";
|
||||
result["timed_out"].set_bool(false);
|
||||
result["output_truncated"].set_bool(false);
|
||||
if(timeout_ms == 0)
|
||||
timeout_ms = 5000;
|
||||
int inpipe[2] = {-1, -1}, outpipe[2] = {-1, -1}, errpipe[2] = {-1, -1};
|
||||
if(pipe(inpipe) != 0 || pipe(outpipe) != 0 || pipe(errpipe) != 0)
|
||||
{
|
||||
for(int fd : {inpipe[0], inpipe[1], outpipe[0], outpipe[1], errpipe[0], errpipe[1]})
|
||||
if(fd >= 0)
|
||||
close(fd);
|
||||
result["stderr"] = "pipe failed";
|
||||
return(result);
|
||||
}
|
||||
ProcessSigchldBlock sigchld;
|
||||
unsigned int child_status_snapshot_value = child_exit_status_snapshot();
|
||||
pid_t pid = fork();
|
||||
if(pid < 0)
|
||||
{
|
||||
close(inpipe[0]); close(inpipe[1]); close(outpipe[0]); close(outpipe[1]); close(errpipe[0]); close(errpipe[1]);
|
||||
result["stderr"] = "fork failed";
|
||||
return(result);
|
||||
}
|
||||
if(pid == 0)
|
||||
{
|
||||
sigchld.restore();
|
||||
setpgid(0, 0);
|
||||
pid_t expected_parent = getppid();
|
||||
prctl(PR_SET_PDEATHSIG, SIGKILL);
|
||||
if(getppid() != expected_parent)
|
||||
_exit(127);
|
||||
dup2(inpipe[0], 0); dup2(outpipe[1], 1); dup2(errpipe[1], 2);
|
||||
close(inpipe[0]); close(inpipe[1]); close(outpipe[0]); close(outpipe[1]); close(errpipe[0]); close(errpipe[1]);
|
||||
for(auto& value : env)
|
||||
setenv(value.first.c_str(), value.second.c_str(), 1);
|
||||
execl("/bin/sh", "sh", "-c", cmd.c_str(), (char*)0);
|
||||
_exit(127);
|
||||
}
|
||||
setpgid(pid, pid);
|
||||
close(inpipe[0]); close(outpipe[1]); close(errpipe[1]);
|
||||
fcntl(inpipe[1], F_SETFL, fcntl(inpipe[1], F_GETFL, 0) | O_NONBLOCK);
|
||||
fcntl(outpipe[0], F_SETFL, fcntl(outpipe[0], F_GETFL, 0) | O_NONBLOCK);
|
||||
fcntl(errpipe[0], F_SETFL, fcntl(errpipe[0], F_GETFL, 0) | O_NONBLOCK);
|
||||
size_t input_offset = 0;
|
||||
bool input_open = true, output_open = true, error_open = true, exited = false, status_valid = false;
|
||||
int status = 0;
|
||||
u64 now_ms = monotonic_ms();
|
||||
u64 deadline = timeout_ms > UINT64_MAX - now_ms ? UINT64_MAX : now_ms + timeout_ms;
|
||||
auto append_output = [&](String key, const char* data, size_t length) {
|
||||
String current = result[key].to_string();
|
||||
u64 captured = result["stdout"].to_string().size() + result["stderr"].to_string().size();
|
||||
size_t accepted = length;
|
||||
if(output_limit > 0)
|
||||
{
|
||||
u64 available = captured < output_limit ? output_limit - captured : 0;
|
||||
accepted = std::min<size_t>(accepted, (size_t)available);
|
||||
if(accepted < length)
|
||||
result["output_truncated"].set_bool(true);
|
||||
}
|
||||
if(accepted > 0)
|
||||
result[key] = current + String(data, accepted);
|
||||
};
|
||||
while(output_open || error_open || !exited)
|
||||
{
|
||||
if(!exited)
|
||||
{
|
||||
pid_t waited = waitpid(pid, &status, WNOHANG);
|
||||
if(waited == pid)
|
||||
{
|
||||
exited = true;
|
||||
status_valid = true;
|
||||
}
|
||||
else if(waited < 0 && errno == ECHILD)
|
||||
{
|
||||
u64 transfer_deadline = monotonic_ms() + 50;
|
||||
do
|
||||
{
|
||||
status_valid = child_exit_status_take(pid, status, child_status_snapshot_value);
|
||||
if(!status_valid)
|
||||
sched_yield();
|
||||
} while(!status_valid && monotonic_ms() < transfer_deadline);
|
||||
exited = true;
|
||||
if(!status_valid)
|
||||
result["stderr"] = result["stderr"].to_string() + "lost child exit status";
|
||||
}
|
||||
}
|
||||
if(input_open)
|
||||
{
|
||||
if(input_offset < input.size())
|
||||
{
|
||||
ssize_t written = write(inpipe[1], input.data() + input_offset, input.size() - input_offset);
|
||||
if(written > 0)
|
||||
input_offset += (size_t)written;
|
||||
else if(written < 0 && errno != EINTR && errno != EAGAIN && errno != EWOULDBLOCK)
|
||||
{
|
||||
close(inpipe[1]);
|
||||
input_open = false;
|
||||
}
|
||||
}
|
||||
else
|
||||
{
|
||||
close(inpipe[1]);
|
||||
input_open = false;
|
||||
}
|
||||
}
|
||||
char buffer[4096];
|
||||
ssize_t length;
|
||||
while((length = read(outpipe[0], buffer, sizeof(buffer))) > 0)
|
||||
append_output("stdout", buffer, (size_t)length);
|
||||
if(length == 0 && output_open)
|
||||
{
|
||||
close(outpipe[0]);
|
||||
output_open = false;
|
||||
}
|
||||
while((length = read(errpipe[0], buffer, sizeof(buffer))) > 0)
|
||||
append_output("stderr", buffer, (size_t)length);
|
||||
if(length == 0 && error_open)
|
||||
{
|
||||
close(errpipe[0]);
|
||||
error_open = false;
|
||||
}
|
||||
if((output_open || error_open || !exited) && monotonic_ms() >= deadline)
|
||||
{
|
||||
result["timed_out"].set_bool(true);
|
||||
kill(-pid, SIGKILL);
|
||||
kill(pid, SIGKILL);
|
||||
if(!exited)
|
||||
status_valid = waitpid(pid, &status, 0) == pid;
|
||||
exited = true;
|
||||
if(input_open) { close(inpipe[1]); input_open = false; }
|
||||
if(output_open) { close(outpipe[0]); output_open = false; }
|
||||
if(error_open) { close(errpipe[0]); error_open = false; }
|
||||
}
|
||||
if(output_open || error_open || !exited)
|
||||
{
|
||||
u64 remaining_ms = deadline > monotonic_ms() ? deadline - monotonic_ms() : 0;
|
||||
if(remaining_ms > 0)
|
||||
usleep((useconds_t)std::min<u64>(1000, remaining_ms * 1000));
|
||||
}
|
||||
}
|
||||
if(status_valid && WIFEXITED(status))
|
||||
result["exit_code"] = (f64)WEXITSTATUS(status);
|
||||
else if(status_valid && WIFSIGNALED(status))
|
||||
result["exit_code"] = (f64)(128 + WTERMSIG(status));
|
||||
return(result);
|
||||
}
|
||||
|
||||
pid_t spawn_subprocess(std::function<void()> exec_after_spawn, char listener_type = 0)
|
||||
{
|
||||
parent_pid = getpid();
|
||||
pid_t p;
|
||||
p = fork();
|
||||
sigset_t blocked;
|
||||
sigset_t previous;
|
||||
sigemptyset(&blocked);
|
||||
sigaddset(&blocked, SIGCHLD);
|
||||
if(sigprocmask(SIG_BLOCK, &blocked, &previous) != 0)
|
||||
{
|
||||
perror("sigprocmask");
|
||||
return(0);
|
||||
}
|
||||
pid_t p = fork();
|
||||
if(p == 0)
|
||||
{
|
||||
sigprocmask(SIG_SETMASK, &previous, 0);
|
||||
my_pid = getpid();
|
||||
//printf("(C) child procress started, PID:%i\n", my_pid);
|
||||
prctl(PR_SET_PDEATHSIG, SIGHUP);
|
||||
exec_after_spawn();
|
||||
return(0);
|
||||
}
|
||||
else
|
||||
if(p < 0)
|
||||
{
|
||||
Worker w;
|
||||
w.pid = p;
|
||||
workers[w.pid] = w;
|
||||
printf("(P) child procress spawned: PID %i\n", p);
|
||||
return(p);
|
||||
perror("fork worker");
|
||||
sigprocmask(SIG_SETMASK, &previous, 0);
|
||||
return(0);
|
||||
}
|
||||
Worker w;
|
||||
w.pid = p;
|
||||
w.listener_type = listener_type;
|
||||
workers[w.pid] = w;
|
||||
sigprocmask(SIG_SETMASK, &previous, 0);
|
||||
printf("(P) child procress spawned: PID %i\n", p);
|
||||
return(p);
|
||||
}
|
||||
|
||||
String runtime_safe_key(String key, String label)
|
||||
@@ -1567,6 +1998,7 @@ pid_t task(String key, std::function<void()> exec_after_spawn, u64 timeout)
|
||||
{
|
||||
close_locked_file(lock_fd);
|
||||
my_pid = getpid();
|
||||
task_child_process = true;
|
||||
// The FastCGI worker handles termination to drain accepted requests.
|
||||
// Generic task children do not run that drain loop, so inheriting those
|
||||
// handlers would turn task_kill(SIGTERM) into a no-op.
|
||||
@@ -1648,6 +2080,7 @@ void on_child_exit(int sig)
|
||||
}
|
||||
else
|
||||
{
|
||||
child_exit_status_publish(pid, status);
|
||||
printf("(P) task child reaped (PID:%i)\n", pid);
|
||||
}
|
||||
}
|
||||
@@ -1676,10 +2109,15 @@ StringMap make_server_settings()
|
||||
cfg["BIN_DIRECTORY"] = "/tmp/uce/work";
|
||||
cfg["WASM_COMPILE_SCRIPT"] = "scripts/compile_wasm_unit";
|
||||
cfg["WASM_BACKEND_VERBOSE"] = "0";
|
||||
cfg["WASM_PROFILE_HOSTCALL_CPU"] = "0";
|
||||
cfg["WASM_PROFILE_THREAD_RUNTIME"] = "0";
|
||||
cfg["WASM_CORE_PATH"] = "";
|
||||
cfg["WASM_MEMORY_LIMIT_BYTES"] = std::to_string(512ull * 1024 * 1024);
|
||||
cfg["WASM_EPOCH_DEADLINE_TICKS"] = "200";
|
||||
cfg["WASM_EPOCH_PERIOD_MS"] = "50";
|
||||
cfg["WASM_SERIALIZE_TIMEOUT_SECONDS"] = "120";
|
||||
cfg["MYSQL_PERSISTENT_POOL_SIZE"] = "8";
|
||||
cfg["MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS"] = "300";
|
||||
cfg["SETUP_TEMPLATE"] = "scripts/setup.h.template";
|
||||
cfg["LIT_ESC"] = "3d5b5_1";
|
||||
cfg["CONTENT_TYPE"] = "text/html; charset=utf-8";
|
||||
@@ -1687,6 +2125,11 @@ StringMap make_server_settings()
|
||||
cfg["FCGI_SOCKET_MODE"] = "0666";
|
||||
cfg["CLI_SOCKET_PATH"] = "/run/uce/cli.sock";
|
||||
cfg["CLI_SOCKET_MODE"] = "0600";
|
||||
// Zero preserves the legacy shared renderer pool. A positive count keeps
|
||||
// trusted CLI/test module caches out of public FastCGI workers.
|
||||
cfg["CLI_WORKER_COUNT"] = "0";
|
||||
// Bound retained CLI/test module state. Zero disables recycling.
|
||||
cfg["CLI_WORKER_MAX_REQUESTS"] = "8";
|
||||
// Command socket the WS broker listens on; workers flush ws_* dispatch
|
||||
// command batches here at workspace teardown.
|
||||
cfg["WS_BROKER_SOCKET_PATH"] = "/run/uce/ws-broker.sock";
|
||||
@@ -1698,9 +2141,13 @@ StringMap make_server_settings()
|
||||
cfg["SESSION_COOKIE_SECURE"] = "0";
|
||||
cfg["COMPILER_SYS_PATH"] = ".";
|
||||
cfg["PRECOMPILE_FILES_IN"] = "";
|
||||
cfg["PRECOMPILE_JOBS"] = "2";
|
||||
cfg["SITE_DIRECTORY"] = "site";
|
||||
cfg["JIT_COMPILE_ON_REQUEST"] = "1";
|
||||
cfg["SHOW_DYNAMIC_COMPILE_ERRORS"] = "1";
|
||||
cfg["SERVE_LAST_KNOWN_GOOD"] = "0";
|
||||
cfg["PROACTIVE_COMPILE_ENABLED"] = "1";
|
||||
cfg["PROACTIVE_COMPILE_JOBS"] = "2";
|
||||
cfg["COMPILE_FAILURE_RETRY_SECONDS"] = std::to_string(10);
|
||||
cfg["PROACTIVE_COMPILE_CHECK_INTERVAL"] = std::to_string(60);
|
||||
cfg["TRANSPORT_MAX_CLIENT_CONNECTIONS"] = std::to_string(256);
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user