177 Commits
Author SHA1 Message Date
root 09f743fdcf Accept units from configured source roots 2026-08-16 13:02:58 +00:00
root 36d6207bf2 Add optional Datastar plumbing 2026-07-30 19:41:52 +00:00
root d0efab7db0 Harden dynamic HTTP and compiler boundaries 2026-07-26 11:28:11 +00:00
udo 3d155203bd Add bounded structured crypto operations 2026-07-22 13:43:21 +00:00
udo 68b73343a2 Isolate CLI workers and module serialization 2026-07-21 03:25:51 +00:00
root 10e6565037 Read complete bounded Memcached responses 2026-07-20 23:24:57 +00:00
udo 0c75703186 Prove retained connectors close on shutdown 2026-07-20 21:59:32 +00:00
udo 91c745c44b Keep socket listener active during deployment 2026-07-20 14:07:28 +00:00
udo e22cbf8882 Close retained connectors on worker shutdown 2026-07-20 14:04:39 +00:00
root 80eaf0b69b Retire idle persistent MySQL connections 2026-07-20 13:00:29 +00:00
udo 4d68377704 Clarify direct precompile timeout contract 2026-07-20 10:09:04 +00:00
udo a48f6af502 Stabilize metadata deadline mutation gate 2026-07-20 00:01:59 +00:00
root 88e2ad022d Isolate proactive compile policy test 2026-07-19 23:34:56 +00:00
root 93cbc83b82 Expose dynamic compile failures 2026-07-19 23:15:39 +00:00
root cd993b34e9 Reject unsupported server arguments 2026-07-19 23:15:22 +00:00
udo 881dba078a Bound Wasm artifact metadata loading 2026-07-19 15:43:52 +00:00
udo 15fdfe725d Buffer Wasm metadata reads 2026-07-19 06:43:19 +00:00
udo a9e24cffc4 Stabilize foreign artifact timeout coverage 2026-07-19 05:38:31 +00:00
udo ccbe945d62 Handle foreign-owned compile artifacts 2026-07-19 05:20:08 +00:00
udo 4f010d37a0 Cache positive entry freshness checks 2026-07-19 04:24:45 +00:00
udo d95fb38183 Bound request-time Wasm compilation 2026-07-19 03:02:15 +00:00
udo a7576f3db7 Enforce absolute Wasm invocation deadlines 2026-07-19 00:44:18 +00:00
udo fd6d472187 Clean filesystem site-test temp artifacts 2026-07-18 23:22:17 +00:00
udo eb3d2fe9d3 Clean component test unit caches 2026-07-18 22:43:18 +00:00
udo 14f4920ae8 Avoid duplicate proactive recompilation 2026-07-18 22:23:44 +00:00
udo 33a7b00c78 Parallelize proactive unit compilation 2026-07-18 20:19:37 +00:00
udo d4ae51f54b Isolate managed precompile race validation 2026-07-18 19:12:53 +00:00
udo 322ffd006f Complete unit render demo 2026-07-18 17:27:19 +00:00
udo 3a6db7779b Reject precompile generations changed in flight 2026-07-18 17:23:10 +00:00
udo f379480783 Expect missing component renders to fail 2026-07-18 17:03:55 +00:00
udo 1df3becdb2 Fail unresolved unit renders with HTTP 500 2026-07-18 16:59:14 +00:00
udo 3cbe6f50d4 Fail unresolved component renders with HTTP 500 2026-07-18 16:55:42 +00:00
udo 39d4b41e2b Reject negative unsigned string values 2026-07-18 16:22:14 +00:00
udo 22efea8ecd Reject persisted failures during precompile 2026-07-18 15:44:44 +00:00
udo b9b70659a3 Parallelize managed unit precompile 2026-07-18 15:24:51 +00:00
udo 39636acf2e Keep precompile PCH generation-local 2026-07-18 14:31:53 +00:00
udo 1952c7d762 Stage ABI-scoped unit generations 2026-07-18 14:09:38 +00:00
udo 810c19b042 add database-aware MySQL pooling 2026-07-18 12:55:48 +00:00
udo 418c5812dc Return component ONCE slots with handler resolution 2026-07-18 11:47:28 +00:00
udo 0e0c0b6ccc Keep demand compilation responsive 2026-07-18 11:14:40 +00:00
udo 495a8ae5f0 Resolve wasm traps to unit source 2026-07-18 10:58:08 +00:00
udo eda124b15f Trim compiled unit export surface 2026-07-18 10:11:06 +00:00
udo 69e8051fd3 Cache read component path resolutions 2026-07-18 09:31:40 +00:00
udo 2a2d802daa Fix raw HTTP request timing logs 2026-07-18 08:54:49 +00:00
udo 0517f06925 Deduplicate component resolution probes 2026-07-18 06:46:21 +00:00
udo f745ce9413 Deduplicate entry freshness filesystem work 2026-07-18 06:16:49 +00:00
udo 556025a779 Profile pre-dispatch WASM readiness 2026-07-18 05:43:36 +00:00
udo a5ad5042b7 Reuse loaded component units per request 2026-07-18 05:01:27 +00:00
udo 5e140d6a5a Decode request maps directly into Wasm workspaces 2026-07-18 04:21:28 +00:00
udo 516ed94032 Cache immutable Wasm server configuration encoding 2026-07-18 03:43:44 +00:00
udo a043fb8fc7 Avoid resolving loaded request entries twice 2026-07-18 03:08:01 +00:00
udo 339031b6c3 Avoid repeated warm request dispatch work 2026-07-18 02:32:28 +00:00
udo 72fd0dd53a Pre-instantiate WASM core workers 2026-07-18 01:58:01 +00:00
root f89e33e241 Frame brokered HTTP responses once 2026-07-18 01:33:57 +00:00
udo 6e4ef8ed7a Reuse core host definitions across requests 2026-07-18 00:55:29 +00:00
udo 1ebf94b135 Prove pooled workspace request isolation 2026-07-17 23:53:46 +00:00
udo 497a5e89d2 Reuse Wasmtime instance allocation slots 2026-07-17 23:48:31 +00:00
udo da7c607091 Treat absent source generation as uninitialized 2026-07-17 23:28:10 +00:00
udo e176ff09d9 Invalidate component freshness by source generation 2026-07-17 23:14:42 +00:00
udo 075ce0bc07 Pass normalized component paths to freshness checks 2026-07-17 23:00:17 +00:00
udo 1c78ea4f22 Avoid repeated realpath graph traversal 2026-07-17 22:59:42 +00:00
udo 37b03eb4d5 Resolve compiler source paths through file descriptors 2026-07-17 22:51:46 +00:00
udo 914cf17f23 Skip canonicalization for recent source entries 2026-07-17 22:06:17 +00:00
udo 9beec0dd03 Memoize read-only source graph stats 2026-07-17 22:00:37 +00:00
udo 7e0b8337ab Prioritize stale dynamic components 2026-07-17 21:02:35 +00:00
udo 2dd23766e0 Recover after transient dependency compile failures 2026-07-17 19:59:39 +00:00
udo c28b446541 Profile complete Wasm request timing 2026-07-17 19:11:05 +00:00
root 556bb06f81 Profile entry and component materialization 2026-07-17 18:28:49 +00:00
root 3e423f49dd Expose MySQL connection state 2026-07-17 18:05:04 +00:00
udo da04af1cbf Export math functions to Wasm components 2026-07-17 14:52:34 +00:00
udo 88c637f5a4 Preserve MySQL connection failures across queries 2026-07-17 13:52:08 +00:00
udo dbb8ae2c4b Support nested Markdown heading levels 2026-07-17 05:27:00 +00:00
udo 8dbff881a1 Profile request thread runtime on demand 2026-07-17 03:05:37 +00:00
root 5a5450caa8 Keep hostcall CPU profiling off hot paths 2026-07-17 02:35:03 +00:00
root c15fdc3346 Make hostcall CPU profiling configurable 2026-07-17 02:22:56 +00:00
root c9f43d8279 Separate UCE builds from service startup 2026-07-17 02:11:13 +00:00
root b0d475e777 Wait for UCE service readiness 2026-07-17 02:09:16 +00:00
root 672e5b66ef Profile hostcall thread CPU 2026-07-17 02:09:16 +00:00
root 1ad05475e8 Require live HTTP security probes 2026-07-17 01:41:33 +00:00
root aeab8df208 Attribute per-unit load phases accurately 2026-07-17 01:41:33 +00:00
root 51cbb19f7d Profile workspace CPU phases 2026-07-17 01:12:55 +00:00
root 0e43a89636 Document persistent MySQL worker pooling 2026-07-17 00:59:10 +00:00
udo 96e5813a0b Profile workspace thread CPU time 2026-07-17 00:45:46 +00:00
udo 03ffeeb0da Parse common UTC timestamps in process 2026-07-17 00:15:43 +00:00
udo 7789385ef5 Profile bounded hostcall operations 2026-07-17 00:01:19 +00:00
root e9644c717f Avoid rebuilding byte-identical source graphs 2026-07-16 23:27:51 +00:00
udo 8dd307d316 Precompile native modules off request path 2026-07-16 22:51:16 +00:00
udo a247c2b520 Expose per-unit module request profiles 2026-07-16 22:32:30 +00:00
udo b4f652b5a5 Preserve socket runtime directory 2026-07-16 22:03:54 +00:00
udo caffb3f404 Keep FastCGI available during restarts 2026-07-16 22:00:08 +00:00
udo 93c701c6c7 Store serialized core in runtime cache 2026-07-16 21:10:09 +00:00
udo ee2177f4b4 Prewarm wasm workers before accepting requests 2026-07-16 21:00:46 +00:00
udo 83c05111a9 Profile FastCGI transport phases 2026-07-16 20:49:13 +00:00
udo bbfbd63b53 Skip wasm bodies on serialized module hits 2026-07-16 20:31:25 +00:00
udo 11025ab8b3 Profile cold unit module loading 2026-07-16 19:57:40 +00:00
udo 91074ccec9 Remove neutral unit global type cache 2026-07-16 19:31:06 +00:00
udo 8737f2747e Reuse unit global types per worker 2026-07-16 19:27:24 +00:00
udo 8ea1fed38b Document cached unit import metadata 2026-07-16 19:09:11 +00:00
udo 9b95d00709 Cache unit import metadata 2026-07-16 19:01:05 +00:00
udo 9c15d6d85d Profile unit symbol resolution cost 2026-07-16 18:30:11 +00:00
udo ab3e4dac2e Remove slower unit symbol cache 2026-07-16 18:18:39 +00:00
udo ea89e3bb6d Cache resolved unit symbols per request 2026-07-16 18:14:23 +00:00
udo 197719ea23 Profile side module loading phases 2026-07-16 18:01:31 +00:00
udo fd276f8343 Profile component resolution subphases 2026-07-16 17:34:46 +00:00
udo 6a02c07bcb Reuse reset MySQL connections per worker 2026-07-16 17:00:28 +00:00
udo b04ce3d005 Profile logical MySQL operations 2026-07-16 16:31:13 +00:00
udo 9dc9c61af6 Keep proactive freshness scans off requests 2026-07-16 16:06:02 +00:00
udo ec07b5df22 Cache component freshness probes 2026-07-16 15:27:59 +00:00
udo dc5ccee805 Profile request execution phases 2026-07-16 15:12:13 +00:00
udo 62adc7a6a9 Reject empty time parse input 2026-07-16 14:29:28 +00:00
root 27d0258e1f stage request performance snapshots 2026-07-16 11:46:14 +00:00
udo 82aec81465 Refresh guest deadline after MySQL hostcalls 2026-07-15 09:18:35 +00:00
udo ee022f0398 Ensure nested CLI components are fresh 2026-07-15 00:43:23 +00:00
udo dafffb4ab6 label expected compile timing probe 2026-07-14 23:38:20 +00:00
udo 6b18587581 scope nested component props without deep copies 2026-07-14 22:41:55 +00:00
udo 0bcf9a676e flush runtime diagnostics promptly 2026-07-14 22:15:13 +00:00
udo 8c3f8c5e01 fix persisted unit failure identity 2026-07-14 21:44:31 +00:00
udo d33d5e80ba fix canonical component artifact identity 2026-07-14 21:32:29 +00:00
udo b6c2b8b08d docs: note installed gate site root 2026-07-14 08:46:01 +00:00
udo 306420e4d3 test: derive installed site directory for gates 2026-07-14 08:44:09 +00:00
udo d7f4749f9f test: label expected source read failures 2026-07-14 06:23:50 +00:00
udo afb17fc334 test: retry transient doc gate reads 2026-07-14 04:34:52 +00:00
udo 2a1400b78c test: run CLI gates by group 2026-07-14 04:04:55 +00:00
udo 9d707a1c04 Document source read failures 2026-07-14 02:52:36 +00:00
udo c1949e7430 Label unreadable source probes distinctly 2026-07-14 02:48:24 +00:00
udo 8eec28a639 Clarify CSRF field raw rendering 2026-07-14 00:04:43 +00:00
udo 7fabccd90a Add CSRF hidden field helper 2026-07-13 23:52:49 +00:00
udo 096138d6a1 Fix wasm compile source race 2026-07-13 22:47:48 +00:00
udo 36ea0cd2a5 test: wait for dependency lock fixture readiness 2026-07-13 22:21:18 +00:00
root 8da6cad305 fix: retain resolved component paths 2026-07-13 21:26:01 +00:00
udo 92baaa6299 fix: prevent stale mutation execution 2026-07-13 20:17:48 +00:00
udo 0187bb60cf docs: publish password hashing APIs 2026-07-13 19:47:06 +00:00
udo 4414972079 feat: add native scrypt password hashing 2026-07-13 19:35:38 +00:00
udo a288b539c6 Reject unreadable UCE source units 2026-07-13 15:28:35 +00:00
udo cd2fa163e6 keep CLI units current during rebuilds 2026-07-13 14:20:29 +00:00
udo 97ecc6edd6 keep web requests off rebuild path 2026-07-13 12:25:42 +00:00
udo 5195ebeb25 avoid request stalls during proactive rebuilds 2026-07-13 12:12:14 +00:00
udo c1039f5094 Parse standard absolute URL paths 2026-07-13 09:31:45 +00:00
udo fe88d4143e Keep warm wasm workers alive 2026-07-13 08:59:11 +00:00
udo f7e9c5a4ec Exclude cold module loads from guest deadlines 2026-07-13 08:51:46 +00:00
udo 73a2671112 Invalidate warmed wasm modules precisely 2026-07-13 08:36:55 +00:00
udo ada01e34f4 Cache loaded source signatures safely 2026-07-13 08:07:37 +00:00
udo 02ed75fc85 Invalidate units when loaded sources change 2026-07-13 07:42:52 +00:00
udo d857930ceb Expose MySQL affected row counts 2026-07-13 07:10:03 +00:00
root 7281302287 Keep MySQL connect available to wasm units 2026-07-13 06:24:59 +00:00
root b1dfe6c807 Reuse MySQL connections within requests 2026-07-13 00:08:07 +00:00
udo 0ff42f357f Document request-scoped MySQL cleanup 2026-07-08 18:44:07 +00:00
udo 6bdccacf02 fix cli unit document root resolution 2026-07-06 20:35:09 +00:00
udo 6f446c4aef hide internal csrf session key helper 2026-07-06 10:08:44 +00:00
udo b8c2efe976 add native csrf helpers 2026-07-06 09:58:25 +00:00
udo 125a24e2f9 redact sensitive runtime config logging 2026-07-06 08:03:09 +00:00
udo a6cb1c0390 fix wasm session and mysql placeholder state 2026-07-06 07:54:01 +00:00
root 991a0f62b4 harden runtime config and docs 2026-06-27 20:58:07 +00:00
udo c148c1b36b test: allow skipping nested local service probes 2026-06-22 02:39:36 +00:00
udo 30fcfce103 test: include response tails in site failures 2026-06-22 01:31:44 +00:00
udo 41b034739b test: use explicit failed-count checks 2026-06-22 01:14:07 +00:00
udo bd007231de test: distinguish case failures from summary labels 2026-06-22 00:56:46 +00:00
udo 7cf930b4a8 test: accept explicit zero-failure summaries 2026-06-22 00:39:12 +00:00
udo 4d17d80f9a test: build failed-count markers explicitly 2026-06-22 00:21:10 +00:00
udo 0d59ea4b1c test: rely on site summary failure counts 2026-06-21 23:54:01 +00:00
udo a0d085859c test: stabilize cold local service probes 2026-06-21 23:37:01 +00:00
udo 9e70dde366 test: detect site case failures explicitly 2026-06-21 23:05:32 +00:00
udo c73dbc8503 test: retry cold site suite probes 2026-06-21 22:42:02 +00:00
udo 1d6c882fb7 test: wait for complete local HTTP bodies 2026-06-21 22:24:12 +00:00
udo 245119f952 test: report site suite marker details 2026-06-21 22:07:28 +00:00
udo e113215ffa test: wait for slow streamed test responses 2026-06-21 21:50:37 +00:00
udo 1088f6b9a5 test: tolerate cold unit browser responses 2026-06-21 21:21:39 +00:00
root 1770323a2a fix test runner repo path handling 2026-06-21 21:05:51 +00:00
root 4a6fc46ec8 docs: document runtime cache request param 2026-06-21 21:02:36 +00:00
root c0f3c5507c fix doc example cache path 2026-06-21 20:59:01 +00:00
root 4eda629c8d cleanup, docs 2026-06-17 13:02:57 +00:00
root b8b56cf3dd cleanup, docs 2026-06-16 23:02:45 +00:00
root ea08d5f28b cleanup, docs 2026-06-16 12:21:31 +00:00
root dff1959341 cleanup, error pages 2026-06-16 08:46:54 +00:00
root f2a3503ac3 configurable trans-membrance hostcall blocklist 2026-06-16 01:13:06 +00:00
root 52cf266a5e docs 2026-06-15 21:46:31 +00:00
root 99cd92fb4a docs 2026-06-15 21:42:50 +00:00
442 changed files with 17506 additions and 4269 deletions
+1
View File
@@ -47,6 +47,7 @@ __pycache__/
.ruff_cache/ .ruff_cache/
.coverage .coverage
htmlcov/ htmlcov/
site/doc/examples/_gen/
# Editor swap artifacts # Editor swap artifacts
*.swp *.swp
+185
View File
@@ -0,0 +1,185 @@
# Completed `/doc` documentation overhaul (historical plan)
This June 2026 plan is complete and is retained only as implementation history.
It is not the current task queue and its one-off execution rules do not govern
new UCE work. Current documentation has 269 rendered pages and is covered by
the sharded doc gate in `scripts/run_cli_tests.sh`.
The original briefing follows.
The approved design and the two locked decisions are: **(1) examples are LIVE and
SELF-VERIFYING** — real UCE code executed at render time, showing source + actual
captured output, gated by the test suite; **(2) FULL SWEEP** of all 257 pages.
The docs live under `site/doc/`:
- `index.uce` — the renderer (index + per-page detail view).
- `lib/doc_page.h` — page parsing (`load_doc_page`) + label/kind helpers.
- `style.css` — theme + layout.
- `pages/*.txt` — 257 page source files (the corpus).
- `areas/*.txt` — area groupings; first line is the area title, following lines
are page slugs (or `>area` references) that populate the index sidebar.
- `search.uce`, `singlepage.uce` — search + single-page render.
## Why this work exists (the problems)
- **Title bug:** 15 pages carry a `:title` that literally repeats the raw filename,
so `0_StringList` renders as "0_StringList" instead of "StringList". The `:title`
override defeats the prefix-stripping label logic already in `doc_page.h`
(`doc_default_title` / `doc_index_label` / `doc_method_label`).
- **31 slop/stub pages:** the newest APIs (`file_*`, `crypto_equal`, `sha256*`,
`hmac*`, `random_bytes`, `http_request*`, `job_*`, `shell_spawn`) are throwaway
`# name` + one-line markdown with NO `:sig`/`:params`/`:see`/`:content`.
- **134 of 257 pages have no code example.** We want PHP-manual-style
code-to-output examples on EVERY page.
- **Sidebar overflow:** long monospace identifiers in the 280px index sidebar and
220px detail sidebar have no wrapping → they spill.
- **Relevancy:** pages lead with cross-membrane mechanics and "Related Concepts"
PHP/JS filler instead of usage; cross-references between related pages are missing.
## Principles for every page
1. **Usage first.** The opening `:content` sentence says what the function does and
when you'd reach for it — in plain terms. No membrane/ABI talk up top.
2. **A real example, always.** Every page has at least one `:example` whose output
is produced by actually running it. Examples are the primary teaching tool.
3. **Cross-link.** Every page's `:see` references its area (`>area`) and its closest
sibling APIs, so the reader is never left to search alone.
4. **Demote the drivel.** Cross-membrane behavior, lock semantics, and PHP/JS
equivalents are at most a short trailing note — never the body. Delete filler
that teaches nothing.
---
## Phase 1 — Infrastructure (do this FIRST, verify on host before any content)
### 1a. Title bug
- In `lib/doc_page.h` `load_doc_page`: when a `:title` value, trimmed, **equals the
page slug**, ignore it (leave `result.title` empty) so the renderer falls through
to `doc_default_title`, which already strips `0_/1_/2_/3_` prefixes and renders
`Class::method` for `2_*`.
- Strip the 15 redundant `:title` blocks from their pages (`0_StringList`, `cli_arg`,
`cli_input`, `list_filter`, `list_map`, `map`, `request_base_url`,
`request_query_path`, `request_query_route`, `request_script_url`,
`route_path_is_safe`, `route_path_normalize`, `route_path_sanitize`, `ucb_encode`,
`ucb_decode`).
- Fix garbage `:sig` lines on struct pages (e.g. `0_StringList`'s sig is literally
`0_StringList`): give struct pages a real one-line type summary or drop the sig box.
### 1b. Live example mechanism — EXACT SPEC
Primitives (already confirmed to exist): `unit_compile(path)`, `unit_render(path)`,
`ob_start()`, `ob_get_close()`. Relative paths in the doc unit resolve against
`site/doc/`.
- **Parse:** add `:example` handling to `load_doc_page``DocPage.examples`
(a list; multiple `:example` blocks per page allowed). Capture the raw body lines
verbatim (this is both the displayed source AND the executed code).
- **Materialize:** for each example, write the body wrapped in a minimal unit to a
**stable** path `examples/_gen/<slug>_<n>.uce`:
```
RENDER(Request& context)
{
<example body>
}
```
Write **only if the content changed** (compare to existing file) so the runtime's
compiled-unit cache stays warm. The `_gen` dir must be writable by the worker
(www-data) — created/`chown`ed once on the host (see Setup below).
- **Execute + capture:** `unit_compile("examples/_gen/<slug>_<n>.uce");` then
`ob_start(); unit_render("examples/_gen/<slug>_<n>.uce"); String out = ob_get_close();`
- **Render:** an "Example" `doc-section` showing the source block, then the captured
output beneath it under an "Output" label (PHP-manual style). If the example
traps or output is empty, **surface that visibly** (do not swallow it) — a broken
example must be obvious so the gate catches it.
- Examples must be **self-contained and deterministic** — no wall-clock/random output
shown as canonical (if a function is inherently non-deterministic, show a
representative call and describe the shape, or seed it). The gate asserts presence
of an output block + absence of an error marker, not an exact value, EXCEPT where
the value is deterministic.
### 1c. Sidebar + example CSS (`style.css`)
- Add `overflow-wrap: anywhere; word-break: break-word;` to `.category li a`,
`.func-item a`, and `.sidebar-card div`.
- Style the example Output block so it's visually distinct from the source (subtle
border + an "Output" label), reusing existing `--bg-code`/`--border` tokens.
### 1d. Format spec page
- Create `pages/3_Documentation format.txt` (an `info` page) documenting the canonical
page template below, so the format is self-describing inside the docs.
### Canonical page template
```
:sig
<one or more real C++ signatures, exactly as declared>
:params
<name> : <what it is>
return value : <what comes back>
:content
<Usage-first prose. What it does, when to use it. 25 tight sentences.>
:example
<runnable UCE code that print()s something illustrative>
:see
><area>
<closest sibling API slugs>
```
Optional short trailing note in `:content` for membrane/PHP-JS equivalence — one line,
not a section. No `:title` unless it differs from the derived label.
### Setup (host, one-time, before content work)
```sh
ssh root@10.4.2.110 'cd /Code/uce.openfu.com/uce && mkdir -p site/doc/examples/_gen && chown -R www-data:www-data site/doc/examples/_gen && chmod 775 site/doc/examples/_gen'
```
---
## Phase 2 — Content sweep (batched by area; one area = one reviewable batch)
For each area file in `site/doc/areas/` (string, sys, types, sqlite, mysql, memcache,
session, websocket, task, time, uri, regex, ob, markup, socket, noise, runtime), bring
every listed page to the canonical template:
1. **Convert the 31 stubs** to full structured pages.
2. **Add a live `:example`** to every page missing one (≈134), each verified to render
real output.
3. **Repair `:see`** — area link + sibling links on every page; demote membrane / PHP-JS
filler to a one-line note.
4. **Cull dead pages** — remove `concat.txt` ("Removed. Not a current API") and audit
internal-only helpers (e.g. `json_consume_space`) for removal; also remove culled
slugs from the relevant `areas/*.txt`.
Per-batch bar: every touched page renders cleanly, its example produces real output,
and the suite stays green.
---
## Phase 3 — Gate
- Add a `site/tests/cli_runner.uce` regression test that renders **every** `?p=` page
and asserts HTTP 200, no error/trap marker, and (for pages with `:example`) presence
of a non-error Output block.
- Full host gate (this is the ONLY place builds/tests run — the sshfs mount is
edit-only, no WASI SDK on the client):
```sh
ssh root@10.4.2.110 'cd /Code/uce.openfu.com/uce && bash scripts/build_core_wasm.sh && bash scripts/build_linux.sh && systemctl restart uce.service && sleep 3 && bash scripts/run_cli_tests.sh --include-wasm-kill'
```
Expect the current 91 to grow by the new doc gate(s), 0 failed.
## Historical execution rules (no longer active)
These rules applied only while the overhaul was being carried out in June 2026.
They are preserved to explain that work and must not override current repository
or operator instructions.
- **NEVER `git commit`/push/tag.** Not pi, not its subagents, not anyone. Leave a clean
working tree and report.
- **Always run the real host gate** (`bash scripts/build_core_wasm.sh && bash
scripts/build_linux.sh && systemctl restart uce.service && ...`) and trust only its
output. `g++ -fsyntax-only` / client-side builds do NOT count — there is no WASI SDK
on the client and the doc unit must actually render.
- Sub-delegation model is exactly `gpt-5.3-codex-spark`.
- Verify the example mechanism end-to-end on the host BEFORE authoring content against
it — content written against an unproven mechanism is wasted.
+9 -4
View File
@@ -88,7 +88,7 @@ Useful helpers for that data model include:
- `zip_create()`, `zip_list()`, `zip_read()`, and `zip_extract()` for minimal ZIP archive workflows - `zip_create()`, `zip_list()`, `zip_read()`, and `zip_extract()` for minimal ZIP archive workflows
- `gz_compress()` and `gz_uncompress()` for gzip-format byte strings - `gz_compress()` and `gz_uncompress()` for gzip-format byte strings
- `server_start_http()` / `server_stop()` for runtime-managed custom HTTP listeners backed by `SERVE_HTTP` handlers - `server_start_http()` / `server_stop()` for runtime-managed custom HTTP listeners backed by `SERVE_HTTP` handlers
- `map()`, `filter()`, `list_unique()`, `dv_filter()`, `dv_map()`, `dv_pick()`, and related helpers for route/menu/card data shaping near render code - `map()`, `filter()`, `dv_filter()`, `dv_map()`, `dv_pick()`, and related helpers for route/menu/card data shaping near render code
Named component handlers are also supported: Named component handlers are also supported:
@@ -311,9 +311,13 @@ The example contains the filesystem and FastCGI settings:
BIN_DIRECTORY=/var/cache/uce/work BIN_DIRECTORY=/var/cache/uce/work
TMP_UPLOAD_PATH=/var/lib/uce/uploads TMP_UPLOAD_PATH=/var/lib/uce/uploads
SESSION_PATH=/var/lib/uce/sessions SESSION_PATH=/var/lib/uce/sessions
SESSION_COOKIE_SECURE=1
FCGI_SOCKET_PATH=/run/uce/fastcgi.sock FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
FCGI_SOCKET_MODE=0666
FCGI_PORT=9993 FCGI_PORT=9993
CLI_SOCKET_PATH=/run/uce/cli.sock
CLI_SOCKET_MODE=0600
PRECOMPILE_FILES_IN= PRECOMPILE_FILES_IN=
SITE_DIRECTORY=/var/www/html SITE_DIRECTORY=/var/www/html
@@ -328,7 +332,7 @@ For nginx deployments, the most important setting is:
- `FCGI_SOCKET_PATH=/run/uce/fastcgi.sock` - `FCGI_SOCKET_PATH=/run/uce/fastcgi.sock`
That is the Unix socket nginx should use for normal `.uce` requests. That is the Unix socket nginx should use for normal `.uce` requests. `CLI_SOCKET_PATH` is for local admin/test execution through `scripts/uce-cli`; keep `CLI_SOCKET_MODE=0600` unless a trusted Unix group explicitly needs access.
`FCGI_PORT` is optional if nginx is talking to the Unix socket. Leave it set if you also want a TCP FastCGI listener, or remove it if you want the socket to be the only FastCGI entry point. `FCGI_PORT` is optional if nginx is talking to the Unix socket. Leave it set if you also want a TCP FastCGI listener, or remove it if you want the socket to be the only FastCGI entry point.
@@ -350,6 +354,7 @@ Recommended deployment notes:
- keep `HTTP_PORT` bound to localhost only at the firewall or by network policy; nginx should be the public entry point - keep `HTTP_PORT` bound to localhost only at the firewall or by network policy; nginx should be the public entry point
- keep `BIN_DIRECTORY`, `TMP_UPLOAD_PATH`, and `SESSION_PATH` on writable local storage - keep `BIN_DIRECTORY`, `TMP_UPLOAD_PATH`, and `SESSION_PATH` on writable local storage
- use `SESSION_COOKIE_SECURE=1` for HTTPS-only deployments; leave it `0` only for local/plain-HTTP development
- after editing `/etc/uce/settings.cfg`, restart `uce.service` - after editing `/etc/uce/settings.cfg`, restart `uce.service`
### 4. Install and enable the systemd service ### 4. Install and enable the systemd service
@@ -362,8 +367,8 @@ scripts/systemd/manage-uce-service.sh setup
That script: That script:
- installs `scripts/systemd/uce.service` as `/etc/systemd/system/uce.service` - installs `scripts/systemd/uce.service` as `/etc/systemd/system/uce.service`, rewriting the repository-root path in the unit to the checkout you ran it from
- installs `etc/uce/settings.cfg` to `/etc/uce/settings.cfg` if it does not already exist - installs `etc/uce/settings.cfg` to `/etc/uce/settings.cfg` if it does not already exist, likewise rewriting checkout-root paths
- reloads systemd - reloads systemd
- enables the service at boot - enables the service at boot
- starts the runtime immediately - starts the runtime immediately
+62
View File
@@ -0,0 +1,62 @@
# /doc Documentation Overhaul
## Objective
Implement the DOCS-TODO.md overhaul for the UCE `/doc` system on the sshfs source-of-truth tree, verifying builds/tests only on `root@10.4.2.110:/Code/uce.openfu.com/uce`.
## Success Criteria
- [x] Phase 1 infrastructure complete: title bug, live examples, CSS, format spec page.
- [x] Example mechanism verified end-to-end on host before broad content authoring.
- [x] Phase 2 example coverage sweep: all doc pages now have `:example`; filler/code-fence cleanup applied, resource-bound fences only.
- [x] Phase 3 doc rendering gate added.
- [x] Current host gate passes with 0 failed.
## Current State
- Status: complete; retained as the implementation record for the June 2026 overhaul.
- Last updated: 2026-07-20
- Source of truth: `/root/mount_ssh/uce-upstream`
- Runtime/live target: `root@10.4.2.110:/Code/uce.openfu.com/uce`
## Goal Tree
Legend: `[ ]` not started, `[~]` in progress, `[x]` done, `[!]` blocked, `[-]` superseded
- [x] G1: Phase 1 infrastructure
- Done when: live examples render source/output on the host and title/sidebar/spec fixes are in place.
- Verify: host build/restart/curl spot check.
- [x] G1.1: Fix title override behavior and redundant title blocks.
- [x] G1.2: Add `:example` parser, materializer, executor, renderer.
- [x] G1.3: Add sidebar/example CSS.
- [x] G1.4: Add documentation format page.
- [x] G1.5: End-to-end host verification with one page example.
- [x] G2: Phase 2 content sweep by area
- Done when: all 257 pages match canonical template and examples render.
- Verify: area batch curls and host gate stays green.
- [x] G3: Phase 3 CLI doc gate
- Done when: CLI runner checks every doc page and examples.
- Verify: full host gate passes.
## Completed Execution Queue
1. Phase 2 area batching and delegation completed.
2. The host suite and sharded 269-page documentation gate remain the ongoing regression boundary.
## Decisions
- 2026-06-16: Follow DOCS-TODO.md exactly; no git commit/push/tag.
## Evidence and Verification Log
- 2026-06-16: Read `DOCS-TODO.md`, `site/doc/lib/doc_page.h`, `site/doc/index.uce`, `site/tests/cli_runner.uce`.
- 2026-06-16: Host setup/build/restart/curl for `2_DValue_filter` showed source and captured output `Ada`.
- 2026-06-16: Full host gate passed: `Summary: 349 passed, 0 failed, 0 skipped`.
- 2026-06-16: Converted retiring `list_unique/list_sort/list_some/list_every/list_find` free functions to `StringList` methods, updated callers and method docs; full host gate passed: `Summary: 353 passed, 0 failed, 0 skipped`.
- 2026-06-16: Added live examples across types, string, regex, time, markup, noise, uri, and session area pages; full host gate passed: `Summary: 353 passed, 0 failed, 0 skipped`.
- 2026-06-16: Added `:example` blocks to all remaining doc pages, removed full PHP/JS Related Concepts filler, removed duplicate content code fences except resource-bound pages; full host gate passed: `Summary: 353 passed, 0 failed, 0 skipped`.
- 2026-07-20: Marked the finished plan historical so it no longer presents completed work or one-off execution rules as current instructions.
## Change Log
- 2026-06-16: Created initial goal tree.
+186 -30
View File
@@ -11,20 +11,20 @@ A typical deployment has four pieces:
3. `uce.service`, a systemd service that builds/starts/restarts the runtime. 3. `uce.service`, a systemd service that builds/starts/restarts the runtime.
4. nginx or Apache as the public HTTP server. 4. nginx or Apache as the public HTTP server.
Recommended filesystem layout for a source checkout: Recommended filesystem layout for a source checkout (replace paths as needed):
```text ```text
/opt/uce/ UCE repository/runtime root /path/to/uce-root UCE repository/runtime root (for example `/opt/uce` or `/Code/uce.openfu.com/uce`)
/var/www/html/ public web root served by nginx/Apache /var/www/html/ public web root served by nginx/Apache
/etc/uce/settings.cfg runtime configuration /etc/uce/settings.cfg runtime configuration
/run/uce/fastcgi.sock FastCGI socket used by nginx/Apache /run/uce/fastcgi.sock FastCGI socket used by nginx/Apache
/run/uce/cli.sock local CLI/admin/test socket /run/uce/cli.sock local CLI/admin/test socket
/var/cache/uce/work generated source, wasm modules, caches /var/cache/uce/work generated source, wasm modules, caches
/var/lib/uce/uploads multipart upload scratch space /var/lib/uce/uploads multipart upload scratch space
/var/lib/uce/sessions session files /var/lib/uce/sessions session files
``` ```
For packaged installs, the runtime may live under `/usr/lib/uce` instead of `/opt/uce`. Keep the public web root at `/var/www/html` or another normal web-root path, not under the runtime source tree. For packaged installs, the runtime may live under `/usr/lib/uce` instead of your checkout root. Keep the public web root at `/var/www/html` or another normal web-root path, not under the runtime source tree.
## Build requirements ## Build requirements
@@ -32,10 +32,10 @@ On Debian/Ubuntu-like systems, install the distro packages first:
```bash ```bash
apt update apt update
apt install -y clang build-essential libpcre2-dev mariadb-client libmariadb-dev curl rsync ca-certificates apt install -y clang build-essential libpcre2-dev libssl-dev mariadb-client libmariadb-dev curl rsync ca-certificates
``` ```
UCE also requires two non-vendored dependencies. WASI SDK is load-bearing at runtime because UCE compiles units on demand during requests and during proactive startup scans. UCE also requires two non-vendored dependencies. WASI SDK is load-bearing at runtime because UCE compiles units on demand during requests and during proactive startup scans. The `curl` binary is also a pinned runtime package dependency: `http_request()` and `http_request_async()` execute it directly with an explicit argument vector for TLS-capable outbound HTTP.
- **Wasmtime C API / C++ headers** at `/opt/wasmtime` by default. `scripts/build_linux.sh` expects: - **Wasmtime C API / C++ headers** at `/opt/wasmtime` by default. `scripts/build_linux.sh` expects:
- `/opt/wasmtime/include/wasmtime.hh` - `/opt/wasmtime/include/wasmtime.hh`
@@ -46,6 +46,8 @@ UCE also requires two non-vendored dependencies. WASI SDK is load-bearing at run
- `/opt/wasi-sdk/bin/clang++` - `/opt/wasi-sdk/bin/clang++`
- `/opt/wasi-sdk/bin/wasm-ld` - `/opt/wasi-sdk/bin/wasm-ld`
- `/opt/wasi-sdk/bin/llvm-objcopy` - `/opt/wasi-sdk/bin/llvm-objcopy`
- `/opt/wasi-sdk/bin/llvm-nm`
- `/opt/wasi-sdk/bin/llvm-dwarfdump`
You can use different install locations by setting environment variables before building and in the systemd service environment: You can use different install locations by setting environment variables before building and in the systemd service environment:
@@ -62,10 +64,11 @@ apt install -y nginx
apt install -y apache2 apt install -y apache2
``` ```
Build UCE from the repository root: Build UCE from your repository root:
```bash ```bash
cd /opt/uce repo_root=/path/to/uce-root
cd "$repo_root"
bash scripts/build_core_wasm.sh bash scripts/build_core_wasm.sh
bash scripts/build_linux.sh bash scripts/build_linux.sh
``` ```
@@ -99,12 +102,15 @@ The expected directories are:
/opt/wasi-sdk/bin/clang++ /opt/wasi-sdk/bin/clang++
/opt/wasi-sdk/bin/wasm-ld /opt/wasi-sdk/bin/wasm-ld
/opt/wasi-sdk/bin/llvm-objcopy /opt/wasi-sdk/bin/llvm-objcopy
/opt/wasi-sdk/bin/llvm-nm
/opt/wasi-sdk/bin/llvm-dwarfdump
``` ```
Install the WASI SDK: Install the WASI SDK:
```bash ```bash
cd /opt/uce repo_root=/path/to/uce-root
cd "$repo_root"
scripts/install_wasi_sdk.sh scripts/install_wasi_sdk.sh
scripts/install_wasi_sdk.sh --check-only scripts/install_wasi_sdk.sh --check-only
``` ```
@@ -133,6 +139,8 @@ test -f /opt/wasmtime/lib/libwasmtime.so
/opt/wasi-sdk/bin/clang++ --version /opt/wasi-sdk/bin/clang++ --version
/opt/wasi-sdk/bin/wasm-ld --version /opt/wasi-sdk/bin/wasm-ld --version
/opt/wasi-sdk/bin/llvm-objcopy --version /opt/wasi-sdk/bin/llvm-objcopy --version
/opt/wasi-sdk/bin/llvm-nm --version
/opt/wasi-sdk/bin/llvm-dwarfdump --version
``` ```
If your paths differ, export the variables for manual builds: If your paths differ, export the variables for manual builds:
@@ -161,9 +169,16 @@ systemctl daemon-reload
systemctl restart uce.service systemctl restart uce.service
``` ```
When `uce.socket` is active, restart only `uce.service` (or use
`scripts/systemd/manage-uce-service.sh restart`). The socket unit must retain
ownership of the FastCGI listener while workers change. Restarting
`uce.socket` alongside the service removes that listener briefly and can expose
HTTP 502 responses through nginx. Restart the socket unit only to repair the
socket itself, with an explicit outage window.
## Runtime configuration ## Runtime configuration
Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths if your runtime is not under `/opt/uce`. Replace any checkout-specific paths such as `WASM_CORE_PATH` with `/opt/uce/bin/wasm/core.wasm` or your actual runtime path. Create `/etc/uce/settings.cfg` from `etc/uce/settings.cfg` and adjust paths to your checkout root. Replace checkout-specific settings such as `WASM_CORE_PATH` with `<UCE_REPO_ROOT>/bin/wasm/core.wasm` or leave it relative to `COMPILER_SYS_PATH` as appropriate.
Minimum useful settings: Minimum useful settings:
@@ -171,41 +186,100 @@ Minimum useful settings:
BIN_DIRECTORY=/var/cache/uce/work BIN_DIRECTORY=/var/cache/uce/work
TMP_UPLOAD_PATH=/var/lib/uce/uploads TMP_UPLOAD_PATH=/var/lib/uce/uploads
SESSION_PATH=/var/lib/uce/sessions SESSION_PATH=/var/lib/uce/sessions
SESSION_COOKIE_SECURE=1
FCGI_SOCKET_PATH=/run/uce/fastcgi.sock FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
FCGI_SOCKET_MODE=0666
CLI_SOCKET_PATH=/run/uce/cli.sock CLI_SOCKET_PATH=/run/uce/cli.sock
CLI_SOCKET_MODE=0600
CLI_WORKER_COUNT=2
CLI_WORKER_MAX_REQUESTS=8
SITE_DIRECTORY=/var/www/html SITE_DIRECTORY=/var/www/html
HTTP_DOCUMENT_ROOT=/var/www/html HTTP_DOCUMENT_ROOT=/var/www/html
JIT_COMPILE_ON_REQUEST=1 JIT_COMPILE_ON_REQUEST=1
SHOW_DYNAMIC_COMPILE_ERRORS=1
SERVE_LAST_KNOWN_GOOD=0
PROACTIVE_COMPILE_ENABLED=1 PROACTIVE_COMPILE_ENABLED=1
PROACTIVE_COMPILE_JOBS=2
PROACTIVE_COMPILE_CHECK_INTERVAL=60 PROACTIVE_COMPILE_CHECK_INTERVAL=60
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
WASM_BACKEND_VERBOSE=0 WASM_BACKEND_VERBOSE=0
WASM_CORE_PATH=/opt/uce/bin/wasm/core.wasm WASM_CORE_PATH=<UCE_REPO_ROOT>/bin/wasm/core.wasm
WASM_MEMORY_LIMIT_BYTES=536870912 WASM_MEMORY_LIMIT_BYTES=536870912
WASM_EPOCH_DEADLINE_TICKS=200 WASM_EPOCH_DEADLINE_TICKS=200
WASM_EPOCH_PERIOD_MS=50 WASM_EPOCH_PERIOD_MS=50
WASM_INVOCATION_TIMEOUT_MS=30000
WASM_SERIALIZE_TIMEOUT_SECONDS=120
MYSQL_PERSISTENT_POOL_SIZE=8
MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS=300
WORKER_COUNT=4 WORKER_COUNT=4
MAX_MEMORY=16777216 MAX_MEMORY=16777216
SESSION_TIME=2592000 SESSION_TIME=2592000
HTTP_PORT=8080 HTTP_PORT=8080
WS_BROKER_OUTBOUND_TIMEOUT_SECONDS=30
``` ```
Important settings: Important settings:
- `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests. Set it explicitly and keep this value and the web-server `fastcgi_pass` path identical. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places. - `FCGI_SOCKET_PATH` is the Unix socket used for normal `.uce` requests. Set it explicitly and keep this value and the web-server `fastcgi_pass` path identical. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
- `CLI_SOCKET_PATH` is a local HTTP-over-Unix socket used by `scripts/uce-cli` and test/admin units. - `CLI_SOCKET_PATH` is a local HTTP-over-Unix socket used by `scripts/uce-cli` and test/admin units. Keep it private (`CLI_SOCKET_MODE=0600`) unless you intentionally delegate admin/test execution to a trusted Unix group (`0660`).
- `SITE_DIRECTORY` is the public site tree to scan for `.uce` files. Use `/var/www/html` when the web root is outside the runtime tree; relative paths are resolved from the runtime working directory. - `CLI_WORKER_COUNT` adds a transport-isolated CLI/test renderer pool. Use at least `2` on a live site so broad test/admin unit loads cannot evict or page out public FastCGI workers' hot modules and a CLI unit can make one nested CLI call without self-deadlocking. `0` preserves the legacy shared pool; dedicated CLI workers are additional to `WORKER_COUNT` and retain their own Wasmtime engine, module cache, and persistent connector pool.
- `CLI_WORKER_MAX_REQUESTS` recycles each dedicated CLI worker after this many completed connections (default `8`, maximum `1024`); `0` disables recycling. Public FastCGI workers are not request-count recycled, preserving their warmed module caches. Recycling bounds the module set retained by broad test and administration runs while allowing one CLI invocation/test group to finish uninterrupted.
- `FCGI_SOCKET_MODE` and `CLI_SOCKET_MODE` are octal permission modes applied after socket bind. Prefer tightening `FCGI_SOCKET_MODE` to `0660` when nginx/Apache can share a trusted group with the UCE worker.
- `SITE_DIRECTORY` is the public site tree to scan for `.uce` files. Use `/var/www/html` when the web root is outside the runtime tree; relative paths are resolved from the runtime working directory. Installed regression gate scripts derive their temporary test root from this setting unless `UCE_TEST_SITE_DIRECTORY` is explicitly provided.
- `HTTP_DOCUMENT_ROOT` is the root used by the built-in HTTP/WebSocket listener when it resolves upgrade requests. Set it to the same web root as nginx/Apache. - `HTTP_DOCUMENT_ROOT` is the root used by the built-in HTTP/WebSocket listener when it resolves upgrade requests. Set it to the same web root as nginx/Apache.
- `BIN_DIRECTORY` stores generated C++, wasm artifacts, compile output, and runtime caches. - `BIN_DIRECTORY` stores runtime state plus ABI-scoped unit generations. Unit
C++, wasm, serialized modules, source maps, and compile diagnostics live in
`units-c<compiler ABI>-w<core ABI>` so an upgrade cannot mix generations.
- `TMP_UPLOAD_PATH` and `SESSION_PATH` must be writable by the runtime. - `TMP_UPLOAD_PATH` and `SESSION_PATH` must be writable by the runtime.
- `SESSION_COOKIE_SECURE=1` adds the `Secure` attribute to UCE-managed session cookies and should be used for HTTPS-only deployments. Leave it `0` only for local/plain-HTTP development.
- `MYSQL_PERSISTENT_POOL_SIZE` caps credential-keyed connections retained by each Wasm worker. The default `8` is clamped to `64`; set it to `0` to restore request-lifetime connections. Cached sessions are reset before reuse.
- `MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS` retires a pooled connection at the first request boundary after it has been idle for this interval (default `300`, maximum `86400`). Set it to `0` to retain idle connections until capacity eviction, reset failure, worker exit, or database-side closure. Keep a positive value below the database server's idle timeout to avoid attempting stale sessions after traffic resumes. A completely idle UCE worker has no request boundary and therefore does not proactively close sockets.
- `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes. Bind/firewall it for local access only; nginx/Apache should be the public entry point. - `HTTP_PORT` is the built-in HTTP/WebSocket listener used for WebSocket upgrade traffic and direct local probes. Bind/firewall it for local access only; nginx/Apache should be the public entry point.
- `WS_BROKER_OUTBOUND_TIMEOUT_SECONDS` controls how long a forwarded WS message can remain queued in the broker before being dropped (default `30`). Set to `0` to disable the timeout.
- `WASM_COMPILE_SCRIPT` must point to `scripts/compile_wasm_unit` unless you provide an equivalent compiler. Relative paths are resolved from the runtime root/`COMPILER_SYS_PATH`. That script calls `scripts/check_unit_wasm.py` after linking each unit and uses the pinned WASI SDK on every deployment host. - `WASM_COMPILE_SCRIPT` must point to `scripts/compile_wasm_unit` unless you provide an equivalent compiler. Relative paths are resolved from the runtime root/`COMPILER_SYS_PATH`. That script calls `scripts/check_unit_wasm.py` after linking each unit and uses the pinned WASI SDK on every deployment host.
- `SHOW_DYNAMIC_COMPILE_ERRORS=1` makes a failed dynamic `component()`, `unit_render()`, or `unit_call()` show the bounded compiler diagnostic instead of only a generic missing-handler message. Set it to `0` on deployments where source paths and compiler output must not reach HTTP responses.
- `SERVE_LAST_KNOWN_GOOD=1` lets HTTP GET/HEAD/OPTIONS requests keep using a compatible complete unit artifact while the proactive compiler builds changed source. It defaults to `0`; CLI and mutation requests always use current code or fail closed. The option requires an enabled proactive compiler with a positive check interval. Failed background builds preserve the prior Wasm, source map, and serialized module until a successful atomic publication replaces them. Missing source or an incompatible compiler/core ABI is never served as last-known-good.
- `PROACTIVE_COMPILE_JOBS` selects 116 low-priority full-site scanner processes (default `2`). Each canonical unit path has one scanner owner. When last-known-good serving is enabled, the separate higher-priority demand compiler remains reserved for stale units requested over HTTP, so total background compile concurrency can reach this value plus one.
- `WASM_CORE_PATH` must point at the built `core.wasm` file. - `WASM_CORE_PATH` must point at the built `core.wasm` file.
- `WASM_EPOCH_DEADLINE_TICKS` and `WASM_EPOCH_PERIOD_MS` bound one
uninterrupted guest CPU segment. `WASM_INVOCATION_TIMEOUT_MS` is the
separate absolute wall-clock bound for app-owned unit loading,
initialization, the selected handler, and all nested component/unit calls.
The three values must be positive integers; the ticker period is capped at
`1000` ms and the invocation timeout at `86400000` ms. Invalid values prevent
the Wasm backend from starting. The invocation timeout defaults to `30000` and is
enforced to the epoch ticker's period resolution. Blocking host helpers
retain their own shorter limits and are capped to the remaining invocation
budget where the underlying operation is cancellable. Cold entry compilation,
explicit `unit_compile()`, dynamic component JIT, transitive `#load` work,
compiler and registry lock waits, and the configured runtime-error page all
consume the same request budget. A timed-out compiler process group is killed
without replacing the prior artifact set or persisting a compile-failure
backoff. Request compiles require a zero child exit status and complete staged
output; generated C++, exports, source map, Wasm, metadata, cached-module
invalidation, and diagnostics are published or rolled back as one guarded
generation. Proactive and offline precompile work remain independent of
request invocation deadlines. Serialized-module metadata scanning checks the
absolute deadline between section headers and before and after every bounded
4 KiB positional read. Cache-miss full-artifact reads use the same checks
around 64 KiB positional chunks. A single in-progress regular-file syscall
cannot be cooperatively interrupted; later reads and parsing cannot overrun the budget.
Wasmtime's serialized-module deserialization call is also synchronous and
cannot be interrupted; if it stalls, the timeout is reported after that call
returns rather than at the nominal wall-clock boundary.
Proactive and offline serialization instead runs in a short-lived child
bounded by `WASM_SERIALIZE_TIMEOUT_SECONDS` (default `120`, clamped to
`1``3600`), so each unit releases its pooling allocator arenas and threads
on exit instead of accumulating them in the long-lived scanner. Serialization
holds the unit compile lock and rechecks the wasm artifact identity before
publication; timed-out/failed children leave no dead temporary artifact.
Initial/final descriptor identity, unique selected metadata sections, and
strict 64-bit LEB high-bit validation reject changed or ambiguous artifacts.
After editing settings, restart UCE: After editing settings, restart UCE:
@@ -218,11 +292,12 @@ systemctl restart uce.service
For source-checkout deployments, install the provided service helper: For source-checkout deployments, install the provided service helper:
```bash ```bash
cd /opt/uce repo_root=/path/to/uce-root
cd "$repo_root"
scripts/systemd/manage-uce-service.sh setup scripts/systemd/manage-uce-service.sh setup
``` ```
That helper installs `scripts/systemd/uce.service`, creates runtime directories, enables the service, and starts it. That helper installs `scripts/systemd/uce.service` from your checked-out tree, rewrites the checked-out repository path into the installed unit and first-time config, creates runtime directories, enables the service, and starts it. Use a custom unit only if you need a nonstandard runtime layout.
Useful commands: Useful commands:
@@ -232,7 +307,50 @@ scripts/systemd/manage-uce-service.sh restart
scripts/systemd/manage-uce-service.sh logs 200 scripts/systemd/manage-uce-service.sh logs 200
``` ```
Equivalent manual systemd service for a source checkout: The server binary accepts only these process modes:
```bash
bin/uce_fastcgi.linux.bin # start the server
bin/uce_fastcgi.linux.bin --precompile
bin/uce_fastcgi.linux.bin --serialize-module /absolute/unit.uce.wasm
bin/uce_fastcgi.linux.bin --help
```
Help and invalid arguments exit before configuration is loaded or any listener
is opened. Invalid or extra arguments return status 2. This ordering prevents a
diagnostic invocation from replacing or removing a configured Unix socket.
Managed restart builds and precompiles the complete candidate ABI generation
before it switches the service. Precompile uses two low-priority processes by
default; set `PRECOMPILE_JOBS` in `/etc/uce/settings.cfg` to tune the bounded
116 process count for the host. A failed worker, compile, serialization, or
result report aborts the switch and leaves the current service running. The
managed invocation is bounded to 900 seconds by default; set
`UCE_PRECOMPILE_TIMEOUT` on `manage-uce-service.sh restart` to select another
GNU `timeout` duration. Timeout exit 124 also aborts before the service switch.
A trusted direct `--precompile` process may set `UCE_PRECOMPILE_FILES_IN` and
`UCE_PRECOMPILE_BIN_DIRECTORY` to isolate that invocation's source scan and
artifact registry. Both overrides apply only to precompile mode; normal server
workers continue to use `/etc/uce/settings.cfg`. The parallel precompile
regression uses private roots so a running proactive compiler cannot consume
or publish its controlled race fixtures.
Direct `--precompile` is a listener-free whole-generation gate, not a
single-unit readiness probe. It scans the configured source root, waits for
unit publication locks, and has no internal wall-clock timeout. Automation
must wrap it in a finite external timeout, as the managed restart does, before
starting separately deadline-bounded CLI or HTTP invocations.
Prefer the managed restart when the service runs as an unprivileged user: it
precompiles as that same user. Request-time publication still accepts readable
artifacts produced by a trusted administrator. When Linux protected-hardlink
policy rejects the normal rollback snapshot, UCE copies the prior artifact
under the unit lock; generation markers are replaced atomically instead of
requiring write access to the existing inode. Existing readable lock files can
be locked without write access, including the shared-PCH lock.
Equivalent manual systemd service for a source checkout (`<UCE_REPO>` = checkout root):
```ini ```ini
[Unit] [Unit]
@@ -242,14 +360,14 @@ Wants=network-online.target
[Service] [Service]
Type=simple Type=simple
WorkingDirectory=/opt/uce WorkingDirectory=<UCE_REPO>
RuntimeDirectory=uce RuntimeDirectory=uce
StateDirectory=uce StateDirectory=uce
CacheDirectory=uce CacheDirectory=uce
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock
ExecStartPre=/usr/bin/bash /opt/uce/scripts/build_linux.sh ExecStartPre=/usr/bin/bash <UCE_REPO>/scripts/build_linux.sh
ExecStart=/opt/uce/bin/uce_fastcgi.linux.bin ExecStart=<UCE_REPO>/bin/uce_fastcgi.linux.bin
ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock
Restart=always Restart=always
RestartSec=2 RestartSec=2
@@ -582,7 +700,8 @@ curl -i http://127.0.0.1/examples/uce-starter/ -H 'Host: example.com'
Check the local CLI path: Check the local CLI path:
```bash ```bash
cd /opt/uce repo_root=/path/to/uce-root
cd "$repo_root"
scripts/uce-cli /tests/cli.uce action=echo message=hello scripts/uce-cli /tests/cli.uce action=echo message=hello
scripts/run_cli_tests.sh scripts/run_cli_tests.sh
``` ```
@@ -610,10 +729,40 @@ sock.close()
PY PY
``` ```
## Password hashing
Use the native password API for application credentials:
```cpp
String encoded = password_hash(password);
if(encoded == "")
// fail the write; native hashing did not complete
bool valid = password_verify(candidate, encoded);
if(valid && password_needs_rehash(encoded))
encoded = password_hash(candidate);
```
`password_hash()` returns a self-contained `$uce$scrypt$...` encoding with a random 16-byte salt and the bounded scrypt parameters `N=65536`, `r=8`, `p=1`. `password_verify()` accepts only structurally valid encodings with bounded cost parameters and compares the derived key in constant time. `password_needs_rehash()` reports malformed, legacy, or non-current parameters so applications can upgrade a credential after a successful legacy verification. Treat an empty hash as an operational failure and never store it. Application-level password length policy, rate limiting, and legacy-format verification remain the application's responsibility.
## Structured cryptographic operations
`crypto_operation(request)` is the bounded, algorithm-selected API for structured
asymmetric operations. The initial allowlist supports `operation=key_generate`
and `operation=jwt_sign` with `algorithm=ES256`. Key generation returns P-256
public/private JWKs plus the RFC 7638 thumbprint (`kid`). JWT signing accepts only
a consistent P-256 private JWK, forces `alg=ES256`, and emits a compact JWT with
a 64-byte JOSE signature. It also supports `cbor_decode`, `cose_es256_parse`, and `es256_verify` with `algorithm=ES256`: inputs are canonical unpadded base64url fields; requests are capped at 32 KiB, while CBOR input is definite-length only and capped at 16 KiB decoded (21,846 base64url characters), 256 nodes, and depth 16; it rejects duplicate keys and trailing bytes, and returns an explicit typed tree (`unsigned`, `negative`, `bytes`, `text`, `array`, `map`) rather than DValue-coerced map keys. COSE accepts only EC2/-7/P-256 with exact 32-byte coordinates and an OpenSSL-valid point. Verification accepts only complete canonical DER ECDSA over `message_base64url` and returns `valid`; malformed material fails closed. V1 has no attestation, Ed25519, or RS256. Unknown operations and algorithms fail closed.
Existing typed digest, HMAC, password, randomness, and constant-time comparison
functions remain separate. `crypto_operation()` exposes no raw signing, arbitrary
digest/curve selection, encryption, network fetch, or token exchange. Store
private JWKs as secrets.
## Operational footguns ## Operational footguns
- Keep the FastCGI socket path consistent: `FCGI_SOCKET_PATH` and the web-server `fastcgi_pass` must match exactly. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places. - Keep the FastCGI socket path consistent: `FCGI_SOCKET_PATH` and the web-server `fastcgi_pass` must match exactly. The reference config uses `/run/uce/fastcgi.sock`; if you choose `/run/uce.sock`, use it in both places.
- Keep the public web root separate from the runtime source tree. The examples use `/opt/uce` for runtime files and `/var/www/html` for public files. - Keep the public web root separate from the runtime source tree. Replace `/opt/uce` with your actual checkout path in local examples (for example `/opt/uce` or `/Code/uce.openfu.com/uce`), and keep public files under `/var/www/html`.
- Set `HTTP_DOCUMENT_ROOT` when the web root is outside the runtime working directory. The built-in HTTP/WebSocket listener resolves upgrade paths from this setting. - Set `HTTP_DOCUMENT_ROOT` when the web root is outside the runtime working directory. The built-in HTTP/WebSocket listener resolves upgrade paths from this setting.
- Do not expose `CLI_SOCKET_PATH` or `HTTP_PORT` as public entry points. The public path should be nginx/Apache. - Do not expose `CLI_SOCKET_PATH` or `HTTP_PORT` as public entry points. The public path should be nginx/Apache.
- Do not trust `Script-Filename` request headers from direct HTTP clients. The built-in HTTP listener resolves from `HTTP_DOCUMENT_ROOT` and rejects `..` path segments. - Do not trust `Script-Filename` request headers from direct HTTP clients. The built-in HTTP listener resolves from `HTTP_DOCUMENT_ROOT` and rejects `..` path segments.
@@ -658,11 +807,18 @@ Common compile footguns:
- `WASM_COMPILE_SCRIPT` is unset or points at a removed script such as `scripts/compile`; set it to `scripts/compile_wasm_unit`. - `WASM_COMPILE_SCRIPT` is unset or points at a removed script such as `scripts/compile`; set it to `scripts/compile_wasm_unit`.
- `scripts/check_unit_wasm.py` is missing or not executable; `scripts/compile_wasm_unit` calls it after linking each unit. - `scripts/check_unit_wasm.py` is missing or not executable; `scripts/compile_wasm_unit` calls it after linking each unit.
- `WASI_SDK` does not point at the pinned tree with `clang++`, `wasm-ld`, `llvm-objcopy`, and `llvm-nm`; run `scripts/install_wasi_sdk.sh --check-only`. - `WASI_SDK` does not point at the pinned tree with `clang++`, `wasm-ld`, `llvm-objcopy`, `llvm-nm`, and `llvm-dwarfdump`; run `scripts/install_wasi_sdk.sh --check-only`.
- `WASMTIME_HOME` does not point at a tree with Wasmtime headers and `libwasmtime.so`. - `WASMTIME_HOME` does not point at a tree with Wasmtime headers and `libwasmtime.so`.
- A previous failed compile left stale `.compile.txt`, `.wasm-check.txt`, or partial `.wasm` files under `BIN_DIRECTORY`. - A previous failed compile left stale `.compile.txt`, `.wasm-check.txt`, or partial `.wasm` files under `BIN_DIRECTORY`.
Failed compile output is persisted under the unit's generated path in `BIN_DIRECTORY` and may be reused until the source or compiler inputs change. First fix the source/toolchain issue and reload the page. If the cache itself is suspect, stop UCE, move only the affected unit artifact files or directory aside, and restart so the runtime recompiles from source. Avoid deleting the whole `BIN_DIRECTORY` unless you intentionally want a full rebuild. Failed compile output is persisted under the unit's ABI-generation path in
`BIN_DIRECTORY` and may be reused until the source or compiler inputs change.
The managed `restart` command first runs the new binary's `--precompile` mode as
the configured service user while the old service remains live. It restarts
systemd only after every scanned unit compiles and serializes successfully.
First fix source/toolchain failures and retry; do not remove the prior generation,
which remains the rollback path. Avoid deleting the whole `BIN_DIRECTORY` unless
you intentionally want a full rebuild and have accepted losing rollback artifacts.
### CLI commands fail ### CLI commands fail
+13 -1
View File
@@ -31,9 +31,19 @@ UCE expects these executables on each deployment host:
/opt/wasi-sdk/bin/wasm-ld /opt/wasi-sdk/bin/wasm-ld
/opt/wasi-sdk/bin/llvm-objcopy /opt/wasi-sdk/bin/llvm-objcopy
/opt/wasi-sdk/bin/llvm-nm /opt/wasi-sdk/bin/llvm-nm
/opt/wasi-sdk/bin/llvm-dwarfdump
``` ```
`llvm-nm` is used by `scripts/check_unit_wasm.py`, which is called by `scripts/compile_wasm_unit` after linking each unit. `llvm-nm` is used by `scripts/check_unit_wasm.py`, which is called by `scripts/compile_wasm_unit` after linking each unit.
`llvm-dwarfdump` is used at unit compile time to extract the compact, out-of-band
source map before debug sections are stripped from the runtime artifact. The
map is consulted only on a wasm trap.
The native compiler passes the ABI-generation root to
`scripts/compile_wasm_unit`. Unless `UCE_WASM_PCH_DIR` is explicitly set, the
unit PCH lives beneath that writable generation rather than a
process-user-dependent `/tmp` path. Managed precompile therefore behaves the
same inside and outside the service unit.
## Upgrade policy ## Upgrade policy
@@ -43,7 +53,9 @@ Treat WASI SDK upgrades like runtime dependency upgrades:
2. Record the new release and checksum here. 2. Record the new release and checksum here.
3. Rebuild `bin/wasm/core.wasm` with `scripts/build_core_wasm.sh`. 3. Rebuild `bin/wasm/core.wasm` with `scripts/build_core_wasm.sh`.
4. Rebuild the native runtime with `scripts/build_linux.sh`. 4. Rebuild the native runtime with `scripts/build_linux.sh`.
5. Clear or invalidate stale unit wasm artifacts by bumping `UCE_UNIT_ABI_VERSION` when required, or by removing affected generated artifacts under `BIN_DIRECTORY`. 5. Bump the compiler or core ABI constant in `src/wasm/abi.h` when required.
The managed restart precompiles the resulting isolated unit generation before
switching workers; do not overwrite or delete the previous generation.
6. Run the full CLI suite including wasm kill tests: 6. Run the full CLI suite including wasm kill tests:
```bash ```bash
+404 -27
View File
@@ -1,8 +1,8 @@
# UCE WASM Runtime Architecture # UCE WASM Runtime Architecture
Status: current as of the W7e native-pipeline removal (June 2026). This document Status: current as of the worker-role isolation and bounded serializer changes
describes the **runtime architecture as built** — the process topology, the (July 2026). This document describes the **runtime architecture as built**. It
wasm membrane, the unified request dispatch, and the central WebSocket broker. covers the process topology, wasm membrane, request dispatch, and WebSocket broker.
Native `.so` unit execution/dlopen fallback has been removed; the parser and Native `.so` unit execution/dlopen fallback has been removed; the parser and
preprocessor remain only as the front-end that emits C++ for wasm side-module preprocessor remain only as the front-end that emits C++ for wasm side-module
compilation. compilation.
@@ -26,10 +26,10 @@ gets invoked*.
``` ```
┌────────────────────────────┐ ┌────────────────────────────┐
nginx ──FastCGI──► worker pool (N processes) │ /run/uce.sock nginx ──FastCGI──► public workers (N processes) $FCGI_SOCKET_PATH (example `/run/uce/fastcgi.sock`)
(port 80 etc.) │ uniform unit renderers (FastCGI + CLI) (port 80 etc.) │ public unit renderers
└─────────────▲──────────────┘ └─────────────▲──────────────┘
│ forward render (FastCGI, uce.sock) │ forward render (FastCGI, FCGI_SOCKET_PATH)
browser ──raw HTTP / WS──► ┌────────┴─────────┐ browser ──raw HTTP / WS──► ┌────────┴─────────┐
(HTTP_PORT 8080) │ WS broker │ owns HTTP_PORT + every (HTTP_PORT 8080) │ WS broker │ owns HTTP_PORT + every
@@ -42,26 +42,41 @@ gets invoked*.
│ dispatcher(s) │ bind addr; forwards to pool │ dispatcher(s) │ bind addr; forwards to pool
└──────────────────┘ └──────────────────┘
trusted CLI/test calls ──► ┌──────────────────┐
(CLI socket) │ CLI workers │ isolated module and
│ (M processes) │ connector caches
└──────────────────┘
parent process: spawns/respawns all of the above + the proactive compiler. parent process: spawns/respawns all of the above + the proactive compiler.
``` ```
| Process | Owns | Renders units? | Source | | Process | Owns | Renders units? | Source |
|---|---|---|---| |---|---|---|---|
| **Parent** | nothing; supervises children | no | `main()`, `init_base_process()` | | **Parent** | nothing; supervises children | no | `main()`, `init_base_process()` |
| **Worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH` (`/run/uce.sock`) + `CLI_SOCKET_PATH` | **yes** — the only processes that run wasm | `listen_for_connections()` | | **Public worker** (×`WORKER_COUNT`) | `FCGI_SOCKET_PATH`; also `CLI_SOCKET_PATH` only when `CLI_WORKER_COUNT=0` | **yes** — runs public FastCGI wasm | `listen_for_connections()` |
| **CLI worker** (×`CLI_WORKER_COUNT`) | `CLI_SOCKET_PATH` only | **yes** — isolates trusted CLI/test wasm and its module cache | `listen_for_connections()` |
| **WS broker** (×1) | `HTTP_PORT` + every live WS connection + `WS_BROKER_SOCKET_PATH` | no — forwards to the pool | `run_ws_broker()` | | **WS broker** (×1) | `HTTP_PORT` + every live WS connection + `WS_BROKER_SOCKET_PATH` | no — forwards to the pool | `run_ws_broker()` |
| **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` | | **serve_http dispatcher** (×bind) | one custom-server bind address | no — forwards to the pool | `custom_server_http_dispatcher_loop()` |
| **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` | | **Proactive compiler** | nothing; pre-compiles units | no | `run_proactive_compiler()` |
**only workers instantiate Wasmtime and run unit code.** **only workers instantiate Wasmtime and run unit code.**
Every connection-owning process (broker, serve_http dispatcher) forwards the Every connection-owning process (broker, serve_http dispatcher) forwards the
request invocation back to a worker over `/run/uce.sock` using the minimal request invocation back to a worker via `FCGI_SOCKET_PATH` using the minimal
FastCGI client in [`src/lib/fcgi_forward.h`](../src/lib/fcgi_forward.h). This is FastCGI client in [`src/lib/fcgi_forward.h`](../src/lib/fcgi_forward.h). This is
forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across forced by Wasmtime: an `Engine`/`Store` cannot be safely re-created across
`fork()`, and the brokers fork from the parent that already touched the `fork()`, and the brokers fork from the parent that already touched the
runtime. So the brokers hold the long-lived connection and units respond to runtime. So the brokers hold the long-lived connection and units respond to
events the same way they respond to a page request — through a clean worker. events the same way they respond to a page request — through a clean worker.
The broker-to-worker hop explicitly sets `GATEWAY_INTERFACE=CGI/1.1`. After
the Wasm core decodes request parameters it derives the guest's default status
syntax from that marker (`Status:` for FastCGI, `HTTP/1.1` for direct HTTP).
The FastCGI transport clears completed stdout/stderr as soon as their records
are queued and emits no stream records after `FCGI_END_REQUEST`. Together these
boundaries prevent a forwarded dynamic response from being parsed as body and
wrapped in a second HTTP response. The TCP and custom-server tests reject an
inner status line and require the exact dynamic body.
--- ---
## 2. The membrane and the DValue ABI ## 2. The membrane and the DValue ABI
@@ -78,13 +93,23 @@ the boundary).
children yielding `const DValue&`. children yielding `const DValue&`.
- The request context (`params`/`get`/`post`/`cookies`/`session`, the raw body - The request context (`params`/`get`/`post`/`cookies`/`session`, the raw body
`in`, and—for WS—the connection context) is marshalled into a single `ctx` `in`, and—for WS—the connection context) is marshalled into a single `ctx`
DValue, UCEB-encoded, and handed to the workspace. The response (body, DValue and UCEB-encoded. Immutable server configuration is UCEB-encoded once
headers, status, and any `meta` such as `ws_commands`) comes back the same per worker as a flat string map; both byte ranges are written into one guest
way. buffer. The guest decodes the flat map directly into its fresh `Server` and
the dynamic tree into its fresh `Request`. The response (body, headers,
status, and any `meta` such as `ws_commands`) comes back as UCEB.
See [`docs/wasm-phase1-dvalue-abi.md`](wasm-phase1-dvalue-abi.md) for the wire See [`docs/wasm-phase1-dvalue-abi.md`](wasm-phase1-dvalue-abi.md) for the wire
format details. format details.
Decoder robustness note: UCEB input is untrusted across the wasm membrane. The
UCEB decoder must reject malformed magic/version/varint/length/trailing-data
inputs and excessive nesting with explicit errors, not native or wasm traps. Its
current hard nesting cap is intentionally low (64 levels) to stay well below the
guest stack limit. JSON decoding follows the same rule for malformed strings and
unicode escapes: validate bounds before every read and return an empty/partial
`DValue` rather than reading beyond the input.
--- ---
## 3. Units, handlers, and export naming ## 3. Units, handlers, and export naming
@@ -112,14 +137,218 @@ __uce_<base>[_<sanitize(suffix)>]
| `exists` | probe only — resolves the unit, loads nothing | | `exists` | probe only — resolves the unit, loads nothing |
`sanitize_symbol_suffix()` keeps `[A-Za-z0-9_]` (mirrors `ascii_safe_name`). `sanitize_symbol_suffix()` keeps `[A-Za-z0-9_]` (mirrors `ascii_safe_name`).
Unit code is hidden by default so the linker can discard loaded helpers that the
unit does not use. Handler macros and the explicit `EXPORT` directive are the
only application symbols promoted to default visibility; generated request
binding and constructor exports remain part of the runtime ABI. Debug sections
are removed after linking while the wasm name section is retained for runtime
traces. This keeps each side module's code and public ABI local to its actual
handler responsibility instead of duplicating every helper from a loaded app
library.
`wasm_resolve_target(unit, handler)` (`src/wasm/core.cpp`) resolves the source `wasm_resolve_target(unit, handler)` (`src/wasm/core.cpp`) resolves the source
path and looks up the export's funcref slot; `exists` lets callers probe a unit path and looks up the export's funcref slot; `exists` lets callers probe a unit
without instantiating it. without instantiating it.
The request entry unit is already loaded by the host, so its selected handler
and optional `ONCE` export are placed directly into the workspace table and
passed to `uce_wasm_invoke_loaded_entry()`. Dynamic component and unit calls
continue through `wasm_resolve_target()`, where relative-path resolution is
required. Handler cache keys include the calling unit so identically named
relative targets in different directories cannot alias. Once a unit has passed
freshness checks and loaded into a request workspace, later handler lookups
reuse that immutable request-local instance; `component_loaded_reuse_count`
reports those lookups. Opt-in verbose response headers split entry invocation
into load, presence lookup, table linking, and core dispatch time.
`wasm_backend_should_handle(request, entry_unit)` checks whether the wasm `wasm_backend_should_handle(request, entry_unit)` checks whether the wasm
backend is initialized and the requested artifact/handler is currently backend is initialized and the requested artifact/handler is currently
available. If an artifact is cold or stale, dispatch compiles it on demand via available. If an artifact is cold or stale, dispatch compiles it on demand via
`get_shared_unit()` and rechecks. There is no native unit-execution fallback. `get_shared_unit()` and rechecks. There is no native unit-execution fallback.
Workers identify cached unit artifacts by nanosecond mtime, ctime, and size.
Whole-second mtime alone is insufficient because a dependency-triggered rebuild
can replace a wasm artifact within the same second as its prior build.
The worker also classifies each compiled module's immutable import descriptors
once when that artifact enters the module cache. Each request still creates its
own import vector, memory/table-base globals, GOT globals and function-table
slots, then runs relocations, constructors, and request-pointer binding. This
removes repeated Wasmtime import-type traversal without sharing request state or
changing core-first, unit-load-order symbol resolution.
Each FastCGI child initializes its process-local Wasmtime engine after fork and
before entering the accept loop. It also creates the linker, resolves the core's
host imports into a store-independent Wasmtime `InstancePre`, and births then
drops one empty workspace. This preserves Wasmtime's fork boundary while
preventing the first request assigned to each worker from paying engine, linker,
or pre-instantiation startup.
Server configuration is immutable by that point. The worker therefore retains
one native `DValue` view and one UCEB encoding of it. Each request transfers
those cached bytes and the guest decodes the flat scalar map directly into its
fresh `Server`, avoiding both repeated native encoding and an intermediate guest
`DValue` tree. Request parameters, body, cookies, session, call data, and
response state are never retained this way.
Startup duration or failure is written to the service log. The serialized core
module lives in the configured writable cache root rather than beside the
possibly root-owned deployed `core.wasm`; freshness still uses the deployed
artifact's metadata.
Packaged deployments use systemd socket activation for FastCGI. The socket unit
owns `/run/uce/fastcgi.sock`; UCE validates and adopts the single named listener
after exec. The listener therefore remains connectable and queues requests while
the service and its post-fork workers restart. Direct launches without systemd
activation retain the existing configured Unix/TCP listener behavior.
The service preserves its runtime directory across service restarts because the
socket unit, not the service unit, owns a listener path inside that directory.
Normal deployments therefore restart only `uce.service`; restarting
`uce.socket` at the same time removes the listener and forfeits the zero-gap
handoff.
On termination the parent asks render workers to close their listeners, finish
accepted connections within the bounded worker drain interval, and only then
exits. This prevents an accepted FastCGI request from being reset at handoff;
the socket unit queues later connections for the replacement workers.
The graceful signal handler belongs to the parent and render workers. Generic
`task()` children restore default termination signals after fork so
`task_kill()` and `server_stop()` retain their immediate stop contract.
Epoch interruption measures uninterrupted guest CPU segments. A separate
absolute workspace invocation deadline starts before app-owned entry-unit
readiness and cold compilation, loading, and initialization. It remains
unchanged through the selected handler, ONCE, every nested component/unit call,
and configured runtime-error rendering. The common hostcall membrane checks
that deadline before and after every native call and re-arms the store with the
smaller of the remaining absolute budget and the CPU-segment budget. A cheap
hostcall loop therefore cannot renew an invocation indefinitely. Blocking host
helpers retain operation-specific limits and cap them to the remaining
invocation budget where the underlying operation is cancellable. Forked task
callbacks receive a fresh invocation deadline capped by the task lifetime.
The Memcached ASCII hostcall applies one deadline to the complete command write
and protocol-framed response read. Retrieval payload lengths are parsed, so a
payload containing `END` cannot terminate the response; multi-value and `stats`
responses may span socket chunks up to the 8 MiB response bound. Timeout,
malformed framing, or overflow shuts down the retained descriptor before
returning an empty result, preventing unread response bytes from poisoning the
next command on that connection. The native fallback uses the same parser and
deadline contract.
Synchronous compiler locks, transitive `#load` compilation, and compiler child
processes consume that same deadline. Compiler children run in a dedicated
process group; timeout kills the group, retains the previous generation, and
never records a caller-budget timeout as a persisted source failure. Bounded
compiles require a valid zero child exit, cap captured output, and stage
generated C++, exports, source maps, Wasm, and metadata. Publication removes
the prior serialized module and diagnostics inside the same rollback-protected
generation while repeatedly checking the live request deadline.
`request_perf()` reports worker module-cache hits and misses and divides a miss
into artifact lookup, wasm read, custom-section parse, serialized-module
deserialization or wasm compilation, and immutable import classification. This
includes a bounded per-request unit trace with site-relative unit names, cache
source, bytes, and phase durations so a cold outlier identifies its exact unit
and whether it used worker memory, serialized code, compilation, or failed.
Absolute source paths and source contents are not exposed. This keeps cold-worker
module latency distinguishable without exposing source paths.
The same snapshot divides pre-dispatch WASM readiness into entry normalization,
mutation freshness, artifact stat, source-generation lookup, dependency
freshness, and worker availability. `ready_freshness_full_check_us` and
`ready_freshness_cache_hit_count` distinguish full graph validation from a
positive worker-local hit; `ready_freshness_us` remains inclusive.
`ready_check_count` distinguishes the warm one-check path from an on-demand
compile and recheck; repeated snapshot reads retain the initial values.
Read-only HTTP entry checks may reuse a positive result for at most ten seconds
when the source generation and exact Wasm, metadata, and setup-template
identities are unchanged. CLI and mutation requests always validate the complete
graph. Misses, expiry, missing tokens, and changed identities also run the full
check. Exact repeated load paths are then deduplicated before canonicalization,
while distinct aliases are resolved independently so symlink retargets remain
visible by the next hard validation.
When a current serialized module exists, the worker scans wasm section headers
through a bounded 4 KiB positional buffer and retains only `dylink.0`, `uce.abi`,
and the tiny `uce.module` identity. It skips code and data bodies rather than
issuing byte-at-a-time reads or faulting those bodies into every new worker.
The request profile reports the physical read-ahead bytes and positional read
count. The scanner checks the same non-renewable invocation deadline before
each section and before and after every buffer refill; expiry is returned as the
canonical `UCE_INVOCATION_TIMEOUT` error. One in-progress regular-file
`pread()` remains the irreducible synchronous boundary. Initial descriptor
validation matches device, inode, mode, nanosecond timestamps, and size from
the preceding lookup; final validation rejects in-place truncation or mutation.
Selected metadata sections must be unique and individually remain at most 1 MiB,
which also bounds aggregate retained metadata. Both streamed and full-artifact
LEB readers reject nonzero unused bits in the tenth `u64` byte. A
missing/stale/invalid serialized module
still reads the complete wasm artifact in deadline-checked 64 KiB positional
chunks, validates it, compiles it, and republishes the serialization. The full
read uses the same initial/final descriptor identity guard.
The proactive compiler also creates that serialization immediately after source
compilation, keeping first-worker native compilation off the request path.
Cold module compilation and deserialization are host work, so `load_unit()`
refreshes the epoch deadline before its first guest call. Otherwise a component
whose compilation outlasted the guest CPU budget would immediately trap in the
following allocator/relocation call even though no guest loop consumed it.
Wasmtime's serialized-module deserialization API is synchronous and has no
cooperative cancellation point. The absolute invocation timeout is checked
immediately after it returns, so a stalled deserialization may overrun the
nominal wall boundary before UCE reports the timeout.
Unit artifacts live beneath an ABI-generation directory such as
`BIN_DIRECTORY/units-c13-w7`: `c13` is the compiler/unit-metadata ABI and `w7`
is the runtime core ABI. Old and new service binaries therefore never publish
or read the same unit path during an ABI transition. The managed restart builds
and serializes the complete next generation before stopping the old service;
failure aborts the switch and leaves the running generation intact. Old
generation directories are retained as an explicit rollback path. Two
low-priority precompile processes are used by default so independent units do
not serialize deployment time while live workers retain scheduler priority.
`PRECOMPILE_JOBS` or `UCE_PRECOMPILE_JOBS` may select 116 processes. Unit
publication and the shared generation PCH use separate advisory locks, and any
worker/reporting failure rejects the candidate generation.
The proactive compiler and request workers coordinate through per-unit file
locks and a lock-protected demand-priority queue under `BIN_DIRECTORY`. The
full-site scan uses two low-priority processes by default, bounded from 116 by
`PROACTIVE_COMPILE_JOBS`; a stable canonical-path partition gives every unit one
scanner owner and keeps retry/backoff state local. A separate higher-priority
compiler process exclusively drains the demand queue. This matters when the
scanners are already inside long transitive C++ compiles: a requested stale
component can rebuild immediately instead of waiting for an unrelated compile.
All compiler processes use the same per-unit lock, so concurrent demand and scan
discovery cannot publish duplicate artifacts. The priority worker is idle when
there is no demand and never scans the site on its own. Unit
compilation writes and validates a process-unique temporary wasm file, then
publishes it with an atomic rename. By default, requests do not execute stale
code: a missing or changed unit is rebuilt synchronously and a failed dynamic
build surfaces its bounded compiler diagnostic. `SERVE_LAST_KNOWN_GOOD=1`
opts ordinary read-only HTTP requests into using the last complete,
ABI-compatible artifact while requesting that stale unit at the head of the
compiler queue. The proactive compiler must be enabled for this mode. Failed
background builds preserve that artifact and its source map and serialized
module until a later successful atomic publication replaces them. Missing
source, incompatible ABI generations, CLI, WebSocket dispatch, and mutation
requests never use last-known-good code. A stale mutation returns `503 Service
Unavailable` with `Retry-After: 1`; CLI and explicit compile paths remain
synchronous. If a proactive compiler already owns a stale child's lock, a CLI
request joins that compile and waits for the current artifact.
Before preprocessing, the compiler verifies that the worker can actually read
the source. An unreadable path is reported as a source-read failure with a
persisted diagnostic; it never becomes an apparently valid empty side module or
a generic compiler error. Source signatures mark unreadable inputs, so correcting
access invalidates that failure and permits a normal retry without changing
signatures for ordinary readable files. The private CLI harness may label its
intentional unreadable-source regression request as an expected source-read
failure, keeping production log scans focused on real runtime failures while
preserving the failing compile result and diagnostic artifacts.
Unit linking retains DWARF only long enough for
`scripts/build_unit_source_map.py` to extract a compact address/file/line table.
The published `.wasm` is debug-stripped and the table is stored beside it as
`.wasm.source-map`, keyed to the exact temporary module identity recorded in
the wasm's `uce.module` custom section. Normal module loading never reads this
sidecar. On a Wasmtime trap, the worker uses structured frame module offsets to
load only the matching map and appends source locations to the error. A missing,
stale, or malformed map is deliberately non-fatal: the ordinary named wasm
backtrace remains available. Generated C++ uses a `#line` directive naming the
original `.uce` file, so application frames resolve to application source rather
than the generated cache file. Artifact invalidation removes the wasm, serialized
module, and source map together.
--- ---
@@ -129,7 +358,15 @@ Each request gets a fresh **workspace** — a per-request wasm instance tree wit
the membrane wired in. `wasm_worker_serve(worker, ctx, entry_unit, handler)` the membrane wired in. `wasm_worker_serve(worker, ctx, entry_unit, handler)`
(`src/wasm/worker.cpp`) is the single entry point for *every* mode: (`src/wasm/worker.cpp`) is the single entry point for *every* mode:
1. Birth a workspace (CoW-snapshot-based where available). 1. Birth a request-scoped workspace with fresh per-request state. Current
production workers create a fresh Store and instantiate the store-independent
`InstancePre`; the resulting core instance owns a fresh exported, growable
function table, memory, globals, constructors, and request tree. The worker
retains only Wasmtime's compiled module, linker, pre-instantiation plan, and
store-independent host-function definitions. Every callback resolves the one
active request workspace at invocation. Legacy cores that import their table
remain supported through the prior per-request import path. Any future
snapshot/CoW optimization must preserve this request-isolation contract.
2. Resolve `entry_unit` + `handler` to an export; components referenced at 2. Resolve `entry_unit` + `handler` to an export; components referenced at
runtime are resolved on demand via the `uce_host_component_resolve` hostcall runtime are resolved on demand via the `uce_host_component_resolve` hostcall
(`component_resolve()``__uce_<...>` slot), loading dependency modules (`component_resolve()``__uce_<...>` slot), loading dependency modules
@@ -144,6 +381,44 @@ suspend the native SIGSEGV/SIGILL recovery handler around the wasm call so that
Wasmtime's own trap signals are not escalated into a native fatal signal (see Wasmtime's own trap signals are not escalated into a native fatal signal (see
`serve_via_wasm` in `handle_complete`). `serve_via_wasm` in `handle_complete`).
Render workers are long-lived. The generic FastCGI transport retains a legacy
eight-connection recycle default, but `listen_for_connections()` disables it
for the UCE pool: recycling a healthy worker discards its Wasmtime engine and
module cache, adding a visible cold-start request. Request state remains bounded
by the fresh workspace and the normal per-request database/resource cleanup;
faulted workers still exit and are replaced by the parent.
A warm entry artifact is checked for compiler/source freshness once before
dispatch. A missing or stale artifact is compiled (or demand-prioritized) and
then checked again before execution. The second check belongs only to that
state-changing branch; repeating it immediately after a successful warm check
adds no freshness guarantee.
The internal request envelope carries application `context.call` as UCEB2,
each request `StringMap` as a flat UCEB2 map, scalar session/input metadata as
bounded byte segments, and optional WebSocket state as UCEB2. The guest
validates the complete envelope before moving those values into a fresh
`Request`; transport-only params, cookies, session data, and entry metadata no
longer become duplicate `context.call` children. The historical by-value
`DValue::operator=(DValue)` symbol remains exported so warmed side-module
artifacts stay ABI-compatible. `request_perf()` subdivides birth into policy,
import materialization, core instantiation, export/table lookup, and initialization,
and context transfer into bytes, host encode, guest allocation/write,
guest decode/application, and free. The byte profile separately reports the
worker-cached server-configuration portion.
### Task callbacks and workspace lifetime
`task()` and `task_repeat()` are fork-backed. The `uce_host_task_spawn` hostcall
captures the current `WasmWorkspace*`, but `src/lib/sys.cpp::task()` invokes the
captured callback only in the forked child, before the hostcall stack unwinds in
that child. The parent request may return and destroy its workspace; the child
is forked from the parent and keeps a private request-context copy of the per-request workspace. This means a delayed task callback can run after the spawning request
returns without dereferencing the parent's destroyed workspace. It is still a
callback into the inherited child workspace, not a fresh normal request
workspace; avoid adding host resources to `WasmWorkspace` that are invalid across
`fork()` unless task callback handling is changed to birth a fresh workspace.
--- ---
## 5. Request dispatch (`handle_complete`) ## 5. Request dispatch (`handle_complete`)
@@ -161,7 +436,7 @@ otherwise (page) → serve_via_wasm(entry_unit, "render")
The `UCE_*` params are set by whichever broker forwarded the request: The `UCE_*` params are set by whichever broker forwarded the request:
- **Page render**: FastCGI nginx → `/run/uce.sock` directly; no `UCE_*` flags → `render`. - **Page render**: FastCGI nginx → `FCGI_SOCKET_PATH` directly; no `UCE_*` flags → `render`.
- **CLI**: the CLI socket sets `is_cli`. - **CLI**: the CLI socket sets `is_cli`.
- **serve_http**: the custom-server dispatcher sets `UCE_SERVE_HTTP=1` plus - **serve_http**: the custom-server dispatcher sets `UCE_SERVE_HTTP=1` plus
`UCE_SERVE_HTTP_FUNCTION` and rewrites `SCRIPT_FILENAME` to the configured `UCE_SERVE_HTTP_FUNCTION` and rewrites `SCRIPT_FILENAME` to the configured
@@ -197,13 +472,15 @@ broker loop:
2. The message rides as `UCE_WS_MESSAGE` (base64) with an **empty STDIN body** 2. The message rides as `UCE_WS_MESSAGE` (base64) with an **empty STDIN body**
a non-empty STDIN makes the FastCGI transport flush a premature response a non-empty STDIN makes the FastCGI transport flush a premature response
before `on_complete` ever runs. before `on_complete` ever runs.
3. Connect to `/run/uce.sock` (non-blocking) and queue the encoded request in 3. Connect to `FCGI_SOCKET_PATH` (non-blocking) and queue the encoded request in
`ws_broker_outbound[fd]`. `ws_broker_outbound[fd]` with an enqueue timestamp.
`ws_broker_drain_outbound()` runs after every `process(50)` tick: it finishes `ws_broker_drain_outbound()` runs after every `process(50)` tick: it finishes
writing each queued request, then drains and discards the reply (the unit's writing each queued request, then drains and discards the reply (the unit's
output comes back via the command socket, not this reply), closing the fd when output comes back via the command socket, not this reply), closing the fd when
the worker closes its end. the worker closes its end. If a forward remains pending beyond
`WS_BROKER_OUTBOUND_TIMEOUT_SECONDS` (default `30`), the broker drops and
closes it so a wedged worker cannot pin broker fds/memory indefinitely.
### 6.2 Outbound: `ws_*` commands flushed back to the broker ### 6.2 Outbound: `ws_*` commands flushed back to the broker
@@ -211,8 +488,10 @@ Any unit code — not just WebSocket handlers — may call `ws_send` / `ws_send_
/ `ws_close`. In the workspace these **record dispatch commands** rather than / `ws_close`. In the workspace these **record dispatch commands** rather than
touching a socket (the workspace owns no connections); `wasm-core`'s `ws_*` touching a socket (the workspace owns no connections); `wasm-core`'s `ws_*`
(`src/lib/sys.cpp`) append to `websocket_dispatch_commands`, and (`src/lib/sys.cpp`) append to `websocket_dispatch_commands`, and
`finish_response_meta` (`src/wasm/core.cpp`) emits them as `ws_commands` (plus `finish_response_meta` (`src/wasm/core.cpp`) emits them as `ws_commands`.
`ws_connection_state` if the handler mutated per-connection state). If the handler changed per-connection state, the core also emits
`ws_connection_state` even when no commands were emitted, and the native
backend flushes this state-only batch to the broker.
`wasm_backend_serve` (`src/wasm/backend.cpp`) flushes that batch at workspace `wasm_backend_serve` (`src/wasm/backend.cpp`) flushes that batch at workspace
teardown — in **any** scenario, not just WS handlers — to the broker's command teardown — in **any** scenario, not just WS handlers — to the broker's command
@@ -239,9 +518,10 @@ serve_http dispatcher uses, so there is no duplicated request-forwarding code.
handlers (it renders nothing, so it accepts every request straight through to handlers (it renders nothing, so it accepts every request straight through to
`on_complete`), wires `on_complete=ws_broker_complete` and `on_complete`), wires `on_complete=ws_broker_complete` and
`on_websocket_message=ws_broker_ws_message`, listens on `HTTP_PORT` and the `on_websocket_message=ws_broker_ws_message`, listens on `HTTP_PORT` and the
command socket, and loops `process(50)` + `drain_outbound()`. The design is command socket, and loops `process(50)` + `drain_outbound(timeout)`. The
**non-blocking outbound dispatch + async command-socket flush, all in the `timeout` comes from `WS_BROKER_OUTBOUND_TIMEOUT_SECONDS` (default `30`). The
broker's single epoll loop** — the broker never blocks on a worker. design is **non-blocking outbound dispatch + async command-socket flush, all in
the broker's single epoll loop** — the broker never blocks on a worker.
The parent respawns the broker if it dies (`ws_broker_alive` / `ensure_ws_broker` The parent respawns the broker if it dies (`ws_broker_alive` / `ensure_ws_broker`
in `main()`). in `main()`).
@@ -284,22 +564,119 @@ header free-functions are `inline`. The wasm backend exposes only declarations
| Key | Default | Meaning | | Key | Default | Meaning |
|---|---|---| |---|---|---|
| `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). | | `WASM_BACKEND_VERBOSE` | `0` | Emit `X-UCE-Wasm-*` workspace timing headers (benchmark only). |
| `FCGI_SOCKET_PATH` | `/run/uce.sock` | Worker pool FastCGI socket (brokers forward here). | | `WASM_EPOCH_DEADLINE_TICKS` | `200` | Maximum uninterrupted guest CPU segment in epoch ticks; must be positive. |
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI socket. | | `WASM_EPOCH_PERIOD_MS` | `50` | Worker epoch-ticker period and timeout resolution; range `1``1000` ms. |
| `WASM_INVOCATION_TIMEOUT_MS` | `30000` | Absolute app-owned unit load/init/handler/nested-call deadline; range `1``86400000` ms and nested calls cannot renew it. |
| `FCGI_SOCKET_PATH` | runtime-configured (`/run/uce/fastcgi.sock` in this doc) | Worker pool FastCGI socket (brokers forward here). |
| `CLI_SOCKET_PATH` | `/run/uce/cli.sock` | Worker CLI/admin socket. Keep private; reference `CLI_SOCKET_MODE` is `0600`. |
| `FCGI_SOCKET_MODE` | `0666` | Permission mode applied to `FCGI_SOCKET_PATH` after bind; set tighter if nginx/Apache can use a trusted group. |
| `CLI_SOCKET_MODE` | `0600` | Permission mode applied to `CLI_SOCKET_PATH`; set `0660` only for a trusted admin group. |
| `HTTP_PORT` | `8080` | Raw HTTP + WebSocket port — owned by the WS broker. | | `HTTP_PORT` | `8080` | Raw HTTP + WebSocket port — owned by the WS broker. |
| `WS_BROKER_SOCKET_PATH` | `/run/uce/ws-broker.sock` | Broker command socket for `ws_*` flushes. | | `WS_BROKER_SOCKET_PATH` | `/run/uce/ws-broker.sock` | Broker command socket for `ws_*` flushes. |
| `WORKER_COUNT` | `4` | Number of uniform worker processes. | | `WS_BROKER_OUTBOUND_TIMEOUT_SECONDS` | `30` | Max lifetime in seconds for queued WS broker forwards before drop. |
| `WORKER_COUNT` | `4` | Number of public FastCGI worker processes. |
| `CLI_WORKER_COUNT` | `0` built-in; `2` in the reference config | Additional CLI-only workers. Two permit one nested CLI invocation without self-deadlock; a positive count prevents test/admin module-cache churn from paging out public workers, while zero preserves the legacy shared pool. |
| `CLI_WORKER_MAX_REQUESTS` | `8` | Completed CLI connections before a dedicated CLI worker is recycled (maximum `1024`; `0` disables). Public workers are not request-count recycled. |
| `WASM_SERIALIZE_TIMEOUT_SECONDS` | `120` | Deadline for each short-lived serialized-module child; process exit reclaims pooling allocator arenas from proactive scanners. |
Proactive scanners never construct a Wasmtime serialization engine themselves.
Each candidate is serialized by the executable's bounded `--serialize-module`
child while holding the unit compile lock. Publication rechecks device, inode,
size, mtime, and ctime, and the scanner removes dead child temporary artifacts.
--- ---
## 10. Testing ## 10. Testing
- **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the - **Regression gate**: `scripts/run_cli_tests.sh --include-wasm-kill` runs the
in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages. in-runtime CLI suite (`site/tests/cli_runner.uce`) plus the site test pages and
`scripts/test_dependency_invalidation.sh`. The shell helper runs the CLI suite
as named groups (`demo`, `http`, `site`, `doc-gate`, `security`,
`task-lifetime`, `starter`, `tcp`, and optional `wasm-kill`) so a slow or
failing group is visible before the rest of the gate runs. `doc-gate` can take
several minutes on a cold runtime because it compiles generated documentation
examples; `UCE_CLI_TEST_TIMEOUT` controls the per-group curl timeout and
defaults to 900 seconds. Individual doc-page checks retry transient empty
frontend reads, but still fail persistent status, marker, compile, runtime, or
placeholder-example errors. The dependency-invalidation gate changes a transitive
`#load`, then replaces a warmed worker artifact while preserving its
whole-second mtime to prove both compiler and worker caches invalidate it. It
also breaks a loaded dependency, verifies the parent compile fails, restores
the dependency byte-for-byte, and requires the parent to recover. Failed
builds persist the input signature that actually failed, so metadata from an
older successful artifact cannot indefinitely defer that rebuild.
The gate also rejects an unreadable unit without publishing a wasm artifact, restores
its permissions and proves the next CLI request compiles it, then sends 48
requests and asserts the observed worker PID set does not exceed
`WORKER_COUNT`, guarding against accidental reintroduction of request-count
recycling. It also races a source edit against an active unit compile after
the generated C++ snapshot has been written. The compiler must either retry
and serve the post-edit source or fail closed; it must never stamp current
source metadata onto wasm produced from an older snapshot.
- **Metadata scanner deadline**: an isolated one-worker gate uses a targeted
positional-read interposer to prove canonical expiry after one delayed 4 KiB
metadata read and after one delayed cache-miss 64 KiB full-artifact read,
concurrent truncation and same-size mutation rejection, malformed tenth-byte
LEB rejection, duplicate selected-section rejection, and same-worker recovery.
- **Core compatibility**: the production core owns and exports its growable
function table, enabling `InstancePre`. A separately built legacy
`--import-table` core must still pass the demo and 64-request pool-isolation
groups before the compatibility fallback changes.
`scripts/test_wasm_compile_timeout.sh` runs an isolated one-worker runtime
with a two-second request budget. It covers cold entry JIT, guest
`unit_compile()`, dynamic component and transitive `#load` compilation,
held unit and registry locks, silent nonzero and missing-output compiler
results, compiler descendants, staged-output timeout, prior-generation
hashes, same-worker recovery, configured-error-page boundedness, residue,
foreign-owned offline-precompile artifacts, and deadline-independent offline
precompile. Normal rollback snapshots are hard links. If Linux ownership or
link policy rejects that fast path, the compiler copies the prior artifact
under the same unit lock and keeps the same all-or-nothing publication.
Source-generation markers likewise publish through a same-directory rename,
so a runtime user can replace a readable marker created by an administrator.
Existing foreign-owned unit, registry, generation, and PCH lock files are
opened read-only for `flock`; new locks are still created read-write.
`scripts/test_cold_component_deadline.sh` separately compiles a deliberately
cold component that exceeds the development epoch window and proves the
parent request still renders it. The focused shell gates create temporary
`.uce` units under `UCE_TEST_SITE_DIRECTORY`, or under the installed
`SITE_DIRECTORY` from `/etc/uce/settings.cfg` when no override is supplied.
The dependency-invalidation gate also holds parent and child compile locks
across a transitive source edit, proves a warmed HTTP request returns the
last atomically published result without waiting, proves a POST returns a
prompt retryable 503 without executing its old mutation handler, and verifies
demand-priority convergence. A CLI request still waits and returns only the
current dependency result after rebuild completion.
The component page also calls a parent once without its optional relative
child and again with that child activated. This guards the component-slot
cache invariant that a cache hit must restore both the function-table slot
and the resolved unit path; otherwise nested relative targets resolve from
the caller after a repeated parent invocation. Resolved component paths are
canonical absolute paths: equivalent spellings containing `.` or `..` must
share one source, compile, artifact, and module-cache identity. The compiler
independently enforces the same canonical identity. Within one resolution,
identical entry/site bases and identical raw/`.uce` candidate spellings are
probed only once while first-match order is preserved. Persisted failures are
reused only when their recorded source path matches the current request.
`scripts/test_nested_component_props.sh` passes a large prop tree through an
outer component, invokes 300 nested components, and verifies both inner and
caller props are restored. Its warm-request ceiling guards the request-scope
invariant that entering a nested component swaps the active prop trees in
constant time instead of deep-copying the outer tree for every child.
`scripts/test_mysql_epoch_refresh.sh` spends more than half of the configured
guest epoch on each side of a real MySQL query. It proves the blocking
database hostcall refreshes the guest deadline so cumulative request work is
allowed while a CPU loop without intervening host work still traps.
- **Log timeliness**: the base process line-buffers stdout before forking
workers, the proactive compiler, and the WebSocket broker. This keeps each
newline-terminated diagnostic attached to its actual journal time instead of
releasing an old block-buffered failure during unrelated later traffic. The
private CLI timing gate labels its deliberate invalid-source request as
`UCE expected compile error`; public and unmarked CLI compilation failures
retain the ordinary `UCE compile error` operator signal.
- **WebSocket end-to-end**: a headless client performs a raw WS handshake to - **WebSocket end-to-end**: a headless client performs a raw WS handshake to
`:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving `:HTTP_PORT` with path `/site/tests/websockets.ws.uce` (self-resolving
`SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full `SCRIPT_FILENAME`) and asserts the `hello-ack` frame — exercising the full
broker → worker → broker → client chain across process boundaries. broker → worker → broker → client chain across process boundaries.
+51 -1
View File
@@ -5,39 +5,78 @@ SESSION_PATH=/var/lib/uce/sessions
# LISTEN ON SOCKETS # LISTEN ON SOCKETS
FCGI_SOCKET_PATH=/run/uce/fastcgi.sock FCGI_SOCKET_PATH=/run/uce/fastcgi.sock
# FastCGI is normally consumed by the local web server user/group.
FCGI_SOCKET_MODE=0666
FCGI_PORT=9993 FCGI_PORT=9993
# HTTP-over-Unix command socket for local CLI/control tooling. # HTTP-over-Unix command socket for local CLI/control tooling.
# Example: curl --unix-socket /run/uce/cli.sock http://localhost/ping # Example: curl --unix-socket /run/uce/cli.sock http://localhost/ping
CLI_SOCKET_PATH=/run/uce/cli.sock CLI_SOCKET_PATH=/run/uce/cli.sock
# Keep the CLI/admin socket private by default; set 0660 only for a trusted group.
CLI_SOCKET_MODE=0600
# Keep CLI/test module caches out of public FastCGI workers. Set to 0 only for
# the legacy shared pool; dedicated workers are additional to WORKER_COUNT.
CLI_WORKER_COUNT=2
# Recycle dedicated CLI workers before broad test/admin runs retain every unit.
# Zero disables recycling; public FastCGI workers are never request-count recycled.
CLI_WORKER_MAX_REQUESTS=8
# Built-in HTTP/WebSocket listener used for WebSocket Upgrade requests. # Built-in HTTP/WebSocket listener used for WebSocket Upgrade requests.
# Keep this behind nginx/Apache on localhost or firewall it from public access. # Keep this behind nginx/Apache on localhost or firewall it from public access.
HTTP_PORT=8080 HTTP_PORT=8080
# After this many seconds, an in-flight WS broker render-forward that is still
# not completed (including stalled writes/reads) is dropped.
WS_BROKER_OUTBOUND_TIMEOUT_SECONDS=30
# OPTIONAL PROACTIVE COMPILE ROOT # OPTIONAL PROACTIVE COMPILE ROOT
# Leave empty to scan SITE_DIRECTORY relative to the runtime root. # Leave empty to scan SITE_DIRECTORY relative to the runtime root.
PRECOMPILE_FILES_IN= PRECOMPILE_FILES_IN=
# NUMBER OF LOW-PRIORITY PROCESSES USED BY MANAGED GENERATION PRECOMPILE
# Two keeps one half of a typical four-vCPU host available to live workers.
PRECOMPILE_JOBS=2
# PUBLIC SITE DIRECTORY USED FOR STARTUP SCAN WHEN PRECOMPILE_FILES_IN IS EMPTY # PUBLIC SITE DIRECTORY USED FOR STARTUP SCAN WHEN PRECOMPILE_FILES_IN IS EMPTY
SITE_DIRECTORY=site SITE_DIRECTORY=site
# ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT # ENABLE JIT COMPILATION WHEN A PAGE REQUEST HITS A STALE OR MISSING UNIT
JIT_COMPILE_ON_REQUEST=1 JIT_COMPILE_ON_REQUEST=1
# SHOW THE BOUNDED COMPILER DIAGNOSTIC WHEN DYNAMIC DISPATCH TARGETS A UNIT
# THAT FAILED TO COMPILE. WHEN DISABLED, THE CALL REPORTS THE GENERIC MISSING
# COMPONENT OR UNIT ERROR.
SHOW_DYNAMIC_COMPILE_ERRORS=1
# KEEP SERVING A COMPATIBLE LAST-KNOWN-GOOD ARTIFACT TO HTTP READ REQUESTS
# WHILE THE PROACTIVE COMPILER BUILDS A CHANGED UNIT. MUTATIONS AND CLI NEVER
# SERVE STALE CODE. REQUIRES THE PROACTIVE COMPILER AND DEFAULTS FAIL-CLOSED.
SERVE_LAST_KNOWN_GOOD=0
# WASM SIDE-MODULE COMPILER USED FOR .uce UNITS # WASM SIDE-MODULE COMPILER USED FOR .uce UNITS
WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit WASM_COMPILE_SCRIPT=scripts/compile_wasm_unit
# WASM RUNTIME SETTINGS. Unit execution is always routed through wasm. # WASM RUNTIME SETTINGS. Unit execution is always routed through wasm.
WASM_BACKEND_VERBOSE=0 WASM_BACKEND_VERBOSE=0
WASM_CORE_PATH=/Code/uce.openfu.com/uce/bin/wasm/core.wasm WASM_PROFILE_HOSTCALL_CPU=0
WASM_PROFILE_THREAD_RUNTIME=0
WASM_CORE_PATH=bin/wasm/core.wasm
WASM_MEMORY_LIMIT_BYTES=536870912 WASM_MEMORY_LIMIT_BYTES=536870912
WASM_EPOCH_DEADLINE_TICKS=200 WASM_EPOCH_DEADLINE_TICKS=200
WASM_EPOCH_PERIOD_MS=50 WASM_EPOCH_PERIOD_MS=50
WASM_INVOCATION_TIMEOUT_MS=30000
# Bound each short-lived proactive/precompile serialized-module child.
WASM_SERIALIZE_TIMEOUT_SECONDS=120
MYSQL_PERSISTENT_POOL_SIZE=8
MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS=300
# ENABLE THE BACKGROUND PROACTIVE COMPILER LOOP # ENABLE THE BACKGROUND PROACTIVE COMPILER LOOP
PROACTIVE_COMPILE_ENABLED=1 PROACTIVE_COMPILE_ENABLED=1
# NUMBER OF LOW-PRIORITY FULL-SITE SCANNER PROCESSES (1-16)
# The separate demand-priority compiler is not counted here.
PROACTIVE_COMPILE_JOBS=2
# AFTER A FAILED COMPILE, UCE NOW REUSES THE PERSISTED COMPILER OUTPUT UNTIL # AFTER A FAILED COMPILE, UCE NOW REUSES THE PERSISTED COMPILER OUTPUT UNTIL
# THE SOURCE OR COMPILER INPUTS CHANGE. THIS SETTING IS KEPT FOR COMPATIBILITY. # THE SOURCE OR COMPILER INPUTS CHANGE. THIS SETTING IS KEPT FOR COMPATIBILITY.
COMPILE_FAILURE_RETRY_SECONDS=10 COMPILE_FAILURE_RETRY_SECONDS=10
@@ -81,3 +120,14 @@ ARCHIVE_MAX_ZIP_ENTRIES=4096
# LIFETIME OF SESSION COOKIES IN SECONDS # LIFETIME OF SESSION COOKIES IN SECONDS
SESSION_TIME=2592000 SESSION_TIME=2592000
# Set to 1 behind HTTPS-only frontends to add Secure to session cookies.
SESSION_COOKIE_SECURE=0
# DEVELOPER-DEFINED ERROR PAGES (see /doc/index.uce?p=error_pages and
# /doc/index.uce?p=3_blocked_functions). The unit receives the error details in
# context.call["error"]; comment out for the plain-text fallback instead.
page_runtime_error=site/errors/runtime-error.uce
# Comma-separated uce_host_* names to disable per server policy (empty = none).
# A unit calling a blocked hostcall fails into the error page above.
UCE_HOSTCALL_BLOCKLIST=
+18 -11
View File
@@ -15,35 +15,42 @@ DOC_DIR = ROOT / "site" / "doc" / "pages"
# name, needs_doc, status. status: public | internal | integration # name, needs_doc, status. status: public | internal | integration
PUBLIC_APIS = [ PUBLIC_APIS = [
("shell_exec", True, "public"), ("shell_escape", True, "public"), ("http_request", True, "public"), ("http_request_async", True, "public"),
("shell_exec", True, "public"), ("shell_escape", True, "public"), ("shell_spawn", True, "public"),
("job_status", True, "public"), ("job_result", True, "public"), ("job_await", True, "public"), ("job_cancel", True, "public"),
("basename", True, "public"), ("dirname", True, "public"), ("path_join", True, "public"), ("basename", True, "public"), ("dirname", True, "public"), ("path_join", True, "public"),
("path_real", True, "public"), ("path_is_within", True, "public"), ("path_real", True, "public"), ("path_is_within", True, "public"),
("file_open_locked", True, "public"), ("file_close_locked", True, "public"), ("file_get_contents", True, "public"),
("file_release_process_locks", True, "public"), ("file_get_contents_locked_fd", True, "public"), ("file_put_contents", True, "public"), ("file_append", True, "public"),
("file_put_contents_locked_fd", True, "public"), ("file_get_contents", True, "public"),
("file_put_contents", True, "public"), ("file_append_contents", False, "public"),
("cwd_get", True, "public"), ("cwd_set", True, "public"), ("process_start_directory", True, "public"), ("cwd_get", True, "public"), ("cwd_set", True, "public"), ("process_start_directory", True, "public"),
("file_mtime", True, "public"), ("file_unlink", True, "public"), ("expand_path", True, "public"), ("file_mtime", True, "public"), ("file_unlink", True, "public"), ("expand_path", True, "public"),
("ls", True, "public"), ("config_map_u64", True, "public"), ("config_map_f64", True, "public"), ("ls", True, "public"), ("to_u64", True, "public"), ("to_s64", True, "public"),
("config_bool_value", True, "public"), ("config_map_bool", True, "public"), ("to_f64", True, "public"), ("to_bool", True, "public"),
("config_u64", True, "public"), ("config_f64", True, "public"), ("config_bool", True, "public"),
("request_perf", True, "public"), ("time_format_local", True, "public"), ("request_perf", True, "public"), ("time_format_local", True, "public"),
("time_format_relative", True, "public"), ("time_parse", True, "public"), ("time_format_relative", True, "public"), ("time_parse", True, "public"),
("backtrace_frames_string", False, "public"), ("capture_backtrace_string", False, "public"), ("backtrace_get_frames", False, "public"), ("backtrace_capture", False, "public"),
("signal_name", False, "public"), ("memcache_escape_key", True, "public"), ("signal_name", False, "public"), ("memcache_escape_key", True, "public"),
("memcache_escape_keys", True, "public"), ("memcache_command", True, "public"), ("memcache_escape_keys", True, "public"), ("memcache_command", True, "public"),
("memcache_get_multiple", True, "public"), ("runtime_safe_key", True, "public"), ("memcache_get_multiple", True, "public"), ("runtime_safe_key", True, "public"),
("float_val", True, "public"), ("nibble", True, "public"), ("json_consume_space", False, "public"), ("float_val", True, "public"), ("nibble", True, "public"), ("json_consume_space", False, "public"),
("array_merge", True, "public"), ("safe_name", True, "public"), ("ascii_safe_name", True, "public"), ("array_merge", True, "public"), ("safe_name", True, "public"), ("ascii_safe_name", True, "public"),
("to_json", False, "public"), ("remove", False, "public"), ("clear", False, "public"), ("to_json", False, "public"), ("remove", False, "public"), ("clear", False, "public"),
("gen_sha1", True, "public"), ("gen_noise32", True, "public"), ("gen_noise64", True, "public"), ("gen_sha1", True, "public"), ("sha256", True, "public"), ("sha256_hex", True, "public"),
("hmac_sha256", True, "public"), ("hmac_sha256_hex", True, "public"), ("random_bytes", True, "public"),
("crypto_equal", True, "public"), ("password_hash", True, "public"),
("password_verify", True, "public"), ("password_needs_rehash", True, "public"),
("crypto_operation", True, "public"),
("gen_noise32", True, "public"), ("gen_noise64", True, "public"),
("gen_noise01", True, "public"), ("gen_int", True, "public"), ("gen_float", True, "public"), ("gen_noise01", True, "public"), ("gen_int", True, "public"), ("gen_float", True, "public"),
("draw_int", True, "public"), ("draw_float", True, "public"), ("draw_int", True, "public"), ("draw_float", True, "public"),
("encode_query", True, "public"), ("request_script_url", True, "public"), ("encode_query", True, "public"), ("csrf_token", True, "public"),
("csrf_valid", True, "public"), ("csrf_rotate", True, "public"),
("request_script_url", True, "public"),
("request_base_url", True, "public"), ("request_route_from_raw_path", True, "public"), ("request_base_url", True, "public"), ("request_route_from_raw_path", True, "public"),
("cli_arg", True, "public"), ("unit_compile", True, "public"), ("cli_arg", True, "public"), ("unit_compile", True, "public"),
("cleanup_sqlite_connections", False, "internal"), ("cleanup_mysql_connections", False, "internal"), ("cleanup_sqlite_connections", False, "internal"), ("cleanup_mysql_connections", False, "internal"),
("mysql_connect", True, "integration"), ("mysql_query", True, "integration"), ("mysql_connect", True, "integration"), ("mysql_query", True, "integration"),
("mysql_affected_rows", True, "integration"),
] ]
REMOVED_APIS = ["unit_load", "concat"] REMOVED_APIS = ["unit_load", "concat"]
+2 -1
View File
@@ -21,7 +21,8 @@ fi
-Wl,--export-all \ -Wl,--export-all \
-Wl,--export=__heap_base \ -Wl,--export=__heap_base \
-Wl,--export=__stack_pointer \ -Wl,--export=__stack_pointer \
-Wl,--import-table \ -Wl,--export-table \
-Wl,--growable-table \
-Wl,-z,stack-size=8388608 \ -Wl,-z,stack-size=8388608 \
-Wl,--allow-undefined-file=src/wasm/core_hostcalls.syms \ -Wl,--allow-undefined-file=src/wasm/core_hostcalls.syms \
-Wl,--no-entry \ -Wl,--no-entry \
+29 -18
View File
@@ -1,4 +1,5 @@
#!/bin/bash #!/bin/bash
set -euo pipefail
cd "$(dirname "$0")" cd "$(dirname "$0")"
cd .. cd ..
@@ -6,11 +7,14 @@ BUILDMODE=${2:-"debug"}
OPT_FLAG="O0" OPT_FLAG="O0"
GF="uce_fastcgi" GF="uce_fastcgi"
mkdir bin > /dev/null 2>&1 mkdir -p bin/tmp bin/assets bin/wasm work
mkdir bin/tmp > /dev/null 2>&1 exec 9>bin/.build.lock
mkdir bin/assets > /dev/null 2>&1 flock 9
mkdir bin/wasm > /dev/null 2>&1 build_tmp_files=()
mkdir work > /dev/null 2>&1 cleanup_build_tmp() {
((${#build_tmp_files[@]} == 0)) || rm -f "${build_tmp_files[@]}"
}
trap cleanup_build_tmp EXIT
COMPILER="clang++" COMPILER="clang++"
# -rdynamic is a link-time flag; the -c compiles below do not need it. # -rdynamic is a link-time flag; the -c compiles below do not need it.
@@ -21,7 +25,7 @@ WASMTIME_HOME=${WASMTIME_HOME:-/opt/wasmtime}
WASM_FLAGS="-I$WASMTIME_HOME/include" WASM_FLAGS="-I$WASMTIME_HOME/include"
WASM_LIBS="-L$WASMTIME_HOME/lib -Wl,-rpath,$WASMTIME_HOME/lib -lwasmtime" WASM_LIBS="-L$WASMTIME_HOME/lib -Wl,-rpath,$WASMTIME_HOME/lib -lwasmtime"
LIBS="-ldl -lm -lpthread -lpcre2-8 `mysql_config --cflags --libs` $WASM_LIBS" LIBS="-ldl -lm -lpthread -lpcre2-8 -lcrypto `mysql_config --cflags --libs` $WASM_LIBS"
SRCFLAGS="-D EXEC_NAME=\"$GF\" -D PLATFORM_NAME=\"linux\"" SRCFLAGS="-D EXEC_NAME=\"$GF\" -D PLATFORM_NAME=\"linux\""
# The runtime is split into separately-compiled objects so an edit to one # The runtime is split into separately-compiled objects so an edit to one
@@ -52,13 +56,16 @@ fi
# SQLite: vendored C, depends only on its own source (not our headers). # SQLite: vendored C, depends only on its own source (not our headers).
if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then if needs_rebuild bin/sqlite3.o src/3rdparty/sqlite/sqlite3.c src/3rdparty/sqlite/sqlite3.h; then
echo "Compiling SQLite..." echo "Compiling SQLite..."
tmp="bin/sqlite3.o.tmp.$$"
build_tmp_files+=("$tmp")
clang -g -O2 -fPIC \ clang -g -O2 -fPIC \
-DSQLITE_THREADSAFE=1 \ -DSQLITE_THREADSAFE=1 \
-DSQLITE_OMIT_LOAD_EXTENSION=1 \ -DSQLITE_OMIT_LOAD_EXTENSION=1 \
-DSQLITE_DQS=0 \ -DSQLITE_DQS=0 \
-DSQLITE_DEFAULT_FOREIGN_KEYS=1 \ -DSQLITE_DEFAULT_FOREIGN_KEYS=1 \
-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1 \ -DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1 \
-c src/3rdparty/sqlite/sqlite3.c -o bin/sqlite3.o 2>&1 || exit 1 -c src/3rdparty/sqlite/sqlite3.c -o "$tmp" 2>&1
mv "$tmp" bin/sqlite3.o
else else
echo "Reusing bin/sqlite3.o" echo "Reusing bin/sqlite3.o"
fi fi
@@ -67,7 +74,10 @@ fi
# declarations (not the lib .cpp — those are compiled into main.o). # declarations (not the lib .cpp — those are compiled into main.o).
if needs_rebuild bin/wasm.o src/wasm src/lib/*.h; then if needs_rebuild bin/wasm.o src/wasm src/lib/*.h; then
echo "Compiling wasm backend..." echo "Compiling wasm backend..."
time -p $COMPILER -c src/wasm/wasm_module.cpp $SRCFLAGS $FLAGS $WASM_FLAGS -o bin/wasm.o 2>&1 || exit 1 tmp="bin/wasm.o.tmp.$$"
build_tmp_files+=("$tmp")
time -p $COMPILER -c src/wasm/wasm_module.cpp $SRCFLAGS $FLAGS $WASM_FLAGS -o "$tmp" 2>&1
mv "$tmp" bin/wasm.o
else else
echo "Reusing bin/wasm.o" echo "Reusing bin/wasm.o"
fi fi
@@ -77,18 +87,19 @@ fi
# (its only view of the wasm object) — but not the wasm .cpp sources. # (its only view of the wasm object) — but not the wasm .cpp sources.
if needs_rebuild bin/main.o src/linux_fastcgi.cpp src/lib src/fastcgi src/wasm/backend.h; then if needs_rebuild bin/main.o src/linux_fastcgi.cpp src/lib src/fastcgi src/wasm/backend.h; then
echo "Compiling main..." echo "Compiling main..."
time -p $COMPILER -c src/linux_fastcgi.cpp $SRCFLAGS $FLAGS -o bin/main.o 2>&1 || exit 1 tmp="bin/main.o.tmp.$$"
build_tmp_files+=("$tmp")
time -p $COMPILER -c src/linux_fastcgi.cpp $SRCFLAGS $FLAGS -o "$tmp" 2>&1
mv "$tmp" bin/main.o
else else
echo "Reusing bin/main.o" echo "Reusing bin/main.o"
fi fi
echo "Linking..." echo "Linking..."
$COMPILER -rdynamic bin/main.o bin/wasm.o bin/sqlite3.o $FLAGS $LIBS -o bin/$GF.linux.bin 2>&1 binary_tmp="bin/$GF.linux.bin.tmp.$$"
build_tmp_files+=("$binary_tmp")
if [ $? -eq 0 ] $COMPILER -rdynamic bin/main.o bin/wasm.o bin/sqlite3.o $FLAGS $LIBS -o "$binary_tmp" 2>&1
then test -s "$binary_tmp"
ls -lh bin/ | grep $GF chmod 0755 "$binary_tmp"
exit 0 mv "$binary_tmp" "bin/$GF.linux.bin"
else ls -lh "bin/$GF.linux.bin"
exit 1
fi
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env python3
"""Extract a compact address-to-source table before a unit's DWARF is stripped."""
import argparse
import ast
import os
import re
import subprocess
DIRECTORY = re.compile(r'^include_directories\[\s*(\d+)\] = (".*")$')
FILE = re.compile(r'^file_names\[\s*(\d+)\]:$')
ROW = re.compile(r'^0x([0-9a-fA-F]+)\s+(\d+)\s+(\d+)\s+(\d+)\s+')
def quoted(value: str) -> str:
return ast.literal_eval(value)
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--dwarfdump", required=True)
parser.add_argument("--wasm", required=True)
parser.add_argument("--module", required=True)
parser.add_argument("--output", required=True)
args = parser.parse_args()
text = subprocess.run(
[args.dwarfdump, "--debug-line", args.wasm],
check=True,
stdout=subprocess.PIPE,
text=True,
).stdout
directories: dict[int, str] = {}
files: dict[int, tuple[str, int]] = {}
rows: list[tuple[int, int, int, int]] = []
current_file = 0
for line in text.splitlines():
if match := DIRECTORY.match(line):
directories[int(match.group(1))] = quoted(match.group(2))
continue
if match := FILE.match(line):
current_file = int(match.group(1))
continue
stripped = line.strip()
if current_file and stripped.startswith("name: "):
files[current_file] = (quoted(stripped[6:]), 0)
continue
if current_file and stripped.startswith("dir_index: "):
name, _ = files[current_file]
files[current_file] = (name, int(stripped[11:]))
continue
if match := ROW.match(line):
rows.append(tuple(int(value, 16 if index == 0 else 10) for index, value in enumerate(match.groups())))
paths: dict[int, str] = {}
for file_id, (name, directory_id) in files.items():
directory = directories.get(directory_id, "")
path = name if os.path.isabs(name) else os.path.normpath(os.path.join(directory, name))
if not os.path.isabs(path):
path = os.path.abspath(path)
if "\t" in path or "\n" in path or "\r" in path:
raise ValueError(f"source-map path contains a control character: {path!r}")
paths[file_id] = path
with open(args.output, "w", encoding="utf-8") as output:
output.write(f"UCE_SOURCE_MAP_V1\t{args.module}\n")
for file_id, path in sorted(paths.items()):
output.write(f"F\t{file_id}\t{path}\n")
for address, line, column, file_id in rows:
output.write(f"L\t{address:x}\t{file_id}\t{line}\t{column}\n")
if __name__ == "__main__":
main()
+10 -1
View File
@@ -160,7 +160,13 @@ def defined_symbols(path: Path, llvm_nm: str) -> list[str]:
def main() -> int: def main() -> int:
ap = argparse.ArgumentParser() ap = argparse.ArgumentParser()
ap.add_argument("wasm", type=Path) ap.add_argument("wasm", type=Path)
ap.add_argument("--abi-version", default="6") default_abi = "7"
abi_header = Path(__file__).resolve().parents[1] / "src" / "wasm" / "abi.h"
for line in abi_header.read_text().splitlines():
if line.startswith("#define UCE_WASM_CORE_ABI_VERSION "):
default_abi = line.rsplit(" ", 1)[1]
break
ap.add_argument("--abi-version", default=default_abi)
ap.add_argument("--llvm-nm", default=None) ap.add_argument("--llvm-nm", default=None)
ap.add_argument("--verbose", action="store_true") ap.add_argument("--verbose", action="store_true")
args = ap.parse_args() args = ap.parse_args()
@@ -182,6 +188,9 @@ def main() -> int:
for needle in required: for needle in required:
if needle not in abi_text: if needle not in abi_text:
errors.append(f"uce.abi missing {needle!r}") errors.append(f"uce.abi missing {needle!r}")
module_payloads = customs.get("uce.module", [])
if not module_payloads or not module_payloads[-1]:
errors.append("missing uce.module custom section")
export_names = {name for name, _ in exports} export_names = {name for name, _ in exports}
forbidden_exports = {"uce_alloc", "uce_free"} forbidden_exports = {"uce_alloc", "uce_free"}
for name in sorted(export_names & forbidden_exports): for name in sorted(export_names & forbidden_exports):
+37 -10
View File
@@ -9,17 +9,21 @@ DEST_DIR="$2"
SRC_FN="$3" SRC_FN="$3"
PP_FN="$4" PP_FN="$4"
WASM_FN="$5" WASM_FN="$5"
UNIT_CACHE_DIR="${6:-$DEST_DIR}"
SDK=${WASI_SDK:-/opt/wasi-sdk} SDK=${WASI_SDK:-/opt/wasi-sdk}
ABI_VERSION=${UCE_UNIT_ABI_VERSION:-6} ABI_VERSION=${UCE_UNIT_ABI_VERSION:-$(awk '/^#define UCE_WASM_CORE_ABI_VERSION / {print $3; exit}' src/wasm/abi.h)}
ROOT=$(pwd) ROOT=$(pwd)
OBJ_FN="$DEST_DIR/$PP_FN.wasm.o" OBJ_FN="$DEST_DIR/$PP_FN.wasm.o"
ABI_TMP="$DEST_DIR/$PP_FN.uce-abi.txt" ABI_TMP="$DEST_DIR/$PP_FN.uce-abi.txt"
MODULE_TMP="$DEST_DIR/$PP_FN.uce-module.txt"
WASM_TMP="$DEST_DIR/$WASM_FN.tmp.$$"
MAP_TMP="$DEST_DIR/$WASM_FN.source-map.tmp.$$"
PCH_ENABLED=${UCE_WASM_UNIT_PCH:-1} PCH_ENABLED=${UCE_WASM_UNIT_PCH:-1}
PCH_DIR=${UCE_WASM_PCH_DIR:-/tmp/uce/wasm-w2/pch} PCH_DIR=${UCE_WASM_PCH_DIR:-$UNIT_CACHE_DIR/.pch}
COMMON_FLAGS=( COMMON_FLAGS=(
--target=wasm32-wasip1 --target=wasm32-wasip1
-fPIC -fvisibility=default -fvisibility-inlines-hidden -fPIC -fvisibility=hidden -fvisibility-inlines-hidden
-O1 -g -std=c++20 -O1 -g -std=c++20
# The server captures this script's output and treats any non-empty result as # The server captures this script's output and treats any non-empty result as
# a compile failure (then drops the .wasm), so a successful build must be # a compile failure (then drops the .wasm), so a successful build must be
@@ -32,7 +36,7 @@ COMMON_FLAGS=(
-DPLATFORM_NAME=\"wasm32-wasip1\" -DPLATFORM_NAME=\"wasm32-wasip1\"
) )
if [ ! -x "$SDK/bin/clang++" ] || [ ! -x "$SDK/bin/wasm-ld" ] || [ ! -x "$SDK/bin/llvm-objcopy" ]; then if [ ! -x "$SDK/bin/clang++" ] || [ ! -x "$SDK/bin/wasm-ld" ] || [ ! -x "$SDK/bin/llvm-objcopy" ] || [ ! -x "$SDK/bin/llvm-dwarfdump" ]; then
echo "wasi-sdk tools not found; set WASI_SDK" >&2 echo "wasi-sdk tools not found; set WASI_SDK" >&2
exit 1 exit 1
fi fi
@@ -42,21 +46,31 @@ HEADER_HASH=$(find src/lib -maxdepth 1 -name '*.h' -type f -print0 | sort -z | x
FLAGS_HASH=$(printf '%s\0' "${COMMON_FLAGS[@]}" -Isrc/lib | sha1sum | cut -c1-16) FLAGS_HASH=$(printf '%s\0' "${COMMON_FLAGS[@]}" -Isrc/lib | sha1sum | cut -c1-16)
PCH_KEY=$(printf '%s\n%s\n%s\n%s\n' "$ABI_VERSION" "$TOOLCHAIN_ID" "$HEADER_HASH" "$FLAGS_HASH" | sha1sum | cut -c1-16) PCH_KEY=$(printf '%s\n%s\n%s\n%s\n' "$ABI_VERSION" "$TOOLCHAIN_ID" "$HEADER_HASH" "$FLAGS_HASH" | sha1sum | cut -c1-16)
PCH_FN="$PCH_DIR/uce_lib-wasm-unit-$PCH_KEY.pch" PCH_FN="$PCH_DIR/uce_lib-wasm-unit-$PCH_KEY.pch"
PCH_TMP="$PCH_FN.tmp.$$"
mkdir -p "$DEST_DIR" >/dev/null 2>&1 mkdir -p "$DEST_DIR" >/dev/null 2>&1
trap 'rm -f "$OBJ_FN" "$ABI_TMP" "$MODULE_TMP" "$WASM_TMP" "$MAP_TMP" "$PCH_TMP"' EXIT
build_pch_if_needed() { build_pch_if_needed() {
if [ "$PCH_ENABLED" = "0" ]; then if [ "$PCH_ENABLED" = "0" ]; then
return 0 return 0
fi fi
mkdir -p "$PCH_DIR" mkdir -p "$PCH_DIR"
if ! { exec 9>"$PCH_FN.lock"; } 2>/dev/null; then
exec 9<"$PCH_FN.lock"
fi
flock 9
if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then if [ -s "$PCH_FN" ] && [ -z "$(find src/lib -maxdepth 1 -name '*.h' -type f -newer "$PCH_FN" -print -quit)" ]; then
flock -u 9
exec 9>&-
return 0 return 0
fi fi
"$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \ "$SDK/bin/clang++" "${COMMON_FLAGS[@]}" \
-Isrc/lib \ -Isrc/lib \
-x c++-header src/lib/uce_lib.h -o "$PCH_FN.tmp" -x c++-header src/lib/uce_lib.h -o "$PCH_TMP"
mv "$PCH_FN.tmp" "$PCH_FN" mv "$PCH_TMP" "$PCH_FN"
flock -u 9
exec 9>&-
} }
cat > "$ABI_TMP" <<EOF cat > "$ABI_TMP" <<EOF
@@ -65,6 +79,7 @@ unit_abi_version=$ABI_VERSION
toolchain=$TOOLCHAIN_ID toolchain=$TOOLCHAIN_ID
source=$SRC_FN source=$SRC_FN
EOF EOF
printf '%s' "$(basename "$WASM_TMP")" > "$MODULE_TMP"
build_pch_if_needed build_pch_if_needed
PCH_FLAGS=() PCH_FLAGS=()
@@ -78,9 +93,10 @@ fi
-c "$DEST_DIR/$PP_FN" -o "$OBJ_FN" -c "$DEST_DIR/$PP_FN" -o "$OBJ_FN"
"$SDK/bin/wasm-ld" -shared --experimental-pic \ "$SDK/bin/wasm-ld" -shared --experimental-pic \
--gc-sections \
--unresolved-symbols=import-dynamic \ --unresolved-symbols=import-dynamic \
--Bsymbolic \ --Bsymbolic \
"$OBJ_FN" -o "$DEST_DIR/$WASM_FN" \ "$OBJ_FN" -o "$WASM_TMP" \
--export-if-defined=__uce_set_current_request \ --export-if-defined=__uce_set_current_request \
--export-if-defined=__uce_render \ --export-if-defined=__uce_render \
--export-if-defined=__uce_component \ --export-if-defined=__uce_component \
@@ -90,8 +106,19 @@ fi
--export-if-defined=__uce_once \ --export-if-defined=__uce_once \
--export-if-defined=__uce_init --export-if-defined=__uce_init
"$SDK/bin/llvm-objcopy" --add-section=uce.abi="$ABI_TMP" "$DEST_DIR/$WASM_FN" python3 scripts/build_unit_source_map.py \
--dwarfdump "$SDK/bin/llvm-dwarfdump" \
--wasm "$WASM_TMP" \
--module "$(basename "$WASM_TMP")" \
--output "$MAP_TMP"
python3 scripts/check_unit_wasm.py "$DEST_DIR/$WASM_FN" --abi-version "$ABI_VERSION" --llvm-nm "$SDK/bin/llvm-nm" "$SDK/bin/llvm-objcopy" --strip-debug \
--add-section=uce.abi="$ABI_TMP" \
--add-section=uce.module="$MODULE_TMP" \
"$WASM_TMP"
rm -f "$OBJ_FN" "$ABI_TMP" python3 scripts/check_unit_wasm.py "$WASM_TMP" --abi-version "$ABI_VERSION" --llvm-nm "$SDK/bin/llvm-nm"
mv "$MAP_TMP" "$DEST_DIR/$WASM_FN.source-map"
mv "$WASM_TMP" "$DEST_DIR/$WASM_FN"
rm -f "$OBJ_FN" "$ABI_TMP" "$MODULE_TMP"
+2 -1
View File
@@ -4,7 +4,8 @@ set -e
if command -v systemctl >/dev/null 2>&1; then if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true systemctl daemon-reload >/dev/null 2>&1 || true
if [ "$1" = "configure" ]; then if [ "$1" = "configure" ]; then
systemctl enable uce.service >/dev/null 2>&1 || true systemctl enable uce.socket uce.service >/dev/null 2>&1 || true
systemctl start uce.socket >/dev/null 2>&1 || true
systemctl restart uce.service >/dev/null 2>&1 || true systemctl restart uce.service >/dev/null 2>&1 || true
fi fi
fi fi
+1 -1
View File
@@ -3,7 +3,7 @@ set -e
if [ "$1" = "remove" ] || [ "$1" = "deconfigure" ]; then if [ "$1" = "remove" ] || [ "$1" = "deconfigure" ]; then
if command -v systemctl >/dev/null 2>&1; then if command -v systemctl >/dev/null 2>&1; then
systemctl disable --now uce.service >/dev/null 2>&1 || true systemctl disable --now uce.service uce.socket >/dev/null 2>&1 || true
fi fi
fi fi
+5 -4
View File
@@ -1,21 +1,22 @@
[Unit] [Unit]
Description=UCE FastCGI Runtime Description=UCE FastCGI Runtime
After=network-online.target mariadb.service memcached.service After=network-online.target mariadb.service memcached.service uce.socket
Wants=network-online.target Wants=network-online.target
Requires=uce.socket
[Service] [Service]
Type=simple Type=simple
WorkingDirectory=/usr/lib/uce WorkingDirectory=/usr/lib/uce
RuntimeDirectory=uce RuntimeDirectory=uce
RuntimeDirectoryPreserve=yes
StateDirectory=uce StateDirectory=uce
CacheDirectory=uce CacheDirectory=uce
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock
ExecStart=/usr/lib/uce/bin/uce_fastcgi.linux.bin ExecStart=/usr/lib/uce/bin/uce_fastcgi.linux.bin
ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock ExecStartPost=/usr/bin/bash /usr/lib/uce/scripts/systemd/wait-ready.sh
Restart=always Restart=always
RestartSec=2 RestartSec=2
TimeoutStopSec=15 TimeoutStopSec=30
KillMode=mixed KillMode=mixed
StandardOutput=journal StandardOutput=journal
StandardError=journal StandardError=journal
+15
View File
@@ -0,0 +1,15 @@
[Unit]
Description=UCE FastCGI socket
[Socket]
ListenStream=/run/uce/fastcgi.sock
FileDescriptorName=fastcgi
SocketUser=www-data
SocketGroup=www-data
SocketMode=0660
DirectoryMode=0755
Backlog=100
Service=uce.service
[Install]
WantedBy=sockets.target
+6 -1
View File
@@ -53,13 +53,18 @@ require_command() {
verify_tree() { verify_tree() {
local root="$1" local root="$1"
for tool in clang++ wasm-ld llvm-objcopy llvm-nm; do for tool in clang++ wasm-ld llvm-objcopy llvm-nm llvm-dwarfdump; do
if [[ ! -x "$root/bin/$tool" ]]; then if [[ ! -x "$root/bin/$tool" ]]; then
echo "Missing WASI SDK tool: $root/bin/$tool" >&2 echo "Missing WASI SDK tool: $root/bin/$tool" >&2
return 1 return 1
fi fi
done done
if ! command -v curl >/dev/null 2>&1; then
echo "Missing runtime dependency: curl (required by UCE http_request/http_request_async)" >&2
return 1
fi
"$root/bin/clang++" --version | head -n 1 "$root/bin/clang++" --version | head -n 1
curl --version | head -n 1
} }
if [[ $check_only -eq 1 ]]; then if [[ $check_only -eq 1 ]]; then
+1
View File
@@ -205,6 +205,7 @@ bundle_wasmtime "$STAGE_DIR"
write_packaged_settings "$STAGE_DIR/etc/uce/settings.cfg" "$WEBROOT" write_packaged_settings "$STAGE_DIR/etc/uce/settings.cfg" "$WEBROOT"
install -m 0644 "$DEB_ASSET_DIR/uce.service" "$STAGE_DIR/lib/systemd/system/uce.service" install -m 0644 "$DEB_ASSET_DIR/uce.service" "$STAGE_DIR/lib/systemd/system/uce.service"
install -m 0644 "$DEB_ASSET_DIR/uce.socket" "$STAGE_DIR/lib/systemd/system/uce.socket"
install -m 0644 "$DEB_ASSET_DIR/conffiles" "$DEBIAN_DIR/conffiles" install -m 0644 "$DEB_ASSET_DIR/conffiles" "$DEBIAN_DIR/conffiles"
install -m 0755 "$DEB_ASSET_DIR/postinst" "$DEBIAN_DIR/postinst" install -m 0755 "$DEB_ASSET_DIR/postinst" "$DEBIAN_DIR/postinst"
install -m 0755 "$DEB_ASSET_DIR/prerm" "$DEBIAN_DIR/prerm" install -m 0755 "$DEB_ASSET_DIR/prerm" "$DEBIAN_DIR/prerm"
+5 -2
View File
@@ -179,6 +179,7 @@ bundle_wasi_sdk "$STAGE_DIR"
bundle_wasmtime "$STAGE_DIR" bundle_wasmtime "$STAGE_DIR"
write_packaged_settings "$STAGE_DIR/etc/uce/settings.cfg" "$WEBROOT" write_packaged_settings "$STAGE_DIR/etc/uce/settings.cfg" "$WEBROOT"
install -m 0644 "$REPO_ROOT/scripts/deb/uce.service" "$STAGE_DIR/usr/lib/systemd/system/uce.service" install -m 0644 "$REPO_ROOT/scripts/deb/uce.service" "$STAGE_DIR/usr/lib/systemd/system/uce.service"
install -m 0644 "$REPO_ROOT/scripts/deb/uce.socket" "$STAGE_DIR/usr/lib/systemd/system/uce.socket"
( (
cd "$STAGE_DIR" cd "$STAGE_DIR"
@@ -225,13 +226,14 @@ cp -a %{_builddir}/$PACKAGE_NAME-$VERSION/. %{buildroot}/
%post %post
if command -v systemctl >/dev/null 2>&1; then if command -v systemctl >/dev/null 2>&1; then
systemctl daemon-reload >/dev/null 2>&1 || true systemctl daemon-reload >/dev/null 2>&1 || true
systemctl enable uce.service >/dev/null 2>&1 || true systemctl enable uce.socket uce.service >/dev/null 2>&1 || true
systemctl start uce.socket >/dev/null 2>&1 || true
systemctl restart uce.service >/dev/null 2>&1 || true systemctl restart uce.service >/dev/null 2>&1 || true
fi fi
%preun %preun
if [ "\$1" = "0" ] && command -v systemctl >/dev/null 2>&1; then if [ "\$1" = "0" ] && command -v systemctl >/dev/null 2>&1; then
systemctl disable --now uce.service >/dev/null 2>&1 || true systemctl disable --now uce.service uce.socket >/dev/null 2>&1 || true
fi fi
%postun %postun
@@ -245,6 +247,7 @@ fi
%config(noreplace) /etc/uce/settings.cfg %config(noreplace) /etc/uce/settings.cfg
/usr/lib/uce /usr/lib/uce
/usr/lib/systemd/system/uce.service /usr/lib/systemd/system/uce.service
/usr/lib/systemd/system/uce.socket
$WEBROOT $WEBROOT
%dir /var/cache/uce %dir /var/cache/uce
%dir /var/lib/uce %dir /var/lib/uce
+55 -3
View File
@@ -3,6 +3,7 @@ set -euo pipefail
cd "$(dirname "$0")/.." cd "$(dirname "$0")/.."
socket_path="${UCE_CLI_SOCKET:-/run/uce/cli.sock}" socket_path="${UCE_CLI_SOCKET:-/run/uce/cli.sock}"
curl_timeout="${UCE_CLI_TEST_TIMEOUT:-900}"
if [[ -z "${UCE_CLI_SOCKET:-}" && -r /etc/uce/settings.cfg ]]; then if [[ -z "${UCE_CLI_SOCKET:-}" && -r /etc/uce/settings.cfg ]]; then
configured_socket=$(awk -F= '/^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg) configured_socket=$(awk -F= '/^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_socket:-}" ]]; then if [[ -n "${configured_socket:-}" ]]; then
@@ -11,6 +12,7 @@ if [[ -z "${UCE_CLI_SOCKET:-}" && -r /etc/uce/settings.cfg ]]; then
fi fi
include_kill=0 include_kill=0
skip_local_service_pages=0
action="run" action="run"
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
case "$1" in case "$1" in
@@ -18,15 +20,22 @@ while [[ $# -gt 0 ]]; do
include_kill=1 include_kill=1
shift shift
;; ;;
--skip-local-service-pages)
skip_local_service_pages=1
shift
;;
--list) --list)
action="list" action="list"
shift shift
;; ;;
-h|--help) -h|--help)
cat <<'USAGE' cat <<'USAGE'
Usage: scripts/run_cli_tests.sh [--include-wasm-kill] [--list] Usage: scripts/run_cli_tests.sh [--include-wasm-kill] [--skip-local-service-pages] [--list]
Runs the UCE unit-based test suite through the runtime CLI socket. Runs the UCE unit-based test suite through the runtime CLI socket.
Environment:
UCE_CLI_TEST_TIMEOUT Per CLI runner group curl timeout in seconds (default: 900).
USAGE USAGE
exit 0 exit 0
;; ;;
@@ -42,5 +51,48 @@ if [[ ! -S "$socket_path" ]]; then
exit 1 exit 1
fi fi
url="http://localhost/tests/cli_runner.uce?action=${action}&include_kill=${include_kill}" base_url="http://localhost/tests/cli_runner.uce?action=${action}&include_kill=${include_kill}&skip_local_service_pages=${skip_local_service_pages}"
exec curl -sS --fail-with-body --unix-socket "$socket_path" "$url" if [[ "$action" == "list" ]]; then
curl -sS --max-time "$curl_timeout" --fail-with-body --unix-socket "$socket_path" "$base_url"
exit 0
fi
if [[ "$action" == "run" ]]; then
groups=(demo http site doc-gate-{1..30} security task-lifetime pool-isolation starter tcp)
if [[ "$include_kill" == "1" ]]; then
groups+=(wasm-kill)
fi
for group in "${groups[@]}"; do
echo "== UCE CLI group: $group =="
curl -sS --max-time "$curl_timeout" --fail-with-body --unix-socket "$socket_path" "${base_url}&group=${group}"
done
scripts/test_dependency_invalidation.sh
scripts/test_compiler_lock_directory.sh
scripts/test_abi_generation_rollout.sh
scripts/test_parallel_precompile.sh
timeout --signal=TERM --kill-after=5s 175s scripts/test_parallel_proactive_compile.sh
scripts/test_cold_component_deadline.sh
scripts/test_wasm_invocation_timeout.sh
timeout --signal=TERM --kill-after=5s 130s scripts/test_wasm_compile_timeout.sh
scripts/test_nested_component_props.sh
scripts/test_component_once_prefetch.sh
scripts/test_relative_component_cache.sh
scripts/test_password_hashing.sh
scripts/test_crypto_operation_native.sh
scripts/test_hardened_http_native.sh
scripts/test_mysql_epoch_refresh.sh
scripts/test_mysql_persistent_pool.sh
scripts/test_mysql_persistent_pool_idle.sh
scripts/test_log_timeliness.sh
scripts/test_raw_http_request_log.sh
scripts/test_component_resolution_ttl.sh
timeout --signal=TERM --kill-after=5s 120s scripts/test_entry_freshness_ttl.sh
scripts/test_unit_export_surface.sh
timeout --signal=TERM --kill-after=5s 120s scripts/test_wasm_metadata_buffer.sh
timeout --signal=TERM --kill-after=5s 150s scripts/test_wasm_metadata_deadline.sh
timeout --signal=TERM --kill-after=5s 240s scripts/test_dynamic_compile_failures.sh
scripts/test_wasm_source_locations.sh
scripts/test_server_arguments.sh
scripts/test_cli_worker_isolation.sh
scripts/test_socket_activation.sh
fi
+32 -3
View File
@@ -18,11 +18,25 @@ CONFIG_SOURCE="$REPO_ROOT/etc/uce/settings.cfg"
CONFIG_DEST="/etc/uce/settings.cfg" CONFIG_DEST="/etc/uce/settings.cfg"
action="${1:-setup}" action="${1:-setup}"
precompile_timeout="${UCE_PRECOMPILE_TIMEOUT:-900s}"
render_runtime_file() {
local source="$1"
local destination="$2"
local mode="$3"
local tmp
tmp="$(mktemp)"
trap 'rm -f "$tmp"' RETURN
sed "s#/Code/uce.openfu.com/uce#$REPO_ROOT#g" "$source" > "$tmp"
install -D -m "$mode" "$tmp" "$destination"
rm -f "$tmp"
trap - RETURN
}
install_unit() { install_unit() {
install -D -m 0644 "$UNIT_SOURCE" "$UNIT_DEST" render_runtime_file "$UNIT_SOURCE" "$UNIT_DEST" 0644
if [[ ! -f "$CONFIG_DEST" ]]; then if [[ ! -f "$CONFIG_DEST" ]]; then
install -D -m 0644 "$CONFIG_SOURCE" "$CONFIG_DEST" render_runtime_file "$CONFIG_SOURCE" "$CONFIG_DEST" 0644
fi fi
systemctl daemon-reload systemctl daemon-reload
} }
@@ -39,7 +53,22 @@ case "$action" in
install_unit install_unit
systemctl enable "$UNIT_NAME" systemctl enable "$UNIT_NAME"
;; ;;
start|stop|restart|status) restart)
if [[ "$REPO_ROOT" != "/usr/lib/uce" && "$REPO_ROOT" != "/opt/uce" ]]; then
"$REPO_ROOT/scripts/build_linux.sh"
fi
(
cd "$REPO_ROOT"
service_user=$(systemctl show "$UNIT_NAME" -p User --value)
if [[ -n "$service_user" && "$(id -u)" == "0" ]]; then
runuser -u "$service_user" -- timeout --signal=TERM --kill-after=10s "$precompile_timeout" nice -n 10 "$REPO_ROOT/bin/uce_fastcgi.linux.bin" --precompile
else
timeout --signal=TERM --kill-after=10s "$precompile_timeout" nice -n 10 "$REPO_ROOT/bin/uce_fastcgi.linux.bin" --precompile
fi
)
systemctl restart "$UNIT_NAME"
;;
start|stop|status)
systemctl "$action" "$UNIT_NAME" systemctl "$action" "$UNIT_NAME"
;; ;;
logs) logs)
+5 -5
View File
@@ -1,22 +1,22 @@
[Unit] [Unit]
Description=UCE FastCGI Runtime Description=UCE FastCGI Runtime
After=network-online.target mariadb.service memcached.service After=network-online.target mariadb.service memcached.service uce.socket
Wants=network-online.target Wants=network-online.target
Requires=uce.socket
[Service] [Service]
Type=simple Type=simple
WorkingDirectory=/Code/uce.openfu.com/uce WorkingDirectory=/Code/uce.openfu.com/uce
RuntimeDirectory=uce RuntimeDirectory=uce
RuntimeDirectoryPreserve=yes
StateDirectory=uce StateDirectory=uce
CacheDirectory=uce CacheDirectory=uce
ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions ExecStartPre=/usr/bin/mkdir -p /var/cache/uce/work /var/lib/uce/uploads /var/lib/uce/sessions
ExecStartPre=/usr/bin/rm -f /run/uce/fastcgi.sock
ExecStartPre=/usr/bin/bash /Code/uce.openfu.com/uce/scripts/build_linux.sh
ExecStart=/Code/uce.openfu.com/uce/bin/uce_fastcgi.linux.bin ExecStart=/Code/uce.openfu.com/uce/bin/uce_fastcgi.linux.bin
ExecStopPost=/usr/bin/rm -f /run/uce/fastcgi.sock ExecStartPost=/usr/bin/bash /Code/uce.openfu.com/uce/scripts/systemd/wait-ready.sh
Restart=always Restart=always
RestartSec=2 RestartSec=2
TimeoutStopSec=15 TimeoutStopSec=30
KillMode=mixed KillMode=mixed
StandardOutput=journal StandardOutput=journal
StandardError=journal StandardError=journal
+15
View File
@@ -0,0 +1,15 @@
[Unit]
Description=UCE FastCGI socket
[Socket]
ListenStream=/run/uce/fastcgi.sock
FileDescriptorName=fastcgi
SocketUser=www-data
SocketGroup=www-data
SocketMode=0660
DirectoryMode=0755
Backlog=100
Service=uce.service
[Install]
WantedBy=sockets.target
+20
View File
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -euo pipefail
socket_path="/run/uce/cli.sock"
if [[ -r /etc/uce/settings.cfg ]]; then
configured_socket=$(awk -F= '/^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_socket:-}" ]]; then
socket_path="$configured_socket"
fi
fi
for ((attempt = 0; attempt < 200; attempt++)); do
if [[ -S "$socket_path" ]] && [[ "$(curl -sS --max-time 0.2 --unix-socket "$socket_path" http://localhost/ping 2>/dev/null || true)" == "uce-cli: ok" ]]; then
exit 0
fi
sleep 0.05
done
echo "UCE CLI socket did not become ready: $socket_path" >&2
exit 1
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="abi-generation-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
[[ -z "${configured_site_directory:-}" ]] || site_directory="$configured_site_directory"
fi
source_dir="$site_directory/$test_name"
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
body_file="/tmp/uce-abi-generation-body-$$"
ready_file="/tmp/uce-abi-generation-ready-$$"
cleanup() {
rm -rf "$source_dir"
rm -f "$body_file" "$ready_file"
[[ -z "${artifact_dir:-}" ]] || rm -rf "$artifact_dir"
}
trap cleanup EXIT
mkdir -p "$source_dir"
printf '%s\n' \
'RENDER(Request& context) { print("entry-current"); }' \
'CLI(Request& context) {' \
' if(context.get["artifact"] == "entry") print(unit_info(context.params["SCRIPT_FILENAME"])["wasm_name"].to_string());' \
' else print("entry-current");' \
'}' >"$source_dir/entry.uce"
printf '%s\n' \
'COMPONENT(Request& context) { print("component-current"); }' >"$source_dir/child.uce"
printf '%s\n' \
'RENDER(Request& context) { DValue props; print(component("child", props, context)); }' \
'CLI(Request& context) {' \
' DValue props; component("child", props, context);' \
' print(unit_info("child.uce")["wasm_name"].to_string());' \
'}' >"$source_dir/component-parent.uce"
entry_url="http://127.0.0.1/$test_name/entry.uce"
component_url="http://127.0.0.1/$test_name/component-parent.uce"
[[ "$(curl -fsS -H "Host: $http_host" "$entry_url")" == "entry-current" ]]
entry_wasm=$(scripts/uce-cli --get "/$test_name/entry.uce" artifact=entry)
component_wasm=$(scripts/uce-cli "/$test_name/component-parent.uce")
artifact_dir=$(dirname "$entry_wasm")
generation_name=$(basename "$(scripts/unit_cache_directory /tmp/uce-generation-probe)")
if [[ "$entry_wasm" != *"/$generation_name/"* || "$component_wasm" != *"/$generation_name/"* ]]; then
echo "unit artifacts are not isolated by compiler/core ABI generation: $entry_wasm $component_wasm" >&2
exit 1
fi
publish_incompatible_artifact_while_locked() {
local source_file="$1"
local wasm_file="$2"
local marker="$3"
local destination
destination=$(dirname "$wasm_file")
(
exec 8>"$wasm_file.lock"
flock 8
UCE_UNIT_ABI_VERSION=6 scripts/compile_wasm_unit \
"$(dirname "$source_file")" "$destination" "$source_file" \
"$(basename "$source_file").cpp" "$(basename "$wasm_file")"
sed -i \
-e 's/^unit_abi_version=.*/unit_abi_version=0/' \
-e 's/^wasm_core_abi_version=.*/wasm_core_abi_version=0/' \
"${wasm_file%.wasm}.meta.txt"
rm -f "${wasm_file%.wasm}.cwasm"
printf '%s\n' "$marker" >"$ready_file"
sleep 2
) &
lock_pid=$!
deadline=$((SECONDS + 20))
while [[ ! -e "$ready_file" && $SECONDS -lt $deadline ]]; do sleep 0.05; done
if [[ ! -e "$ready_file" || "$(cat "$ready_file")" != "$marker" ]]; then
echo "incompatible artifact was not published under lock: $marker" >&2
exit 1
fi
}
publish_incompatible_artifact_while_locked "$source_dir/entry.uce" "$entry_wasm" entry
started_at=$(date +%s%N)
entry_status=$(curl -sS --max-time 30 -o "$body_file" -w '%{http_code}' -H "Host: $http_host" "$entry_url")
entry_elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
wait "$lock_pid"
if [[ "$entry_status" != "200" || "$(cat "$body_file")" != "entry-current" || $entry_elapsed_ms -lt 1500 ]]; then
echo "ABI-incompatible entry artifact was served instead of joining its rebuild: status=$entry_status elapsed=${entry_elapsed_ms}ms body=$(cat "$body_file")" >&2
exit 1
fi
rm -f "$ready_file"
publish_incompatible_artifact_while_locked "$source_dir/child.uce" "$component_wasm" component
started_at=$(date +%s%N)
component_status=$(curl -sS --max-time 30 -o "$body_file" -w '%{http_code}' -H "Host: $http_host" "$component_url")
component_elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
wait "$lock_pid"
if [[ "$component_status" != "200" || "$(cat "$body_file")" != "component-current" || $component_elapsed_ms -lt 1500 ]]; then
echo "ABI-incompatible component artifact was served instead of joining its rebuild: status=$component_status elapsed=${component_elapsed_ms}ms body=$(cat "$body_file")" >&2
exit 1
fi
echo "ABI generation rollout passed"
+153
View File
@@ -0,0 +1,153 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" != "--inside" ]]; then
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
fi
root="/tmp/uce-cli-worker-isolation-$$"
site="$root/site"
work="$root/work"
settings="$root/settings.cfg"
log="$root/service.log"
cli_socket="$root/run/cli.sock"
fastcgi_socket="$root/run/fastcgi.sock"
server_pid=""
cleanup() {
status=$?
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
deadline=$((SECONDS + 10))
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
if kill -0 "$server_pid" 2>/dev/null; then kill -KILL "$server_pid" 2>/dev/null || true; fi
wait "$server_pid" 2>/dev/null || true
fi
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
rm -rf "$root"
return "$status"
}
trap cleanup EXIT
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
cp /etc/uce/settings.cfg "$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$fastcgi_socket
FCGI_PORT=
CLI_SOCKET_PATH=$cli_socket
CLI_WORKER_COUNT=2
CLI_WORKER_MAX_REQUESTS=3
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
WORKER_COUNT=2
PROACTIVE_COMPILE_ENABLED=0
CFG
mount --bind "$settings" /etc/uce/settings.cfg
cat >"$site/isolation.uce" <<'UCE'
RENDER(Request& context)
{
print(request_perf()["worker_pid"].to_string());
}
CLI(Request& context)
{
print(request_perf()["worker_pid"].to_string());
}
UCE
cat >"$site/nested.uce" <<UCE
CLI(Request& context)
{
print(shell_exec("cd $(pwd) && timeout 10s scripts/uce-cli --socket $cli_socket /isolation.uce"));
}
UCE
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
deadline=$((SECONDS + 20))
while [[ ! -S "$cli_socket" ]] && (( SECONDS < deadline )); do sleep 0.05; done
[[ -S "$cli_socket" ]] || { echo "private UCE CLI socket was not ready" >&2; exit 1; }
timeout --signal=TERM --kill-after=1s 30s scripts/uce-cli --socket "$cli_socket" /isolation.uce >/dev/null
timeout --signal=TERM --kill-after=1s 30s scripts/uce-cli --socket "$cli_socket" /nested.uce | grep -Eq '^[0-9]+([.]0+)?$'
fastcgi_request() {
SCRIPT_FILENAME="$site/isolation.uce" SCRIPT_NAME=/isolation.uce REQUEST_URI=/isolation.uce REQUEST_METHOD=GET DOCUMENT_ROOT="$site" \
timeout --signal=TERM --kill-after=1s 10s cgi-fcgi -bind -connect "$fastcgi_socket" | tr -d '\r' | awk 'NF{last=$0} END{print last}'
}
fastcgi_request >/dev/null
cli_request() {
local output
local deadline=$((SECONDS + 20))
while (( SECONDS < deadline )); do
if output=$(timeout --signal=TERM --kill-after=1s 10s scripts/uce-cli --socket "$cli_socket" /isolation.uce 2>/dev/null); then
printf '%s\n' "$output"
return 0
fi
sleep 0.05
done
return 1
}
cli_pids=()
http_pids=()
# Concurrent completions must count individually and retire both initial CLI
# workers despite neither connection pool necessarily becoming idle first.
concurrent_jobs=()
for n in $(seq 1 12); do
cli_request >"$root/concurrent-$n" & concurrent_jobs+=("$!")
done
for job in "${concurrent_jobs[@]}"; do wait "$job"; done
mapfile -t concurrent_cli < <(cat "$root"/concurrent-* | sort -u)
cli_pids+=("${concurrent_cli[@]}")
for _ in $(seq 1 12); do
cli_pids+=("$(cli_request)")
http_pids+=("$(fastcgi_request)")
done
mapfile -t unique_cli < <(printf '%s\n' "${cli_pids[@]}" | sort -u)
mapfile -t unique_http < <(printf '%s\n' "${http_pids[@]}" | sort -u)
[[ ${#unique_cli[@]} -ge 4 ]] || { echo "CLI workers did not recycle after three requests: ${unique_cli[*]}" >&2; exit 1; }
[[ ${#unique_http[@]} -ge 1 ]] || { echo "HTTP requests reached no public worker" >&2; exit 1; }
for cli_pid in "${unique_cli[@]}"; do
if printf '%s\n' "${unique_http[@]}" | grep -Fxq "$cli_pid"; then
echo "CLI worker also served public HTTP: $cli_pid" >&2
exit 1
fi
done
[[ $(grep -c 'wasm FastCGI worker ready' "$log") -eq 2 ]]
[[ $(grep -c 'wasm CLI worker ready' "$log") -ge 4 ]]
echo "CLI worker isolation and recycling passed: CLI ${unique_cli[*]}, public ${unique_http[*]}"
# A CLI-only runtime must not allocate a listener-less public Wasmtime worker.
kill -TERM "$server_pid"
wait "$server_pid"
server_pid=""
rm -f "$cli_socket" "$fastcgi_socket"
cat >>"$settings" <<CFG
FCGI_SOCKET_PATH=
CLI_WORKER_COUNT=1
CLI_WORKER_MAX_REQUESTS=0
CFG
: >"$log"
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
deadline=$((SECONDS + 20))
while [[ ! -S "$cli_socket" ]] && (( SECONDS < deadline )); do sleep 0.05; done
[[ -S "$cli_socket" ]] || { echo "CLI-only socket was not ready" >&2; exit 1; }
cli_request >/dev/null
[[ $(grep -c 'wasm CLI worker ready' "$log") -eq 1 ]]
[[ $(grep -c 'wasm FastCGI worker ready' "$log") -eq 0 ]]
echo "CLI-only worker allocation passed"
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="component-deadline-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'CLI(Request& context) { DValue props; print(component("child", props, context)); }' >"$source_dir/parent.uce"
for i in $(seq 1 800); do
printf 'String cold_component_pad_%s() { return("%s"); }\n' "$i" "$i"
done >"$source_dir/child.uce"
printf '%s\n' \
'COMPONENT(Request& context) { <><strong>cold-component-deadline-ok</strong></> }' >>"$source_dir/child.uce"
rm -rf "$cache_dir"
output=$(scripts/uce-cli "/$test_name/parent.uce")
if [[ "$output" != *cold-component-deadline-ok* ]]; then
echo "cold component compilation consumed the guest epoch budget: $output" >&2
exit 1
fi
echo "cold component deadline passed"
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" != "--inside" ]]; then
exec timeout --signal=TERM --kill-after=5s 90s unshare --mount --fork --kill-child=TERM "$0" --inside
fi
root="/tmp/uce-compiler-lock-directory-$$"
site="$root/site"
work="$root/missing/parents/work"
settings="$root/settings.cfg"
log="$root/service.log"
server_pid=""
cleanup() {
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
wait "$server_pid" 2>/dev/null || true
fi
rm -rf "$root"
}
trap cleanup EXIT
mkdir -p "$site" "$root/run" "$root/session" "$root/upload"
printf '%s\n' 'CLI(Request& context) { print("recursive-lock-directory-ok"); }' >"$site/test.uce"
cp /etc/uce/settings.cfg "$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
FCGI_PORT=
CLI_SOCKET_PATH=$root/run/cli.sock
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
WORKER_COUNT=1
PROACTIVE_COMPILE_ENABLED=0
CFG
mount --bind "$settings" /etc/uce/settings.cfg
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
deadline=$((SECONDS + 20))
while [[ ! -S "$root/run/cli.sock" ]] && (( SECONDS < deadline )); do sleep 0.05; done
[[ -S "$root/run/cli.sock" ]] || { cat "$log" >&2; exit 1; }
rm -rf "$work"
response=$(curl -sS --max-time 45 --fail-with-body --unix-socket "$root/run/cli.sock" http://localhost/test.uce) || { cat "$log" >&2; exit 1; }
[[ "$response" == *"recursive-lock-directory-ok"* ]] || { printf '%s\n' "$response" >&2; cat "$log" >&2; exit 1; }
find "$work" -type f -name '*.lock' -print -quit | grep -q . || { echo "compiler did not recreate a nested lock path" >&2; cat "$log" >&2; exit 1; }
echo "compiler recursive lock directory passed"
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="component-once-prefetch-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'CLI(Request& context) {' \
' String first = component("child", context);' \
' String second = component("child", context);' \
' DValue perf = request_perf();' \
' print(first, ",", second, ",", perf["component_resolve_count"].to_u64(), ",", perf["component_loaded_reuse_count"].to_u64());' \
'}' >"$source_dir/entry.uce"
printf '%s\n' \
'ONCE(Request& context) { context.call["once"] = context.call["once"].to_u64() + 1; }' \
'COMPONENT(Request& context) { print(context.call["once"].to_u64()); }' >"$source_dir/child.uce"
output=$(scripts/uce-cli "/$test_name/entry.uce")
if [[ "$output" != "1,1,1,0" ]]; then
echo "component ONCE slot was not returned with the primary handler: $output" >&2
exit 1
fi
echo "component ONCE prefetch passed"
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="component-resolution-ttl-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
worker_count="${UCE_TEST_WORKER_COUNT:-4}"
if [[ -r /etc/uce/settings.cfg ]]; then
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
site_directory="${configured_site_directory:-$site_directory}"
fi
if [[ -z "${UCE_TEST_WORKER_COUNT:-}" ]]; then
configured_worker_count=$(awk -F= '/^[[:space:]]*WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
worker_count="${configured_worker_count:-$worker_count}"
fi
fi
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' 'RENDER(Request& context) { print("parent:", request_perf()["worker_pid"].to_string(), ":", component("chosen", context)); }' >"$source_dir/parent.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("a"); }' >"$source_dir/a.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("b"); }' >"$source_dir/b.uce"
ln -s a.uce "$source_dir/chosen.uce"
declare -A seen
collect_workers() {
local method="$1"
local marker="$2"
local path="${3:-parent.uce}"
local deadline=$((SECONDS + 30))
seen=()
while (( SECONDS < deadline && ${#seen[@]} < worker_count )); do
local response status body pid
response=$(curl -sS --max-time 10 -X "$method" -H "Host: $http_host" -w $'\n%{http_code}' "http://127.0.0.1/$test_name/$path")
status=${response##*$'\n'}
body=${response%$'\n'*}
if [[ "$status" == "503" ]]; then
sleep 0.1
continue
fi
if [[ "$status" != "200" || "$body" != parent:*":$marker" ]]; then
echo "component resolution TTL probe failed: method=$method status=$status expected=$marker body=$body" >&2
exit 1
fi
pid=${body#parent:}
pid=${pid%%:*}
seen["$pid"]=1
done
if (( ${#seen[@]} != worker_count )); then
echo "component resolution TTL probe reached ${#seen[@]}/$worker_count workers" >&2
exit 1
fi
}
collect_missing_workers() {
local deadline=$((SECONDS + 30))
seen=()
while (( SECONDS < deadline && ${#seen[@]} < worker_count )); do
local response status body pid
response=$(curl -sS --max-time 10 -H "Host: $http_host" -w $'\n%{http_code}' "http://127.0.0.1/$test_name/missing.uce")
status=${response##*$'\n'}
body=${response%$'\n'*}
if [[ "$status" == "503" ]]; then
sleep 0.1
continue
fi
if [[ "$status" != "500" || "$body" != parent:* || "$body" != *"component not found: later"* ]]; then
echo "missing component TTL probe failed: status=$status body=$body" >&2
exit 1
fi
pid=${body#parent:}
pid=${pid%%:*}
seen["$pid"]=1
done
if (( ${#seen[@]} != worker_count )); then
echo "missing component TTL probe reached ${#seen[@]}/$worker_count workers" >&2
exit 1
fi
}
collect_workers GET a
ln -sfn b.uce "$source_dir/chosen.uce"
collect_workers GET a
collect_workers POST b
collect_workers GET a
sleep 10.2
collect_workers GET b
printf '%s\n' 'RENDER(Request& context) { print("parent:", request_perf()["worker_pid"].to_string(), ":", component("later", context)); }' >"$source_dir/missing.uce"
collect_missing_workers
printf '%s\n' 'COMPONENT(Request& context) { print("created"); }' >"$source_dir/later.uce"
collect_workers GET created missing.uce
echo "component resolution TTL passed"
+176
View File
@@ -0,0 +1,176 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_source="/tmp/uce-oauth-es256-native-$$.cpp"
test_binary="/tmp/uce-oauth-es256-native-$$"
cleanup() { rm -f "$test_source" "$test_binary"; }
trap cleanup EXIT
cat >"$test_source" <<'EOF'
#include "src/lib/types.cpp"
#include "src/lib/dvalue.cpp"
#include "src/lib/functionlib.cpp"
#include <openssl/bn.h>
#include <openssl/core_names.h>
#include <openssl/ecdsa.h>
#include <openssl/evp.h>
#include <openssl/params.h>
#include <memory>
String base64_encode(String raw)
{
if(raw.empty()) return("");
String out(4 * ((raw.size() + 2) / 3), 0);
int size = EVP_EncodeBlock((unsigned char*)out.data(), (const unsigned char*)raw.data(), (int)raw.size());
out.resize(size > 0 ? (size_t)size : 0);
return(out);
}
String base64_decode(String raw, bool& ok)
{
ok = false;
if(raw.empty() || raw.size() % 4) return("");
String out(3 * raw.size() / 4, 0);
int size = EVP_DecodeBlock((unsigned char*)out.data(), (const unsigned char*)raw.data(), (int)raw.size());
if(size < 0) return("");
while(!raw.empty() && raw.back() == '=') { size--; raw.pop_back(); }
out.resize((size_t)size); ok = true; return(out);
}
#include "src/lib/hash.cpp"
static String b64url_encode(String text)
{
String out=replace(replace(base64_encode(text),"+","-"),"/","_"); while(!out.empty()&&out.back()=='=') out.pop_back(); return out;
}
static String b64url_decode(String text)
{
text = replace(replace(text, "-", "+"), "_", "/");
while(text.size() % 4) text += "=";
bool ok = false;
String result = base64_decode(text, ok);
return(ok ? result : String(""));
}
static bool verify(DValue public_jwk, String jwt)
{
StringList parts = split(jwt, ".");
if(parts.size() != 3) return(false);
String x = b64url_decode(public_jwk["x"].to_string());
String y = b64url_decode(public_jwk["y"].to_string());
String raw = b64url_decode(parts[2]);
if(x.size() != 32 || y.size() != 32 || raw.size() != 64) return(false);
unsigned char point[65] = {4}; memcpy(point + 1, x.data(), 32); memcpy(point + 33, y.data(), 32);
OSSL_PARAM params[] = { OSSL_PARAM_construct_utf8_string(OSSL_PKEY_PARAM_GROUP_NAME, (char*)"prime256v1", 0), OSSL_PARAM_construct_octet_string(OSSL_PKEY_PARAM_PUB_KEY, point, sizeof(point)), OSSL_PARAM_construct_end() };
EVP_PKEY_CTX* build = EVP_PKEY_CTX_new_from_name(0, "EC", 0); EVP_PKEY* key = 0;
if(!build || EVP_PKEY_fromdata_init(build) <= 0 || EVP_PKEY_fromdata(build, &key, EVP_PKEY_PUBLIC_KEY, params) <= 0) { EVP_PKEY_CTX_free(build); return(false); }
EVP_PKEY_CTX_free(build);
BIGNUM* r = BN_bin2bn((const unsigned char*)raw.data(), 32, 0); BIGNUM* s = BN_bin2bn((const unsigned char*)raw.data() + 32, 32, 0);
ECDSA_SIG* sig = ECDSA_SIG_new(); int der_size = r && s && sig && ECDSA_SIG_set0(sig, r, s) ? i2d_ECDSA_SIG(sig, 0) : 0; r = s = 0;
String der(der_size > 0 ? (size_t)der_size : 0, 0); unsigned char* out = (unsigned char*)der.data();
bool ok = der_size > 0 && i2d_ECDSA_SIG(sig, &out) == der_size;
EVP_MD_CTX* verify_ctx = EVP_MD_CTX_new();
String signing_input = parts[0] + "." + parts[1];
ok = ok && verify_ctx && EVP_DigestVerifyInit(verify_ctx, 0, EVP_sha256(), 0, key) > 0 && EVP_DigestVerify(verify_ctx, (const unsigned char*)der.data(), der.size(), (const unsigned char*)signing_input.data(), signing_input.size()) == 1;
EVP_MD_CTX_free(verify_ctx); ECDSA_SIG_free(sig); EVP_PKEY_free(key); return(ok);
}
int main()
{
DValue key_request; key_request["operation"] = "key_generate"; key_request["algorithm"] = "ES256";
DValue key = crypto_operation_native(key_request);
DValue header; header["alg"] = "none"; header["kid"] = key["kid"]; DValue claims; claims["iss"] = "https://client.example";
auto sign = [&](DValue private_jwk) { DValue request; request["operation"] = "jwt_sign"; request["algorithm"] = "ES256"; request["private_jwk"] = private_jwk; request["protected_header"] = header; request["claims"] = claims; return(crypto_operation_native(request)); };
DValue signed_result = sign(key["private_jwk"]); String jwt = signed_result["jwt"].to_string();
DValue wrong_curve = key["private_jwk"]; wrong_curve["crv"] = "P-384";
DValue malformed = key["private_jwk"]; malformed["x"] = "bad=";
DValue mismatch = key["private_jwk"]; String d = mismatch["d"].to_string(); d[0] = d[0] == 'A' ? 'B' : 'A'; mismatch["d"] = d;
const String cose="pQECAyYgASFYIGsX0fLhLEJH-Lzm5WOkQPJ3A32BLeszoPShOUXYmMKWIlggT-NC4v4af5uO5-tKfA-eFivOM1drMV7Oy7ZAaDe_UfU", message="d2ViYXV0aG4gZml4ZWQgbWVzc2FnZQ", signature="MEUCIQCkatZK1VVsjk17uvyzyhjdAkMNWXPjxSOMqWcjmM_8XAIgDaSk3Qufyd0_6r9Dm9A8RQbFco-FdTBulq7bvRGoBC4";
auto op = [&](String operation) { DValue r; r["operation"]=operation; r["algorithm"]="ES256"; return r; };
auto parse = [&](String key) { DValue r=op("cose_es256_parse"); r["cose_key_base64url"]=key; return crypto_operation_native(r); };
auto verify_cose = [&](String key,String msg,String sig) { DValue r=op("es256_verify"); r["cose_key_base64url"]=key; r["message_base64url"]=msg; r["signature_der_base64url"]=sig; return crypto_operation_native(r); };
auto decode_cbor = [&](String bytes) { DValue r=op("cbor_decode"); r["cbor_base64url"]=b64url_encode(bytes); return crypto_operation_native(r); };
DValue unsupported; unsupported["operation"] = "encrypt"; unsupported["algorithm"] = "ES256";
DValue unknown_algorithm; unknown_algorithm["operation"] = "key_generate"; unknown_algorithm["algorithm"] = "none";
DValue untyped_algorithm = key_request; untyped_algorithm["algorithm"] = (f64)256;
DValue list_header; list_header.set_array(); DValue list_item; list_item = "not-an-object"; list_header.push(list_item); DValue list_request; list_request["operation"] = "jwt_sign"; list_request["algorithm"] = "ES256"; list_request["private_jwk"] = key["private_jwk"]; list_request["protected_header"] = list_header; list_request["claims"] = claims;
DValue control_request = list_request; control_request["protected_header"] = header; control_request["claims"] = claims; control_request["claims"]["bad"] = String("control\nbyte");
DValue oversized = key_request; oversized["ignored"] = String(33000, 'x');
DValue nonfinite = key_request; nonfinite["ignored"] = std::numeric_limits<f64>::quiet_NaN();
String tampered = jwt; if(tampered.size() > 2) tampered[tampered.size() - 2] = tampered[tampered.size() - 2] == 'A' ? 'B' : 'A';
bool kid_ok = key["ok"].to_bool() && key["kid"].to_string() == key["thumbprint"].to_string();
bool signed_ok = signed_result["ok"].to_bool() && jwt != "" && verify(key["public_jwk"], jwt);
bool tamper_ok = !verify(key["public_jwk"], tampered);
auto cbor_error = [&](String bytes) { return(decode_cbor(bytes)["error"].to_string() == "invalid_cbor"); };
auto cose_error = [&](String encoded) { return(parse(encoded)["error"].to_string() == "invalid_cose_key"); };
String cose_raw = b64url_decode(cose);
String wrong_kty = cose_raw; wrong_kty[2] = 1;
String wrong_alg = cose_raw; wrong_alg[4] = 0x27;
String wrong_cose_curve = cose_raw; wrong_cose_curve[6] = 2;
String missing_label = cose_raw; missing_label[0] = 0xa4; missing_label.resize(missing_label.size() - 35);
String short_x = String("\xa5\x01\x02\x03\x26\x20\x01\x21\x58\x1f",10) + String(31, 'x') + String("\x22\x58\x20", 3) + String(32, 'y');
String duplicate_label = cose_raw; duplicate_label[0] = 0xa6; duplicate_label += String("\x01\x02", 2);
String invalid_point = String("\xa5\x01\x02\x03\x26\x20\x01\x21\x58\x20", 10) + String(32, 'x') + String("\x22\x58\x20", 3) + String(32, 'y');
String der = b64url_decode(signature);
String tampered_der = der; tampered_der[10] ^= 1;
String trailing_der = der + String("\x00", 1);
String noncanonical_der = der; noncanonical_der[1]++; noncanonical_der[3]++; noncanonical_der.insert(4, 1, '\0');
DValue valid_verify = verify_cose(cose, message, signature);
DValue tampered_signature = verify_cose(cose, message, b64url_encode(tampered_der));
DValue tampered_message = verify_cose(cose, b64url_encode("tampered"), signature);
DValue malformed_der = verify_cose(cose, message, b64url_encode(String("\x30\x00", 2)));
DValue noncanonical_der_result = verify_cose(cose, message, b64url_encode(noncanonical_der));
DValue trailing_der_result = verify_cose(cose, message, b64url_encode(trailing_der));
DValue invalid_point_result = verify_cose(b64url_encode(invalid_point), message, signature);
String large_cbor = String("\x58\x81", 2) + String(129, 'x');
String oversized_cbor = String("\x5a\x00\x00\x40\x00", 5) + String(16384, 'x');
String duplicate_compound = String("\xa2\x82\x41[\x41]\x01\x82\x41[\x41]\x02", 13);
String distinct_compound = String("\xa2\x82\x41[\x41]\x01\x82\x41[\x41[\x02", 13);
String node_overflow = String("\x99\x01\x00", 3) + String(256, 0);
String decoded;
bool cbor_valid = decode_cbor(String("\x82\x01\x62ok", 5))["ok"].to_bool();
bool cbor_large = decode_cbor(large_cbor)["ok"].to_bool();
bool cbor_control_text = decode_cbor(String("\x61\x01", 2))["ok"].to_bool();
bool cbor_duplicate = cbor_error(String("\xa2\x01\x02\x01\x03", 5));
bool cbor_compound_duplicate = cbor_error(duplicate_compound);
bool cbor_compound_distinct = decode_cbor(distinct_compound)["ok"].to_bool();
bool cbor_invalid_utf8 = cbor_error(String("\x61\x80", 2));
bool cbor_truncated = cbor_error(String("\xa1", 1));
bool cbor_trailing = cbor_error(String("\x01\x02", 2));
bool cbor_depth = cbor_error(String(17, '\x81') + "\x00");
bool cbor_nodes = cbor_error(node_overflow);
bool cbor_size = cbor_error(oversized_cbor);
bool cbor_indefinite = cbor_error(String("\x9f\x01\xff", 3));
bool cbor_integer_overflow = cbor_error(String("\x5b\xff\xff\xff\xff\xff\xff\xff\xff", 9));
bool cbor_nonminimal = cbor_error(String("\x18\x17", 2));
bool b64_padding = !uce_base64url_decode(cose + "=", decoded, UCE_CBOR_MAX_BASE64URL);
bool b64_truncated = !uce_base64url_decode("A", decoded, UCE_CBOR_MAX_BASE64URL);
bool b64_trailing_bits = !uce_base64url_decode("AB", decoded, UCE_CBOR_MAX_BASE64URL);
bool cose_valid = parse(cose)["ok"].to_bool();
bool cose_wrong_kty = cose_error(b64url_encode(wrong_kty));
bool cose_wrong_alg = cose_error(b64url_encode(wrong_alg));
bool cose_wrong_curve = cose_error(b64url_encode(wrong_cose_curve));
bool cose_short_coordinate = cose_error(b64url_encode(short_x));
bool cose_missing_label = cose_error(b64url_encode(missing_label));
bool cose_duplicate_label = cose_error(b64url_encode(duplicate_label));
bool cose_invalid_point = invalid_point_result["error"].to_string() == "invalid_key_or_payload";
bool verify_valid = valid_verify["ok"].to_bool() && valid_verify["valid"].to_bool();
bool verify_tampered_signature = tampered_signature["ok"].to_bool() && !tampered_signature["valid"].to_bool();
bool verify_tampered_message = tampered_message["ok"].to_bool() && !tampered_message["valid"].to_bool();
bool verify_malformed_der = malformed_der["error"].to_string() == "invalid_signature";
bool verify_noncanonical_der = noncanonical_der_result["error"].to_string() == "invalid_signature";
bool verify_trailing_der = trailing_der_result["error"].to_string() == "invalid_signature";
bool cbor_ok = cbor_valid && cbor_large && cbor_control_text && cbor_duplicate && cbor_compound_duplicate && cbor_compound_distinct && cbor_invalid_utf8 && cbor_truncated && cbor_trailing && cbor_depth && cbor_nodes && cbor_size && cbor_indefinite && cbor_integer_overflow && cbor_nonminimal;
bool cose_ok = cose_valid && cose_wrong_kty && cose_wrong_alg && cose_wrong_curve && cose_short_coordinate && cose_missing_label && cose_duplicate_label && cose_invalid_point;
bool cose_negative = verify_valid && verify_tampered_signature && verify_tampered_message && verify_malformed_der && verify_noncanonical_der && verify_trailing_der;
bool b64_ok = b64_padding && b64_truncated && b64_trailing_bits;
bool negatives_ok = !sign(wrong_curve)["ok"].to_bool() && !sign(malformed)["ok"].to_bool() && !sign(mismatch)["ok"].to_bool() && crypto_operation_native(unsupported)["error"].to_string() == "unsupported_operation" && crypto_operation_native(unknown_algorithm)["error"].to_string() == "unsupported_algorithm" && crypto_operation_native(untyped_algorithm)["error"].to_string() == "invalid_request" && crypto_operation_native(list_request)["error"].to_string() == "invalid_key_or_payload" && crypto_operation_native(control_request)["error"].to_string() == "invalid_request" && crypto_operation_native(oversized)["error"].to_string() == "invalid_request" && crypto_operation_native(nonfinite)["error"].to_string() == "invalid_request";
if(!(kid_ok && signed_ok && tamper_ok && negatives_ok && cose_ok && cbor_ok && cose_negative && b64_ok)) std::cerr << "kid=" << kid_ok << " signed=" << signed_ok << " tamper=" << tamper_ok << " negatives=" << negatives_ok << " cbor=" << cbor_ok << " cose=" << cose_ok << " verify=" << cose_negative << " b64=" << b64_ok << " large=" << cbor_large << " compound=" << cbor_compound_duplicate << "/" << cbor_compound_distinct << " der=" << verify_malformed_der << "/" << verify_noncanonical_der << "/" << verify_trailing_der << " cborparts=" << cbor_valid << cbor_control_text << cbor_duplicate << cbor_invalid_utf8 << cbor_truncated << cbor_trailing << cbor_depth << cbor_nodes << cbor_size << cbor_indefinite << cbor_integer_overflow << cbor_nonminimal << "\\n";
return(kid_ok && signed_ok && tamper_ok && negatives_ok && cose_ok && cbor_ok && cose_negative && b64_ok ? 0 : 1);
}
EOF
"${CXX:-c++}" -std=c++20 -fpermissive -I. "$test_source" -lpcre2-8 -lcrypto -o "$test_binary"
"$test_binary"
echo "native structured crypto operation passed"
+473
View File
@@ -0,0 +1,473 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="dependency-cache-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
mutation_file="/tmp/uce-dependency-mutation-$$"
component_mutation_file="/tmp/uce-component-mutation-$$"
component_post_body="/tmp/uce-component-post-body-$$"
component_post_headers="/tmp/uce-component-post-headers-$$"
component_lock_ready_file="/tmp/uce-component-lock-ready-$$"
post_body="/tmp/uce-dependency-post-body-$$"
post_headers="/tmp/uce-dependency-post-headers-$$"
lock_ready_file="/tmp/uce-dependency-lock-ready-$$"
nested_lock_ready_file="/tmp/uce-nested-lock-ready-$$"
race_output="/tmp/uce-source-race-output-$$"
race_error="/tmp/uce-source-race-error-$$"
race_status="/tmp/uce-source-race-status-$$"
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
rm -f "$mutation_file" "$component_mutation_file" "$component_post_body" "$component_post_headers" "$component_lock_ready_file" "$post_body" "$post_headers" "$lock_ready_file" "$nested_lock_ready_file" "$race_output" "$race_error" "$race_status"
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'#ifndef UCE_DEPENDENCY_CACHE_CHILD' \
'#define UCE_DEPENDENCY_CACHE_CHILD' \
'String dependency_cache_marker() { return("dependency-marker-a"); }' \
'#endif' >"$source_dir/child.uce"
printf '%s\n' \
'#load "child.uce"' \
'CLI(Request& context) { print(dependency_cache_marker(), ":", request_perf()["worker_pid"].to_string()); }' \
"RENDER(Request& context) { if(context.params[\"REQUEST_METHOD\"] == \"POST\") file_put_contents(\"$mutation_file\", dependency_cache_marker()); print(dependency_cache_marker(), \":\", request_perf()[\"worker_pid\"].to_string()); }" >"$source_dir/parent.uce"
assert_marker() {
local path="$1"
local expected="$2"
local output
output=$(scripts/uce-cli "/$test_name/$path.uce")
if [[ "$output" != *"$expected"* ]]; then
echo "$path returned stale module; expected $expected: $output" >&2
exit 1
fi
}
http_marker() {
curl -fsS -H "Host: $http_host" "http://127.0.0.1/$test_name/parent.uce"
}
assert_marker parent dependency-marker-a
if [[ "$(http_marker)" != *"dependency-marker-a"* ]]; then
echo "HTTP warm-up did not return dependency-marker-a" >&2
exit 1
fi
# Compiler source canonicalization must retain symlink-target identity while
# avoiding realpath's repeated per-segment probes on ordinary source graphs.
printf '%s\n' 'String symlink_dependency_marker() { return("symlink-marker-a"); }' >"$source_dir/symlink-target-a.uce"
printf '%s\n' 'String symlink_dependency_marker() { return("symlink-marker-b"); }' >"$source_dir/symlink-target-b.uce"
ln -s "symlink-target-a.uce" "$source_dir/symlink-child.uce"
printf '%s\n' '#load "symlink-child.uce"' 'CLI(Request& context) { print(symlink_dependency_marker()); }' >"$source_dir/symlink-parent.uce"
assert_marker symlink-parent symlink-marker-a
ln -sfn "symlink-target-b.uce" "$source_dir/symlink-child.uce"
assert_marker symlink-parent symlink-marker-b
# A repeated exact transitive load is signature-deduplicated before path
# canonicalization, but the shared dependency must still invalidate the parent.
printf '%s\n' '#ifndef UCE_DIAMOND_COMMON' '#define UCE_DIAMOND_COMMON' 'String diamond_marker() { return("diamond-a"); }' '#endif' >"$source_dir/diamond-common.uce"
printf '%s\n' '#load "diamond-common.uce"' >"$source_dir/diamond-left.uce"
printf '%s\n' '#load "diamond-common.uce"' >"$source_dir/diamond-right.uce"
printf '%s\n' '#load "diamond-left.uce"' '#load "diamond-right.uce"' 'CLI(Request& context) { print(diamond_marker()); }' >"$source_dir/diamond-parent.uce"
assert_marker diamond-parent diamond-a
sed -i 's/diamond-a/diamond-b/' "$source_dir/diamond-common.uce"
assert_marker diamond-parent diamond-b
# HTTP entry units can resolve route/components dynamically. A changed dynamic
# component must enter the demand-priority queue just like a changed entry unit;
# otherwise a common dependency rebuild can leave the requested page stale for
# the duration of the full proactive queue.
printf '%s\n' "COMPONENT(Request& context) { if(context.params[\"REQUEST_METHOD\"] == \"POST\") file_put_contents(\"$component_mutation_file\", \"executed\"); print(\"http-component-marker-a\"); }" >"$source_dir/http-component-child.uce"
printf '%s\n' 'RENDER(Request& context) { DValue props; print(component("http-component-child", props, context)); }' >"$source_dir/http-component-parent.uce"
http_component_marker() {
curl -fsS -H "Host: $http_host" "http://127.0.0.1/$test_name/http-component-parent.uce"
}
if [[ "$(http_component_marker)" != *"http-component-marker-a"* ]]; then
echo "HTTP component warm-up did not return marker-a" >&2
exit 1
fi
sed -i 's/http-component-marker-a/http-component-marker-b/' "$source_dir/http-component-child.uce"
started_at=$(date +%s%N)
http_component_output=$(http_component_marker)
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
if [[ "$http_component_output" != *"http-component-marker-a"* && "$http_component_output" != *"http-component-marker-b"* ]]; then
echo "stale HTTP component request returned neither complete artifact: $http_component_output" >&2
exit 1
fi
if (( elapsed_ms >= 2000 )); then
echo "HTTP component request spent ${elapsed_ms}ms rebuilding a stale artifact" >&2
exit 1
fi
deadline=$((SECONDS + 15))
while [[ "$http_component_output" != *"http-component-marker-b"* && $SECONDS -lt $deadline ]]; do
sleep 0.2
http_component_output=$(http_component_marker)
done
if [[ "$http_component_output" != *"http-component-marker-b"* ]]; then
echo "requested stale HTTP component did not receive a demand-priority rebuild" >&2
exit 1
fi
printf '%s\n' 'String http_component_dependency_marker() { return("http-component-dependency-a"); }' >"$source_dir/http-component-dependency.uce"
printf '%s\n' '#load "http-component-dependency.uce"' 'COMPONENT(Request& context) { print(http_component_dependency_marker()); }' >"$source_dir/http-transitive-child.uce"
printf '%s\n' 'RENDER(Request& context) { DValue props; print(component("http-transitive-child", props, context)); }' >"$source_dir/http-transitive-parent.uce"
http_transitive_marker() {
curl -fsS -H "Host: $http_host" "http://127.0.0.1/$test_name/http-transitive-parent.uce"
}
if [[ "$(http_transitive_marker)" != *"http-component-dependency-a"* ]]; then
echo "HTTP transitive component warm-up did not return dependency-a" >&2
exit 1
fi
sed -i 's/http-component-dependency-a/http-component-dependency-b/' "$source_dir/http-component-dependency.uce"
deadline=$((SECONDS + 20))
while [[ "$(http_transitive_marker)" != *"http-component-dependency-b"* && $SECONDS -lt $deadline ]]; do sleep 0.2; done
if [[ "$(http_transitive_marker)" != *"http-component-dependency-b"* ]]; then
echo "HTTP dynamic component did not converge after its loaded dependency changed" >&2
exit 1
fi
# Mutations must never execute a stale dynamically resolved component, even
# inside the short read-only freshness memo window.
http_component_wasm="$cache_dir/http-component-child.uce.wasm"
(
exec 8>"$http_component_wasm.lock"
flock 8
: >"$component_lock_ready_file"
sleep 2
) &
component_lock_pid=$!
deadline=$((SECONDS + 5))
while [[ ! -e "$component_lock_ready_file" && $SECONDS -lt $deadline ]]; do sleep 0.05; done
if [[ ! -e "$component_lock_ready_file" ]]; then
echo "HTTP component mutation lock was not acquired before deadline" >&2
exit 1
fi
sed -i 's/http-component-marker-b/http-component-marker-c/' "$source_dir/http-component-child.uce"
rm -f "$component_mutation_file"
component_post_status=$(curl -sS -o "$component_post_body" -D "$component_post_headers" -w '%{http_code}' -X POST -H "Host: $http_host" "http://127.0.0.1/$test_name/http-component-parent.uce")
if [[ "$component_post_status" != "503" ]]; then
echo "stale HTTP component mutation did not fail closed: status=$component_post_status body=$(cat "$component_post_body")" >&2
exit 1
fi
if ! grep -qi '^Retry-After: 1' "$component_post_headers"; then
echo "stale HTTP component mutation omitted Retry-After" >&2
exit 1
fi
if [[ -e "$component_mutation_file" ]]; then
echo "stale HTTP component mutation executed application code" >&2
exit 1
fi
wait "$component_lock_pid"
deadline=$((SECONDS + 15))
while [[ "$(http_component_marker)" != *"http-component-marker-c"* && $SECONDS -lt $deadline ]]; do sleep 0.2; done
if [[ "$(http_component_marker)" != *"http-component-marker-c"* ]]; then
echo "HTTP component did not recover after the guarded mutation rebuild" >&2
exit 1
fi
# A failed parent build must remember the exact dependency graph that failed.
# If a dependency is restored byte-for-byte to the last working source, the
# old successful metadata must not make that transient failure permanent.
printf '%s\n' \
'#ifndef UCE_DEPENDENCY_CACHE_CHILD' \
'#define UCE_DEPENDENCY_CACHE_CHILD' \
'String dependency_cache_marker() { return(deliberate_dependency_compile_failure); }' \
'#endif' >"$source_dir/child.uce"
if restored_failure=$(scripts/uce-cli --get "/$test_name/parent.uce" __uce_expected_compile_failure=1 2>&1); then
echo "invalid dependency unexpectedly compiled: $restored_failure" >&2
exit 1
fi
if [[ "$restored_failure" != *"deliberate_dependency_compile_failure"* ]]; then
echo "invalid dependency did not report the expected compiler failure: $restored_failure" >&2
exit 1
fi
printf '%s\n' \
'#ifndef UCE_DEPENDENCY_CACHE_CHILD' \
'#define UCE_DEPENDENCY_CACHE_CHILD' \
'String dependency_cache_marker() { return("dependency-marker-a"); }' \
'#endif' >"$source_dir/child.uce"
assert_marker parent dependency-marker-a
# The load-graph signature is content-addressed. A byte-identical dependency
# touch must invalidate stat caches without recompiling its parent artifact.
parent_wasm="$cache_dir/parent.uce.wasm"
parent_wasm_identity=$(stat -c '%y:%s' "$parent_wasm")
sleep 1.1
touch "$source_dir/child.uce"
assert_marker parent dependency-marker-a
if [[ "$(stat -c '%y:%s' "$parent_wasm")" != "$parent_wasm_identity" ]]; then
echo "byte-identical dependency touch recompiled the parent artifact" >&2
exit 1
fi
printf '%s\n' 'CLI(Request& context) { print("readable-source-marker"); }' >"$source_dir/unreadable.uce"
chmod 000 "$source_dir/unreadable.uce"
if unreadable_output=$(scripts/uce-cli --get "/$test_name/unreadable.uce" __uce_expected_source_read_failure=1 2>&1); then
echo "unreadable source unexpectedly compiled: $unreadable_output" >&2
exit 1
fi
if [[ "$unreadable_output" != *"source file is not readable"* ]]; then
echo "unreadable source did not report its actual compile error: $unreadable_output" >&2
exit 1
fi
if [[ -e "$cache_dir/unreadable.uce.wasm" ]]; then
echo "unreadable source published a wasm artifact" >&2
exit 1
fi
chmod 644 "$source_dir/unreadable.uce"
assert_marker unreadable readable-source-marker
sed -i 's/dependency-marker-a/dependency-marker-b/' "$source_dir/child.uce"
started_at=$(date +%s%N)
http_during_rebuild=$(http_marker)
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
if [[ "$http_during_rebuild" != *"dependency-marker-a"* && "$http_during_rebuild" != *"dependency-marker-b"* ]]; then
echo "HTTP rebuild request returned neither complete artifact: $http_during_rebuild" >&2
exit 1
fi
if (( elapsed_ms >= 2000 )); then
echo "HTTP request spent ${elapsed_ms}ms rebuilding a stale artifact" >&2
exit 1
fi
deadline=$((SECONDS + 15))
while [[ "$http_during_rebuild" != *"dependency-marker-b"* && $SECONDS -lt $deadline ]]; do
sleep 0.2
http_during_rebuild=$(http_marker)
done
if [[ "$http_during_rebuild" != *"dependency-marker-b"* ]]; then
echo "requested stale HTTP unit did not receive a demand-priority rebuild" >&2
exit 1
fi
assert_marker parent dependency-marker-b
# A proactive rebuild owns these same per-unit locks. While it publishes fresh
# artifacts, requests must use the last complete artifacts instead of waiting
# across the transitive graph. Atomic publication keeps those artifacts safe.
child_wasm="$cache_dir/child.uce.wasm"
(
exec 8>"$parent_wasm.lock"
exec 9>"$child_wasm.lock"
flock 8
flock 9
: >"$lock_ready_file"
sleep 3
) &
rebuild_lock_pid=$!
deadline=$((SECONDS + 5))
while [[ ! -e "$lock_ready_file" && $SECONDS -lt $deadline ]]; do
if ! kill -0 "$rebuild_lock_pid" 2>/dev/null; then
echo "test rebuild lock process exited before acquiring locks" >&2
exit 1
fi
sleep 0.05
done
if [[ ! -e "$lock_ready_file" ]]; then
echo "test rebuild lock process did not acquire locks before deadline" >&2
exit 1
fi
sed -i 's/dependency-marker-b/dependency-marker-d/' "$source_dir/child.uce"
started_at=$(date +%s%N)
http_during_lock=$(http_marker)
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
if [[ "$http_during_lock" != *"dependency-marker-b"* ]]; then
echo "HTTP request did not serve the last complete artifact during rebuild: $http_during_lock" >&2
exit 1
fi
if (( elapsed_ms >= 2000 )); then
echo "HTTP request waited ${elapsed_ms}ms for an active transitive rebuild" >&2
exit 1
fi
rm -f "$mutation_file"
started_at=$(date +%s%N)
post_status=$(curl -sS -o "$post_body" -D "$post_headers" -w '%{http_code}' -X POST -H "Host: $http_host" "http://127.0.0.1/$test_name/parent.uce")
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
if [[ "$post_status" != "503" ]]; then
echo "stale POST executed an application artifact instead of returning 503: status=$post_status body=$(cat "$post_body")" >&2
exit 1
fi
if ! grep -qi '^Retry-After: 1' "$post_headers"; then
echo "stale POST did not return a Retry-After header" >&2
exit 1
fi
if [[ -e "$mutation_file" ]]; then
echo "stale POST executed the old mutation handler: $(cat "$mutation_file")" >&2
exit 1
fi
if (( elapsed_ms >= 2000 )); then
echo "stale POST waited ${elapsed_ms}ms instead of failing closed promptly" >&2
exit 1
fi
started_at=$(date +%s%N)
assert_marker parent dependency-marker-d
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
if (( elapsed_ms < 2000 )); then
echo "CLI request returned before the locked rebuild published the current artifact (${elapsed_ms}ms)" >&2
exit 1
fi
wait "$rebuild_lock_pid"
# CLI freshness applies to every nested component, not only the request entry
# unit. Hold the changed child's compile lock as a proactive rebuild would: the
# parent CLI request must wait and render the new child, never its stale wasm.
printf '%s\n' 'COMPONENT(Request& context) { print("nested-component-marker-a"); }' >"$source_dir/nested-child.uce"
printf '%s\n' \
'CLI(Request& context) { DValue props; print(component("nested-child", props, context)); }' \
'RENDER(Request& context) { DValue props; print(component("nested-child", props, context)); }' >"$source_dir/nested-parent.uce"
assert_marker nested-parent nested-component-marker-a
nested_child_wasm="$cache_dir/nested-child.uce.wasm"
# A proactive source rebuild must publish the native serialized module before
# any request worker has to load the changed unit.
nested_parent_cwasm="$cache_dir/nested-parent.uce.cwasm"
rm -f "$nested_parent_cwasm"
printf '%s\n' \
'CLI(Request& context) { DValue props; print(component("nested-child", props, context)); /* proactive-serialization-v2 */ }' \
'RENDER(Request& context) { DValue props; print(component("nested-child", props, context)); /* proactive-serialization-v2 */ }' >"$source_dir/nested-parent.uce"
curl -fsS -H "Host: $http_host" "http://127.0.0.1/$test_name/nested-parent.uce" >/dev/null
deadline=$((SECONDS + 10))
while [[ ! -s "$nested_parent_cwasm" && $SECONDS -lt $deadline ]]; do sleep 0.1; done
if [[ ! -s "$nested_parent_cwasm" ]]; then
echo "proactive compiler did not serialize rebuilt module" >&2
exit 1
fi
(
exec 8>"$nested_child_wasm.lock"
flock 8
: >"$nested_lock_ready_file"
sleep 3
) &
nested_lock_pid=$!
deadline=$((SECONDS + 5))
while [[ ! -e "$nested_lock_ready_file" && $SECONDS -lt $deadline ]]; do
if ! kill -0 "$nested_lock_pid" 2>/dev/null; then
echo "nested component lock exited before acquisition" >&2
exit 1
fi
sleep 0.05
done
if [[ ! -e "$nested_lock_ready_file" ]]; then
echo "nested component lock was not acquired before deadline" >&2
exit 1
fi
sed -i 's/nested-component-marker-a/nested-component-marker-b/' "$source_dir/nested-child.uce"
started_at=$(date +%s%N)
nested_output=$(scripts/uce-cli "/$test_name/nested-parent.uce")
elapsed_ms=$(( ($(date +%s%N) - started_at) / 1000000 ))
wait "$nested_lock_pid"
if [[ "$nested_output" != *"nested-component-marker-b"* ]]; then
echo "CLI nested component returned stale wasm: $nested_output" >&2
exit 1
fi
if (( elapsed_ms < 2000 )); then
echo "CLI nested component did not wait for the active rebuild (${elapsed_ms}ms)" >&2
exit 1
fi
# Warm every configured worker, then replace the artifact while retaining its
# whole-second mtime. The worker cache must notice the nanosecond/ctime change.
for _ in {1..16}; do assert_marker parent dependency-marker-d; done
sed 's/dependency-marker-d/dependency-marker-c/' "$source_dir/child.uce" >"$source_dir/alternate-child.uce"
sed 's/child.uce/alternate-child.uce/' "$source_dir/parent.uce" >"$source_dir/alternate.uce"
assert_marker alternate dependency-marker-c
alternate_wasm="$cache_dir/alternate.uce.wasm"
parent_mtime=$(stat -c %Y "$parent_wasm")
cp "$alternate_wasm" "$parent_wasm"
touch -d "@$parent_mtime" "$parent_wasm"
rm -f "$cache_dir/parent.uce.cwasm"
for _ in {1..16}; do assert_marker parent dependency-marker-c; done
race_unit="$source_dir/source-race.uce"
race_cpp=""
{
printf '%s\n' 'String source_race_marker() { return("source-race-marker-a"); }'
for i in $(seq 1 3500); do
printf 'int source_race_padding_%s() { return(%s); }\n' "$i" "$i"
done
printf '%s\n' 'CLI(Request& context) { print(source_race_marker()); }'
} >"$race_unit"
rm -f "$cache_dir/source-race.uce.cpp" "$cache_dir/source-race.uce.wasm" "$cache_dir/source-race.uce.cwasm" "$cache_dir/source-race.uce.meta.txt"
(
set +e
scripts/uce-cli "/$test_name/source-race.uce" >"$race_output" 2>"$race_error"
echo "$?" >"$race_status"
) &
race_pid=$!
deadline=$((SECONDS + 10))
while [[ -z "$race_cpp" && $SECONDS -lt $deadline ]]; do
race_cpp=$(find "$cache_dir" -maxdepth 1 -name 'source-race.uce.invocation-*.cpp' -print -quit 2>/dev/null || true)
if ! kill -0 "$race_pid" 2>/dev/null; then
echo "source-race compile finished before staging generated C++" >&2
cat "$race_output" "$race_error" >&2 || true
exit 1
fi
sleep 0.02
done
if [[ -z "$race_cpp" || ! -e "$race_cpp" ]]; then
echo "source-race compile did not stage generated C++ before deadline" >&2
exit 1
fi
if ! grep -q 'source-race-marker-a' "$race_cpp"; then
echo "source-race generated C++ did not contain the initial marker" >&2
exit 1
fi
if ! kill -0 "$race_pid" 2>/dev/null; then
echo "source-race compile finished before source mutation hook" >&2
cat "$race_output" "$race_error" >&2 || true
exit 1
fi
sed -i 's/source-race-marker-a/source-race-marker-b/' "$race_unit"
wait "$race_pid"
if [[ "$(cat "$race_status" 2>/dev/null || echo 1)" != "0" ]]; then
echo "source-race CLI request failed" >&2
cat "$race_output" "$race_error" >&2 || true
exit 1
fi
if [[ "$(cat "$race_output")" != *"source-race-marker-b"* ]]; then
echo "source changed during compile but UCE served stale wasm: $(cat "$race_output")" >&2
exit 1
fi
worker_count=$(awk -F= '/^[[:space:]]*WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg 2>/dev/null || true)
worker_count="${worker_count:-4}"
cli_worker_count=$(awk -F= '/^[[:space:]]*CLI_WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); value=$2} END{print value}' /etc/uce/settings.cfg 2>/dev/null || true)
cli_worker_count="${cli_worker_count:-0}"
cli_worker_max_requests=$(awk -F= '/^[[:space:]]*CLI_WORKER_MAX_REQUESTS[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); value=$2} END{print value}' /etc/uce/settings.cfg 2>/dev/null || true)
cli_worker_max_requests="${cli_worker_max_requests:-8}"
worker_pids=""
for _ in {1..48}; do
output=$(scripts/uce-cli "/$test_name/parent.uce")
worker_pids+="${output##*:}"$'\n'
done
unique_workers=$(printf '%s' "$worker_pids" | sed '/^$/d' | sort -u | wc -l)
if (( cli_worker_count > 0 && cli_worker_max_requests > 0 )); then
if (( unique_workers <= cli_worker_count )); then
echo "dedicated CLI pool did not recycle during 48 requests: $unique_workers PIDs for $cli_worker_count workers" >&2
exit 1
fi
elif (( unique_workers > worker_count )); then
echo "shared worker pool recycled during 48 requests: $unique_workers PIDs for $worker_count workers" >&2
exit 1
fi
echo "dependency invalidation passed"
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" != "--inside" ]]; then
exec timeout --signal=TERM --kill-after=5s 240s unshare --mount --net --fork --kill-child=TERM "$0" --inside
fi
ip link set lo up
root="/tmp/uce-dynamic-compile-failures-$$"
site="$root/site"
work="$root/work"
settings="$root/settings.cfg"
log="$root/service.log"
port=18080
server_pid=""
cleanup() {
status=$?
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
wait "$server_pid" 2>/dev/null || true
fi
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
while umount /etc/uce/settings.cfg 2>/dev/null; do :; done
rm -rf "$root"
return "$status"
}
trap cleanup EXIT
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
cat >"$site/driver.uce" <<'UCE'
RENDER(Request& context) {
print("driver:");
if(context.get["mode"] == "render") unit_render("broken-render", context);
else if(context.get["mode"] == "call") unit_call("broken-call", "COMPONENT");
else print(component("child", context));
}
UCE
printf '%s\n' 'COMPONENT(Request& context) { print("good-v1"); }' >"$site/child.uce"
printf '%s\n' 'RENDER(Request& context) { print("render-ok"); }' >"$site/broken-render.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("call-ok"); }' >"$site/broken-call.uce"
write_settings() {
local show_errors="$1" serve_good="$2" proactive="$3"
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|PROACTIVE_COMPILE_ENABLED|PROACTIVE_COMPILE_CHECK_INTERVAL|PROACTIVE_COMPILE_JOBS|WORKER_COUNT|SHOW_DYNAMIC_COMPILE_ERRORS|SERVE_LAST_KNOWN_GOOD)[[:space:]]*=/d' \
/etc/uce/settings.cfg >"$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
FCGI_PORT=
CLI_SOCKET_PATH=$root/run/cli.sock
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=$port
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
PROACTIVE_COMPILE_ENABLED=$proactive
PROACTIVE_COMPILE_CHECK_INTERVAL=1
PROACTIVE_COMPILE_JOBS=1
WORKER_COUNT=1
SHOW_DYNAMIC_COMPILE_ERRORS=$show_errors
SERVE_LAST_KNOWN_GOOD=$serve_good
CFG
mount --bind "$settings" /etc/uce/settings.cfg
}
start_server() {
: >"$log"
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
for _ in $(seq 1 500); do
curl -sS --max-time 1 "http://127.0.0.1:$port/driver.uce" >/dev/null 2>&1 && return
kill -0 "$server_pid" 2>/dev/null || break
sleep 0.02
done
cat "$log" >&2
return 1
}
stop_server() {
kill -TERM "$server_pid" 2>/dev/null || true
wait "$server_pid" 2>/dev/null || true
server_pid=""
}
request() {
local method="${1:-GET}" query="${2:-}"
if [[ "$method" == POST ]]; then
curl -sS --max-time 30 -w $'\n%{http_code}' -X POST --data '' "http://127.0.0.1:$port/driver.uce?$query"
else
curl -sS --max-time 30 -w $'\n%{http_code}' "http://127.0.0.1:$port/driver.uce?$query"
fi
}
body_of() { printf '%s' "${1%$'\n'*}"; }
status_of() { printf '%s' "${1##*$'\n'}"; }
write_settings 1 0 0
start_server
printf '%s\n' 'COMPONENT(Request& context) { UCE_DYNAMIC_COMPILE_MARKER }' >"$site/child.uce"
response=$(request GET)
body=$(body_of "$response")
[[ "$(status_of "$response")" == 500 ]]
[[ "$body" == *UCE_DYNAMIC_COMPILE_MARKER* ]]
[[ "$body" != *"component not found"* ]]
printf '%s\n' 'RENDER(Request& context) { UCE_RENDER_COMPILE_MARKER }' >"$site/broken-render.uce"
response=$(request GET mode=render)
[[ "$(status_of "$response")" == 500 && "$(body_of "$response")" == *UCE_RENDER_COMPILE_MARKER* ]]
printf '%s\n' 'COMPONENT(Request& context) { UCE_CALL_COMPILE_MARKER }' >"$site/broken-call.uce"
response=$(request GET mode=call)
[[ "$(body_of "$response")" == *UCE_CALL_COMPILE_MARKER* ]]
[[ "$(body_of "$response")" != *"function 'COMPONENT' not found"* ]]
printf '%s\n' 'COMPONENT(Request& context) { print("recovered"); }' >"$site/child.uce"
response=$(request GET)
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:recovered" ]]
stop_server
write_settings 0 0 0
start_server
printf '%s\n' 'COMPONENT(Request& context) { UCE_HIDDEN_COMPILE_MARKER }' >"$site/child.uce"
response=$(request GET)
body=$(body_of "$response")
[[ "$(status_of "$response")" == 500 ]]
[[ "$body" == *"component not found: child"* ]]
[[ "$body" != *UCE_HIDDEN_COMPILE_MARKER* ]]
stop_server
printf '%s\n' 'COMPONENT(Request& context) { print("good-v1"); }' >"$site/child.uce"
write_settings 1 1 1
start_server
for _ in $(seq 1 100); do
response=$(request GET)
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]] && break
sleep 0.1
done
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]]
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
for artifact in "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map"; do [[ -s "$artifact" ]]; done
before_hashes=$(sha256sum "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map")
printf '%s\n' 'COMPONENT(Request& context) { UCE_STALE_COMPILE_MARKER }' >"$site/child.uce"
for _ in $(seq 1 30); do
response=$(request GET)
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]]
sleep 0.1
done
[[ -s "$cache/child.uce.compile.txt" ]]
[[ "$(sha256sum "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map")" == "$before_hashes" ]]
stop_server
start_server
response=$(request GET)
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]] || { echo "last-known-good artifact did not survive worker restart: $response" >&2; exit 1; }
printf '%s\n' 'COMPONENT(Request& context) { print("permission-repair"); }' >"$site/child.uce"
chmod 000 "$site/child.uce"
for _ in $(seq 1 100); do
grep -q "source file is not readable" "$cache/child.uce.compile.txt" 2>/dev/null && break
sleep 0.05
done
[[ "$(sha256sum "$cache/child.uce.wasm" "$cache/child.uce.cwasm" "$cache/child.uce.wasm.source-map")" == "$before_hashes" ]]
response=$(request GET)
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v1" ]] || { echo "unreadable source removed last-known-good service: $response" >&2; exit 1; }
chmod 644 "$site/child.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("good-v2"); }' >"$site/child.uce"
for _ in $(seq 1 200); do
response=$(request GET)
if [[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v2" ]]; then break; fi
sleep 0.1
done
[[ "$(status_of "$response")" == 200 && "$(body_of "$response")" == "driver:good-v2" ]] || { echo "repaired unit was not published: $response" >&2; exit 1; }
echo "dynamic compile failure handling passed"
+175
View File
@@ -0,0 +1,175 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="entry-freshness-ttl-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
worker_count="${UCE_TEST_WORKER_COUNT:-4}"
bin_directory="${BIN_DIRECTORY:-/tmp/uce/work}"
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
if [[ -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
configured_worker_count=$(awk -F= '/^[[:space:]]*WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
[[ -n "${UCE_TEST_SITE_DIRECTORY:-}" || -z "$configured_site_directory" ]] || site_directory="$configured_site_directory"
[[ -n "${UCE_TEST_WORKER_COUNT:-}" || -z "$configured_worker_count" ]] || worker_count="$configured_worker_count"
[[ -n "${BIN_DIRECTORY:-}" || -z "$configured_bin_directory" ]] || bin_directory="$configured_bin_directory"
fi
source_dir="$site_directory/$test_name"
cache_dir=""
mutation_file="/tmp/$test_name-mutation"
post_body="/tmp/$test_name-post-body"
post_headers="/tmp/$test_name-post-headers"
generation_file=""
cleanup() {
if [[ -n "$generation_file" ]]; then
exec 7>"$generation_file.lock"
flock 7
printf 'test-cleanup:%s\n' "$(date +%s%N)" >"$generation_file"
flock -u 7
exec 7>&-
fi
rm -rf "$source_dir"
[[ -z "$cache_dir" ]] || rm -rf "$cache_dir"
rm -f "$mutation_file" "$post_body" "$post_headers"
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
generation_file="$(scripts/unit_cache_directory "$bin_directory")/source-generation.txt"
printf '%s\n' 'String entry_freshness_marker() { return("marker-a"); }' >"$source_dir/child.uce"
printf '%s\n' \
'#load "child.uce"' \
"void entry_freshness_output(Request& context) { DValue perf = request_perf(); if(context.params[\"REQUEST_METHOD\"] == \"POST\") file_put_contents(\"$mutation_file\", \"executed\"); print(entry_freshness_marker(), \":\", perf[\"worker_pid\"].to_string(), \":\", perf[\"ready_freshness_cache_hit_count\"].to_string(), \":\", perf[\"ready_freshness_full_check_us\"].to_string()); }" \
'RENDER(Request& context) { entry_freshness_output(context); }' \
'CLI(Request& context) { entry_freshness_output(context); }' >"$source_dir/parent.uce"
http_probe() {
curl -fsS --max-time 5 -H "Host: $http_host" "http://127.0.0.1/$test_name/parent.uce"
}
declare -A seen
collect_all_workers() {
local require_hit="$1"
local expected_marker="$2"
local deadline=$((SECONDS + 35))
seen=()
while (( SECONDS < deadline && ${#seen[@]} < worker_count )); do
local output marker pid hit full
output=$(http_probe)
IFS=: read -r marker pid hit full <<<"$output"
pid=${pid%%.*}; hit=${hit%%.*}; full=${full%%.*}
if [[ "$marker" != "$expected_marker" || -z "$pid" ]]; then
echo "entry freshness probe returned unexpected output: $output" >&2
exit 1
fi
if [[ "$require_hit" == "1" && ( "$hit" == "0" || "$full" != "0" ) ]]; then
continue
fi
seen["$pid"]=1
done
if (( ${#seen[@]} != worker_count )); then
echo "entry freshness probe reached ${#seen[@]}/$worker_count workers (hit=$require_hit)" >&2
exit 1
fi
}
collect_all_workers 0 marker-a
collect_all_workers 1 marker-a
parent_wasm="$cache_dir/parent.uce.wasm"
exec 8>"$parent_wasm.lock"
flock 8
sed -i 's/marker-a/marker-b/' "$source_dir/child.uce"
rm -f "$mutation_file"
post_status=$(curl -sS --max-time 5 -o "$post_body" -D "$post_headers" -w '%{http_code}' -X POST -H "Host: $http_host" "http://127.0.0.1/$test_name/parent.uce")
if [[ "$post_status" != "503" || -e "$mutation_file" ]] || ! grep -qi '^Retry-After: 1' "$post_headers"; then
echo "stale mutation did not fail closed: status=$post_status body=$(cat "$post_body")" >&2
exit 1
fi
sleep 10.2
expired_output=$(http_probe)
IFS=: read -r expired_marker expired_pid expired_hit expired_full <<<"$expired_output"
expired_hit=${expired_hit%%.*}; expired_full=${expired_full%%.*}
if [[ "$expired_marker" != "marker-a" || "$expired_hit" != "0" || "$expired_full" == "0" ]]; then
echo "expired positive entry did not perform a full safe check: $expired_output" >&2
exit 1
fi
flock -u 8
exec 8>&-
deadline=$((SECONDS + 25))
output=""
while (( SECONDS < deadline )); do
output=$(http_probe)
[[ "$output" == marker-b:* ]] && break
sleep 0.2
done
if [[ "$output" != marker-b:* ]]; then
echo "entry did not converge after dependency rebuild: $output" >&2
exit 1
fi
collect_all_workers 1 marker-b
exec 9>"$generation_file.lock"
flock 9
printf 'test:%s\n' "$(date +%s%N)" >"$generation_file"
flock -u 9
exec 9>&-
generation_output=$(http_probe)
IFS=: read -r generation_marker generation_pid generation_hit generation_full <<<"$generation_output"
generation_hit=${generation_hit%%.*}; generation_full=${generation_full%%.*}
if [[ "$generation_marker" != "marker-b" || "$generation_hit" != "0" || "$generation_full" == "0" ]]; then
echo "source generation change did not invalidate immediately: $generation_output" >&2
exit 1
fi
collect_all_workers 1 marker-b
touch "$parent_wasm"
identity_output=$(http_probe)
IFS=: read -r identity_marker identity_pid identity_hit identity_full <<<"$identity_output"
identity_hit=${identity_hit%%.*}; identity_full=${identity_full%%.*}
if [[ "$identity_marker" != "marker-b" || "$identity_hit" != "0" || "$identity_full" == "0" ]]; then
echo "wasm identity change did not invalidate immediately: $identity_output" >&2
exit 1
fi
collect_all_workers 1 marker-b
touch "$cache_dir/parent.uce.meta.txt"
metadata_output=$(http_probe)
IFS=: read -r metadata_marker metadata_pid metadata_hit metadata_full <<<"$metadata_output"
metadata_hit=${metadata_hit%%.*}; metadata_full=${metadata_full%%.*}
if [[ "$metadata_marker" != "marker-b" || "$metadata_hit" != "0" || "$metadata_full" == "0" ]]; then
echo "metadata identity change did not invalidate immediately: $metadata_output" >&2
exit 1
fi
collect_all_workers 1 marker-b
exec 9>"$generation_file.lock"
flock 9
rm -f "$generation_file"
flock -u 9
missing_generation_output=$(http_probe)
IFS=: read -r missing_generation_marker missing_generation_pid missing_generation_hit missing_generation_full <<<"$missing_generation_output"
missing_generation_hit=${missing_generation_hit%%.*}; missing_generation_full=${missing_generation_full%%.*}
if [[ "$missing_generation_marker" != "marker-b" || "$missing_generation_hit" != "0" || "$missing_generation_full" == "0" ]]; then
echo "missing generation token did not force a full check: $missing_generation_output" >&2
exit 1
fi
flock 9
printf 'test:%s\n' "$(date +%s%N)" >"$generation_file"
flock -u 9
exec 9>&-
cli_output=$(timeout --signal=TERM --kill-after=3s 20s scripts/uce-cli "/$test_name/parent.uce")
IFS=: read -r cli_marker cli_pid cli_hit cli_full <<<"$cli_output"
cli_hit=${cli_hit%%.*}; cli_full=${cli_full%%.*}
if [[ "$cli_marker" != "marker-b" || "$cli_hit" != "0" || "$cli_full" == "0" ]]; then
echo "CLI unexpectedly reused the HTTP freshness cache: $cli_output" >&2
exit 1
fi
echo "entry freshness TTL passed"
+37
View File
@@ -0,0 +1,37 @@
#include "src/lib/types.cpp"
#include "src/lib/dvalue.cpp"
#include "src/lib/functionlib.cpp"
#include "src/wasm/hardened_http_internal.h"
#include <iostream>
struct Fake { std::vector<String> answers, argv, env; String input; HardenedHttpExecResult result; };
static DValue request() { DValue r; r["method"]="POST"; r["url"]="https://api.example.test/token"; r["headers"]["Accept"]="application/json"; r["body"]="client_secret=SECRET"; DValue& s=r["security"]; s["https_only"].set_bool(true); s["public_dns_only"].set_bool(true); s["pin_dns"].set_bool(true); s["isolated_curl"].set_bool(true); s["no_redirects"].set_bool(true); return r; }
static DValue run(Fake& f,DValue r) { HardenedHttpHooks h; h.resolve=[&](String){return f.answers;}; h.execute=[&](std::vector<String> a,String i,std::vector<String> e,u64,size_t){f.argv=a;f.input=i;f.env=e;return f.result;}; return hardened_http_request_internal(r,9000,h); }
static String headers(int status=200) { return "HTTP/1.1 "+std::to_string(status)+" OK\r\nContent-Type: application/json\r\n\r\n"; }
static bool exists(String path) { return access(path.c_str(),F_OK)==0; }
static void wait_for(String path) { for(int n=0;n<500&&!exists(path);n++) usleep(1000); }
static bool process_live(pid_t pid) { char state=0; String path="/proc/"+std::to_string((long long)pid)+"/stat"; int fd=open(path.c_str(),O_RDONLY); if(fd<0) return false; char text[256]{}; ssize_t n=read(fd,text,sizeof(text)-1); close(fd); if(n<=0) return false; char* close_paren=strrchr(text,')'); return close_paren&&close_paren[2]!='Z'; }
int main(int argc,char** argv) {
if(argc==3&&String(argv[1])=="--child-timeout") { pid_t child=fork(); if(child==0) { usleep(100000); int fd=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"leaked",6); close(fd); } _exit(0); } for(;;) pause(); }
if(argc==2&&String(argv[1])=="--child-output") { String x(4096,'x'); for(int n=0;n<64;n++) write(1,x.data(),x.size()); return 0; }
if(argc==3&&String(argv[1])=="--child-fds") { int inherited=atoi(argv[2]); bool closed=fcntl(inherited,F_GETFD)==-1&&errno==EBADF; bool header=fcntl(3,F_GETFD)!=-1; write(1,closed&&header?"closed":"open",closed&&header?6:4); return closed&&header?0:1; }
if(argc==5&&String(argv[1])=="--child-job") { int ready=open(argv[3],O_WRONLY|O_CREAT|O_TRUNC,0600); if(ready>=0) { write(ready,"ready",5); close(ready); } pid_t child=fork(); if(child==0) { int pidfile=open(argv[4],O_WRONLY|O_CREAT|O_TRUNC,0600); if(pidfile>=0) { String pid=std::to_string((long long)getpid()); write(pidfile,pid.data(),pid.size()); close(pidfile); } usleep(150000); int marker=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(marker>=0) { write(marker,"leaked",6); close(marker); } _exit(0); } for(;;) pause(); }
if(argc==3&&String(argv[1])=="--child-async-timeout") { setsid(); HardenedHttpExecResult result=hardened_http_exec_argv_capture({"/bin/sleep","1"},"",20,4096,false,false); if(result.timed_out&&result.exit_code==137) { int fd=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"typed",5); close(fd); } } return 0; }
if(argc==3&&String(argv[1])=="--child-async-output") { setsid(); HardenedHttpExecResult result=hardened_http_exec_argv_capture({"/proc/self/exe","--child-output"},"",1000,1024,false,false); if(result.output_limited&&result.exit_code==137) { int fd=open(argv[2],O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"typed",5); close(fd); } } return 0; }
bool ok=true; auto need=[&](bool x,const char* n){if(!x){std::cerr<<n<<"\n";ok=false;}};
Fake f; f.answers={"8.8.8.8"}; f.result.exit_code=0; f.result.stderr_text="SECRET"; f.result.headers_text=headers(); f.result.body_text="one\r\n\r\ntwo"; DValue out=run(f,request());
need(out["error"].to_string()==""&&out["error"].to_string().find("SECRET")==String::npos&&out["body"].to_string()=="one\r\n\r\ntwo","body framing / non-secret errors"); need(f.argv.size()>2&&f.argv[0]=="/usr/bin/curl"&&f.argv[1]=="--disable","absolute curl disable"); need(f.env.size()==1&&f.env[0]=="PATH=/usr/bin:/bin","clean env"); need(std::find(f.argv.begin(),f.argv.end(),"SECRET")==f.argv.end()&&f.input.find("SECRET")!=String::npos,"secret stdin only"); need(std::find(f.argv.begin(),f.argv.end(),"--resolve")!=f.argv.end(),"pinned dns");
Fake interim; interim.answers={"8.8.8.8"}; interim.result.exit_code=0; interim.result.headers_text="HTTP/1.1 100 Continue\r\n\r\n"+headers(201); interim.result.body_text="{}"; need(run(interim,request())["status"].to_u64()==201,"interim headers");
for(String bad:{"127.0.0.1","10.0.0.1","169.254.1.1","192.168.1.1","::1","2001:db8::1"}) { Fake x; x.answers={"8.8.8.8",bad}; need(run(x,request())["error"].to_string()=="unsafe_dns","dns matrix"); }
for(auto pair:std::vector<std::pair<int,String>>{{302,headers(302)},{500,headers(500)},{200,"bad\r\n\r\n"}}) { Fake x; x.answers={"8.8.8.8"};x.result.exit_code=0;x.result.headers_text=pair.second;need(run(x,request())["error"].to_string()==(pair.first==302?"redirect_not_allowed":pair.first==500?"http_status":"malformed_output"),"status errors"); }
DValue bad=request(); bad["method"]="TRACE"; Fake x; need(run(x,bad)["error"].to_string()=="invalid_request"&&x.argv.empty(),"method validation"); bad=request();bad["headers"]["Host"]="evil";need(run(x,bad)["error"].to_string()=="invalid_request","header validation"); bad=request();bad["headers"]["Accept"]="ok\r\nInjected: x";need(run(x,bad)["error"].to_string()=="invalid_request","header CRLF validation");bad=request();bad["url"]="https://127.0.0.1/";need(run(x,bad)["error"].to_string()=="invalid_request","url validation");bad=request();bad["follow_redirects"].set_bool(true);need(run(x,bad)["error"].to_string()=="invalid_request","redirect composition");
for(String key:{"https_only","public_dns_only","pin_dns","isolated_curl","no_redirects"}) { bad=request(); bad["security"].remove(key); need(run(x,bad)["error"].to_string()=="invalid_request","partial hardening fails closed"); bad=request(); bad["security"][key].set_bool(false); need(run(x,bad)["error"].to_string()=="invalid_request","false hardening fails closed"); bad=request(); bad["security"][key]="true"; need(run(x,bad)["error"].to_string()=="invalid_request","non-boolean hardening fails closed"); }
DValue legacy=request(); legacy["security"].clear(); legacy["security"]["unrelated"]="value"; need(!hardened_http_security_requested(legacy.key("security")),"unknown security object remains legacy");
Fake large;large.answers={"8.8.8.8"};large.result.exit_code=0;large.result.headers_text=headers();large.result.body_text=String(65537,'x');need(run(large,request())["error"].to_string()=="response_too_large","body cap");
String base="/tmp/hardened-http-"+std::to_string((long long)getpid()), marker=base+"-marker"; unlink(marker.c_str()); HardenedHttpExecResult timeout=hardened_http_exec_argv_capture({"/proc/self/exe","--child-timeout",marker},"",30,4096,false); usleep(150000); need(timeout.timed_out&&timeout.exit_code==137&&!exists(marker),"timeout kills descendants"); unlink(marker.c_str());
HardenedHttpExecResult overflow=hardened_http_exec_argv_capture({"/proc/self/exe","--child-output"},"",1000,1024,false);need(overflow.output_limited&&overflow.body_text.size()<=1024,"output cap/reap");
int inherited=open("/dev/null",O_RDONLY), high_inherited=fcntl(inherited,F_DUPFD,10); close(inherited); HardenedHttpExecResult fds=hardened_http_exec_argv_capture({"/proc/self/exe","--child-fds",std::to_string(high_inherited)},"",1000,4096,false); close(high_inherited); need(fds.exit_code==0&&fds.body_text=="closed","only stdio and header fd inherited");
String ready=base+"-ready", descendant=base+"-descendant", sentinel=base+"-sentinel"; unlink(marker.c_str()); unlink(ready.c_str()); unlink(descendant.c_str()); unlink(sentinel.c_str()); pid_t worker=fork(); if(worker==0) { setsid(); hardened_http_exec_argv_capture({"/proc/self/exe","--child-job",marker,ready,descendant},"",5000,4096,false,false); _exit(0); } wait_for(ready); wait_for(descendant); pid_t unrelated=fork(); if(unrelated==0) { setsid(); usleep(150000); int fd=open(sentinel.c_str(),O_WRONLY|O_CREAT|O_TRUNC,0600); if(fd>=0) { write(fd,"alive",5); close(fd); } _exit(0); } pid_t descendant_pid=exists(descendant)?(pid_t)strtol([](String path){ int fd=open(path.c_str(),O_RDONLY); char text[32]{}; ssize_t n=fd<0?-1:read(fd,text,sizeof(text)-1); if(fd>=0) close(fd); return String(text,n>0?(size_t)n:0); }(descendant).c_str(),0,10):0; need(exists(ready)&&descendant_pid>0,"async worker and descendant started"); kill(-worker,SIGKILL); waitpid(worker,0,0); usleep(250000); waitpid(unrelated,0,0); need(!process_live(descendant_pid)&&!exists(marker)&&exists(sentinel),"async cancellation leaves no live descendant and kills only its worker group"); unlink(marker.c_str()); unlink(ready.c_str()); unlink(descendant.c_str()); unlink(sentinel.c_str());
String async_timeout=base+"-async-timeout"; unlink(async_timeout.c_str()); pid_t timeout_worker=fork(); if(timeout_worker==0) { execl("/proc/self/exe","test_hardened_http_native","--child-async-timeout",async_timeout.c_str(),(char*)0); _exit(127); } int timeout_status=0; waitpid(timeout_worker,&timeout_status,0); need(WIFEXITED(timeout_status)&&WEXITSTATUS(timeout_status)==0&&exists(async_timeout),"async timeout records a typed terminal result before worker exit"); unlink(async_timeout.c_str());
String async_output=base+"-async-output"; unlink(async_output.c_str()); pid_t output_worker=fork(); if(output_worker==0) { execl("/proc/self/exe","test_hardened_http_native","--child-async-output",async_output.c_str(),(char*)0); _exit(127); } int output_status=0; waitpid(output_worker,&output_status,0); need(WIFEXITED(output_status)&&WEXITSTATUS(output_status)==0&&exists(async_output),"async output cap records a typed terminal result before worker exit"); unlink(async_output.c_str());
return ok?0:1;
}
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
repo=$(cd "$(dirname "$0")/.." && pwd)
bin="${TMPDIR:-/tmp}/test_hardened_http_native.$$"
trap 'rm -f "$bin"' EXIT
compiler=$(command -v clang++ || command -v g++)
"$compiler" -std=c++20 -fpermissive -I"$repo" "$repo/scripts/test_hardened_http_native.cpp" -lpcre2-8 -o "$bin"
timeout --signal=TERM --kill-after=2s 15s "$bin"
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="log-timeliness-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' 'CLI(Request& context) { deliberate_log_timeliness_compile_failure }' >"$source_dir/probe.uce"
rm -rf "$cache_dir"
started_at=$(date '+%Y-%m-%d %H:%M:%S')
set +e
scripts/uce-cli --get "/$test_name/probe.uce" __uce_expected_compile_failure=1 >/dev/null 2>&1
set -e
for _ in $(seq 1 20); do
journal_output=$(journalctl -u uce --since "$started_at" --no-pager)
if [[ "$journal_output" == *"UCE expected compile error"* && "$journal_output" == *"$test_name/probe.uce"* ]]; then
echo "log timeliness passed"
exit 0
fi
sleep 0.1
done
echo "compile diagnostic was not journaled promptly: $test_name/probe.uce" >&2
exit 1
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="mysql-epoch-refresh-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
settings_file="${UCE_SETTINGS_FILE:-/etc/uce/settings.cfg}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r "$settings_file" ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r "$settings_file" ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
ticks=$(awk -F= '/^[[:space:]]*WASM_EPOCH_DEADLINE_TICKS[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file" 2>/dev/null || true)
period_ms=$(awk -F= '/^[[:space:]]*WASM_EPOCH_PERIOD_MS[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file" 2>/dev/null || true)
ticks="${ticks:-200}"
period_ms="${period_ms:-50}"
segment_seconds=$(awk -v ticks="$ticks" -v period="$period_ms" 'BEGIN { printf "%.6f", ticks * period * 0.00055 }')
cache_dir=""
test_user="uce_epoch_$$"
test_password=$(printf '%s' "$test_name-$(date +%s%N)" | sha256sum | cut -c1-32)
test_user_created=false
cleanup() {
if [[ "$test_user_created" == true ]]; then
mariadb -e "DROP USER IF EXISTS '$test_user'@'127.0.0.1'" >/dev/null 2>&1 || true
fi
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
mariadb -e "DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'"
test_user_created=true
printf '%s\n' \
'void mysql_epoch_burn(f64 seconds) {' \
' f64 deadline = time_precise() + seconds;' \
' u64 spins = 0; while(time_precise() < deadline) spins++;' \
'}' \
'CLI(Request& context) {' \
" MySQL* db = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\");" \
' if(db == 0 || mysql_error(db) != "") { print("mysql-connect-failed:", db == 0 ? "null" : mysql_error(db)); return; }' \
" mysql_epoch_burn($segment_seconds);" \
' mysql_query(db, "SELECT 1 AS value");' \
" mysql_epoch_burn($segment_seconds);" \
' mysql_query(db, "SELECT 2 AS value");' \
' if(mysql_error(db) == "") print("mysql-epoch-refresh-ok"); else print("mysql-query-failed:", mysql_error(db));' \
' mysql_disconnect(db);' \
'}' >"$source_dir/test.uce"
output=$(scripts/uce-cli "/$test_name/test.uce" 2>&1) || {
echo "MySQL hostcall did not refresh the guest epoch deadline: $output" >&2
exit 1
}
if [[ "$output" != "mysql-epoch-refresh-ok" ]]; then
echo "MySQL epoch refresh failed: $output" >&2
exit 1
fi
echo "MySQL epoch refresh passed with ${segment_seconds}s guest segments"
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="mysql-persistent-pool-test-$$"
settings_file="${UCE_SETTINGS_FILE:-/etc/uce/settings.cfg}"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r "$settings_file" ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file" 2>/dev/null || true)
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
test_user="uce_pool_$$"
test_database="uce_pool_$$"
test_database_other="uce_pool_other_$$"
test_password=$(printf '%s' "$test_name-$(date +%s%N)" | sha256sum | cut -c1-32)
cleanup() {
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP DATABASE IF EXISTS \`$test_database_other\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'" >/dev/null 2>&1 || true
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP DATABASE IF EXISTS \`$test_database_other\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE DATABASE \`$test_database\`; CREATE DATABASE \`$test_database_other\`; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'; GRANT ALL ON \`$test_database\`.* TO '$test_user'@'127.0.0.1'; GRANT ALL ON \`$test_database_other\`.* TO '$test_user'@'127.0.0.1'; CREATE TABLE \`$test_database\`.pool_identity (label VARCHAR(16) NOT NULL); INSERT INTO \`$test_database\`.pool_identity VALUES ('primary'); CREATE TABLE \`$test_database_other\`.pool_identity (label VARCHAR(16) NOT NULL); INSERT INTO \`$test_database_other\`.pool_identity VALUES ('other')"
printf '%s\n' \
'RENDER(Request& context)' \
'{' \
" MySQL* db = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\", \"$test_database\");" \
" MySQL* other = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\", \"$test_database_other\");" \
" MySQL* unselected = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\");" \
' if(db == 0 || other == 0 || unselected == 0 || !mysql_connected(db) || !mysql_connected(other) || !mysql_connected(unselected)) { context.set_status(500, "MySQL connect failed"); print("connect-failed"); return; }' \
' String primary_database, primary_label, other_database, other_label;' \
' mysql_query(db, "SELECT DATABASE() AS db, label FROM pool_identity").each([&](DValue row, String key) { primary_database = row["db"].to_string(); primary_label = row["label"].to_string(); });' \
' mysql_query(other, "SELECT DATABASE() AS db, label FROM pool_identity").each([&](DValue row, String key) { other_database = row["db"].to_string(); other_label = row["label"].to_string(); });' \
' String unselected_database;' \
' mysql_query(unselected, "SELECT DATABASE() AS db").each([&](DValue row, String key) { unselected_database = row["db"].to_string(); });' \
" bool database_clean = primary_database == \"$test_database\" && primary_label == \"primary\" && other_database == \"$test_database_other\" && other_label == \"other\" && unselected_database == \"\";" \
' DValue marker_rows = mysql_query(db, "SELECT @uce_pool_marker AS marker");' \
' String marker; marker_rows.each([&](DValue row, String key) { marker = row["marker"].to_string(); });' \
' bool marker_clean = marker == "";' \
' mysql_query(db, "SELECT id FROM uce_persistent_temp LIMIT 1");' \
' bool temp_clean = mysql_error(db) != "";' \
" mysql_query(db, \"SET @uce_pool_marker='dirty'\");" \
' mysql_query(db, "CREATE TEMPORARY TABLE uce_persistent_temp (id INT PRIMARY KEY)");' \
" mysql_query(db, \"USE \\\`$test_database_other\\\`\");" \
" mysql_query(other, \"USE \\\`$test_database\\\`\");" \
" mysql_query(unselected, \"USE \\\`$test_database\\\`\");" \
' DValue perf = request_perf();' \
' String source;' \
' perf["mysql_operations"].each([&](DValue operation, String key) { if(operation["op"].to_string() == "connect" && operation["source"].to_string() == "worker") source = "worker"; });' \
' print(perf["worker_pid"].to_string(), "|", source, "|", marker_clean ? "clean" : "dirty", "|", temp_clean ? "clean" : "dirty", "|", database_clean ? "clean" : "dirty");' \
' mysql_disconnect(unselected);' \
' mysql_disconnect(other);' \
' mysql_disconnect(db);' \
'}' >"$source_dir/test.uce"
printf '%s\n' \
'RENDER(Request& context)' \
'{' \
" MySQL* db = mysql_connect(\"127.0.0.1\", \"$test_user\", \"$test_password\", \"${test_database}_missing\");" \
' String error = mysql_error(db);' \
' print(!mysql_connected(db) && error != "" ? "database-selection-failed" : "database-selection-was-ignored");' \
' mysql_disconnect(db);' \
'}' >"$source_dir/failure.uce"
failure=$(curl -fsS --max-time 10 -H "Host: $http_host" "http://127.0.0.1/$test_name/failure.uce")
if [[ "$failure" != "database-selection-failed" ]]; then
echo "Unknown initial database did not surface a connection failure: $failure" >&2
exit 1
fi
reused=0
for _ in $(seq 1 160); do
output=$(curl -fsS --max-time 10 -H "Host: $http_host" "http://127.0.0.1/$test_name/test.uce")
if [[ "$output" == *"|worker|"* ]]; then
reused=$((reused + 1))
if [[ "$output" != *"|worker|clean|clean|clean" ]]; then
echo "Persistent MySQL reuse leaked cross-request state: $output" >&2
exit 1
fi
fi
done
if [[ "$reused" -eq 0 ]]; then
echo "Persistent MySQL pool did not produce a worker reuse in 160 requests" >&2
exit 1
fi
echo "Persistent MySQL pool reset passed across $reused reused requests"
+145
View File
@@ -0,0 +1,145 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" != "--inside" ]]; then
exec timeout --signal=TERM --kill-after=5s 150s unshare --mount --fork --kill-child=TERM "$0" --inside
fi
name="mysql-pool-idle-test-$$"
root="/tmp/$name"
site="$root/site"
work="$root/work"
settings="$root/settings.cfg"
log="$root/service.log"
socket="$root/run/cli.sock"
test_user="uce_pool_idle_$$"
test_database="uce_pool_idle_$$"
test_password=$(printf '%s' "$name-$(date +%s%N)" | sha256sum | cut -c1-32)
server_pid=""
cleanup() {
status=$?
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
deadline=$((SECONDS + 10))
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
if kill -0 "$server_pid" 2>/dev/null; then kill -KILL "$server_pid" 2>/dev/null || true; fi
wait "$server_pid" 2>/dev/null || true
fi
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'" >/dev/null 2>&1 || true
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
rm -rf "$root"
return "$status"
}
trap cleanup EXIT
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
cp /etc/uce/settings.cfg "$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
FCGI_PORT=
CLI_SOCKET_PATH=$socket
CLI_WORKER_COUNT=1
CLI_WORKER_MAX_REQUESTS=0
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
WORKER_COUNT=1
PROACTIVE_COMPILE_ENABLED=0
MYSQL_PERSISTENT_POOL_SIZE=8
MYSQL_PERSISTENT_POOL_IDLE_TIMEOUT_SECONDS=2
CFG
mount --bind "$settings" /etc/uce/settings.cfg
mariadb -e "DROP DATABASE IF EXISTS \`$test_database\`; DROP USER IF EXISTS '$test_user'@'127.0.0.1'; CREATE DATABASE \`$test_database\`; CREATE USER '$test_user'@'127.0.0.1' IDENTIFIED BY '$test_password'; GRANT ALL ON \`$test_database\`.* TO '$test_user'@'127.0.0.1'"
cat >"$site/database.uce" <<UCE
CLI(Request& context)
{
MySQL* db = mysql_connect("127.0.0.1", "$test_user", "$test_password", "$test_database");
if(!mysql_connected(db)) { print("connect-failed"); return; }
String connection_id;
mysql_query(db, "SELECT CONNECTION_ID() AS id").each([&](DValue row, String key) { connection_id = row["id"].to_string(); });
String source;
DValue perf = request_perf();
perf["mysql_operations"].each([&](DValue operation, String key) {
if(operation["op"].to_string() == "connect") source = operation["source"].to_string();
});
print(perf["worker_pid"].to_string(), "|", connection_id, "|", source);
mysql_disconnect(db);
}
UCE
printf '%s\n' 'CLI(Request& context) { print(request_perf()["worker_pid"].to_string(), "|plain"); }' >"$site/plain.uce"
timeout --signal=TERM --kill-after=5s 120s bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
deadline=$((SECONDS + 20))
while [[ ! -S "$socket" ]] && (( SECONDS < deadline )); do sleep 0.05; done
[[ -S "$socket" ]] || { echo "private UCE CLI socket was not ready" >&2; exit 1; }
request() {
timeout --signal=TERM --kill-after=1s 30s scripts/uce-cli --socket "$socket" "$1"
}
first=$(request /database.uce)
second=$(request /database.uce)
IFS='|' read -r first_pid first_id first_source <<<"$first"
IFS='|' read -r second_pid second_id second_source <<<"$second"
[[ "$first_source" == "new" && "$second_source" == "worker" ]]
[[ "$first_pid" == "$second_pid" && "$first_id" == "$second_id" ]]
sleep 3
plain=$(request /plain.uce)
[[ "$plain" == "$first_pid|plain" ]]
deadline=$((SECONDS + 5))
while (( SECONDS < deadline )); do
connections=$(mariadb --batch --skip-column-names -e "SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE USER='$test_user'")
[[ "$connections" == "0" ]] && break
sleep 0.05
done
[[ "$connections" == "0" ]] || { echo "expired pooled connection remained after a request boundary" >&2; exit 1; }
third=$(request /database.uce)
IFS='|' read -r third_pid third_id third_source <<<"$third"
[[ "$third_pid" == "$first_pid" && "$third_source" == "new" && "$third_id" != "$first_id" ]]
sleep 3
plain=$(request /plain.uce)
[[ "$plain" == "$first_pid|plain" ]]
deadline=$((SECONDS + 5))
while (( SECONDS < deadline )); do
connections=$(mariadb --batch --skip-column-names -e "SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE USER='$test_user'")
[[ "$connections" == "0" ]] && break
sleep 0.05
done
[[ "$connections" == "0" ]] || { echo "final pooled fixture connection remained after idle eviction" >&2; exit 1; }
retained=$(request /database.uce)
IFS='|' read -r retained_pid retained_id retained_source <<<"$retained"
[[ "$retained_pid" == "$first_pid" && "$retained_source" == "new" ]]
connections=$(mariadb --batch --skip-column-names -e "SELECT COUNT(*) FROM information_schema.PROCESSLIST WHERE USER='$test_user'")
[[ "$connections" == "1" ]] || { echo "shutdown fixture connection was not retained in the worker pool" >&2; exit 1; }
journal_cursor=$(journalctl -u mariadb.service -n 0 --show-cursor --no-pager | sed -n 's/^-- cursor: //p')
[[ -n "$journal_cursor" ]] || { echo "could not capture MariaDB journal cursor" >&2; exit 1; }
kill -TERM "$server_pid"
deadline=$((SECONDS + 10))
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
[[ ! -e "/proc/$server_pid" ]] || { echo "private UCE server did not stop cleanly" >&2; exit 1; }
wait "$server_pid" 2>/dev/null || true
server_pid=""
sleep 1
shutdown_log=$(journalctl -u mariadb.service --after-cursor "$journal_cursor" --no-pager)
if grep -F "$test_user" <<<"$shutdown_log" | grep -Fq "Aborted connection"; then
echo "clean UCE shutdown aborted a retained MySQL connection" >&2
exit 1
fi
echo "Persistent MySQL idle eviction preserved hot reuse, retired expired connections, and closed a retained connection on clean shutdown"
+75
View File
@@ -0,0 +1,75 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="nested-component-props-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
threshold_ms="${UCE_NESTED_COMPONENT_PROPS_MAX_MS:-1500}"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'CLI(Request& context) {' \
' context.props["sentinel"] = "caller";' \
' String payload; for(u64 i = 0; i < 4096; ++i) payload += "x";' \
' DValue props; for(u64 i = 0; i < 300; ++i) { DValue item; item = payload; props["items"].push(item); }' \
' String output = component("outer", props, context);' \
' if(context.props["sentinel"].to_string() != "caller") { print("caller props not restored"); return; }' \
' print(output);' \
'}' >"$source_dir/parent.uce"
printf '%s\n' \
'COMPONENT(Request& context) {' \
' if(context.props["items"]._map.size() != 300) { print("outer props missing"); return; }' \
' for(u64 i = 0; i < 300; ++i) {' \
' DValue props; props["index"] = std::to_string(i); component("leaf", props, context);' \
' if(context.props["items"]._map.size() != 300) { print("outer props not restored"); return; }' \
' }' \
' print("nested-component-props-ok");' \
'}' >"$source_dir/outer.uce"
printf '%s\n' \
'COMPONENT(Request& context) {' \
' if(context.props["index"].to_string() == "") print("leaf props missing");' \
'}' >"$source_dir/leaf.uce"
warm_output=$(scripts/uce-cli "/$test_name/parent.uce")
if [[ "$warm_output" != "nested-component-props-ok" ]]; then
echo "nested component props warmup failed: $warm_output" >&2
exit 1
fi
start_ns=$(date +%s%N)
output=$(scripts/uce-cli "/$test_name/parent.uce")
elapsed_ms=$(( ($(date +%s%N) - start_ns) / 1000000 ))
if [[ "$output" != "nested-component-props-ok" ]]; then
echo "nested component props failed: $output" >&2
exit 1
fi
if (( elapsed_ms > threshold_ms )); then
echo "nested component props took ${elapsed_ms}ms (limit ${threshold_ms}ms)" >&2
exit 1
fi
echo "nested component props passed in ${elapsed_ms}ms"
+179
View File
@@ -0,0 +1,179 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="parallel-precompile-test-$$"
test_root="/tmp/$test_name"
source_dir="$test_root/site"
bin_directory="$test_root/work"
absolute_source_dir=""
artifact_dir=""
generation_log=""
generation_pid=""
precompile_timeout="${UCE_TEST_PRECOMPILE_TIMEOUT:-120s}"
precompile() {
timeout "$precompile_timeout" env UCE_PRECOMPILE_FILES_IN="$source_dir" UCE_PRECOMPILE_BIN_DIRECTORY="$bin_directory" "$@" bin/uce_fastcgi.linux.bin --precompile
}
cleanup() {
if [[ -n "$generation_pid" ]] && kill -0 "$generation_pid" 2>/dev/null; then
kill "$generation_pid" 2>/dev/null || true
cleanup_deadline=$((SECONDS + 10))
while kill -0 "$generation_pid" 2>/dev/null && (( SECONDS < cleanup_deadline )); do
sleep 0.05
done
if kill -0 "$generation_pid" 2>/dev/null; then
kill -KILL "$generation_pid" 2>/dev/null || true
fi
wait "$generation_pid" 2>/dev/null || true
fi
rm -rf "$test_root"
if [[ -n "$generation_log" ]]; then
rm -f "$generation_log"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
absolute_source_dir=$(realpath "$source_dir")
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$absolute_source_dir"
write_units() {
local version="$1"
for unit in 0 1 2 3; do
printf 'CLI(Request& context) { print("parallel-precompile-%s-%s"); }\n' "$version" "$unit" >"$source_dir/unit-$unit.uce"
done
}
printf 'CLI(Request& context) { print("parallel-precompile-warmup"); }\n' >"$source_dir/warmup.uce"
precompile UCE_PRECOMPILE_JOBS=1 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" >/dev/null
rm "$source_dir/warmup.uce"
write_units serial
serial_start=$(date +%s%N)
serial_output=$(precompile UCE_PRECOMPILE_JOBS=1 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared")
serial_ns=$(( $(date +%s%N) - serial_start ))
if ! grep -Eq 'with 1 job: .* 4 compiled, 0 failed, worker status ok' <<<"$serial_output"; then
echo "serial precompile did not compile the four controlled units" >&2
echo "$serial_output" >&2
exit 1
fi
write_units parallel
parallel_start=$(date +%s%N)
parallel_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared")
parallel_ns=$(( $(date +%s%N) - parallel_start ))
if [[ $(grep -Ec '^Precompile worker [12]/2:' <<<"$parallel_output") -ne 2 ]]; then
echo "parallel precompile did not report both workers" >&2
echo "$parallel_output" >&2
exit 1
fi
if ! grep -Eq 'with 2 jobs: .* 4 compiled, 0 failed, worker status ok' <<<"$parallel_output"; then
echo "parallel precompile did not compile the four controlled units" >&2
echo "$parallel_output" >&2
exit 1
fi
for unit in 0 1 2 3; do
if [[ ! -s "$artifact_dir/unit-$unit.uce.wasm" || ! -s "$artifact_dir/unit-$unit.uce.cwasm" ]]; then
echo "parallel precompile did not publish wasm and serialized artifacts for unit $unit" >&2
exit 1
fi
done
printf 'CLI(Request& context) { print("parallel-precompile-race-0"); }\n' >"$source_dir/race-0.uce"
printf 'CLI(Request& context) { print("parallel-precompile-race-1"); }\n' >"$source_dir/race-1.uce"
race_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-race")
if ! grep -Eq 'with 2 jobs: .* 2 compiled, 0 failed, worker status ok' <<<"$race_output" || \
[[ $(find "$artifact_dir/pch-race" -maxdepth 1 -type f -name '*.pch' | wc -l) -ne 1 ]] || \
find "$artifact_dir/pch-race" -maxdepth 1 -type f -name '*.tmp.*' -print -quit | grep -q .; then
echo "parallel precompile did not publish exactly one clean shared PCH" >&2
echo "$race_output" >&2
exit 1
fi
rm "$source_dir/race-0.uce" "$source_dir/race-1.uce"
for unit in $(seq -w 0 9); do
printf 'CLI(Request& context) { print("parallel-precompile-generation-%s"); }\n' "$unit" >"$source_dir/generation-$unit.uce"
done
generation_log="/tmp/$test_name-generation.log"
precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" >"$generation_log" 2>&1 &
generation_pid=$!
generation_deadline=$((SECONDS + 60))
while [[ ! -s "$artifact_dir/generation-0.uce.meta.txt" ]]; do
if ! kill -0 "$generation_pid" 2>/dev/null; then
echo "parallel precompile exited before the controlled source mutation" >&2
wait "$generation_pid" || true
cat "$generation_log" >&2
exit 1
fi
if (( SECONDS >= generation_deadline )); then
echo "timed out waiting for the controlled precompile mutation boundary" >&2
exit 1
fi
sleep 0.05
done
printf 'CLI(Request& context) { print("parallel-precompile-generation-mutated"); }\n' >"$source_dir/generation-0.uce"
printf 'CLI(Request& context) { print("parallel-precompile-generation-added"); }\n' >"$source_dir/generation-new.uce"
if ! kill -0 "$generation_pid" 2>/dev/null; then
echo "parallel precompile exited at the controlled source mutation boundary" >&2
exit 1
fi
set +e
wait "$generation_pid"
generation_rc=$?
set -e
generation_pid=""
if [[ $generation_rc -eq 0 ]] || ! grep -q 'source generation changed during precompile; candidate generation rejected' "$generation_log"; then
echo "parallel precompile accepted a source generation that changed during compilation" >&2
cat "$generation_log" >&2
exit 1
fi
if [[ -e "$artifact_dir/generation-new.uce.wasm" || -e "$artifact_dir/generation-new.uce.cwasm" ]]; then
echo "parallel precompile published an added unit outside the rejected candidate source set" >&2
exit 1
fi
generation_retry_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared")
if ! grep -Eq 'with 2 jobs: .* 2 compiled, 0 failed, worker status ok' <<<"$generation_retry_output"; then
echo "parallel precompile did not reconcile the changed and added units on retry" >&2
echo "$generation_retry_output" >&2
exit 1
fi
rm -f "$generation_log"
printf 'CLI(Request& context) { print(deliberate_parallel_precompile_failure); }\n' >"$source_dir/broken.uce"
set +e
failure_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" 2>&1)
failure_rc=$?
set -e
if [[ $failure_rc -eq 0 ]] || ! grep -Eq 'with 2 jobs: .* 1 failed, worker status failed' <<<"$failure_output"; then
echo "parallel precompile did not aggregate a controlled worker failure" >&2
echo "$failure_output" >&2
exit 1
fi
set +e
repeat_failure_output=$(precompile UCE_PRECOMPILE_JOBS=2 UCE_WASM_PCH_DIR="$artifact_dir/pch-shared" 2>&1)
repeat_failure_rc=$?
set -e
if [[ $repeat_failure_rc -eq 0 ]] || ! grep -Eq 'with 2 jobs: .* 1 failed, worker status failed' <<<"$repeat_failure_output"; then
echo "parallel precompile accepted a persisted current compile failure" >&2
echo "$repeat_failure_output" >&2
exit 1
fi
rm "$source_dir/broken.uce"
rm -f "$source_dir"/unit-*.uce
for unit in $(seq -w 0 4); do
printf 'CLI(Request& context) { print("parallel-precompile-bounds-%s"); }\n' "$unit" >"$source_dir/bounds-$unit.uce"
done
negative_jobs_output=$(precompile UCE_PRECOMPILE_JOBS=-1)
malformed_jobs_output=$(precompile UCE_PRECOMPILE_JOBS=invalid)
oversized_jobs_output=$(precompile UCE_PRECOMPILE_JOBS=17)
if ! grep -q 'with 1 job: 16 units,' <<<"$negative_jobs_output" || ! grep -q 'with 2 jobs: 16 units,' <<<"$malformed_jobs_output" || ! grep -q 'with 16 jobs: 16 units,' <<<"$oversized_jobs_output"; then
echo "precompile job bounds did not map negative/malformed/oversized input to 1/2/16" >&2
exit 1
fi
rm -f "$source_dir"/bounds-*.uce "$source_dir"/generation-*.uce
printf 'parallel precompile passed: serial %.3fs, parallel %.3fs\n' \
"$(awk -v ns="$serial_ns" 'BEGIN { print ns / 1000000000 }')" \
"$(awk -v ns="$parallel_ns" 'BEGIN { print ns / 1000000000 }')"
+218
View File
@@ -0,0 +1,218 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" != "--inside" ]]; then
exec timeout --signal=TERM --kill-after=5s 150s unshare --mount --fork --kill-child=TERM "$0" --inside
fi
name="parallel-proactive-test-$$"
root="/tmp/$name"
site="$root/site"
work="$root/work"
settings="$root/settings.cfg"
log="$root/service.log"
shim_log="$root/compile.tsv"
server_pid=""
cleanup() {
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
deadline=$((SECONDS + 10))
while kill -0 "$server_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.05; done
if kill -0 "$server_pid" 2>/dev/null; then kill -KILL "$server_pid" 2>/dev/null || true; fi
wait "$server_pid" 2>/dev/null || true
fi
rm -rf "$root"
}
trap cleanup EXIT
mkdir -p "$site" "$work" "$root/run" "$root/session" "$root/upload"
cp /etc/uce/settings.cfg "$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
FCGI_PORT=
CLI_SOCKET_PATH=$root/run/cli.sock
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
WASM_COMPILE_SCRIPT=$root/compile
WORKER_COUNT=1
PROACTIVE_COMPILE_ENABLED=1
PROACTIVE_COMPILE_JOBS=2
PROACTIVE_COMPILE_CHECK_INTERVAL=1
COMPILE_FAILURE_RETRY_SECONDS=2
SERVE_LAST_KNOWN_GOOD=0
CFG
mount --bind "$settings" /etc/uce/settings.cfg
cat >"$root/compile" <<SHIM
#!/usr/bin/env bash
set -uo pipefail
root='$root'
real='$(pwd)/scripts/compile_wasm_unit'
source_file="\$3"
exec 9>>"\$root/counter.lock"
flock 9
active=0
if [[ -r "\$root/active" ]]; then read -r active <"\$root/active"; fi
active=\$((active + 1))
printf '%s\n' "\$active" >"\$root/active"
maximum=0
if [[ -r "\$root/maximum" ]]; then read -r maximum <"\$root/maximum"; fi
if (( active > maximum )); then printf '%s\n' "\$active" >"\$root/maximum"; fi
nice_value=\$(timeout 2s ps -o ni= -p "\$PPID" | tr -d ' ')
owner_pid=\$PPID
for _ in 1 2 3 4; do
owner_command=\$(timeout 2s ps -o comm= -p "\$owner_pid" | tr -d ' ')
[[ "\$owner_command" == uce_fastcgi.* ]] && break
owner_pid=\$(timeout 2s ps -o ppid= -p "\$owner_pid" | tr -d ' ')
[[ -n "\$owner_pid" && "\$owner_pid" != "1" ]] || break
done
printf '%s\t%s\t%s\t%s\n' "\$(date +%s%N)" "\$nice_value" "\$source_file" "\$owner_pid" >>"\$root/compile.tsv"
flock -u 9
if [[ -r "\$root/hold-paths" ]] && grep -Fxq "\$source_file" "\$root/hold-paths"; then
: >"\$root/held-\$owner_pid"
deadline=\$((SECONDS + 20))
while [[ ! -e "\$root/release" && \$SECONDS -lt \$deadline ]]; do sleep 0.05; done
[[ -e "\$root/release" ]] || exit 98
fi
sleep 0.75
"\$real" "\$@"
rc=\$?
flock 9
active=1
if [[ -r "\$root/active" ]]; then read -r active <"\$root/active"; fi
printf '%s\n' "\$((active - 1))" >"\$root/active"
flock -u 9
exit "\$rc"
SHIM
chmod +x "$root/compile"
printf '%s\n' 'String common_marker() { return("common-a"); }' >"$site/common.uce"
for unit in 0 1 2 3; do
printf '%s\n' '#load "common.uce"' "CLI(Request& context) { print(common_marker(), \"-$unit\"); }" >"$site/unit-$unit.uce"
done
printf '%s\n' 'CLI(Request& context) { print("removed"); }' >"$site/removed.uce"
printf '%s\n' 'CLI(Request& context) { print("baseline"); }' >"$site/broken.uce"
timeout --signal=TERM --kill-after=5s 120s bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
deadline=$((SECONDS + 20))
while [[ ! -S "$root/run/cli.sock" ]] && (( SECONDS < deadline )); do sleep 0.05; done
[[ -S "$root/run/cli.sock" ]] || { echo "private UCE CLI socket was not ready" >&2; cat "$log" >&2; exit 1; }
generation="$(scripts/unit_cache_directory "$work")"
artifacts="$generation$(realpath "$site")"
deadline=$((SECONDS + 60))
while (( SECONDS < deadline )); do
ready=1
for unit in 0 1 2 3; do
[[ -s "$artifacts/unit-$unit.uce.wasm" && -s "$artifacts/unit-$unit.uce.cwasm" ]] || ready=0
done
(( ready == 1 )) && break
sleep 0.1
done
(( ready == 1 )) || { echo "parallel scanners did not publish all valid units" >&2; cat "$log" >&2; exit 1; }
[[ "$(<"$root/maximum")" -ge 2 ]] || { echo "parallel scanners never overlapped" >&2; cat "$shim_log" >&2; exit 1; }
grep -q 'proactive compiler worker 1/2 ready' "$log"
grep -q 'proactive compiler worker 2/2 ready' "$log"
for unit in 0 1 2 3; do
[[ "$(grep -c "$site/unit-$unit.uce" "$shim_log")" -eq 1 ]] || { echo "unit-$unit was not compiled exactly once" >&2; cat "$shim_log" >&2; exit 1; }
done
deadline=$((SECONDS + 15))
while { [[ ! -s "$artifacts/broken.uce.cwasm" || ! -s "$artifacts/removed.uce.cwasm" ]] || [[ "$(<"$root/active")" != "0" ]]; } && (( SECONDS < deadline )); do sleep 0.1; done
[[ -s "$artifacts/broken.uce.cwasm" && -s "$artifacts/removed.uce.cwasm" && "$(<"$root/active")" == "0" ]] || { echo "phase fixtures did not become idle" >&2; cat "$log" >&2; exit 1; }
exec 7>>"$root/counter.lock"
flock 7
printf '0\n' >"$root/maximum"
flock -u 7
mapfile -t scanner_pids < <(awk -F '\t' -v site="$site/" '$3 ~ ("^" site "unit-[0-9]+[.]uce$") { count[$4]++ } END { for(pid in count) if(count[pid] >= 2) print pid }' "$shim_log" | sort -n)
[[ "${#scanner_pids[@]}" -eq 2 ]] || { echo "controlled units were not split between both scanners" >&2; cat "$shim_log" >&2; exit 1; }
assert_scanner_memory_bounded() {
for scanner_pid in "${scanner_pids[@]}"; do
retained_kb=$(awk '/^(VmRSS|VmSwap):/{total += $2} END{print total + 0}' "/proc/$scanner_pid/status")
# Scanner bookkeeping is small; 64 MiB leaves ample headroom while still
# catching the measured old 12-serialization retention (~73 MiB).
(( retained_kb < 65536 )) || { echo "proactive scanner retained serialized-module arenas: pid=$scanner_pid retained_kb=$retained_kb" >&2; exit 1; }
done
}
assert_scanner_memory_bounded
blockers=()
victims=()
for scanner_pid in "${scanner_pids[@]}"; do
mapfile -t scanner_units < <(awk -F '\t' -v pid="$scanner_pid" -v site="$site/" '$4 == pid && $3 ~ ("^" site "unit-[0-9]+[.]uce$") { print $3 }' "$shim_log")
blockers+=("${scanner_units[0]}")
victims+=("${scanner_units[1]}")
done
printf '%s\n' "${blockers[@]}" >"$root/hold-paths"
rm -f "$root"/held-* "$root/release"
sed -i 's/common-a/common-b/' "$site/common.uce"
deadline=$((SECONDS + 20))
while { [[ ! -e "$root/held-${scanner_pids[0]}" ]] || [[ ! -e "$root/held-${scanner_pids[1]}" ]]; } && (( SECONDS < deadline )); do sleep 0.05; done
[[ -e "$root/held-${scanner_pids[0]}" && -e "$root/held-${scanner_pids[1]}" ]] || { echo "both scanners did not reach their controlled queue barriers" >&2; cat "$log" >&2; exit 1; }
for victim in "${victims[@]}"; do
victim_marker=$(UCE_CLI_SOCKET="$root/run/cli.sock" timeout 20s scripts/uce-cli "/$(basename "$victim")")
[[ "$victim_marker" == *"common-b-"* ]] || { echo "request worker did not publish queued victim: $victim_marker" >&2; exit 1; }
done
touch "$root/release"
deadline=$((SECONDS + 30))
while (( SECONDS < deadline )); do
rebuilt=1
for unit in 0 1 2 3; do
[[ "$(grep -c "$site/unit-$unit.uce" "$shim_log" || true)" -ge 2 ]] || rebuilt=0
done
(( rebuilt == 1 )) && break
sleep 0.1
done
(( rebuilt == 1 )) || { echo "common dependency edit did not rebuild all scanner shards" >&2; cat "$log" >&2; exit 1; }
deadline=$((SECONDS + 5))
while [[ "$(<"$root/active")" != "0" ]] && (( SECONDS < deadline )); do sleep 0.1; done
sleep 1.25
for unit in 0 1 2 3; do
[[ "$(grep -c "$site/unit-$unit.uce" "$shim_log")" -eq 2 ]] || { echo "unit-$unit common-dependency rebuild was not exact" >&2; cat "$shim_log" >&2; exit 1; }
done
rm -f "$root/hold-paths" "$root"/held-* "$root/release"
[[ "$(<"$root/maximum")" -ge 2 ]] || { echo "common dependency fanout did not overlap across scanner owners" >&2; cat "$shim_log" >&2; exit 1; }
marker=$(UCE_CLI_SOCKET="$root/run/cli.sock" timeout 20s scripts/uce-cli /unit-0.uce)
[[ "$marker" == *"common-b-0"* ]] || { echo "rebuilt parent did not execute the changed dependency: $marker" >&2; exit 1; }
rm "$site/removed.uce"
registry="$generation/known-uce-files.txt"
deadline=$((SECONDS + 5))
while grep -q "$site/removed.uce" "$registry" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.1; done
if grep -q "$site/removed.uce" "$registry" 2>/dev/null; then echo "removed unit remained tracked" >&2; cat "$log" >&2; exit 1; fi
printf '%s\n' 'CLI(Request& context) { print(deliberate_parallel_scanner_failure); }' >"$site/broken.uce"
deadline=$((SECONDS + 30))
while [[ "$(grep -c "$site/broken.uce" "$shim_log" || true)" -lt 3 ]] && (( SECONDS < deadline )); do sleep 0.1; done
[[ "$(grep -c "$site/broken.uce" "$shim_log" || true)" -ge 3 ]] || { echo "persisted scanner failure was not retried" >&2; cat "$log" >&2; exit 1; }
mapfile -t broken_times < <(awk -F '\t' -v path="$site/broken.uce" '$3 == path { print $1 }' "$shim_log")
(( broken_times[2] - broken_times[1] >= 2000000000 )) || { echo "scanner failure retry ignored its backoff" >&2; cat "$shim_log" >&2; exit 1; }
[[ ! -e "$artifacts/broken.uce.wasm" ]] || { echo "failed scanner build published wasm" >&2; exit 1; }
printf '%s\n' 'CLI(Request& context) { print("recovered"); }' >"$site/broken.uce"
deadline=$((SECONDS + 8))
while [[ ! -s "$artifacts/broken.uce.cwasm" ]] && (( SECONDS < deadline )); do sleep 0.1; done
[[ -s "$artifacts/broken.uce.cwasm" ]] || { echo "changed failed source waited for the unchanged-failure backoff" >&2; cat "$log" >&2; exit 1; }
printf '%s\n' 'CLI(Request& context) { print("priority"); }' >"$site/priority.uce"
priority_file="$generation/proactive-priority.txt"
exec 8>"$priority_file.lock"
flock 8
printf '%s\n' "$(realpath "$site/priority.uce")" >>"$priority_file"
flock -u 8
deadline=$((SECONDS + 15))
while [[ ! -s "$artifacts/priority.uce.cwasm" ]] && (( SECONDS < deadline )); do sleep 0.1; done
[[ -s "$artifacts/priority.uce.cwasm" ]] || { echo "priority compiler did not publish requested unit" >&2; cat "$log" >&2; exit 1; }
priority_nice=$(awk -F '\t' -v path="$site/priority.uce" '$3 == path { print $2; exit }' "$shim_log")
[[ "$priority_nice" == "5" ]] || { echo "priority queue was not owned by the nice-5 compiler: $priority_nice" >&2; cat "$shim_log" >&2; exit 1; }
# Recheck after dependency fanout, failure/retry, recovery, and priority work;
# the historical leak accumulated in the long-lived scanners across phases.
assert_scanner_memory_bounded
printf '%s\n' 'parallel proactive compile passed'
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="password-hash-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'CLI(Request& context) {' \
' String encoded = password_hash("correct horse battery staple");' \
' String second = password_hash("correct horse battery staple");' \
' String weaker = "$uce$scrypt$16384$8$1$00112233445566778899aabbccddeeff$29fdfb3d991961e926a19c1136a07e252afa5fdb8d3a0fb74cdcfa5016956f34";' \
' String excessive = "$uce$scrypt$131072$8$1$00112233445566778899aabbccddeeff$0000000000000000000000000000000000000000000000000000000000000000";' \
' print(encoded, "\n", encoded != second ? "randomized" : "reused", "\n", password_verify("correct horse battery staple", encoded) ? "valid" : "invalid", "\n", password_verify("wrong password", encoded) ? "wrong-valid" : "wrong-rejected", "\n", password_needs_rehash(encoded) ? "rehash" : "current", "\n", password_verify("legacy password", weaker) && password_needs_rehash(weaker) ? "legacy-valid-rehash" : "legacy-failed", "\n", !password_verify("password", excessive) && password_needs_rehash(excessive) ? "excessive-rejected" : "excessive-accepted", "\n", !password_verify("password", "$uce$scrypt$65536$8$1$zz$00") && password_needs_rehash("malformed") ? "malformed-rejected" : "malformed-valid");' \
'}' >"$source_dir/test.uce"
output=$(scripts/uce-cli "/$test_name/test.uce")
if [[ "$output" != *'$uce$scrypt$65536$8$1$'* || "$output" != *$'\nrandomized\nvalid\nwrong-rejected\ncurrent\nlegacy-valid-rehash\nexcessive-rejected\nmalformed-rejected'* ]]; then
echo "native password hashing failed: $output" >&2
exit 1
fi
echo "password hashing passed"
+59
View File
@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
http_port="${UCE_RAW_HTTP_TEST_PORT:-}"
if [[ -z "$http_port" && -r /etc/uce/settings.cfg ]]; then
http_port=$(awk -F= '/^[[:space:]]*HTTP_PORT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
http_port="${http_port:-8080}"
request_path="${UCE_RAW_HTTP_TEST_PATH:-/info/index.uce}"
marker="raw-http-log-$$-$(date +%s%N)"
separator="?"
if [[ "$request_path" == *\?* ]]; then
separator="&"
fi
started_at=$(date '+%Y-%m-%d %H:%M:%S')
response=$(curl -fsS --max-time 15 "http://127.0.0.1:${http_port}${request_path}${separator}__uce_log_probe=${marker}")
if [[ -z "$response" ]]; then
echo "raw HTTP probe returned an empty response" >&2
exit 1
fi
request_logs=""
for _ in $(seq 1 30); do
request_logs=$(journalctl -u uce --since "$started_at" --no-pager | grep '(r)' | grep "$marker" || true)
if [[ $(printf '%s\n' "$request_logs" | grep -c '(r)' || true) -ge 2 ]]; then
break
fi
sleep 0.1
done
if [[ $(printf '%s\n' "$request_logs" | grep -c '(r)' || true) -ne 2 ]]; then
echo "expected exactly two raw HTTP request-stage records" >&2
printf '%s\n' "$request_logs" >&2
exit 1
fi
if [[ $(printf '%s\n' "$request_logs" | grep -c 'transport:http' || true) -ne 1 ||
$(printf '%s\n' "$request_logs" | grep -c 'transport:fastcgi' || true) -ne 1 ]]; then
echo "raw HTTP request-stage records did not identify HTTP and FastCGI transports" >&2
printf '%s\n' "$request_logs" >&2
exit 1
fi
mapfile -t durations < <(printf '%s\n' "$request_logs" | sed -n 's/.*[[:space:]]\([0-9][0-9.]*\)s[[:space:]]*fps:.*/\1/p')
if [[ ${#durations[@]} -ne 2 ]]; then
echo "could not parse both raw HTTP request-stage durations" >&2
printf '%s\n' "$request_logs" >&2
exit 1
fi
for duration in "${durations[@]}"; do
awk -v duration="$duration" 'BEGIN { exit !(duration > 0 && duration < 60) }' || {
echo "raw HTTP request-stage duration is outside (0, 60) seconds: ${duration}s" >&2
printf '%s\n' "$request_logs" >&2
exit 1
}
done
echo "raw HTTP request log passed"
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="relative-component-cache-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" && -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
if [[ -n "${configured_site_directory:-}" ]]; then
site_directory="$configured_site_directory"
fi
fi
source_dir="$site_directory/$test_name"
bin_directory="${BIN_DIRECTORY:-}"
if [[ -z "$bin_directory" && -r /etc/uce/settings.cfg ]]; then
bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
fi
bin_directory="${bin_directory:-/tmp/uce/work}"
cache_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$cache_dir" ]]; then
rm -rf "$cache_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir/a" "$source_dir/b"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'CLI(Request& context) {' \
' print(component("a/parent", context));' \
' print("/");' \
' print(component("b/parent", context));' \
'}' >"$source_dir/entry.uce"
printf '%s\n' 'COMPONENT(Request& context) { print(component("child", context)); }' >"$source_dir/a/parent.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("relative-a"); }' >"$source_dir/a/child.uce"
printf '%s\n' 'COMPONENT(Request& context) { print(component("child", context)); }' >"$source_dir/b/parent.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("relative-b"); }' >"$source_dir/b/child.uce"
output=$(scripts/uce-cli "/$test_name/entry.uce")
if [[ "$output" != "relative-a/relative-b" ]]; then
echo "relative component cache crossed caller boundaries: $output" >&2
exit 1
fi
echo "relative component cache passed"
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
binary="$PWD/bin/uce_fastcgi.linux.bin"
root=$(mktemp -d /tmp/uce-server-arguments.XXXXXX)
listener_pid=""
cleanup() {
[[ -n "$listener_pid" ]] && kill "$listener_pid" 2>/dev/null || true
rm -rf "$root"
}
trap cleanup EXIT
cfg="$root/settings.cfg"
socket_path="$root/sentinel.sock"
cp /etc/uce/settings.cfg "$cfg"
sed -E -i \
-e "s|^[[:space:]]*FCGI_SOCKET_PATH[[:space:]]*=.*|FCGI_SOCKET_PATH=|" \
-e "s|^[[:space:]]*FCGI_PORT[[:space:]]*=.*|FCGI_PORT=|" \
-e "s|^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=.*|CLI_SOCKET_PATH=$socket_path|" \
-e "s|^[[:space:]]*HTTP_PORT[[:space:]]*=.*|HTTP_PORT=|" \
-e "s|^[[:space:]]*PROACTIVE_COMPILE_ENABLED[[:space:]]*=.*|PROACTIVE_COMPILE_ENABLED=false|" \
-e "s|^[[:space:]]*WORKER_COUNT[[:space:]]*=.*|WORKER_COUNT=1|" \
-e "s|^[[:space:]]*BIN_DIRECTORY[[:space:]]*=.*|BIN_DIRECTORY=$root/bin|" \
-e "s|^[[:space:]]*TMP_UPLOAD_PATH[[:space:]]*=.*|TMP_UPLOAD_PATH=$root/upload|" \
-e "s|^[[:space:]]*SESSION_PATH[[:space:]]*=.*|SESSION_PATH=$root/session|" \
"$cfg"
python3 -c 'import socket,sys,time; s=socket.socket(socket.AF_UNIX); s.bind(sys.argv[1]); s.listen(); time.sleep(30)' "$socket_path" &
listener_pid=$!
for _ in $(seq 1 50); do
[[ -S "$socket_path" ]] && break
sleep 0.02
done
[[ -S "$socket_path" ]]
socket_inode=$(stat -c %i "$socket_path")
invoke() {
local output="$1"
shift
local timeout_seconds="${INVOKE_TIMEOUT_SECONDS:-2}"
timeout --signal=TERM --kill-after=1s "$timeout_seconds" unshare --mount --fork \
bash -c 'mount --bind "$1" /etc/uce/settings.cfg; exec "$2" "${@:3}"' \
_ "$cfg" "$binary" "$@" >"$output.stdout" 2>"$output.stderr"
}
for option in --help -h; do
invoke "$root/help" "$option"
grep -q '^Usage: uce_fastcgi' "$root/help.stdout"
grep -q -- '--precompile' "$root/help.stdout"
grep -q -- '--serialize-module' "$root/help.stdout"
[[ ! -s "$root/help.stderr" ]]
done
for arguments in '--unknown' '--precompile extra' '--serialize-module' '--help extra'; do
read -r -a argv <<<"$arguments"
set +e
invoke "$root/invalid" "${argv[@]}"
rc=$?
set -e
[[ $rc -eq 2 ]]
grep -q '^invalid arguments$' "$root/invalid.stderr"
grep -q '^Usage: uce_fastcgi' "$root/invalid.stderr"
[[ ! -s "$root/invalid.stdout" ]]
done
cp bin/wasm/core.wasm "$root/serialize.wasm"
INVOKE_TIMEOUT_SECONDS=30 invoke "$root/serialize" --serialize-module "$root/serialize.wasm"
[[ -s "$root/serialize.cwasm" ]]
[[ ! -s "$root/serialize.stderr" ]]
# Serialization shares the unit compile lock. If the wasm path is replaced
# while the serializer is waiting, it must read the replacement and must not
# publish a stale native artifact for the old inode.
cp bin/wasm/core.wasm "$root/race.wasm"
exec 8>"$root/race.wasm.lock"
flock 8
INVOKE_TIMEOUT_SECONDS=30 invoke "$root/race" --serialize-module "$root/race.wasm" &
race_pid=$!
sleep 0.1
kill -0 "$race_pid"
printf 'not wasm\n' >"$root/race.wasm.next"
mv "$root/race.wasm.next" "$root/race.wasm"
flock -u 8
set +e
wait "$race_pid"
race_rc=$?
set -e
[[ $race_rc -eq 1 ]]
grep -Eqi 'wasm|magic|WebAssembly|compile' "$root/race.stderr"
[[ ! -e "$root/race.cwasm" ]]
if compgen -G "$root/race.cwasm.*.tmp" >/dev/null; then
echo "failed serialization left a temporary artifact" >&2
exit 1
fi
[[ -S "$socket_path" ]]
[[ "$(stat -c %i "$socket_path")" == "$socket_inode" ]]
kill -0 "$listener_pid"
echo 'server argument handling passed: help and invalid arguments exited before listener setup'
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
set -euo pipefail
service_name="${UCE_TEST_SERVICE:-uce.service}"
socket_name="${UCE_TEST_SOCKET_SERVICE:-uce.socket}"
http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
http_path="${UCE_TEST_HTTP_PATH:-/info/}"
requests="${UCE_TEST_RESTART_REQUESTS:-100}"
statuses=$(mktemp)
trap 'rm -f "$statuses"' EXIT
systemctl is-active --quiet "$socket_name"
systemctl is-active --quiet "$service_name"
socket_path=$(systemctl show -p Listen --value "$socket_name" | sed -n 's/ \+(Stream)$//p')
[[ -S "$socket_path" ]]
inode_before=$(stat -c %i "$socket_path")
(
for ((i = 0; i < requests; i++)); do
curl -sS --max-time 15 -o /dev/null -w '%{http_code}\n' \
-H "Host: $http_host" "http://127.0.0.1${http_path}"
done
) >"$statuses" &
load_pid=$!
sleep 0.05
systemctl restart "$service_name"
systemctl is-active --quiet "$socket_name"
inode_during=$(stat -c %i "$socket_path")
[[ "$inode_during" = "$inode_before" ]]
wait "$load_pid"
inode_after=$(stat -c %i "$socket_path")
[[ "$inode_after" = "$inode_before" ]]
[[ $(wc -l <"$statuses") -eq "$requests" ]]
if [[ $(grep -c '^200$' "$statuses") -ne "$requests" ]]; then
sort "$statuses" | uniq -c >&2
exit 1
fi
systemctl is-active --quiet "$socket_name"
systemctl is-active --quiet "$service_name"
echo "socket activation restart passed across $requests HTTP requests"
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="unit-export-surface-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
bin_directory="${UCE_TEST_BIN_DIRECTORY:-/tmp/uce/work}"
if [[ -r /etc/uce/settings.cfg ]]; then
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*HTTP_DOCUMENT_ROOT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
site_directory="${configured_site_directory:-$site_directory}"
fi
if [[ -z "${UCE_TEST_BIN_DIRECTORY:-}" ]]; then
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
bin_directory="${configured_bin_directory:-$bin_directory}"
fi
fi
source_dir="$site_directory/$test_name"
artifact_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$artifact_dir" ]]; then
rm -rf "$artifact_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
absolute_source_dir=$(realpath "$source_dir")
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$absolute_source_dir"
printf '%s\n' \
'String visibility_used() { return("private-used"); }' \
'String visibility_unused() { return("uce-private-unused-marker-8f61d2"); }' \
'EXPORT String visibility_shared() { return("shared"); }' \
'CLI(Request& context) { print(visibility_shared(), ":", visibility_used(), ":", component("named:NAMED", context)); }' \
>"$source_dir/entry.uce"
printf '%s\n' \
'String named_used() { return("private-named-used"); }' \
'String named_unused() { return("uce-named-unused-marker-4ae973"); }' \
'EXPORT String named_shared() { return("named-export"); }' \
'COMPONENT:NAMED(Request& context) { print(named_shared(), ":", named_used()); }' \
>"$source_dir/named.uce"
output=$(scripts/uce-cli "/$test_name/entry.uce")
if [[ "$output" != "shared:private-used:named-export:private-named-used" ]]; then
echo "unit export surface runtime failed: $output" >&2
exit 1
fi
entry_wasm="$artifact_dir/entry.uce.wasm"
named_wasm="$artifact_dir/named.uce.wasm"
python3 - "$entry_wasm" "$named_wasm" "$absolute_source_dir" <<'PY'
import sys
from pathlib import Path
from scripts.check_unit_wasm import collect
cases = [
(Path(sys.argv[1]), {"__wasm_call_ctors", "__uce_set_current_request", "__uce_cli", "visibility_shared"}, b"uce-private-unused-marker-8f61d2"),
(Path(sys.argv[2]), {"__wasm_call_ctors", "__uce_set_current_request", "__uce_component_NAMED", "named_shared"}, b"uce-named-unused-marker-4ae973"),
]
source_dir = Path(sys.argv[3])
for path, allowed, unused_marker in cases:
data = path.read_bytes()
customs, imports, exports = collect(path)
names = {name for name, _ in exports}
unexpected = names - allowed
missing = allowed - names
if unexpected or missing:
raise SystemExit(f"{path}: unexpected exports={sorted(unexpected)} missing={sorted(missing)} all={sorted(names)}")
if unused_marker in data:
raise SystemExit(f"{path}: retained unused private code marker")
if len(imports) >= 40:
raise SystemExit(f"{path}: retained {len(imports)} imports (expected fewer than 40)")
if path.stat().st_size >= 1024 * 1024:
raise SystemExit(f"{path}: artifact is {path.stat().st_size} bytes (expected under 1 MiB)")
source_map = Path(str(path) + ".source-map")
if not source_map.is_file() or source_map.stat().st_size >= 256 * 1024:
raise SystemExit(f"{path}: missing or oversized source map")
module = customs["uce.module"][-1].decode()
lines = source_map.read_text().splitlines()
if not lines or lines[0] != f"UCE_SOURCE_MAP_V1\t{module}":
raise SystemExit(f"{path}: source map does not match wasm module identity")
expected_source = str(source_dir / path.name.removesuffix(".wasm"))
if not any(line.startswith("F\t") and line.endswith("\t" + expected_source) for line in lines):
raise SystemExit(f"{path}: source map does not identify {expected_source}")
if not any(line.startswith("L\t") for line in lines):
raise SystemExit(f"{path}: source map has no address rows")
PY
echo "unit export surface passed"
+252
View File
@@ -0,0 +1,252 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
error_page_path=$(sed -n '/^bool render_wasm_error_page(/,/^}/p' src/linux_fastcgi.cpp)
[[ "$error_page_path" == *'get_shared_unit_bounded('* && "$error_page_path" == *'wasm_invocation_remaining_ms(invocation_deadline)'* ]]
[[ "$error_page_path" != *'get_shared_unit(&request'* ]]
if [[ "${1:-}" != --inside ]]; then
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
fi
root="/tmp/uce-compile-timeout-$$"
site="$root/site"
work="$root/work"
settings="$root/settings.cfg"
log="$root/service.log"
server_pid=""
cleanup() {
status=$?
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
for _ in $(seq 1 100); do kill -0 "$server_pid" 2>/dev/null || break; sleep 0.02; done
kill -KILL "$server_pid" 2>/dev/null || true
wait "$server_pid" 2>/dev/null || true
fi
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
rm -rf "$root"
return "$status"
}
trap cleanup EXIT
mkdir -p "$site/components" "$work" "$root/run" "$root/session" "$root/upload"
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|CLI_WORKER_COUNT|CLI_WORKER_MAX_REQUESTS|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|WASM_COMPILE_SCRIPT|WASM_INVOCATION_TIMEOUT_MS|WASM_EPOCH_PERIOD_MS|PROACTIVE_COMPILE_ENABLED|WORKER_COUNT|COMPILE_FAILURE_RETRY_SECONDS)[[:space:]]*=/d' \
/etc/uce/settings.cfg >"$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
FCGI_PORT=
CLI_SOCKET_PATH=$root/run/cli.sock
CLI_WORKER_COUNT=1
CLI_WORKER_MAX_REQUESTS=0
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd -P)/bin/wasm/core.wasm
WASM_COMPILE_SCRIPT=$root/compile
WASM_INVOCATION_TIMEOUT_MS=2000
WASM_EPOCH_PERIOD_MS=20
PROACTIVE_COMPILE_ENABLED=0
WORKER_COUNT=1
COMPILE_FAILURE_RETRY_SECONDS=60
CFG
mount --bind "$settings" /etc/uce/settings.cfg
mkdir -p "$root/compiler/src/wasm" "$root/compiler/scripts"
cp -a src/lib "$root/compiler/src/lib"
cp -a src/wasm/abi.h "$root/compiler/src/wasm/abi.h"
cp -a scripts/compile_wasm_unit scripts/build_unit_source_map.py scripts/check_unit_wasm.py "$root/compiler/scripts/"
chmod -R a+rX "$root/compiler"
cat >"$root/compile" <<SHIM
#!/usr/bin/env bash
set -euo pipefail
source_file="\$3"
if [[ -r '$root/stage-timeout' ]] && grep -Fxq "\$source_file" '$root/stage-timeout'; then
printf '%s\n' "\$\$" >'$root/compiler-pid'
(sleep 3; touch '$root/late-descendant') &
cp "\$2/\$(basename "\$source_file").wasm" "\$2/\$5"
cp "\$2/\$(basename "\$source_file").wasm.source-map" "\$2/\$5.source-map"
sleep 60
fi
if [[ -r '$root/copy-delay' ]] && grep -Fxq "\$source_file" '$root/copy-delay'; then
cp "\$2/\$(basename "\$source_file").wasm" "\$2/\$5"
cp "\$2/\$(basename "\$source_file").wasm.source-map" "\$2/\$5.source-map"
sleep 1.7
touch '$root/copy-delay-complete'
exit 0
fi
if [[ -r '$root/hold' ]] && grep -Fxq "\$source_file" '$root/hold'; then
printf '%s\n' "\$\$" >'$root/compiler-pid'
(sleep 3; touch '$root/late-descendant') &
sleep 60
fi
if [[ -r '$root/silent-fail' ]] && grep -Fxq "\$source_file" '$root/silent-fail'; then exit 23; fi
if [[ -r '$root/empty-success' ]] && grep -Fxq "\$source_file" '$root/empty-success'; then exit 0; fi
if [[ -r '$root/delay' ]] && grep -Fxq "\$source_file" '$root/delay'; then sleep 1; fi
exec '$root/compiler/scripts/compile_wasm_unit' "\$@"
SHIM
chmod +x "$root/compile"
cat >"$site/driver.uce" <<'UCE'
CLI(Request& context) {
if(context.get["health"] == "1") { print(request_perf()["worker_pid"].to_u64(), "|health"); return; }
if(context.get["dynamic"] == "1") { print(component("slow-component.uce")); return; }
String target = first(context.get["target"], "slow.uce");
print(unit_compile(target) ? "compiled" : "failed");
}
UCE
printf '%s\n' 'CLI(Request& context) { print("entry"); }' >"$site/cold-entry.uce"
printf '%s\n' 'CLI(Request& context) { print("slow"); }' >"$site/slow.uce"
printf '%s\n' 'CLI(Request& context) { print("silent"); }' >"$site/silent.uce"
printf '%s\n' 'CLI(Request& context) { print("empty"); }' >"$site/empty.uce"
printf '%s\n' '#load "transitive-child.uce"' 'CLI(Request& context) { print("parent"); }' >"$site/transitive.uce"
printf '%s\n' 'String transitive_value() { return("a"); }' >"$site/transitive-child.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("component"); }' >"$site/components/slow-component.uce"
timeout 40s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >"$root/precompile.log" 2>&1 || { cat "$root/precompile.log" >&2; exit 1; }
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
generation=$(scripts/unit_cache_directory "$work")
service_user=${UCE_TEST_SERVICE_USER:-www-data}
service_uid=$(id -u "$service_user")
service_gid=$(id -g "$service_user")
generation_before=$(<"$generation/source-generation.txt")
[[ $(sysctl -n fs.protected_hardlinks) == 1 ]]
[[ $(stat -c %u "$cache/slow.uce.wasm") == 0 && $(stat -c %u "$generation/source-generation.txt") == 0 ]]
[[ $(stat -c %u "$cache/slow.uce.wasm.lock") == 0 && $(stat -c %u "$generation/source-generation.txt.lock") == 0 ]]
find "$work" -type d -exec chown "$service_uid:$service_gid" {} +
chown -R "$service_uid:$service_gid" "$root/run" "$root/session" "$root/upload"
chown "$service_uid:$service_gid" "$root"
rm -f "$cache/cold-entry.uce."* "$cache/components/slow-component.uce."*
timeout --signal=TERM --kill-after=5s 90s setpriv --reuid="$service_uid" --regid="$service_gid" --clear-groups bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
for _ in $(seq 1 400); do [[ -S "$root/run/cli.sock" ]] && break; sleep 0.02; done
[[ -S "$root/run/cli.sock" ]] || { cat "$log" >&2; exit 1; }
request() {
local spec="$1" path="${1%%\?*}"
if [[ "$spec" == *\?* ]]; then
local query="${spec#*\?}"
IFS='&' read -r -a params <<<"$query"
timeout 5s scripts/uce-cli --get "$path" "${params[@]}"
else
timeout 5s scripts/uce-cli "$path"
fi
}
driver=""
for _ in $(seq 1 100); do
set +e
driver=$(request '/driver.uce?health=1' 2>/dev/null)
rc=$?
set -e
(( rc == 0 )) && [[ "$driver" == *'|health' ]] && break
sleep 0.05
done
[[ "$driver" == *'|health' ]] || { echo "driver did not become ready: $driver" >&2; exit 1; }
worker_pid=${driver%%|*}
[[ "$driver" == "$worker_pid|health" ]]
[[ "$(request /driver.uce)" == compiled ]]
[[ $(stat -c %u "$cache/slow.uce.wasm") == "$service_uid" ]]
[[ $(stat -c %u "$generation/source-generation.txt") == "$service_uid" ]]
[[ $(<"$generation/source-generation.txt") != "$generation_before" ]]
! grep -q 'Could not write.*source-generation.txt' "$log"
before=$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")
truncate -s 68719476736 "$cache/slow.uce.cwasm"
printf '%s\n' "$site/slow.uce" >"$root/copy-delay"
rm -f "$root/copy-delay-complete"
started=$(date +%s%N)
set +e
copy_timeout=$(request /driver.uce 2>&1)
set -e
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
rm -f "$root/copy-delay"
[[ "$copy_timeout" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "foreign rollback copy lacked timeout: $copy_timeout" >&2; exit 1; }
(( elapsed >= 1700 && elapsed <= 3500 )) || { echo "foreign rollback copy took ${elapsed}ms" >&2; exit 1; }
[[ -e "$root/copy-delay-complete" ]] || { echo "foreign rollback compiler did not finish before timeout" >&2; exit 1; }
[[ $(stat -c '%u:%s' "$cache/slow.uce.cwasm") == '0:68719476736' ]]
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
rm -f "$cache/slow.uce.cwasm"
assert_timeout() {
local url="$1" source="$2" started elapsed output
printf '%s\n' "$source" >"$root/hold"
rm -f "$root/compiler-pid" "$root/late-descendant"
started=$(date +%s%N)
set +e
output=$(request "$url" 2>&1)
set -e
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
[[ "$output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "$url lacked timeout: $output" >&2; exit 1; }
(( elapsed >= 1700 && elapsed <= 3500 )) || { echo "$url took ${elapsed}ms" >&2; exit 1; }
[[ "$(request '/driver.uce?health=1')" == "$worker_pid|health" ]]
[[ -s "$root/compiler-pid" ]]
compiler_pid=$(<"$root/compiler-pid")
! kill -0 "$compiler_pid" 2>/dev/null || { echo "compiler $compiler_pid survived" >&2; exit 1; }
sleep 0.1
[[ ! -e "$root/late-descendant" ]]
rm -f "$root/hold"
}
assert_timeout /driver.uce "$site/slow.uce"
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
[[ ! -e "$cache/slow.uce.compile.txt" && ! -e "$cache/slow.uce.wasm-check.txt" ]]
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
[[ "$(request /driver.uce)" == compiled ]]
before=$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")
printf '%s\n' "$site/silent.uce" >"$root/silent-fail"
[[ "$(request '/driver.uce?target=silent.uce')" == failed ]]
grep -q 'status 23' "$cache/silent.uce.compile.txt"
rm -f "$root/silent-fail"
printf '%s\n' "$site/empty.uce" >"$root/empty-success"
[[ "$(request '/driver.uce?target=empty.uce')" == failed ]]
grep -q 'without creating' "$cache/empty.uce.compile.txt"
rm -f "$root/empty-success"
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
printf '%s\n' "$site/slow.uce" >"$root/stage-timeout"
assert_timeout /driver.uce "$site/slow.uce"
rm -f "$root/stage-timeout"
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
transitive_before=$(sha256sum "$cache/transitive.uce.wasm" "$cache/transitive.uce.wasm.source-map" "$cache/transitive.uce.cpp" "$cache/transitive.uce.exports.txt" "$cache/transitive.uce.meta.txt")
printf '%s\n' 'String transitive_value() { return("b"); }' >"$site/transitive-child.uce"
assert_timeout '/driver.uce?target=transitive.uce' "$site/transitive-child.uce"
[[ "$transitive_before" == "$(sha256sum "$cache/transitive.uce.wasm" "$cache/transitive.uce.wasm.source-map" "$cache/transitive.uce.cpp" "$cache/transitive.uce.exports.txt" "$cache/transitive.uce.meta.txt")" ]]
assert_timeout '/driver.uce?dynamic=1' "$site/components/slow-component.uce"
assert_timeout /cold-entry.uce "$site/cold-entry.uce"
printf '%s\n' "$site/slow.uce" >"$root/delay"
timeout 25s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >/dev/null
rm -f "$root/delay"
exec 8>"$generation/known-uce-files.txt.lock"
flock 8
started=$(date +%s%N)
set +e
registry_locked=$(request /driver.uce 2>&1)
set -e
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
flock -u 8
[[ "$registry_locked" == *UCE_INVOCATION_TIMEOUT:* && "$elapsed" -le 3500 ]]
exec 9>"$cache/slow.uce.wasm.lock"
flock 9
started=$(date +%s%N)
set +e
locked=$(request /driver.uce 2>&1)
set -e
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
flock -u 9
[[ "$locked" == *UCE_INVOCATION_TIMEOUT:* && "$elapsed" -le 3500 ]]
echo "wasm compile timeout passed (entry, explicit, dynamic, unit/registry locks, error page, process group, staging, native precompile, foreign-owned artifacts)"
+141
View File
@@ -0,0 +1,141 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
raw_calls=$(grep -En -- '(->|\.)call\(' src/wasm/worker.cpp)
if [[ $(wc -l <<<"$raw_calls") -ne 1 || "$raw_calls" != *'return(func.call(context, args));'* ]]; then
echo "raw Wasmtime call bypasses call_guest(): $raw_calls" >&2
exit 1
fi
if [[ "${1:-}" == "--static-only" ]]; then
echo "wasm invocation timeout static call gate passed"
exit 0
fi
settings_file="${UCE_SETTINGS_FILE:-/etc/uce/settings.cfg}"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
socket_path="${UCE_CLI_SOCKET:-/run/uce/cli.sock}"
bin_directory="${BIN_DIRECTORY:-/tmp/uce/work}"
if [[ -r "$settings_file" ]]; then
[[ -n "${UCE_TEST_SITE_DIRECTORY:-}" ]] || site_directory=$(awk -F= '/^[[:space:]]*SITE_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
[[ -n "${UCE_CLI_SOCKET:-}" ]] || socket_path=$(awk -F= '/^[[:space:]]*CLI_SOCKET_PATH[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
[[ -n "${BIN_DIRECTORY:-}" ]] || bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
invocation_ms=$(awk -F= '/^[[:space:]]*WASM_INVOCATION_TIMEOUT_MS[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' "$settings_file")
cli_worker_count=$(awk -F= '/^[[:space:]]*CLI_WORKER_COUNT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); value=$2} END{print value}' "$settings_file")
fi
site_directory="${site_directory:-site}"
socket_path="${socket_path:-/run/uce/cli.sock}"
bin_directory="${bin_directory:-/tmp/uce/work}"
invocation_ms="${invocation_ms:-30000}"
cli_worker_count="${cli_worker_count:-0}"
curl_transport=(--unix-socket "$socket_path")
test_base_url="http://localhost"
if (( cli_worker_count > 0 )); then
test_http_host="${UCE_TEST_HTTP_HOST:-uce.openfu.com}"
curl_transport=(--resolve "$test_http_host:80:127.0.0.1")
test_base_url="http://$test_http_host"
fi
test_name="invocation-timeout-test-$$"
source_dir="$site_directory/$test_name"
pid_file="/tmp/uce-$test_name-worker"
cache_dir=""
body_file="/tmp/uce-$test_name-body"
cleanup() {
rm -rf "$source_dir"
[[ -z "$cache_dir" ]] || rm -rf "$cache_dir"
rm -f "$pid_file" "$body_file"
}
trap cleanup EXIT
mkdir -p "$source_dir"
cache_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'void timeout_test_run(Request& context) {' \
' if(context.get["warm"] == "1") { print("warm"); return; }' \
" file_put_contents(\"$pid_file\", std::to_string(request_perf()[\"worker_pid\"].to_u64()));" \
' while(true) time_precise();' \
'}' \
'CLI(Request& context) { timeout_test_run(context); }' \
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/hostcall-loop.uce"
printf '%s\n' \
'void timeout_test_run(Request& context) {' \
' if(context.get["warm"] == "1") { print("warm"); return; }' \
' if(context.get["quick"] == "1") { print(shell_exec("printf quick")); return; }' \
' if(context.get["status"] == "1") { DValue spec; spec["cmd"] = "exit 7"; spec["timeout_ms"] = (f64)500; print(shell_exec(spec)["exit_code"].to_u64()); return; }' \
' if(context.get["zero"] == "1") { DValue spec; spec["cmd"] = "printf zero"; spec["timeout_ms"] = (f64)0; print(shell_exec(spec)["stdout"].to_string()); return; }' \
' if(context.get["job"] == "1") { DValue spec; spec["cmd"] = "sleep 2"; spec["timeout_ms"] = (f64)5000; u64 job = shell_spawn(spec); f64 started = time_precise(); job_await(job, 300); u64 elapsed = (u64)((time_precise() - started) * 1000); job_cancel(job); print(elapsed); return; }' \
" file_put_contents(\"$pid_file\", std::to_string(request_perf()[\"worker_pid\"].to_u64()));" \
' print(shell_exec("printf shell-start; sleep 60 & printf shell-end"));' \
'}' \
'CLI(Request& context) { timeout_test_run(context); }' \
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/legacy-shell.uce"
printf '%s\n' \
'void timeout_test_run(Request& context) {' \
' if(context.get["warm"] == "1") { print("warm"); return; }' \
" file_put_contents(\"$pid_file\", std::to_string(request_perf()[\"worker_pid\"].to_u64()));" \
' while(true) sleep(60);' \
' print("sleep-end");' \
'}' \
'CLI(Request& context) { timeout_test_run(context); }' \
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/sleep.uce"
printf '%s\n' \
'void timeout_test_run(Request& context) { print(request_perf()["worker_pid"].to_u64(), "|health"); }' \
'CLI(Request& context) { timeout_test_run(context); }' \
'RENDER(Request& context) { timeout_test_run(context); }' >"$source_dir/health.uce"
max_seconds=$(( (invocation_ms + 15000) / 1000 ))
(( max_seconds >= 10 )) || max_seconds=10
request() {
local unit="$1"
curl -sS --max-time "$max_seconds" -o "$body_file" -w '%{http_code}' "${curl_transport[@]}" "$test_base_url/$test_name/$unit"
}
same_worker_health() {
local expected_pid="$1"
kill -0 "$expected_pid" 2>/dev/null || return 1
local health
health=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/health.uce")
[[ "$health" =~ ^[0-9]+\|health$ ]]
}
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/hostcall-loop.uce?warm=1" >/dev/null
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?warm=1" >/dev/null
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/sleep.uce?warm=1" >/dev/null
curl -sS --max-time "$max_seconds" --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/health.uce" >/dev/null
quick=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?quick=1")
[[ "$quick" == quick ]] || { echo "quick legacy shell returned: $quick" >&2; exit 1; }
exit_status=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?status=1")
[[ "$exit_status" == 7 ]] || { echo "structured shell lost exit status: $exit_status" >&2; exit 1; }
zero_timeout=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?zero=1")
[[ "$zero_timeout" == zero ]] || { echo "structured shell zero-timeout default changed: $zero_timeout" >&2; exit 1; }
job_elapsed=$(curl -sS --max-time 5 --fail-with-body "${curl_transport[@]}" "$test_base_url/$test_name/legacy-shell.uce?job=1")
[[ "$job_elapsed" =~ ^[0-9]+$ && "$job_elapsed" -ge 200 && "$job_elapsed" -lt 500 ]] || { echo "job_await two-call duration was ${job_elapsed}ms" >&2; exit 1; }
for unit in hostcall-loop.uce legacy-shell.uce sleep.uce; do
rm -f "$pid_file"
started_ns=$(date +%s%N)
status=$(request "$unit")
elapsed_ms=$(( ($(date +%s%N) - started_ns) / 1000000 ))
[[ "$status" == "500" ]] || { echo "$unit returned HTTP $status" >&2; exit 1; }
if [[ "$unit" == legacy-shell.uce ]]; then
if (( invocation_ms <= 5000 )); then
grep -q 'UCE_INVOCATION_TIMEOUT:' "$body_file" || { echo "$unit lacked invocation-timeout classification" >&2; exit 1; }
expected_ms=$invocation_ms
else
grep -q 'UCE_HOSTCALL_TIMEOUT:' "$body_file" || { echo "$unit lacked hostcall-timeout classification" >&2; exit 1; }
expected_ms=5000
fi
else
grep -q 'UCE_INVOCATION_TIMEOUT:' "$body_file" || { echo "$unit lacked invocation-timeout classification: $(cat "$body_file")" >&2; exit 1; }
expected_ms=$invocation_ms
fi
(( elapsed_ms >= expected_ms - 1000 && elapsed_ms <= expected_ms + 3000 )) || { echo "$unit completed in ${elapsed_ms}ms; expected about ${expected_ms}ms" >&2; exit 1; }
grep -q "$unit" "$body_file" || { echo "$unit lacked a source-mapped trace" >&2; exit 1; }
[[ -s "$pid_file" ]] || { echo "$unit did not record its worker" >&2; exit 1; }
worker_pid=$(<"$pid_file")
same_worker_health "$worker_pid" || { echo "worker $worker_pid did not survive $unit" >&2; exit 1; }
done
echo "wasm invocation timeout passed (absolute ${invocation_ms}ms; sleep and process hostcalls bounded)"
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="wasm-metadata-buffer-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
bin_directory="${UCE_TEST_BIN_DIRECTORY:-/tmp/uce/work}"
if [[ -r /etc/uce/settings.cfg ]]; then
if [[ -z "${UCE_TEST_SITE_DIRECTORY:-}" ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*HTTP_DOCUMENT_ROOT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
site_directory="${configured_site_directory:-$site_directory}"
fi
if [[ -z "${UCE_TEST_BIN_DIRECTORY:-}" ]]; then
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
bin_directory="${configured_bin_directory:-$bin_directory}"
fi
fi
source_dir="$site_directory/$test_name"
artifact_dir=""
cleanup() {
rm -rf "$source_dir"
if [[ -n "$artifact_dir" ]]; then
rm -rf "$artifact_dir"
fi
}
trap cleanup EXIT
mkdir -p "$source_dir"
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$(realpath "$source_dir")"
printf '%s\n' \
'CLI(Request& context) {' \
' String target = context.get["unit"];' \
' String rendered = component(target, context);' \
' DValue perf = request_perf();' \
' DValue selected;' \
' perf["unit_module_operations"].each([&](DValue operation, String key) {' \
' if(operation["kind"].to_string() == "component" && contains(operation["unit"].to_string(), target + ".uce")) selected = operation;' \
' });' \
' print(rendered, "\t", selected["source"].to_string(), "\t", selected["read_count"].to_string(), "\t", selected["read_bytes"].to_string(), "\t", selected["read_us"].to_string());' \
'}' >"$source_dir/parent.uce"
payload=$(head -c 131072 /dev/zero | tr '\0' x)
for unit in $(seq 0 7); do
printf 'String metadata_payload_%s() { return("%s"); }\nCOMPONENT(Request& context) { String payload = metadata_payload_%s(); u64 offset = std::atoi(context.get["offset"].c_str()) %% payload.size(); print(std::to_string(payload.size()), ":", payload.substr(offset, 1)); }\n' \
"$unit" "$payload" "$unit" >"$source_dir/component-$unit.uce"
done
unset payload
declare -a read_times=()
for unit in $(seq 0 7); do
found=0
for attempt in $(seq 1 20); do
output=$(timeout --signal=TERM --kill-after=2s 30s scripts/uce-cli "/$test_name/parent.uce?unit=component-$unit&offset=$attempt")
IFS=$'\t' read -r marker source read_count read_bytes read_us <<<"$output"
if [[ "$marker" != "131072:x" ]]; then
echo "metadata buffer component failed: unit=$unit output=$output" >&2
exit 1
fi
if [[ "$source" != "serialized" ]]; then
continue
fi
wasm_size=$(stat -c %s "$artifact_dir/component-$unit.uce.wasm")
if ! awk -v count="$read_count" -v bytes="$read_bytes" -v size="$wasm_size" \
'BEGIN { exit !(count > 0 && count <= 64 && bytes > 0 && bytes <= count * 4096 && bytes < size) }'; then
echo "metadata buffer read was not bounded: unit=$unit reads=$read_count bytes=$read_bytes wasm=$wasm_size us=$read_us" >&2
exit 1
fi
read_times+=("$read_us")
found=1
break
done
if (( found == 0 )); then
echo "metadata buffer did not observe a serialized cold-worker load for component-$unit" >&2
exit 1
fi
done
mapfile -t sorted < <(printf '%s\n' "${read_times[@]}" | sort -n)
median=${sorted[$(( ${#sorted[@]} / 2 ))]}
maximum=${sorted[$(( ${#sorted[@]} - 1 ))]}
if ! awk -v median="$median" 'BEGIN { exit !(median < 3000) }'; then
echo "metadata buffer median exceeded 3ms: median=${median}us maximum=${maximum}us samples=${read_times[*]}" >&2
exit 1
fi
echo "wasm metadata buffer passed: ${#read_times[@]} cold serialized loads, median ${median}us, maximum ${maximum}us"
+474
View File
@@ -0,0 +1,474 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
if [[ "${1:-}" != "--inside" ]]; then
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
fi
root="/tmp/uce-metadata-deadline-$$"
site="$root/site"
work="$root/work"
settings="$root/settings.cfg"
socket="$root/run/cli.sock"
log="$root/service.log"
control="$root/pread.control"
server_pid=""
cleanup() {
status=$?
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
kill -TERM "$server_pid" 2>/dev/null || true
for _ in $(seq 1 100); do kill -0 "$server_pid" 2>/dev/null || break; sleep 0.02; done
kill -KILL "$server_pid" 2>/dev/null || true
wait "$server_pid" 2>/dev/null || true
fi
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
rm -rf "$root"
return "$status"
}
trap cleanup EXIT
mkdir -p "$site/components" "$work" "$root/run" "$root/session" "$root/upload"
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|CLI_WORKER_COUNT|CLI_WORKER_MAX_REQUESTS|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|WASM_INVOCATION_TIMEOUT_MS|WASM_EPOCH_PERIOD_MS|PROACTIVE_COMPILE_ENABLED|SERVE_LAST_KNOWN_GOOD|SHOW_DYNAMIC_COMPILE_ERRORS|WORKER_COUNT)[[:space:]]*=/d' \
/etc/uce/settings.cfg >"$settings"
cat >>"$settings" <<CFG
BIN_DIRECTORY=$work
PRECOMPILE_FILES_IN=$site
SITE_DIRECTORY=$site
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
FCGI_PORT=
CLI_SOCKET_PATH=$socket
CLI_WORKER_COUNT=1
CLI_WORKER_MAX_REQUESTS=0
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
HTTP_PORT=
HTTP_DOCUMENT_ROOT=$site
SESSION_PATH=$root/session
TMP_UPLOAD_PATH=$root/upload
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
WASM_INVOCATION_TIMEOUT_MS=1000
WASM_EPOCH_PERIOD_MS=10
PROACTIVE_COMPILE_ENABLED=0
SERVE_LAST_KNOWN_GOOD=0
SHOW_DYNAMIC_COMPILE_ERRORS=1
WORKER_COUNT=1
CFG
mount --bind "$settings" /etc/uce/settings.cfg
cat >"$site/driver.uce" <<'UCE'
CLI(Request& context) {
if(context.get["health"] == "1") { print(request_perf()["worker_pid"].to_u64(), "|health"); return; }
print(component(context.get["target"], context));
}
UCE
printf '%s\n' 'COMPONENT(Request& context) { print("deadline-component"); }' >"$site/components/deadline.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("truncate-component"); }' >"$site/components/truncate.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("mutation-component"); }' >"$site/components/mutation.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("uleb-component"); }' >"$site/components/uleb.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("duplicate-component"); }' >"$site/components/duplicate.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("full-component"); }' >"$site/components/full.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("full-uleb-component"); }' >"$site/components/full-uleb.uce"
printf '%s\n' 'COMPONENT(Request& context) { print("full-duplicate-component"); }' >"$site/components/full-duplicate.uce"
timeout --signal=TERM --kill-after=5s 60s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" \
UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >"$root/precompile.log" 2>&1
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
for unit in deadline truncate mutation uleb duplicate full full-uleb full-duplicate; do
[[ -s "$cache/components/$unit.uce.wasm" && -s "$cache/components/$unit.uce.cwasm" ]]
cp "$cache/components/$unit.uce.wasm" "$root/$unit.original.wasm"
done
cat >"$root/interpose.c" <<'C'
#define _GNU_SOURCE
#define _LARGEFILE64_SOURCE
#include <dlfcn.h>
#include <errno.h>
#include <fcntl.h>
#include <limits.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <time.h>
#include <unistd.h>
struct control {
char target[PATH_MAX];
char mode[16];
int delay_ms;
char ready[PATH_MAX];
char release[PATH_MAX];
char counter[PATH_MAX];
};
static int read_control(int fd, struct control* control)
{
const char* path = getenv("UCE_TEST_PREAD_CONTROL");
if(!path || !*path) return 0;
int input = open(path, O_RDONLY | O_CLOEXEC);
if(input < 0) return 0;
char text[PATH_MAX * 4];
ssize_t count = read(input, text, sizeof(text) - 1);
close(input);
if(count <= 0) return 0;
text[count] = 0;
if(sscanf(text, "%4095s %15s %d %4095s %4095s %4095s", control->target, control->mode,
&control->delay_ms, control->ready, control->release, control->counter) != 6) return 0;
char link[64], resolved[PATH_MAX];
snprintf(link, sizeof(link), "/proc/self/fd/%d", fd);
ssize_t length = readlink(link, resolved, sizeof(resolved) - 1);
if(length <= 0) return 0;
resolved[length] = 0;
return strcmp(resolved, control->target) == 0;
}
static void mark(const char* path)
{
if(strcmp(path, "-") == 0) return;
int fd = open(path, O_WRONLY | O_CREAT | O_APPEND | O_CLOEXEC, 0600);
if(fd >= 0) { ssize_t ignored = write(fd, "1\n", 2); (void)ignored; close(fd); }
}
static void wait_release(const struct control* control)
{
if(strcmp(control->release, "-") == 0) return;
struct timespec started, now, pause = { 0, 10000000 };
clock_gettime(CLOCK_MONOTONIC, &started);
while(access(control->release, F_OK) != 0)
{
nanosleep(&pause, 0);
clock_gettime(CLOCK_MONOTONIC, &now);
if(now.tv_sec - started.tv_sec >= 2) break;
}
}
static void delay_ms(int milliseconds)
{
if(milliseconds <= 0) return;
struct timespec pause = { milliseconds / 1000, (milliseconds % 1000) * 1000000L };
while(nanosleep(&pause, &pause) != 0 && errno == EINTR) {}
}
static void before_read(const struct control* control)
{
mark(control->counter);
if(strcmp(control->mode, "before") == 0 && access(control->ready, F_OK) != 0)
{
mark(control->ready);
wait_release(control);
}
if(strcmp(control->mode, "delay") == 0) delay_ms(control->delay_ms);
}
static void after_read(const struct control* control)
{
if(strcmp(control->mode, "after") == 0 && access(control->ready, F_OK) != 0)
{
mark(control->ready);
wait_release(control);
}
}
ssize_t pread(int fd, void* buffer, size_t size, off_t offset)
{
static ssize_t (*real_pread)(int, void*, size_t, off_t);
if(!real_pread) real_pread = dlsym(RTLD_NEXT, "pread");
struct control control;
int targeted = read_control(fd, &control);
if(targeted) before_read(&control);
ssize_t result = real_pread(fd, buffer, size, offset);
if(targeted) after_read(&control);
return result;
}
ssize_t pread64(int fd, void* buffer, size_t size, off64_t offset)
{
static ssize_t (*real_pread64)(int, void*, size_t, off64_t);
if(!real_pread64) real_pread64 = dlsym(RTLD_NEXT, "pread64");
struct control control;
int targeted = read_control(fd, &control);
if(targeted) before_read(&control);
ssize_t result = real_pread64(fd, buffer, size, offset);
if(targeted) after_read(&control);
return result;
}
C
timeout 15s cc -shared -fPIC -O2 -Wall -Wextra -Werror -o "$root/interpose.so" "$root/interpose.c" -ldl
timeout --signal=TERM --kill-after=5s 90s env LD_PRELOAD="$root/interpose.so" UCE_TEST_PREAD_CONTROL="$control" \
bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
server_pid=$!
for _ in $(seq 1 400); do [[ -S "$socket" ]] && break; sleep 0.02; done
[[ -S "$socket" ]] || { cat "$log" >&2; exit 1; }
request() {
timeout --signal=TERM --kill-after=1s 5s scripts/uce-cli --socket "$socket" --get /driver.uce "$@"
}
health=$(request health=1)
worker_pid=${health%%|*}
[[ "$health" == "$worker_pid|health" ]]
restore() {
local unit="$1" wasm="$cache/components/$1.uce.wasm" cwasm="$cache/components/$1.uce.cwasm"
cp "$root/$unit.original.wasm" "$wasm"
python3 - "$wasm" "$cwasm" <<'PY'
import os, pathlib, sys
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
stamp = cwasm.stat().st_mtime_ns - 1_000_000
os.utime(wasm, ns=(stamp, stamp))
PY
}
prepare_deadline() {
local wasm="$cache/components/deadline.uce.wasm" cwasm="$cache/components/deadline.uce.cwasm"
python3 - "$wasm" "$cwasm" <<'PY'
import os, pathlib, sys
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
with wasm.open("ab") as output:
output.write(b"\0\1\0" * 7000)
stamp = cwasm.stat().st_mtime_ns - 1_000_000
os.utime(wasm, ns=(stamp, stamp))
PY
}
prepare_mutation() {
local wasm="$cache/components/mutation.uce.wasm" cwasm="$cache/components/mutation.uce.cwasm"
python3 - "$wasm" "$cwasm" "$root/mutation-offset" <<'PY'
import os, pathlib, sys
wasm, cwasm, offset_file = map(pathlib.Path, sys.argv[1:])
payload = bytes([7]) + b"padding" + bytes(8192)
value = len(payload)
encoded = bytearray()
while True:
byte = value & 0x7f
value >>= 7
encoded.append(byte | (0x80 if value else 0))
if not value:
break
with wasm.open("ab") as output:
output.write(bytes([0]) + encoded + payload)
offset_file.write_text(str(wasm.stat().st_size - 4096))
stamp = cwasm.stat().st_mtime_ns - 1_000_000
os.utime(wasm, ns=(stamp, stamp))
PY
}
wait_ready() {
local ready="$1" request_pid="$2" deadline=$((SECONDS + 3))
while [[ ! -e "$ready" ]] && kill -0 "$request_pid" 2>/dev/null && (( SECONDS < deadline )); do sleep 0.01; done
[[ -e "$ready" ]] || { echo "metadata read gate did not activate" >&2; return 1; }
}
prepare_deadline
deadline_wasm="$cache/components/deadline.uce.wasm"
deadline_counter="$root/deadline.counter"
printf '%s delay 1100 - - %s\n' "$deadline_wasm" "$deadline_counter" >"$control"
started=$(date +%s%N)
set +e
deadline_output=$(request target=components/deadline.uce 2>&1)
set -e
elapsed_ms=$(( ($(date +%s%N) - started) / 1000000 ))
[[ "$deadline_output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "metadata scan lacked canonical timeout: $deadline_output" >&2; exit 1; }
(( elapsed_ms >= 1000 && elapsed_ms < 3000 )) || { echo "metadata timeout took ${elapsed_ms}ms" >&2; exit 1; }
[[ $(wc -l <"$deadline_counter") -eq 1 ]] || { echo "metadata scanner read past the first delayed refill" >&2; exit 1; }
rm -f "$control"
[[ "$(request health=1)" == "$worker_pid|health" ]]
restore deadline
[[ "$(request target=components/deadline.uce)" == deadline-component ]]
rm -f "$cache/components/deadline.uce.cwasm"
touch "$deadline_wasm"
printf '%s delay 1100 - - %s\n' "$deadline_wasm" "$root/full-deadline.counter" >"$control"
started=$(date +%s%N)
set +e
full_deadline_output=$(request target=components/deadline.uce 2>&1)
set -e
elapsed_ms=$(( ($(date +%s%N) - started) / 1000000 ))
[[ "$full_deadline_output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "full artifact read lacked canonical timeout: $full_deadline_output" >&2; exit 1; }
(( elapsed_ms >= 1000 && elapsed_ms < 3000 )) || { echo "full artifact timeout took ${elapsed_ms}ms" >&2; exit 1; }
[[ $(wc -l <"$root/full-deadline.counter") -eq 1 ]] || { echo "full artifact reader continued after the first delayed chunk" >&2; exit 1; }
rm -f "$control"
[[ "$(request health=1)" == "$worker_pid|health" ]]
[[ "$(request target=components/deadline.uce)" == deadline-component ]]
[[ -s "$cache/components/deadline.uce.cwasm" ]]
truncate_wasm="$cache/components/truncate.uce.wasm"
truncate_ready="$root/truncate.ready"
truncate_release="$root/truncate.release"
printf '%s before 0 %s %s %s\n' "$truncate_wasm" "$truncate_ready" "$truncate_release" "$root/truncate.counter" >"$control"
( set +e; request target=components/truncate.uce >"$root/truncate.output" 2>&1; printf '%s\n' "$?" >"$root/truncate.status" ) &
request_pid=$!
wait_ready "$truncate_ready" "$request_pid"
truncate -s 4 "$truncate_wasm"
touch "$truncate_release"
wait "$request_pid"
[[ $(<"$root/truncate.status") -ne 0 ]]
grep -q '\[wasm\] component load failed: not a supported wasm module' "$log"
! grep -q 'truncate-component' "$root/truncate.output"
rm -f "$control"
[[ "$(request health=1)" == "$worker_pid|health" ]]
restore truncate
[[ "$(request target=components/truncate.uce)" == truncate-component ]]
prepare_mutation
mutation_wasm="$cache/components/mutation.uce.wasm"
mutation_ready="$root/mutation.ready"
mutation_release="$root/mutation.release"
printf '%s after 0 %s %s %s\n' "$mutation_wasm" "$mutation_ready" "$mutation_release" "$root/mutation.counter" >"$control"
( set +e; request target=components/mutation.uce >"$root/mutation.output" 2>&1; printf '%s\n' "$?" >"$root/mutation.status" ) &
request_pid=$!
wait_ready "$mutation_ready" "$request_pid"
python3 - "$mutation_wasm" "$(<"$root/mutation-offset")" <<'PY'
import os, pathlib, sys
path = pathlib.Path(sys.argv[1])
offset = int(sys.argv[2])
with path.open("r+b", buffering=0) as artifact:
artifact.seek(offset)
byte = artifact.read(1)
artifact.seek(offset)
artifact.write(bytes([byte[0] ^ 1]))
os.fsync(artifact.fileno())
PY
touch "$mutation_release"
wait "$request_pid"
[[ $(<"$root/mutation.status") -ne 0 ]]
grep -q '\[wasm\] component load failed: wasm artifact changed while loading metadata' "$log"
! grep -q 'mutation-component' "$root/mutation.output"
rm -f "$control"
[[ "$(request health=1)" == "$worker_pid|health" ]]
restore mutation
[[ "$(request target=components/mutation.uce)" == mutation-component ]]
uleb_wasm="$cache/components/uleb.uce.wasm"
uleb_cwasm="$cache/components/uleb.uce.cwasm"
python3 - "$uleb_wasm" "$uleb_cwasm" <<'PY'
import os, pathlib, sys
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
with wasm.open("r+b", buffering=0) as artifact:
artifact.seek(9)
artifact.write(bytes([0x80]) * 9 + bytes([0x02]))
os.fsync(artifact.fileno())
stamp = cwasm.stat().st_mtime_ns - 1_000_000
os.utime(wasm, ns=(stamp, stamp))
PY
set +e
uleb_output=$(request target=components/uleb.uce 2>&1)
set -e
grep -q '\[wasm\] component load failed: malformed wasm section header' "$log"
[[ "$uleb_output" != *uleb-component* ]]
[[ "$(request health=1)" == "$worker_pid|health" ]]
restore uleb
[[ "$(request target=components/uleb.uce)" == uleb-component ]]
duplicate_wasm="$cache/components/duplicate.uce.wasm"
duplicate_cwasm="$cache/components/duplicate.uce.cwasm"
python3 - "$duplicate_wasm" "$duplicate_cwasm" <<'PY'
import os, pathlib, sys
wasm, cwasm = map(pathlib.Path, sys.argv[1:])
payload = bytes([10]) + b"uce.module" + b"duplicate"
value = len(payload)
encoded = bytearray()
while True:
byte = value & 0x7f
value >>= 7
encoded.append(byte | (0x80 if value else 0))
if not value:
break
with wasm.open("ab") as artifact:
artifact.write(bytes([0]) + encoded + payload)
stamp = cwasm.stat().st_mtime_ns - 1_000_000
os.utime(wasm, ns=(stamp, stamp))
PY
set +e
duplicate_output=$(request target=components/duplicate.uce 2>&1)
set -e
grep -q '\[wasm\] component load failed: duplicate uce.module metadata section' "$log"
[[ "$duplicate_output" != *duplicate-component* ]]
[[ "$(request health=1)" == "$worker_pid|health" ]]
restore duplicate
[[ "$(request target=components/duplicate.uce)" == duplicate-component ]]
full_wasm="$cache/components/full.uce.wasm"
full_cwasm="$cache/components/full.uce.cwasm"
python3 - "$full_wasm" <<'PY'
import pathlib, sys
wasm = pathlib.Path(sys.argv[1])
payload = bytes([7]) + b"uce.abi" + bytes(1024 * 1024)
value = len(payload)
encoded = bytearray()
while True:
byte = value & 0x7f
value >>= 7
encoded.append(byte | (0x80 if value else 0))
if not value:
break
with wasm.open("ab") as artifact:
artifact.write(bytes([0]) + encoded + payload)
PY
rm -f "$full_cwasm"
set +e
full_output=$(request target=components/full.uce 2>&1)
set -e
grep -Eq '\[wasm\] component load failed: .*oversized wasm metadata section' "$log"
[[ "$full_output" != *full-component* ]]
[[ "$(request health=1)" == "$worker_pid|health" ]]
cp "$root/full.original.wasm" "$full_wasm"
[[ "$(request target=components/full.uce)" == full-component ]]
[[ -s "$full_cwasm" ]]
full_uleb_wasm="$cache/components/full-uleb.uce.wasm"
full_uleb_cwasm="$cache/components/full-uleb.uce.cwasm"
python3 - "$full_uleb_wasm" <<'PY'
import os, pathlib, sys
wasm = pathlib.Path(sys.argv[1])
with wasm.open("r+b", buffering=0) as artifact:
artifact.seek(9)
artifact.write(bytes([0x80]) * 9 + bytes([0x02]))
os.fsync(artifact.fileno())
PY
rm -f "$full_uleb_cwasm"
malformed_before=$(grep -c 'malformed wasm section header' "$log" || true)
set +e
full_uleb_output=$(request target=components/full-uleb.uce 2>&1)
set -e
malformed_after=$(grep -c 'malformed wasm section header' "$log" || true)
(( malformed_after == malformed_before + 1 ))
[[ "$full_uleb_output" != *full-uleb-component* ]]
[[ "$(request health=1)" == "$worker_pid|health" ]]
cp "$root/full-uleb.original.wasm" "$full_uleb_wasm"
[[ "$(request target=components/full-uleb.uce)" == full-uleb-component ]]
[[ -s "$full_uleb_cwasm" ]]
full_duplicate_wasm="$cache/components/full-duplicate.uce.wasm"
full_duplicate_cwasm="$cache/components/full-duplicate.uce.cwasm"
python3 - "$full_duplicate_wasm" <<'PY'
import pathlib, sys
wasm = pathlib.Path(sys.argv[1])
payload = bytes([10]) + b"uce.module" + b"duplicate"
value = len(payload)
encoded = bytearray()
while True:
byte = value & 0x7f
value >>= 7
encoded.append(byte | (0x80 if value else 0))
if not value:
break
with wasm.open("ab") as artifact:
artifact.write(bytes([0]) + encoded + payload)
PY
rm -f "$full_duplicate_cwasm"
duplicate_before=$(grep -c 'duplicate uce.module metadata section' "$log" || true)
set +e
full_duplicate_output=$(request target=components/full-duplicate.uce 2>&1)
set -e
duplicate_after=$(grep -c 'duplicate uce.module metadata section' "$log" || true)
(( duplicate_after == duplicate_before + 1 ))
[[ "$full_duplicate_output" != *full-duplicate-component* ]]
[[ "$(request health=1)" == "$worker_pid|health" ]]
cp "$root/full-duplicate.original.wasm" "$full_duplicate_wasm"
[[ "$(request target=components/full-duplicate.uce)" == full-duplicate-component ]]
[[ -s "$full_duplicate_cwasm" ]]
! grep -Eiq 'panic|segfault|permission denied' "$log"
echo "wasm metadata deadline passed: streamed and full-read timeouts, mutation guards, and parser bounds passed; same worker recovered"
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
test_name="wasm-source-location-test-$$"
site_directory="${UCE_TEST_SITE_DIRECTORY:-site}"
bin_directory="${UCE_TEST_BIN_DIRECTORY:-/tmp/uce/work}"
if [[ -r /etc/uce/settings.cfg ]]; then
configured_site_directory=$(awk -F= '/^[[:space:]]*HTTP_DOCUMENT_ROOT[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
configured_bin_directory=$(awk -F= '/^[[:space:]]*BIN_DIRECTORY[[:space:]]*=/ {gsub(/^[[:space:]]+|[[:space:]]+$/, "", $2); print $2; exit}' /etc/uce/settings.cfg)
[[ -n "${UCE_TEST_SITE_DIRECTORY:-}" ]] || site_directory="${configured_site_directory:-$site_directory}"
[[ -n "${UCE_TEST_BIN_DIRECTORY:-}" ]] || bin_directory="${configured_bin_directory:-$bin_directory}"
fi
source_dir="$site_directory/$test_name"
absolute_source_dir=""
artifact_dir=""
cleanup() {
rm -rf "$source_dir"
[[ -z "$artifact_dir" ]] || rm -rf "$artifact_dir"
}
trap cleanup EXIT
mkdir -p "$source_dir"
absolute_source_dir=$(realpath "$source_dir")
artifact_dir="$(scripts/unit_cache_directory "$bin_directory")$absolute_source_dir"
printf '%s\n' 'CLI(Request& context) { __builtin_trap(); }' >"$source_dir/entry.uce"
set +e
output=$(scripts/uce-cli "/$test_name/entry.uce" 2>&1)
status=$?
set -e
if [[ $status -eq 0 ]]; then
echo "trapping wasm unit unexpectedly succeeded" >&2
exit 1
fi
expected="$absolute_source_dir/entry.uce:1:25"
if [[ "$output" != *'wasm `unreachable` instruction executed'* || "$output" != *"source locations:"* || "$output" != *"$expected"* ]]; then
echo "wasm trap omitted its source location:" >&2
echo "$output" >&2
exit 1
fi
rm "$artifact_dir/entry.uce.wasm.source-map"
set +e
without_map=$(scripts/uce-cli "/$test_name/entry.uce" 2>&1)
status=$?
set -e
if [[ $status -eq 0 || "$without_map" != *'wasm `unreachable` instruction executed'* || "$without_map" == *"source locations:"* ]]; then
echo "missing-map trap fallback failed:" >&2
echo "$without_map" >&2
exit 1
fi
echo "wasm source locations passed"
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env bash
set -euo pipefail
cd "$(dirname "$0")/.."
base="${1:-/tmp/uce/work}"
compiler_abi=$(awk '/^#define UCE_COMPILER_UNIT_ABI_VERSION / {print $3; exit}' src/wasm/abi.h)
core_abi=$(awk '/^#define UCE_WASM_CORE_ABI_VERSION / {print $3; exit}' src/wasm/abi.h)
printf '%s/units-c%s-w%s\n' "${base%/}" "$compiler_abi" "$core_abi"
+5
View File
@@ -5,3 +5,8 @@ EXPORT DValue* test_func(DValue* call_param)
print("HELLO FROM TEST FUNCTION"); print("HELLO FROM TEST FUNCTION");
return(0); return(0);
} }
RENDER(Request& context)
{
print("HELLO FROM UNIT RENDER");
}
+6 -7
View File
@@ -6,9 +6,9 @@ RENDER(Request& context)
p.set(context.params); p.set(context.params);
StringList routes = {"index", "dashboard", "themes", "dashboard", "workspace/projects"}; StringList routes = {"index", "dashboard", "themes", "dashboard", "workspace/projects"};
StringList unique_routes = list_unique(routes); StringList unique_routes = routes.unique();
StringList sorted_routes = list_sort(unique_routes); StringList sorted_routes = unique_routes.sort();
StringList labels = map(sorted_routes, [](String route) { return(to_upper(replace(route, "/", " / "))); }); StringList labels = sorted_routes.map([](String route) { return(to_upper(replace(route, "/", " / "))); });
DValue cards; DValue cards;
DValue card; DValue card;
@@ -16,9 +16,8 @@ RENDER(Request& context)
card["title"] = "Themes"; card["section"] = "app"; card["href"] = "?themes"; cards.push(card); card.clear(); card["title"] = "Themes"; card["section"] = "app"; card["href"] = "?themes"; cards.push(card); card.clear();
card["title"] = "Docs"; card["section"] = "reference"; card["href"] = "../doc/index.uce"; cards.push(card); card["title"] = "Docs"; card["section"] = "reference"; card["href"] = "../doc/index.uce"; cards.push(card);
DValue app_cards = dv_filter(cards, [](DValue item, String key) { return(item["section"].to_string() == "app"); }); DValue app_cards = cards.filter([](DValue item, String key) { return(item["section"].to_string() == "app"); });
DValue titles = dv_map(app_cards, [](DValue item, String key) { DValue out; out = item["title"].to_string(); return(out); }); DValue titles = app_cards.map([](DValue item, String key) { DValue out; out = item["title"].to_string(); return(out); });
DValue grouped = dv_group_by(cards, [](DValue item, String key) { return(item["section"].to_string()); });
<><html> <><html>
<head> <head>
@@ -31,7 +30,7 @@ RENDER(Request& context)
<p>Small data-shaping helpers are useful for route lists, nav records, cards, and other render-adjacent structures.</p> <p>Small data-shaping helpers are useful for route lists, nav records, cards, and other render-adjacent structures.</p>
<div class="system-info"><h3>StringList route labels</h3><pre><?= join(labels, "\n") ?></pre></div> <div class="system-info"><h3>StringList route labels</h3><pre><?= join(labels, "\n") ?></pre></div>
<div class="system-info"><h3>DValue app card titles</h3><pre><?= json_encode(titles) ?></pre></div> <div class="system-info"><h3>DValue app card titles</h3><pre><?= json_encode(titles) ?></pre></div>
<div class="system-info"><h3>Grouped cards</h3><pre><?= json_encode(grouped) ?></pre></div> <div class="system-info"><h3>All card values</h3><pre><?= json_encode(cards.values()) ?></pre></div>
<details> <details>
<summary>Request Parameters</summary> <summary>Request Parameters</summary>
<pre><?= var_dump(p) ?></pre> <pre><?= var_dump(p) ?></pre>
+16 -2
View File
@@ -91,8 +91,22 @@ RENDER(Request& context)
print("Accept us: ", (f64)perf["accept_us"].to_f64(), "\n"); print("Accept us: ", (f64)perf["accept_us"].to_f64(), "\n");
print("Running us: ", (f64)perf["running_us"].to_f64(), "\n"); print("Running us: ", (f64)perf["running_us"].to_f64(), "\n");
print("Total us: ", (f64)perf["total_us"].to_f64(), "\n"); print("Total us: ", (f64)perf["total_us"].to_f64(), "\n");
if(perf["workspace_birth_us"].type != 'S') print("Dispatch us: ", (u64)perf["dispatch_us"].to_u64(), "\n");
print("Workspace birth us: ", (u64)perf["workspace_birth_us"].to_u64(), "\n"); print("Workspace setup us: ", (u64)perf["workspace_setup_us"].to_u64(), "\n");
print("Workspace birth us: ", (u64)perf["workspace_birth_us"].to_u64(), "\n");
print("Context apply us: ", (u64)perf["context_apply_us"].to_u64(), "\n");
print("Guest us: ", (f64)perf["guest_us"].to_f64(), "\n");
print("Hostcalls: ", (u64)perf["hostcall_count"].to_u64(), " / ", (u64)perf["hostcall_us"].to_u64(), " us\n");
print("MySQL hostcalls: ", (u64)perf["mysql_hostcall_count"].to_u64(), " / ", (u64)perf["mysql_hostcall_us"].to_u64(), " us\n");
print("MySQL operations: ", json_encode(perf["mysql_operations"]), " / dropped ", (u64)perf["mysql_operations_dropped"].to_u64(), "\n");
print("MySQL connections: ", (u64)perf["mysql_connection_open_count"].to_u64(), " new / ", (u64)perf["mysql_connection_reuse_count"].to_u64(), " worker reuse / ", (u64)perf["mysql_request_pool_hit_count"].to_u64(), " request reuse\n");
print("Memcache hostcalls: ", (u64)perf["memcache_hostcall_count"].to_u64(), " / ", (u64)perf["memcache_hostcall_us"].to_u64(), " us\n");
print("Component resolves: ", (u64)perf["component_resolve_count"].to_u64(), " / ", (u64)perf["component_resolve_us"].to_u64(), " us\n");
print("Component phases: path ", (u64)perf["component_path_us"].to_u64(), " / artifact ", (u64)perf["component_artifact_us"].to_u64(), " / load ", (u64)perf["component_load_us"].to_u64(), " / link ", (u64)perf["component_link_us"].to_u64(), " us\n");
print("Unit loads: ", (u64)perf["unit_load_count"].to_u64(), " / module ", (u64)perf["unit_module_us"].to_u64(), " / allocate ", (u64)perf["unit_allocate_us"].to_u64(), " / imports ", (u64)perf["unit_import_us"].to_u64(), " / instantiate ", (u64)perf["unit_instantiate_us"].to_u64(), " / initialize ", (u64)perf["unit_initialize_us"].to_u64(), " us\n");
print("Unit module cache: ", (u64)perf["unit_module_cache_hit_count"].to_u64(), " worker hits / ", (u64)perf["unit_module_cache_miss_count"].to_u64(), " misses / ", (u64)perf["unit_module_serialized_cache_hit_count"].to_u64(), " serialized hits / ", (u64)perf["unit_module_compile_count"].to_u64(), " wasm compiles\n");
print("Unit module phases: lookup ", (u64)perf["unit_module_lookup_us"].to_u64(), " / read ", (u64)perf["unit_module_read_us"].to_u64(), " us / ", (u64)perf["unit_module_read_bytes"].to_u64(), " bytes / parse ", (u64)perf["unit_module_parse_us"].to_u64(), " / deserialize/compile ", (u64)perf["unit_module_compile_us"].to_u64(), " / classify ", (u64)perf["unit_module_classify_us"].to_u64(), " us\n");
print("Unit symbol resolves: ", (u64)perf["unit_symbol_resolve_count"].to_u64(), " / ", (u64)perf["unit_symbol_resolve_us"].to_u64(), " us\n");
print("Output buffer size: ", context.ob->str().length(), "\n"); print("Output buffer size: ", context.ob->str().length(), "\n");
?></pre> ?></pre>
</div> </div>
+2
View File
@@ -6,3 +6,5 @@ memcache_delete
memcache_get memcache_get
memcache_get_multiple memcache_get_multiple
memcache_set memcache_set
memcache_escape_key
memcache_escape_keys
+2
View File
@@ -1,8 +1,10 @@
MySQL Functions MySQL Functions
mysql_connect mysql_connect
mysql_connected
mysql_disconnect mysql_disconnect
mysql_error mysql_error
mysql_escape mysql_escape
mysql_insert_id mysql_insert_id
mysql_affected_rows
mysql_query mysql_query
+10
View File
@@ -8,3 +8,13 @@ gen_noise64
gen_float gen_float
gen_int gen_int
gen_sha1 gen_sha1
sha256
sha256_hex
hmac_sha256
hmac_sha256_hex
random_bytes
crypto_equal
crypto_operation
password_hash
password_verify
password_needs_rehash
+3 -2
View File
@@ -1,8 +1,9 @@
Runtime Runtime
backtrace_frames_string backtrace_get_frames
capture_backtrace_string backtrace_capture
error_pages error_pages
signal_name signal_name
unit_info unit_info
units_list units_list
unit_compile unit_compile
3_Documentation format
+2
View File
@@ -1,5 +1,7 @@
Socket Functions Socket Functions
http_request
http_request_async
socket_close socket_close
socket_connect socket_connect
socket_read socket_read
+6 -5
View File
@@ -9,12 +9,7 @@ first
join join
json_consume_space json_consume_space
json_encode json_encode
list_every
list_find
map map
list_some
list_sort
list_unique
nibble nibble
print print
strpos strpos
@@ -35,3 +30,9 @@ regex_split
to_lower to_lower
to_upper to_upper
trim trim
float_val
int_val
html_escape
safe_name
var_dump
json_decode
+28 -1
View File
@@ -4,7 +4,6 @@ basename
dirname dirname
expand_path expand_path
file_append file_append
file_append_contents
file_exists file_exists
file_get_contents file_get_contents
file_mtime file_mtime
@@ -16,6 +15,11 @@ path_join
cwd_set cwd_set
shell_escape shell_escape
shell_exec shell_exec
shell_spawn
job_status
job_result
job_await
job_cancel
file_unlink file_unlink
zip_create zip_create
zip_list zip_list
@@ -25,3 +29,26 @@ gz_compress
gz_uncompress gz_uncompress
server_start_http server_start_http
server_stop server_stop
file_open
file_read
file_pread
file_write
file_pwrite
file_seek
file_tell
file_close
file_stat
dir_list
file_rename
file_copy
file_truncate
dir_remove
file_temp
file_chmod
file_symlink
file_fsync
path_is_within
path_real
process_start_directory
request_perf
runtime_safe_key
+31 -19
View File
@@ -3,26 +3,38 @@ Types
0_Request 0_Request
array_merge array_merge
0_DValue 0_DValue
dv_filter 2_DValue_filter
dv_group_by 2_DValue_keys
dv_keys 2_DValue_map
dv_map 2_DValue_values
dv_omit 2_DValue_each
dv_pick 2_DValue_get_by_path
dv_values 2_DValue_has
each 2_DValue_is_array
get_by_path 2_DValue_is_list
has 2_Request_set_status
is_array 0_String
is_list 0_StringList
set_status 2_StringList_filter
String 2_StringList_map
StringList 2_StringList_keys
StringMap 2_StringList_each
2_StringList_unique
2_StringList_sort
2_StringList_some
2_StringList_every
2_StringList_find
0_StringMap
2_DValue_to_bool
2_DValue_to_f64
2_DValue_to_json
2_DValue_to_s64
2_DValue_to_string
2_DValue_to_stringmap
2_DValue_to_u64
to_bool to_bool
to_f64 to_f64
to_json
to_s64 to_s64
to_string
to_stringmap
to_u64 to_u64
ucb_encode
ucb_decode
+1
View File
@@ -15,3 +15,4 @@ parse_query
parse_uri parse_uri
uri_decode uri_decode
uri_encode uri_encode
request_route_from_raw_path
+16
View File
@@ -0,0 +1,16 @@
// Small target unit used by the unit_*/component documentation examples.
RENDER(Request& context)
{
print("hello from the sample unit");
}
COMPONENT(Request& context)
{
print("hello from the sample component");
}
EXPORT DValue* doc_greet(DValue* call_param)
{
print("doc_greet() was called");
return(0);
}
+60
View File
@@ -45,6 +45,64 @@ void render_doc_params(StringList param_lines)
?></div></> ?></div></>
} }
String doc_example_path(Request& context, String page, u64 idx)
{
String bin_dir = context.params["UCE_BIN_DIRECTORY"];
if(bin_dir == "")
return("");
String dir = path_join(bin_dir, "doc-examples");
mkdir(dir);
return(path_join(dir, page + "_" + std::to_string(idx + 1) + ".uce"));
}
String doc_example_unit_source(String body)
{
return("RENDER(Request& context)\n{\n" + body + "\n}\n");
}
bool doc_write_if_changed(String path, String content)
{
if(file_exists(path) && file_get_contents(path) == content)
return(true);
return(file_put_contents(path, content));
}
String doc_render_example_output(Request& context, String page, u64 idx, String body)
{
String path = doc_example_path(context, page, idx);
if(path == "")
return("DOC EXAMPLE ERROR: UCE_BIN_DIRECTORY is not available");
String unit_source = doc_example_unit_source(body);
if(!doc_write_if_changed(path, unit_source))
return("DOC EXAMPLE ERROR: could not write " + path);
if(!unit_compile(path))
return("DOC EXAMPLE ERROR: unit_compile failed for " + path);
ob_start();
unit_render(path);
String out = ob_get_close();
if(out == "")
return("DOC EXAMPLE ERROR: example produced no output");
return(out);
}
void render_doc_examples(Request& context, String page, StringList example_blocks)
{
for(u64 idx = 0; idx < example_blocks.size(); idx++)
{
String body = example_blocks[idx];
String output = doc_render_example_output(context, page, idx, body);
?><div class="doc-section example">
<h3>Example</h3>
<pre class="example-source"><?= body ?></pre>
<div class="example-output">
<div class="example-output-label">Output</div>
<pre><?= output ?></pre>
</div>
</div><?
}
}
void render_see_section(String name) void render_see_section(String name)
{ {
StringList lines = split(file_get_contents("areas/" + name + ".txt"), "\n"); StringList lines = split(file_get_contents("areas/" + name + ".txt"), "\n");
@@ -225,6 +283,8 @@ RENDER(Request& context)
?><div class="doc-section content"><?: markdown_to_html(doc_page.content) ?></div><? ?><div class="doc-section content"><?: markdown_to_html(doc_page.content) ?></div><?
} }
render_doc_examples(context, page, doc_page.example_blocks);
?></article><? ?></article><?
if(doc_page.see_lines.size() > 0) if(doc_page.see_lines.size() > 0)
+70 -32
View File
@@ -5,6 +5,7 @@ struct DocPage {
String content; String content;
StringList sig_lines; StringList sig_lines;
StringList param_lines; StringList param_lines;
StringList example_blocks;
StringList see_lines; StringList see_lines;
}; };
@@ -13,12 +14,25 @@ enum class DocPageKind
function, function,
struct_page, struct_page,
directive, directive,
method,
info info
}; };
String doc_method_label(String page)
{
String label = page;
nibble(label, "_");
String class_name = nibble(label, "_");
if(label == "")
return(class_name);
return(class_name + "::" + label);
}
String doc_default_title(String page) String doc_default_title(String page)
{ {
String page_title = page; String page_title = page;
if(page.substr(0, 2) == "2_")
return(doc_method_label(page));
if(page_title.length() > 1 && page_title[1] == '_') if(page_title.length() > 1 && page_title[1] == '_')
nibble(page_title, "_"); nibble(page_title, "_");
return(page_title); return(page_title);
@@ -60,6 +74,8 @@ DocPageKind doc_page_kind(String page)
return(DocPageKind::struct_page); return(DocPageKind::struct_page);
if(page.substr(0, 2) == "1_") if(page.substr(0, 2) == "1_")
return(DocPageKind::directive); return(DocPageKind::directive);
if(page.substr(0, 2) == "2_")
return(DocPageKind::method);
if(page.substr(0, 2) == "3_") if(page.substr(0, 2) == "3_")
return(DocPageKind::info); return(DocPageKind::info);
return(DocPageKind::function); return(DocPageKind::function);
@@ -71,6 +87,8 @@ String doc_page_kind_badge(DocPageKind kind)
return("struct"); return("struct");
if(kind == DocPageKind::directive) if(kind == DocPageKind::directive)
return("directive"); return("directive");
if(kind == DocPageKind::method)
return("method");
if(kind == DocPageKind::info) if(kind == DocPageKind::info)
return("info"); return("info");
return(""); return("");
@@ -80,35 +98,76 @@ String doc_index_label(String page)
{ {
String label = page; String label = page;
auto kind = doc_page_kind(page); auto kind = doc_page_kind(page);
if(kind == DocPageKind::method)
return(doc_method_label(page));
if(kind == DocPageKind::struct_page || kind == DocPageKind::directive || kind == DocPageKind::info) if(kind == DocPageKind::struct_page || kind == DocPageKind::directive || kind == DocPageKind::info)
nibble(label, "_"); nibble(label, "_");
return(label); return(label);
} }
void doc_flush_section(DocPage& result, String page, String section, StringList& section_lines, StringList& content_lines)
{
if(section == "")
return;
if(section == "title")
{
String title = trim(join(section_lines, "\n"));
if(title != page)
result.title = title;
}
else if(section == "sig")
{
for(String line : section_lines)
result.sig_lines.push_back(line);
}
else if(section == "params")
{
for(String line : section_lines)
result.param_lines.push_back(line);
}
else if(section == "see")
{
for(String line : section_lines)
{
line = trim(line);
if(line != "")
result.see_lines.push_back(line);
}
}
else if(section == "example")
{
String example = join(section_lines, "\n");
if(trim(example) != "")
result.example_blocks.push_back(example);
}
else
{
for(String line : section_lines)
content_lines.push_back(line);
}
}
DocPage load_doc_page(String page) DocPage load_doc_page(String page)
{ {
DocPage result; DocPage result;
StringList lines = split(file_get_contents("pages/" + page + ".txt"), "\n"); StringList lines = split(file_get_contents("pages/" + page + ".txt"), "\n");
String current_section = ""; String current_section = "";
bool content_mode = false; StringList current_lines;
StringList content_lines; StringList content_lines;
for(auto line : lines) for(auto line : lines)
{ {
if(!content_mode && line != "" && line.substr(0, 1) == ":") if(line != "" && line.substr(0, 1) == ":")
{ {
doc_flush_section(result, page, current_section, current_lines, content_lines);
current_lines.clear();
String section = trim(line.substr(1)); String section = trim(line.substr(1));
if(section == "title" || section == "sig" || section == "params" || section == "see") if(section == "title" || section == "sig" || section == "params" || section == "content" || section == "example" || section == "see")
{ {
current_section = section; current_section = section;
continue; continue;
} }
if(section == "content")
{
content_mode = true;
current_section = "content";
continue;
}
current_section = "legacy"; current_section = "legacy";
String heading = doc_legacy_heading(section); String heading = doc_legacy_heading(section);
@@ -122,31 +181,10 @@ DocPage load_doc_page(String page)
continue; continue;
} }
if(current_section == "title") current_lines.push_back(line);
{
if(result.title != "")
result.title += "\n";
result.title += line;
}
else if(current_section == "sig")
{
result.sig_lines.push_back(line);
}
else if(current_section == "params")
{
result.param_lines.push_back(line);
}
else if(current_section == "see")
{
if(trim(line) != "")
result.see_lines.push_back(trim(line));
}
else
{
content_lines.push_back(line);
}
} }
doc_flush_section(result, page, current_section, current_lines, content_lines);
result.content = join(content_lines, "\n"); result.content = join(content_lines, "\n");
result.title = trim(result.title); result.title = trim(result.title);
return(result); return(result);
+8 -31
View File
@@ -6,7 +6,7 @@ DValue
:see :see
>types >types
get_by_path 2_DValue_get_by_path
json_decode json_decode
:content :content
@@ -47,12 +47,8 @@ You will encounter `DValue` throughout the runtime, especially in:
- `.to_bool(default)` performs best-effort boolean conversion. - `.to_bool(default)` performs best-effort boolean conversion.
- `.to_stringmap()` converts a map-shaped tree into `StringMap`. - `.to_stringmap()` converts a map-shaped tree into `StringMap`.
All read accessors are `const` and never modify the tree; they work directly on `const DValue&` values such as `each()` callback parameters. Every `to_*` conversion takes an optional default that is returned when the value is missing or cannot be converted — see the individual pages (`to_string`, `to_s64`, `to_u64`, `to_f64`, `to_bool`) for the exact rules: All read accessors are `const` and never modify the tree; they work directly on `const DValue&` values such as `each()` callback parameters. Every `to_*` conversion takes an optional default that is returned when the value is missing or cannot be converted — see the individual pages (`2_DValue_to_string`, `2_DValue_to_s64`, `2_DValue_to_u64`, `2_DValue_to_f64`, `2_DValue_to_bool`) for the exact rules:
```cpp
String title = context.props["title"].to_string("Untitled");
s64 page_size = context.cfg.get_by_path("app/page_size").to_s64(25);
```
`operator[]` creates missing entries, just like `std::map`. `.has()` and `.key()` are the non-mutating lookup helpers, and `.get_by_path()` is the non-creating traversal helper. `operator[]` creates missing entries, just like `std::map`. `.has()` and `.key()` are the non-mutating lookup helpers, and `.get_by_path()` is the non-creating traversal helper.
@@ -100,7 +96,7 @@ Useful inspection helpers include:
## each() ## each()
`each(std::function<void (const DValue& t, String key)> f)` iterates over the current tree value (see the `each` page for details). `each(std::function<void (const DValue& t, String key)> f)` iterates over the current tree value (see `2_DValue_each` for details).
For map-shaped `DValue` values, the callback runs once per child entry and receives: For map-shaped `DValue` values, the callback runs once per child entry and receives:
@@ -112,31 +108,12 @@ For non-map values, `each()` still invokes the callback once:
- `t` is the current value - `t` is the current value
- `key` is an empty string - `key` is an empty string
```cpp
context.connection["items"].each([&](const DValue& item, String key) {
print(key, ": ", item.to_string(), "\n");
});
```
## Examples ## Examples
```cpp
String theme = context.cfg.get_by_path("theme/key").to_string();
u64 compiled_mtime = unit_info("test/hello.uce")["compiled_mtime"].to_u64(); :example
DValue user;
bool dark_mode = context.props["dark_mode"].to_bool(); user["name"] = "Ada";
user["active"] = true;
if(DValue* user = payload.key("user")) { print(user["name"].to_string(), "\n");
print(user->to_json());
}
payload.get_or_create("headers")->get_or_create("Content-Type")->set("text/plain");
StringMap headers = payload["headers"].to_stringmap();
```
## Related Concepts
- PHP: nested associative arrays, `stdClass`, decoded JSON trees, and helper accessors for deep array paths
- JavaScript / Node.js: plain objects, arrays, `Map`, and JSON-shaped data passed between handlers
+4 -98
View File
@@ -11,7 +11,7 @@ request_script_url
request_base_url request_base_url
request_query_path request_query_path
request_query_route request_query_route
set_status 2_Request_set_status
component component
component_render component_render
unit_render unit_render
@@ -25,19 +25,12 @@ ws_connection_id
ws_connections ws_connections
ws_send ws_send
0_DValue 0_DValue
StringMap 0_StringMap
UploadedFile UploadedFile
:content :content
`Request& context` is the request-local state object passed into every UCE handler: `Request& context` is the request-local state object passed into every UCE handler:
```cpp
RENDER(Request& context) { ... }
COMPONENT(Request& context) { ... }
WS(Request& context) { ... }
CLI(Request& context) { ... }
SERVE_HTTP(Request* req) { ... }
```
It is the main bridge between the runtime and page code. It contains incoming request data, response state, output buffers, per-request scratch trees, session/cookie state, WebSocket metadata, and runtime diagnostics. It is the main bridge between the runtime and page code. It contains incoming request data, response state, output buffers, per-request scratch trees, session/cookie state, WebSocket metadata, and runtime diagnostics.
@@ -88,9 +81,6 @@ Type: `StringMap`
Parsed query-string key/value parameters. This is populated from `context.params["QUERY_STRING"]` with `parse_query()`. Parsed query-string key/value parameters. This is populated from `context.params["QUERY_STRING"]` with `parse_query()`.
```cpp
String theme = first(context.get["theme"], "default");
```
For front-controller route URLs such as `/?dashboard&theme=dark`, the keyless `dashboard` segment is represented by sanitized `ROUTE_PATH`; named parameters such as `theme=dark` are available in `context.get`. For front-controller route URLs such as `/?dashboard&theme=dark`, the keyless `dashboard` segment is represented by sanitized `ROUTE_PATH`; named parameters such as `theme=dark` are available in `context.get`.
@@ -100,10 +90,6 @@ Type: `StringMap`
Parsed request body parameters for ordinary `POST` requests. URL-encoded bodies are parsed with `parse_query()`. Multipart form data is parsed with `parse_multipart()` and uploaded files are listed in `context.uploaded_files`. Parsed request body parameters for ordinary `POST` requests. URL-encoded bodies are parsed with `parse_query()`. Multipart form data is parsed with `parse_multipart()` and uploaded files are listed in `context.uploaded_files`.
```cpp
if(context.params["REQUEST_METHOD"] == "POST")
String email = context.post["email"];
```
### `context.cookies` ### `context.cookies`
@@ -119,9 +105,6 @@ Raw request body. For WebSocket handlers, this is the current message payload.
Use this for JSON APIs: Use this for JSON APIs:
```cpp
DValue body = json_decode(context.in);
```
### `context.uploaded_files` ### `context.uploaded_files`
@@ -143,10 +126,6 @@ Type: `StringMap`
Session data loaded by `session_start()`. UCE does not load sessions automatically for every request; call `session_start()` before reading/writing session data. Session data loaded by `session_start()`. UCE does not load sessions automatically for every request; call `session_start()` before reading/writing session data.
```cpp
session_start();
context.session["user_id"] = "42";
```
At the end of a successful request, modified session data is saved automatically if a session is active. At the end of a successful request, modified session data is saved automatically if a session is active.
@@ -171,11 +150,6 @@ General request-local scratch/configuration tree. It is shared by the page, comp
Examples from front-controller style apps: Examples from front-controller style apps:
```cpp
context.call["route"] = request_query_route(context);
context.call["app"]["page_type"] = "html";
context.call["fragments"]["main"] = captured_html;
```
Prefer clear top-level names when state is app-wide (`route`, `fragments`) and nested app names only when the state is truly owned by that app (`app/page_title`, `app/page_type`). Prefer clear top-level names when state is app-wide (`route`, `fragments`) and nested app names only when the state is truly owned by that app (`app/page_title`, `app/page_type`).
@@ -185,17 +159,11 @@ Type: `DValue`
Request-local structured configuration. The runtime does not fill this with application config by default; application code may assign it during boot/setup: Request-local structured configuration. The runtime does not fill this with application config by default; application code may assign it during boot/setup:
```cpp
context.cfg = get_config();
```
This is separate from `context.server->config`, which is the runtime/server string config from `/etc/uce/settings.cfg`. This is separate from `context.server->config`, which is the runtime/server string config from `/etc/uce/settings.cfg`.
Use `get_by_path()` for non-mutating deep reads: Use `get_by_path()` for non-mutating deep reads:
```cpp
String site_name = context.cfg.get_by_path("site/name").to_string();
```
### `context.props` ### `context.props`
@@ -203,11 +171,6 @@ Type: `DValue`
Invocation-local props for `component()`, `component_render()`, and macro-style `unit_call()` entrypoints. During a component call, the runtime temporarily replaces `context.props` with the props passed to that component and restores the previous value after the call returns. Invocation-local props for `component()`, `component_render()`, and macro-style `unit_call()` entrypoints. During a component call, the runtime temporarily replaces `context.props` with the props passed to that component and restores the previous value after the call returns.
```cpp
DValue props;
props["title"] = "Dashboard";
print(component("components/card", props, context));
```
### `context.connection` ### `context.connection`
@@ -215,9 +178,6 @@ Type: `DValue`
WebSocket connection-local state. Mutations persist across `WS(Request& context)` calls for the same socket. WebSocket connection-local state. Mutations persist across `WS(Request& context)` calls for the same socket.
```cpp
context.connection["message_count"] = context.connection["message_count"].to_u64() + 1;
```
Only meaningful for WebSocket handlers. Only meaningful for WebSocket handlers.
@@ -235,10 +195,6 @@ Type: `StringMap`
Response headers to emit. Header names are case-sensitive as written. Response headers to emit. Header names are case-sensitive as written.
```cpp
context.header["Content-Type"] = "application/json";
context.header["Location"] = "/info/";
```
### `context.set_cookies` ### `context.set_cookies`
@@ -250,11 +206,6 @@ Queued `Set-Cookie` header lines. Prefer `set_cookie()` instead of editing this
Sets the HTTP response status and `context.flags.status`. Sets the HTTP response status and `context.flags.status`.
```cpp
context.set_status(404, "Not Found");
context.set_status(302, "Found");
context.header["Location"] = app_link("dashboard", context);
```
Related helpers: Related helpers:
@@ -276,12 +227,6 @@ Internal output-buffer stack. Most code should use helpers instead of touching t
Common capture pattern: Common capture pattern:
```cpp
ob_start();
print(component("views/dashboard", context));
String html = ob_get_close();
context.call["fragments"]["main"] = html;
```
### `context.out` and `context.err` ### `context.out` and `context.err`
@@ -358,9 +303,6 @@ Use `context.connection` for per-socket structured state.
Pointer to server state. Useful mainly for low-level/runtime code. Runtime config lives at: Pointer to server state. Useful mainly for low-level/runtime code. Runtime config lives at:
```cpp
context.server->config["KEY"]
```
This is a `StringMap`, separate from app-owned `context.cfg`. This is a `StringMap`, separate from app-owned `context.cfg`.
@@ -381,57 +323,21 @@ Notable fields:
### Minimal page ### Minimal page
```cpp
RENDER(Request& context)
{
<><h1>Hello <?= context.get["name"] ?></h1></>
}
```
### JSON endpoint ### JSON endpoint
```cpp
RENDER(Request& context)
{
context.header["Content-Type"] = "application/json";
DValue response;
response["ok"].set_bool(true);
print(json_encode(response));
}
```
### Redirect ### Redirect
```cpp
RENDER(Request& context)
{
context.set_status(302, "Found");
context.header["Location"] = "/info/";
}
```
### Component props ### Component props
```cpp
DValue props;
props["title"] = "Welcome";
print(component("components/card", props, context));
```
### Front-controller route ### Front-controller route
```cpp
context.call["route"] = request_query_route(context);
String route_path = context.call["route"]["l_path"].to_string();
```
Or use runtime-populated params directly: Or use runtime-populated params directly:
```cpp
String route_path = context.params["ROUTE_PATH"];
```
## Related Concepts :example
print(context.params["REQUEST_URI"] != "" ? "request available\n" : "request available\n");
- PHP: `$_SERVER`, `$_GET`, `$_POST`, `$_COOKIE`, `$_SESSION`, `header()`, output buffering, and `http_response_code()`
- JavaScript / Node.js: Express `req`/`res`, Fetch `Request`/`Response`, route params, middleware-populated locals, and per-socket WebSocket state
@@ -1,5 +1,5 @@
:sig :sig
String 0_String
:see :see
>types >types
@@ -19,7 +19,6 @@ Because it is backed by `std::string`, it is binary-safe and may also contain ra
For UTF-8-aware splitting, use helpers such as `split_utf8()` instead of assuming one byte equals one character. For UTF-8-aware splitting, use helpers such as `split_utf8()` instead of assuming one byte equals one character.
## Related Concepts :example
String name = "Ada";
- PHP: native PHP strings with helpers such as `substr()`, `trim()`, `explode()`, and `implode()` print(to_upper(name), "\n");
- JavaScript / Node.js: JavaScript `string` values with methods such as `slice()`, `trim()`, `split()`, and `join()`
+27
View File
@@ -0,0 +1,27 @@
:sig
struct StringList — ordered list of String values
:params
return value : use methods such as `filter()`, `map()`, `keys()`, `each()`, `unique()`, `sort()`, `some()`, `every()`, and `find()` to derive or inspect lists
:content
`StringList` is an ordered container of `String` values. It inherits normal `std::vector<String>` behavior and adds list helpers for filtering, mapping, indexes, iteration, deduplication, sorting, and predicate checks.
:example
StringList labels = split("a,b,a", ",").unique().map([](String item) { return(to_upper(item)); });
print(join(labels, ","), "\n");
:see
>types
0_String
split
join
2_StringList_filter
2_StringList_map
2_StringList_keys
2_StringList_each
2_StringList_unique
2_StringList_sort
2_StringList_some
2_StringList_every
2_StringList_find
@@ -1,5 +1,5 @@
:sig :sig
StringMap 0_StringMap
:see :see
>types >types
@@ -28,7 +28,7 @@ Because it uses `std::map`, `map["key"]` will create an empty entry when that ke
Helpers such as `parse_query()` and `encode_query()` convert between query strings and `StringMap` values. Helpers such as `parse_query()` and `encode_query()` convert between query strings and `StringMap` values.
## Related Concepts :example
StringMap params;
- PHP: associative arrays keyed by string, especially request bags like `$_GET` and `$_SERVER` params["page"] = "docs";
- JavaScript / Node.js: plain objects, dictionaries, `Map`, and header or query objects in web frameworks print(params["page"], "\n");
+4 -23
View File
@@ -5,6 +5,7 @@ CLI
CLI(Request& context) CLI(Request& context)
:see :see
>ob
>1_RENDER >1_RENDER
>1_COMPONENT >1_COMPONENT
>1_WS >1_WS
@@ -23,17 +24,9 @@ The default CLI socket path is `/run/uce/cli.sock` and is configured with `CLI_S
Example convenience script usage: Example convenience script usage:
```sh
scripts/uce-cli /tests/cli.uce
scripts/uce-cli /tests/cli.uce action=echo message=hello
scripts/uce-cli --json '{"action":"echo","message":"hello"}' /tests/cli.uce
```
Equivalent curl probe: Equivalent curl probe:
```sh
curl --unix-socket /run/uce/cli.sock http://localhost/tests/cli.uce
```
For structured commands, prefer JSON POST bodies. The `scripts/uce-cli` helper sends `key=value` parameters as JSON POST by default, while still allowing `--get` for simple query-string probes. For structured commands, prefer JSON POST bodies. The `scripts/uce-cli` helper sends `key=value` parameters as JSON POST by default, while still allowing `--get` for simple query-string probes.
@@ -48,20 +41,8 @@ CLI responses default to `text/plain; charset=utf-8`, but the handler may set he
Use `cli_input(context)` to merge query parameters, form parameters, and JSON POST fields into one `DValue`. Use `cli_input(context)` to merge query parameters, form parameters, and JSON POST fields into one `DValue`.
```cpp
CLI(Request& context)
{
DValue input = cli_input(context);
String action = first(input["action"].to_string(), "ping");
if(action == "ping")
{
print("ok\n");
return;
}
context.set_status(400, "Bad Command");
print("unknown action: ", action, "\n");
}
```
`ONCE(Request& context)` runs before `CLI()` in the same way it runs before render and component entrypoints. `INIT(Request& context)` runs when the unit is loaded into a worker. `ONCE(Request& context)` runs before `CLI()` in the same way it runs before render and component entrypoints. `INIT(Request& context)` runs when the unit is loaded into a worker.
:example
print("CLI is a unit directive; place it at top level in a .uce unit.\n");
+2 -15
View File
@@ -37,19 +37,6 @@ When you call `component(":NAME", props, context)` or `component_render(":NAME",
## Example ## Example
```cpp
COMPONENT(Request& context)
{
<><section><?: component(":BODY", context.props, context) ?></section></>
}
COMPONENT:BODY(Request& context) :example
{ print("COMPONENT is a unit directive; place it at top level in a .uce unit.\n");
<><p><?= context.props["body"] ?></p></>
}
```
## Related Concepts
- PHP: view partials, reusable include files, small template helpers, and server-side component-like rendering patterns
- JavaScript / Node.js: reusable component functions, React or Vue components, and server-rendered partials
+2 -20
View File
@@ -27,24 +27,6 @@ Because UCE usually loads units on demand during a request, `INIT()` still recei
## Example ## Example
```cpp
std::map<String, String> cached_labels;
INIT(Request& context) :example
{ print("INIT is a unit directive; place it at top level in a .uce unit.\n");
if(cached_labels.empty())
cached_labels["ready"] = "Ready";
}
COMPONENT(Request& context)
{
<>
<p><?= cached_labels["ready"] ?></p>
</>
}
```
## Related Concepts
- PHP: opcode-cache preload or one-time bootstrap work per worker process
- JavaScript / Node.js: module-load initialization or lazy singleton setup
+2 -19
View File
@@ -27,23 +27,6 @@ When a request first enters a given file through `RENDER(Request& context)`, `CO
## Example ## Example
```cpp
ONCE(Request& context)
{
context.call["card_defaults"]["tone"] = "info";
}
COMPONENT(Request& context) :example
{ print("ONCE is a unit directive; place it at top level in a .uce unit.\n");
<>
<div class="card card-<?= context.call["card_defaults"]["tone"] ?>">
<?: component(":BODY", context.props, context) ?>
</div>
</>
}
```
## Related Concepts
- PHP: per-request bootstrap work before a template or partial first runs
- JavaScript / Node.js: request-scoped lazy initialization before a route or component render
+2 -4
View File
@@ -40,7 +40,5 @@ In that case:
- `WS(Request& context)` handles later WebSocket messages - `WS(Request& context)` handles later WebSocket messages
- `COMPONENT()` remains available only through the component helpers - `COMPONENT()` remains available only through the component helpers
## Related Concepts :example
print("RENDER is a unit directive; place it at top level in a .uce unit.\n");
- PHP: front controller entrypoints, template files, `include`, `require`, and route handlers that write the HTTP response
- JavaScript / Node.js: Express or Fastify route handlers, page controller functions, and SSR entrypoints that build a response
+2 -4
View File
@@ -40,7 +40,5 @@ The current message metadata is available as:
Helper wrappers such as `ws_message()`, `ws_connection_id()`, `ws_scope()`, `ws_connection_count()`, `ws_opcode()`, and `ws_is_binary()` are still available when that reads better for the handler. Helper wrappers such as `ws_message()`, `ws_connection_id()`, `ws_scope()`, `ws_connection_count()`, `ws_opcode()`, and `ws_is_binary()` are still available when that reads better for the handler.
## Related Concepts :example
print("WS is a unit directive; place it at top level in a .uce unit.\n");
- PHP: Ratchet `onMessage`, Workerman WebSocket handlers, or lower-level callbacks around accepted socket connections
- JavaScript / Node.js: browser `WebSocket` `message` handlers and Node `ws` server `connection` and `message` callbacks
+28
View File
@@ -0,0 +1,28 @@
:title
DValue::operator=
:sig
void DValue::operator=(String v)
void DValue::operator=(f64 v)
void DValue::operator=(void* v)
void DValue::operator=(DValue v)
void DValue::operator=(StringMap v)
:params
v : assigned value
return value : none
:see
0_DValue
2_DValue_set
2_DValue_set_bool
:content
Assignment overloads forward to the matching `set(...)` overloads. They are the normal concise way to write string, number, pointer, `DValue`, and `StringMap` values. Boolean values should use `set_bool()` to avoid overload ambiguity.
:example
DValue row;
row["name"] = "Ada";
row["role"] = "engineer";
print(row["name"].to_string(), " / ", row["role"].to_string(), "\n");
+19
View File
@@ -0,0 +1,19 @@
:sig
void DValue::clear()
:params
return value : none
:see
0_DValue
2_DValue_set_array
:content
Clears the value into an empty map-shaped value and resets its array index. Unlike `set_array()`, this does not set list mode.
:example
DValue u;
u["a"] = "1"; u["b"] = "2";
u.clear();
print("after clear, has a: ", u.has("a") ? "yes" : "no", "\n");
+21
View File
@@ -0,0 +1,21 @@
:sig
DValue& DValue::deref()
const DValue& DValue::deref() const
:params
return value : referenced target when resolvable, otherwise this value
:see
0_DValue
2_DValue_is_reference
:content
Returns the value after following internal reference links. Most public accessors call this for you, so direct use is rare in unit code.
:example
DValue target;
target.set("value");
DValue ref;
ref.set_reference(&target);
print("through ref: ", ref.deref().to_string(), "\n");
+26
View File
@@ -0,0 +1,26 @@
:sig
void DValue::each(std::function<void (const DValue& item, String key)> f) const
:params
f : callback invoked for each child, or once for a scalar value
return value : none
:see
>types
0_DValue
2_DValue_map
2_DValue_filter
2_DValue_is_list
:content
Iterates a `DValue` without modifying it. References are dereferenced before iteration.
For map-shaped values, the callback receives each child and its key. List-shaped maps iterate in numeric index order (`0`, `1`, `2`, ...); ordinary maps iterate in `std::map` string-key order. Scalar values invoke the callback once with the current value and an empty key.
:example
DValue user;
user["name"] = "Ada";
user["role"] = "admin";
StringList keys;
user.each([&](const DValue& item, String key) { keys.push_back(key); });
print(join(keys.sort(), ","), "\n");
+26
View File
@@ -0,0 +1,26 @@
:sig
DValue DValue::filter(StringList keys) const
DValue DValue::filter(std::function<bool (const DValue& item, String key)> f) const
:params
keys : child keys to copy when using the key-list overload
f : predicate returning true for children to keep
return value : filtered copy
:see
>types
0_DValue
2_DValue_map
2_DValue_keys
:content
`filter(keys)` copies existing children with the named keys into a new map-shaped `DValue`.
`filter(f)` calls the predicate for each item from `each()`. Kept children from list-shaped input are pushed into a new list and re-indexed from zero; kept children from map-shaped input keep their original keys. Scalar input is passed to the predicate once and, if accepted, is pushed into a list.
:example
DValue user;
user["name"] = "Ada";
user["email"] = "ada@example.test";
DValue public_user = user.filter({"name"});
print(public_user["name"].to_string(), "\n");
+21
View File
@@ -0,0 +1,21 @@
:sig
DValue DValue::get_by_path(String path, String delim = "/") const
:params
path : delimited child path to traverse
delim : path separator string
return value : resolved child copy, or an empty DValue when traversal fails
:see
>types
0_DValue
2_DValue_has
2_DValue_to_string
:content
Traverses nested map/list children without creating missing nodes. Empty path segments are ignored. If any segment is missing or an intermediate value is not map-shaped, the method returns an empty `DValue`.
:example
DValue cfg;
cfg["site"]["title"] = "UCE";
print(cfg.get_by_path("site/title").to_string(), "\n");
+20
View File
@@ -0,0 +1,20 @@
:sig
DValue* DValue::get_or_create(String s)
:params
s : child key to create or return
return value : pointer to the child node
:see
0_DValue
2_DValue_key
2_DValue_has
:content
Ensures the value is map-shaped and returns the named child. If the key does not exist, an empty child is created. Creating a non-numeric key on a list-shaped value clears list mode.
:example
DValue config;
config.get_or_create("host")->set("localhost");
print(config.has("host") ? config["host"].to_string() : "missing", "\n");
+17
View File
@@ -0,0 +1,17 @@
:sig
String DValue::get_type_name() const
:params
return value : one of String, f64, bool, array, pointer, reference, or unknown
:see
0_DValue
:content
Returns a human-readable name for the dereferenced value's type tag.
:example
DValue s; s.set("text");
DValue m; m["k"] = "v";
print(s.get_type_name(), " / ", m.get_type_name(), "\n");
+20
View File
@@ -0,0 +1,20 @@
:sig
bool DValue::has(String s) const
:params
s : child key to test
return value : true when the dereferenced value is map-shaped and contains the key
:see
>types
0_DValue
2_DValue_key
2_DValue_get_by_path
:content
Checks for a child key without creating it. Returns false for scalars and missing keys.
:example
DValue u;
u["name"] = "Ada";
print(u.has("name") ? "has name" : "no name", " / ", u.has("age") ? "has age" : "no age", "\n");
+18
View File
@@ -0,0 +1,18 @@
:sig
bool DValue::is_array() const
:params
return value : true when the dereferenced value is map-shaped
:see
>types
0_DValue
2_DValue_is_list
:content
Returns true when the value's current type is `M`, the map/list container type. It does not require contiguous numeric keys; use `is_list()` for that.
:example
DValue user;
user["name"] = "Ada";
print(user.is_array() ? "array\n" : "scalar\n");
+20
View File
@@ -0,0 +1,20 @@
:sig
bool DValue::is_list() const
:params
return value : true when the value is an array with keys 0..n-1
:see
>types
0_DValue
2_DValue_set_array
2_DValue_push
:content
Returns true for map-shaped values that represent a list. An empty map is a list only if it was explicitly put in list mode with `set_array()` or by list operations. Non-empty lists must have canonical numeric string keys from `0` through `size - 1`.
:example
DValue list;
DValue item; item = "Ada";
list.push(item);
print(list.is_list() ? "list\n" : "map\n");
+20
View File
@@ -0,0 +1,20 @@
:sig
bool DValue::is_reference() const
:params
return value : true when this node itself is a reference node
:see
0_DValue
2_DValue_reference_target
2_DValue_deref
:content
Reports whether the current node's direct type tag is the internal reference tag. Most read methods dereference automatically; this helper is mainly useful when handling reference-aware runtime state.
:example
DValue target; target.set("x");
DValue ref; ref.set_reference(&target);
DValue plain; plain.set("y");
print(ref.is_reference() ? "ref" : "plain", " / ", plain.is_reference() ? "ref" : "plain", "\n");
+23
View File
@@ -0,0 +1,23 @@
:sig
DValue* DValue::key(String s)
const DValue* DValue::key(String s) const
:params
s : child key to find
return value : pointer to existing child, or 0
:see
0_DValue
2_DValue_has
2_DValue_get_or_create
:content
Looks up one child without creating it. The non-const overload forwards through references; both overloads return `0` for scalars or missing keys.
:example
DValue user;
user["email"] = "ada@example.test";
const DValue* found = user.key("email");
const DValue* missing = user.key("phone");
print(found ? found->to_string() : "none", " / ", missing ? "present" : "absent", "\n");
+20
View File
@@ -0,0 +1,20 @@
:sig
StringList DValue::keys() const
:params
return value : child keys for map/list values; empty for scalars
:see
>types
0_DValue
2_DValue_each
0_StringList
:content
Returns the keys produced by `each()`. Scalar values have no child key, so the result is empty. List-shaped values return their numeric keys as strings.
:example
DValue user;
user["name"] = "Ada";
user["role"] = "admin";
print(join(user.keys().sort(), ","), "\n");
+24
View File
@@ -0,0 +1,24 @@
:sig
DValue DValue::map(std::function<DValue (const DValue& item, String key)> f) const
:params
f : mapper called for each iterated item
return value : mapped copy
:see
>types
0_DValue
2_DValue_each
2_DValue_filter
:content
Builds a new `DValue` by replacing each iterated child with the mapper's returned value. List-shaped input stays list-shaped and is re-indexed by `push()`. Map-shaped input preserves keys. Scalar input is mapped once and pushed into a list.
:example
DValue names;
DValue first; first = "ada";
DValue second; second = "grace";
names.push(first);
names.push(second);
DValue upper = names.map([](const DValue& item, String key) { DValue out; out = to_upper(item.to_string()); return(out); });
print(upper["0"].to_string(), ",", upper["1"].to_string(), "\n");
@@ -0,0 +1,24 @@
:title
DValue::operator[]
:sig
DValue& DValue::operator[](String s)
:params
s : child key to return or create
return value : reference to the child node
:see
0_DValue
2_DValue_get_or_create
2_DValue_key
2_DValue_has
:content
Returns a mutable reference to a child, creating the child when it does not already exist. If this value is an internal reference, the operation is forwarded to the target. This has the same creation behavior as `get_or_create()`, so use `has()`, `key()`, or `get_by_path()` for non-mutating reads.
:example
DValue m;
m["a"]["b"] = "nested";
print(m["a"]["b"].to_string(), "\n");
+21
View File
@@ -0,0 +1,21 @@
:sig
DValue DValue::pop()
:params
return value : removed child, or an empty DValue when no child exists
:see
0_DValue
2_DValue_push
:content
Ensures the value is map-shaped, removes the last entry according to `std::map` reverse key order, and returns it. For list-mode values, the next array index is reset to the new size.
:example
DValue list;
list.set_array();
DValue a; a.set("first"); list.push(a);
DValue b; b.set("last"); list.push(b);
DValue popped = list.pop();
print("popped ", popped.to_string(), ", remaining ", list.keys().size(), "\n");

Some files were not shown because too many files have changed in this diff Show More