198 lines
10 KiB
Markdown
198 lines
10 KiB
Markdown
# Phase 0 findings — toolchain & runtime spike
|
|
|
|
- **Status: EXIT CRITERION PASSED** (2026-06-12, on k-uce)
|
|
- **Runtime selected: Wasmtime** (v45.0.1, C API). WAMR rejected — evidence below.
|
|
- Exceptions decision (§11.1, error codes / `-fno-exceptions`) **confirmed viable**:
|
|
both the stubs and two real generated units compile with `-fno-exceptions`,
|
|
no try/catch blocker anywhere.
|
|
|
|
The exit criterion ran end-to-end: a core stub (libc/libc++ statically linked,
|
|
owns memory/allocator) and a unit stub (PIC, `dylink.0`) were linked **at
|
|
runtime** by `loader.cpp` and produced:
|
|
|
|
```
|
|
hello from unit; unit-data-segment-ok; counter=7; mapsum=3; core-string+unit[cb:42][got-func-ok][fn:42]
|
|
core_counter (in linear memory): before=7 after=8
|
|
PHASE0 EXIT CRITERION: PASS
|
|
```
|
|
|
|
which validates, in one render call: unit data-segment relocation
|
|
(`__memory_base`), GOT.mem read **and write** of a core global, `std::string`
|
|
/`std::map` in unit code on the core's heap, a heap C++ object created in core
|
|
and mutated/read by the unit, function pointers crossing unit→core→unit
|
|
through the shared table, GOT.func resolution, and a `std::function` lambda
|
|
allocated in the unit and invoked by core. That is the §3.4 contract
|
|
("DValue inside the workspace: no serialization, ever") demonstrated at the
|
|
ABI level.
|
|
|
|
## Toolchain pins
|
|
|
|
| What | Version | Where on k-uce |
|
|
|---|---|---|
|
|
| wasi-sdk | 33 (clang 22.1.0-wasi-sdk) | `/opt/wasi-sdk` |
|
|
| target triple | `wasm32-wasip1` (`wasm32-wasi` is deprecated) | — |
|
|
| Wasmtime C API | v45.0.1 (prebuilt x86_64-linux release) | `/opt/wasmtime` |
|
|
| WAMR (rejected) | WAMR-2.4.4, built from source | `/opt/wamr` |
|
|
| cmake / ninja | 3.31.6 / 1.12.1 (apt) | — |
|
|
|
|
## Runtime selection: why not WAMR
|
|
|
|
WAMR was tried first per §9 ("preferred ... use Wasmtime only if blocked").
|
|
We are blocked, on the load-bearing requirement itself:
|
|
|
|
1. **WAMR's wasm-c-api ignores imported memories and tables.** At unit
|
|
instantiation it logs `"doesn't support import memories and tables for
|
|
now, ignore them"` (`wasm_c_api.c`) and gives the instance its own
|
|
memory/table — which silently destroys the shared-workspace model.
|
|
2. **Host-side `wasm_table_grow` / `wasm_memory_grow` are explicitly
|
|
unsupported** ("Only allow growing a table via the opcode table.grow").
|
|
3. Its build banner lists *Import/Export of Mutable Globals* as unsupported —
|
|
the dylink ABI imports `__stack_pointer` and every `GOT.*` entry as a
|
|
mutable global.
|
|
|
|
Its multi-module feature is name-based auto-resolution, not host-orchestrated
|
|
dylink (no host-computed `__memory_base`/`__table_base`, no GOT). Making WAMR
|
|
fit means implementing import binding through the c-api layer and runtime
|
|
internals — a runtime-development project, not a patch.
|
|
|
|
**Wasmtime v45.0.1 passed everything on the first run** through the standard
|
|
`wasm.h` C API: host-created funcref table shared by both instances, exported
|
|
memory imported by the unit, host-created (mutable) globals, cross-instance
|
|
export→import wiring. The remaining §9 criteria also favor it: AOT artifacts
|
|
(`.cwasm` precompilation) for the unit cache, epoch interruption for CPU
|
|
limits, and built-in copy-on-write memory-image instantiation for the §6 core
|
|
snapshot (machinery we'd have had to build ourselves on WAMR).
|
|
|
|
Trade-off accepted (was already in §10): Rust codebase, heavier to
|
|
vendor/patch. Pin the release artifact (lib + headers, checksummed) the way
|
|
sqlite is vendored; building from source stays possible but is not the
|
|
default path. The C API .so is ~27 MB.
|
|
|
|
## Module build recipe (what `build_modules.sh` settled on)
|
|
|
|
Core (non-PIC reactor, owns libc/libc++/allocator):
|
|
```
|
|
clang++ --target=wasm32-wasip1 -mexec-model=reactor -O1 -fno-exceptions \
|
|
core.cpp -o core.wasm \
|
|
-Wl,--export-all -Wl,--import-table \
|
|
-Wl,--export=__stack_pointer -Wl,--export=__heap_base \
|
|
-Wl,--undefined=_ZTVN10__cxxabiv117__class_type_infoE
|
|
```
|
|
|
|
Unit (PIC side module):
|
|
```
|
|
clang++ --target=wasm32-wasip1 -fPIC -fvisibility=default \
|
|
-fvisibility-inlines-hidden -O1 -fno-exceptions -c unit.cpp
|
|
wasm-ld -shared --experimental-pic --unresolved-symbols=import-dynamic \
|
|
--Bsymbolic unit.o -o unit.wasm --export=<entry>
|
|
```
|
|
|
|
Hard-won flag findings:
|
|
|
|
1. **`--unresolved-symbols=import-dynamic`** is required for the side-module
|
|
link; undefined symbols then become `env.*` function imports and `GOT.*`
|
|
globals exactly per the Emscripten dylink ABI.
|
|
2. **`-fvisibility-inlines-hidden` is mandatory.** Without it one libc++
|
|
vague-linkage lambda (`std::map` tree-emplace internals, missing libc++'s
|
|
usual hide-from-ABI attribute) is emitted as *both* an export and an
|
|
import of the unit — a self-import the loader cannot satisfy at
|
|
instantiation time without lazy-binding trampolines. `--Bsymbolic` alone
|
|
did **not** bind it.
|
|
3. **Core symbol closure**: `--export-all` only exports what got *linked*.
|
|
The unit needed `__cxxabiv1::__class_type_info`'s vtable (RTTI machinery
|
|
behind `std::function`), which the core never references — forced in with
|
|
`--undefined=`. The production core needs a closure strategy:
|
|
`--whole-archive` for libc/libc++/libc++abi, or a curated keep-list. The
|
|
loader also implements the complementary fallback (resolve `GOT.mem` of
|
|
weak data from the unit's *own* exports post-instantiation, patching the
|
|
provisional GOT global).
|
|
4. **`--import-table` on the core** + a **host-created table** is the right
|
|
shape (see loader notes); `--export-table`/`--growable-table` was the
|
|
first attempt and died on WAMR's host-grow limitation, but host-created
|
|
stays the better design under Wasmtime too: the loader picks table size
|
|
(core's declared minimum + headroom) before any instantiation.
|
|
5. `wasm32-wasi` triple is deprecated in wasi-sdk 33 → use `wasm32-wasip1`.
|
|
|
|
## Loader notes (`loader.cpp`, ~450 lines, standard wasm-c-api)
|
|
|
|
Sequence proven: instantiate core (45 WASI imports satisfied with named trap
|
|
stubs — none was ever called) → parse `dylink.0` (`mem_info`: memsize/align,
|
|
tablesize) → `__memory_base` = call core's exported `malloc` → `__table_base`
|
|
= bump pointer starting at core's table-import minimum → build the unit's
|
|
import vector (memory/table/`__stack_pointer` shared from core; `env.*`
|
|
functions from core exports; `GOT.mem.*` as host mutable i32 globals holding
|
|
addresses read from core's exported data-symbol globals) → instantiate →
|
|
patch deferred GOT entries → `__wasm_apply_data_relocs` → `__wasm_call_ctors`
|
|
→ call the entry export.
|
|
|
|
- **Erratum (found in Phase 3): self-resolved `GOT.mem` values must add
|
|
`__memory_base`.** A PIC module's exported data symbols are i32 globals
|
|
holding offsets *relative to its `__memory_base`*, not absolute addresses —
|
|
the linker adds the base when patching deferred GOT entries (there is no
|
|
`__wasm_apply_global_relocs` export to do it). Copying the export verbatim
|
|
reads/writes core memory at low addresses and renders silently wrong values;
|
|
the Phase 3 fixture's `self-got`/`callback` markers exist to catch exactly
|
|
this. `GOT.mem` entries resolved from the *core's* exports are absolute
|
|
already (the core is non-PIC) and need no adjustment.
|
|
- **GOT.func is resolved guest-side**: the core exports a helper returning
|
|
`(intptr_t)&func` — taking the address forces a link-time elem entry and a
|
|
wasm function pointer *is* its table index. No host-side funcref injection
|
|
is needed at all (it was WAMR-unsupported; under Wasmtime it would work but
|
|
the guest-side registry is simpler and runtime-agnostic). The production
|
|
core should carry a name→funcptr registry (dlsym-shaped) for its API
|
|
surface.
|
|
- **wasi-libc gotcha**: `_initialize` has a double-init guard ending in
|
|
`__builtin_trap()`. WAMR runs `_initialize` automatically at instantiation
|
|
(so calling it again traps "unreachable"); Wasmtime does not (so you must
|
|
call it). Cost one debugging round; recorded here so it never costs another.
|
|
- Export-name `wasm_name_t` may include the trailing NUL in `size` (WAMR
|
|
did); trim when indexing exports by name.
|
|
|
|
## Real generated units (delegated grind — full log in `realunit-report.md`)
|
|
|
|
`site/demo/collections.uce.cpp` and `hello.uce.cpp` (taken verbatim from the
|
|
live unit cache at `/tmp/uce/work/...`) both compile and link as PIC side
|
|
modules with `dylink.0`, **no allocator definitions**, with only shim-level
|
|
intervention. `collections.wasm`: 42 KB, 52 imports — including exactly the
|
|
predicted `GOT.mem.context` for the global `Request*`. Friction points that
|
|
become Phase 2 work items:
|
|
|
|
1. **`types.h` defines global `operator new/delete` in every unit** — must be
|
|
gated (`#ifdef`) out of side-module builds; allocator ownership belongs to
|
|
the core (§3.2: "the one fatal misconfiguration").
|
|
2. **`sys.h` includes `<signal.h>`** → wasi needs `-D_WASI_EMULATED_SIGNAL`
|
|
(+ `-lwasi-emulated-signal` in the core) or an `#ifdef __wasm__` carve-out;
|
|
signals/fork/exec/sockets in `sys.h` have no wasi equivalent and move
|
|
behind hostcalls anyway (§5.1).
|
|
3. **Generated units include `uce_lib.h` by absolute path** — the
|
|
preprocessor should emit a logical include so the wasm build can supply
|
|
its own include order.
|
|
4. **Header-inline connector wrappers (MySQL etc.) get pulled into every
|
|
unit** regardless of use; the §3.3 membrane split (thin wasm-side shim,
|
|
host-side implementation) resolves this and shrinks unit import lists.
|
|
5. The real-unit compile used the pre-`-fvisibility-inlines-hidden` flag set
|
|
(parallel work); the final unit flag set above should be used from
|
|
Phase 2 on.
|
|
|
|
## Implications for the next phases
|
|
|
|
- **Phase 1 (DValue C ABI, native)**: unaffected by any of this; proceed as
|
|
written.
|
|
- **Phase 2 (core module + membrane)**: add the closure strategy
|
|
(whole-archive), the GOT.func name→funcptr registry, the `types.h`
|
|
allocator gate, the signal emulation define, and the preprocessor include
|
|
change. Compile `uce_lib` with the core recipe above.
|
|
- **Phase 3 (loader)**: `loader.cpp` here is the skeleton — dylink parsing,
|
|
base allocation, GOT resolution, and init sequencing are all proven; what
|
|
remains is the registry/dispatch layer, ABI stamping, and multi-unit
|
|
placement.
|
|
- **Phase 4**: use Wasmtime's epoch interruption for CPU limits and its
|
|
memory-image/CoW instantiation for the core snapshot rather than building
|
|
either by hand.
|
|
|
|
## Artifacts (on k-uce, not in git)
|
|
|
|
- `/tmp/uce/wasm-phase0/{core.wasm,unit.wasm,loader}` — exit-criterion run
|
|
- `/tmp/uce/wasm-phase0/realunit/` — real-unit compiles + shim tree + inspector dumps
|
|
- `/opt/wasi-sdk`, `/opt/wasmtime`, `/opt/wamr` — toolchains/runtimes
|