Bound request-time Wasm compilation
This commit is contained in:
+10
-1
@@ -237,7 +237,16 @@ Important settings:
|
|||||||
the Wasm backend from starting. The invocation timeout defaults to `30000` and is
|
the Wasm backend from starting. The invocation timeout defaults to `30000` and is
|
||||||
enforced to the epoch ticker's period resolution. Blocking host helpers
|
enforced to the epoch ticker's period resolution. Blocking host helpers
|
||||||
retain their own shorter limits and are capped to the remaining invocation
|
retain their own shorter limits and are capped to the remaining invocation
|
||||||
budget where the underlying operation is cancellable.
|
budget where the underlying operation is cancellable. Cold entry compilation,
|
||||||
|
explicit `unit_compile()`, dynamic component JIT, transitive `#load` work,
|
||||||
|
compiler and registry lock waits, and the configured runtime-error page all
|
||||||
|
consume the same request budget. A timed-out compiler process group is killed
|
||||||
|
without replacing the prior artifact set or persisting a compile-failure
|
||||||
|
backoff. Request compiles require a zero child exit status and complete staged
|
||||||
|
output; generated C++, exports, source map, Wasm, metadata, cached-module
|
||||||
|
invalidation, and diagnostics are published or rolled back as one guarded
|
||||||
|
generation. Proactive and offline precompile work remain independent of
|
||||||
|
request invocation deadlines.
|
||||||
|
|
||||||
After editing settings, restart UCE:
|
After editing settings, restart UCE:
|
||||||
|
|
||||||
|
|||||||
@@ -202,14 +202,23 @@ The graceful signal handler belongs to the parent and render workers. Generic
|
|||||||
|
|
||||||
Epoch interruption measures uninterrupted guest CPU segments. A separate
|
Epoch interruption measures uninterrupted guest CPU segments. A separate
|
||||||
absolute workspace invocation deadline starts before app-owned entry-unit
|
absolute workspace invocation deadline starts before app-owned entry-unit
|
||||||
loading and initialization and remains unchanged through the selected handler,
|
readiness and cold compilation, loading, and initialization. It remains
|
||||||
ONCE, and every nested component/unit call. The common hostcall membrane checks
|
unchanged through the selected handler, ONCE, every nested component/unit call,
|
||||||
|
and configured runtime-error rendering. The common hostcall membrane checks
|
||||||
that deadline before and after every native call and re-arms the store with the
|
that deadline before and after every native call and re-arms the store with the
|
||||||
smaller of the remaining absolute budget and the CPU-segment budget. A cheap
|
smaller of the remaining absolute budget and the CPU-segment budget. A cheap
|
||||||
hostcall loop therefore cannot renew an invocation indefinitely. Blocking host
|
hostcall loop therefore cannot renew an invocation indefinitely. Blocking host
|
||||||
helpers retain operation-specific limits and cap them to the remaining
|
helpers retain operation-specific limits and cap them to the remaining
|
||||||
invocation budget where the underlying operation is cancellable. Forked task
|
invocation budget where the underlying operation is cancellable. Forked task
|
||||||
callbacks receive a fresh invocation deadline capped by the task lifetime.
|
callbacks receive a fresh invocation deadline capped by the task lifetime.
|
||||||
|
Synchronous compiler locks, transitive `#load` compilation, and compiler child
|
||||||
|
processes consume that same deadline. Compiler children run in a dedicated
|
||||||
|
process group; timeout kills the group, retains the previous generation, and
|
||||||
|
never records a caller-budget timeout as a persisted source failure. Bounded
|
||||||
|
compiles require a valid zero child exit, cap captured output, and stage
|
||||||
|
generated C++, exports, source maps, Wasm, and metadata. Publication removes
|
||||||
|
the prior serialized module and diagnostics inside the same rollback-protected
|
||||||
|
generation while repeatedly checking the live request deadline.
|
||||||
|
|
||||||
`request_perf()` reports worker module-cache hits and misses and divides a miss
|
`request_perf()` reports worker module-cache hits and misses and divides a miss
|
||||||
into artifact lookup, wasm read, custom-section parse, serialized-module
|
into artifact lookup, wasm read, custom-section parse, serialized-module
|
||||||
@@ -556,6 +565,13 @@ header free-functions are `inline`. The wasm backend exposes only declarations
|
|||||||
function table, enabling `InstancePre`. A separately built legacy
|
function table, enabling `InstancePre`. A separately built legacy
|
||||||
`--import-table` core must still pass the demo and 64-request pool-isolation
|
`--import-table` core must still pass the demo and 64-request pool-isolation
|
||||||
groups before the compatibility fallback changes.
|
groups before the compatibility fallback changes.
|
||||||
|
`scripts/test_wasm_compile_timeout.sh` runs an isolated one-worker runtime
|
||||||
|
with a two-second request budget. It covers cold entry JIT, guest
|
||||||
|
`unit_compile()`, dynamic component and transitive `#load` compilation,
|
||||||
|
held unit and registry locks, silent nonzero and missing-output compiler
|
||||||
|
results, compiler descendants, staged-output timeout, prior-generation
|
||||||
|
hashes, same-worker recovery, configured-error-page boundedness, residue,
|
||||||
|
and deadline-independent offline precompile.
|
||||||
`scripts/test_cold_component_deadline.sh` separately compiles a deliberately
|
`scripts/test_cold_component_deadline.sh` separately compiles a deliberately
|
||||||
cold component that exceeds the development epoch window and proves the
|
cold component that exceeds the development epoch window and proves the
|
||||||
parent request still renders it. The focused shell gates create temporary
|
parent request still renders it. The focused shell gates create temporary
|
||||||
|
|||||||
@@ -72,6 +72,7 @@ if [[ "$action" == "run" ]]; then
|
|||||||
timeout --signal=TERM --kill-after=5s 175s scripts/test_parallel_proactive_compile.sh
|
timeout --signal=TERM --kill-after=5s 175s scripts/test_parallel_proactive_compile.sh
|
||||||
scripts/test_cold_component_deadline.sh
|
scripts/test_cold_component_deadline.sh
|
||||||
scripts/test_wasm_invocation_timeout.sh
|
scripts/test_wasm_invocation_timeout.sh
|
||||||
|
timeout --signal=TERM --kill-after=5s 130s scripts/test_wasm_compile_timeout.sh
|
||||||
scripts/test_nested_component_props.sh
|
scripts/test_nested_component_props.sh
|
||||||
scripts/test_component_once_prefetch.sh
|
scripts/test_component_once_prefetch.sh
|
||||||
scripts/test_relative_component_cache.sh
|
scripts/test_relative_component_cache.sh
|
||||||
|
|||||||
@@ -398,7 +398,7 @@ rm -f "$cache_dir/parent.uce.cwasm"
|
|||||||
for _ in {1..16}; do assert_marker parent dependency-marker-c; done
|
for _ in {1..16}; do assert_marker parent dependency-marker-c; done
|
||||||
|
|
||||||
race_unit="$source_dir/source-race.uce"
|
race_unit="$source_dir/source-race.uce"
|
||||||
race_cpp="$cache_dir/source-race.uce.cpp"
|
race_cpp=""
|
||||||
{
|
{
|
||||||
printf '%s\n' 'String source_race_marker() { return("source-race-marker-a"); }'
|
printf '%s\n' 'String source_race_marker() { return("source-race-marker-a"); }'
|
||||||
for i in $(seq 1 3500); do
|
for i in $(seq 1 3500); do
|
||||||
@@ -414,16 +414,17 @@ rm -f "$cache_dir/source-race.uce.cpp" "$cache_dir/source-race.uce.wasm" "$cache
|
|||||||
) &
|
) &
|
||||||
race_pid=$!
|
race_pid=$!
|
||||||
deadline=$((SECONDS + 10))
|
deadline=$((SECONDS + 10))
|
||||||
while [[ ! -e "$race_cpp" && $SECONDS -lt $deadline ]]; do
|
while [[ -z "$race_cpp" && $SECONDS -lt $deadline ]]; do
|
||||||
|
race_cpp=$(find "$cache_dir" -maxdepth 1 -name 'source-race.uce.invocation-*.cpp' -print -quit 2>/dev/null || true)
|
||||||
if ! kill -0 "$race_pid" 2>/dev/null; then
|
if ! kill -0 "$race_pid" 2>/dev/null; then
|
||||||
echo "source-race compile finished before publishing generated C++" >&2
|
echo "source-race compile finished before staging generated C++" >&2
|
||||||
cat "$race_output" "$race_error" >&2 || true
|
cat "$race_output" "$race_error" >&2 || true
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
sleep 0.02
|
sleep 0.02
|
||||||
done
|
done
|
||||||
if [[ ! -e "$race_cpp" ]]; then
|
if [[ -z "$race_cpp" || ! -e "$race_cpp" ]]; then
|
||||||
echo "source-race compile did not publish generated C++ before deadline" >&2
|
echo "source-race compile did not stage generated C++ before deadline" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
if ! grep -q 'source-race-marker-a' "$race_cpp"; then
|
if ! grep -q 'source-race-marker-a' "$race_cpp"; then
|
||||||
|
|||||||
Executable
+206
@@ -0,0 +1,206 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
|
||||||
|
error_page_path=$(sed -n '/^bool render_wasm_error_page(/,/^}/p' src/linux_fastcgi.cpp)
|
||||||
|
[[ "$error_page_path" == *'get_shared_unit_bounded('* && "$error_page_path" == *'wasm_invocation_remaining_ms(invocation_deadline)'* ]]
|
||||||
|
[[ "$error_page_path" != *'get_shared_unit(&request'* ]]
|
||||||
|
|
||||||
|
if [[ "${1:-}" != --inside ]]; then
|
||||||
|
exec timeout --signal=TERM --kill-after=5s 120s unshare --mount --fork --kill-child=TERM "$0" --inside
|
||||||
|
fi
|
||||||
|
|
||||||
|
root="/tmp/uce-compile-timeout-$$"
|
||||||
|
site="$root/site"
|
||||||
|
work="$root/work"
|
||||||
|
settings="$root/settings.cfg"
|
||||||
|
log="$root/service.log"
|
||||||
|
server_pid=""
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
status=$?
|
||||||
|
if [[ -n "$server_pid" ]] && kill -0 "$server_pid" 2>/dev/null; then
|
||||||
|
kill -TERM "$server_pid" 2>/dev/null || true
|
||||||
|
for _ in $(seq 1 100); do kill -0 "$server_pid" 2>/dev/null || break; sleep 0.02; done
|
||||||
|
kill -KILL "$server_pid" 2>/dev/null || true
|
||||||
|
wait "$server_pid" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
if (( status != 0 )) && [[ -r "$log" ]]; then cat "$log" >&2; fi
|
||||||
|
rm -rf "$root"
|
||||||
|
return "$status"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
mkdir -p "$site/components" "$work" "$root/run" "$root/session" "$root/upload"
|
||||||
|
sed -E '/^[[:space:]]*(BIN_DIRECTORY|PRECOMPILE_FILES_IN|SITE_DIRECTORY|FCGI_SOCKET_PATH|FCGI_PORT|CLI_SOCKET_PATH|WS_BROKER_SOCKET_PATH|HTTP_PORT|HTTP_DOCUMENT_ROOT|SESSION_PATH|TMP_UPLOAD_PATH|WASM_CORE_PATH|WASM_COMPILE_SCRIPT|WASM_INVOCATION_TIMEOUT_MS|WASM_EPOCH_PERIOD_MS|PROACTIVE_COMPILE_ENABLED|WORKER_COUNT|COMPILE_FAILURE_RETRY_SECONDS)[[:space:]]*=/d' \
|
||||||
|
/etc/uce/settings.cfg >"$settings"
|
||||||
|
cat >>"$settings" <<CFG
|
||||||
|
BIN_DIRECTORY=$work
|
||||||
|
PRECOMPILE_FILES_IN=$site
|
||||||
|
SITE_DIRECTORY=$site
|
||||||
|
FCGI_SOCKET_PATH=$root/run/fastcgi.sock
|
||||||
|
FCGI_PORT=
|
||||||
|
CLI_SOCKET_PATH=$root/run/cli.sock
|
||||||
|
WS_BROKER_SOCKET_PATH=$root/run/ws.sock
|
||||||
|
HTTP_PORT=
|
||||||
|
HTTP_DOCUMENT_ROOT=$site
|
||||||
|
SESSION_PATH=$root/session
|
||||||
|
TMP_UPLOAD_PATH=$root/upload
|
||||||
|
WASM_CORE_PATH=$(pwd)/bin/wasm/core.wasm
|
||||||
|
WASM_COMPILE_SCRIPT=$root/compile
|
||||||
|
WASM_INVOCATION_TIMEOUT_MS=2000
|
||||||
|
WASM_EPOCH_PERIOD_MS=20
|
||||||
|
PROACTIVE_COMPILE_ENABLED=0
|
||||||
|
WORKER_COUNT=1
|
||||||
|
COMPILE_FAILURE_RETRY_SECONDS=60
|
||||||
|
CFG
|
||||||
|
mount --bind "$settings" /etc/uce/settings.cfg
|
||||||
|
|
||||||
|
cat >"$root/compile" <<SHIM
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
source_file="\$3"
|
||||||
|
if [[ -r '$root/stage-timeout' ]] && grep -Fxq "\$source_file" '$root/stage-timeout'; then
|
||||||
|
printf '%s\n' "\$\$" >'$root/compiler-pid'
|
||||||
|
(sleep 3; touch '$root/late-descendant') &
|
||||||
|
cp "\$2/\$(basename "\$source_file").wasm" "\$2/\$5"
|
||||||
|
cp "\$2/\$(basename "\$source_file").wasm.source-map" "\$2/\$5.source-map"
|
||||||
|
sleep 60
|
||||||
|
fi
|
||||||
|
if [[ -r '$root/hold' ]] && grep -Fxq "\$source_file" '$root/hold'; then
|
||||||
|
printf '%s\n' "\$\$" >'$root/compiler-pid'
|
||||||
|
(sleep 3; touch '$root/late-descendant') &
|
||||||
|
sleep 60
|
||||||
|
fi
|
||||||
|
if [[ -r '$root/silent-fail' ]] && grep -Fxq "\$source_file" '$root/silent-fail'; then exit 23; fi
|
||||||
|
if [[ -r '$root/empty-success' ]] && grep -Fxq "\$source_file" '$root/empty-success'; then exit 0; fi
|
||||||
|
if [[ -r '$root/delay' ]] && grep -Fxq "\$source_file" '$root/delay'; then sleep 1; fi
|
||||||
|
exec '$(pwd)/scripts/compile_wasm_unit' "\$@"
|
||||||
|
SHIM
|
||||||
|
chmod +x "$root/compile"
|
||||||
|
|
||||||
|
cat >"$site/driver.uce" <<'UCE'
|
||||||
|
CLI(Request& context) {
|
||||||
|
if(context.get["health"] == "1") { print(request_perf()["worker_pid"].to_u64(), "|health"); return; }
|
||||||
|
if(context.get["dynamic"] == "1") { print(component("slow-component.uce")); return; }
|
||||||
|
String target = first(context.get["target"], "slow.uce");
|
||||||
|
print(unit_compile(target) ? "compiled" : "failed");
|
||||||
|
}
|
||||||
|
UCE
|
||||||
|
printf '%s\n' 'CLI(Request& context) { print("entry"); }' >"$site/cold-entry.uce"
|
||||||
|
printf '%s\n' 'CLI(Request& context) { print("slow"); }' >"$site/slow.uce"
|
||||||
|
printf '%s\n' 'CLI(Request& context) { print("silent"); }' >"$site/silent.uce"
|
||||||
|
printf '%s\n' 'CLI(Request& context) { print("empty"); }' >"$site/empty.uce"
|
||||||
|
printf '%s\n' '#load "transitive-child.uce"' 'CLI(Request& context) { print("parent"); }' >"$site/transitive.uce"
|
||||||
|
printf '%s\n' 'String transitive_value() { return("a"); }' >"$site/transitive-child.uce"
|
||||||
|
printf '%s\n' 'COMPONENT(Request& context) { print("component"); }' >"$site/components/slow-component.uce"
|
||||||
|
|
||||||
|
timeout 40s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >"$root/precompile.log" 2>&1 || { cat "$root/precompile.log" >&2; exit 1; }
|
||||||
|
cache="$(scripts/unit_cache_directory "$work")$(realpath "$site")"
|
||||||
|
generation=$(scripts/unit_cache_directory "$work")
|
||||||
|
rm -f "$cache/cold-entry.uce."* "$cache/components/slow-component.uce."*
|
||||||
|
|
||||||
|
timeout --signal=TERM --kill-after=5s 90s bin/uce_fastcgi.linux.bin >"$log" 2>&1 &
|
||||||
|
server_pid=$!
|
||||||
|
for _ in $(seq 1 400); do [[ -S "$root/run/cli.sock" ]] && break; sleep 0.02; done
|
||||||
|
[[ -S "$root/run/cli.sock" ]] || { cat "$log" >&2; exit 1; }
|
||||||
|
|
||||||
|
request() {
|
||||||
|
local spec="$1" path="${1%%\?*}"
|
||||||
|
if [[ "$spec" == *\?* ]]; then
|
||||||
|
local query="${spec#*\?}"
|
||||||
|
IFS='&' read -r -a params <<<"$query"
|
||||||
|
timeout 5s scripts/uce-cli --get "$path" "${params[@]}"
|
||||||
|
else
|
||||||
|
timeout 5s scripts/uce-cli "$path"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
driver=""
|
||||||
|
for _ in $(seq 1 100); do
|
||||||
|
set +e
|
||||||
|
driver=$(request '/driver.uce?health=1' 2>/dev/null)
|
||||||
|
rc=$?
|
||||||
|
set -e
|
||||||
|
(( rc == 0 )) && [[ "$driver" == *'|health' ]] && break
|
||||||
|
sleep 0.05
|
||||||
|
done
|
||||||
|
[[ "$driver" == *'|health' ]] || { echo "driver did not become ready: $driver" >&2; exit 1; }
|
||||||
|
worker_pid=${driver%%|*}
|
||||||
|
[[ "$driver" == "$worker_pid|health" ]]
|
||||||
|
[[ "$(request /driver.uce)" == compiled ]]
|
||||||
|
before=$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")
|
||||||
|
|
||||||
|
assert_timeout() {
|
||||||
|
local url="$1" source="$2" started elapsed output
|
||||||
|
printf '%s\n' "$source" >"$root/hold"
|
||||||
|
rm -f "$root/compiler-pid" "$root/late-descendant"
|
||||||
|
started=$(date +%s%N)
|
||||||
|
set +e
|
||||||
|
output=$(request "$url" 2>&1)
|
||||||
|
set -e
|
||||||
|
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||||
|
[[ "$output" == *UCE_INVOCATION_TIMEOUT:* ]] || { echo "$url lacked timeout: $output" >&2; exit 1; }
|
||||||
|
(( elapsed >= 1700 && elapsed <= 3500 )) || { echo "$url took ${elapsed}ms" >&2; exit 1; }
|
||||||
|
[[ "$(request '/driver.uce?health=1')" == "$worker_pid|health" ]]
|
||||||
|
[[ -s "$root/compiler-pid" ]]
|
||||||
|
compiler_pid=$(<"$root/compiler-pid")
|
||||||
|
! kill -0 "$compiler_pid" 2>/dev/null || { echo "compiler $compiler_pid survived" >&2; exit 1; }
|
||||||
|
sleep 0.1
|
||||||
|
[[ ! -e "$root/late-descendant" ]]
|
||||||
|
rm -f "$root/hold"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_timeout /driver.uce "$site/slow.uce"
|
||||||
|
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
|
||||||
|
[[ ! -e "$cache/slow.uce.compile.txt" && ! -e "$cache/slow.uce.wasm-check.txt" ]]
|
||||||
|
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
|
||||||
|
[[ "$(request /driver.uce)" == compiled ]]
|
||||||
|
before=$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")
|
||||||
|
|
||||||
|
printf '%s\n' "$site/silent.uce" >"$root/silent-fail"
|
||||||
|
[[ "$(request '/driver.uce?target=silent.uce')" == failed ]]
|
||||||
|
grep -q 'status 23' "$cache/silent.uce.compile.txt"
|
||||||
|
rm -f "$root/silent-fail"
|
||||||
|
printf '%s\n' "$site/empty.uce" >"$root/empty-success"
|
||||||
|
[[ "$(request '/driver.uce?target=empty.uce')" == failed ]]
|
||||||
|
grep -q 'without creating' "$cache/empty.uce.compile.txt"
|
||||||
|
rm -f "$root/empty-success"
|
||||||
|
! find "$cache" -name '*.invocation-*' -print -quit | grep -q .
|
||||||
|
|
||||||
|
printf '%s\n' "$site/slow.uce" >"$root/stage-timeout"
|
||||||
|
assert_timeout /driver.uce "$site/slow.uce"
|
||||||
|
rm -f "$root/stage-timeout"
|
||||||
|
[[ "$before" == "$(sha256sum "$cache/slow.uce.wasm" "$cache/slow.uce.wasm.source-map" "$cache/slow.uce.cpp" "$cache/slow.uce.exports.txt" "$cache/slow.uce.meta.txt")" ]]
|
||||||
|
|
||||||
|
transitive_before=$(sha256sum "$cache/transitive.uce.wasm" "$cache/transitive.uce.wasm.source-map" "$cache/transitive.uce.cpp" "$cache/transitive.uce.exports.txt" "$cache/transitive.uce.meta.txt")
|
||||||
|
printf '%s\n' 'String transitive_value() { return("b"); }' >"$site/transitive-child.uce"
|
||||||
|
assert_timeout '/driver.uce?target=transitive.uce' "$site/transitive-child.uce"
|
||||||
|
[[ "$transitive_before" == "$(sha256sum "$cache/transitive.uce.wasm" "$cache/transitive.uce.wasm.source-map" "$cache/transitive.uce.cpp" "$cache/transitive.uce.exports.txt" "$cache/transitive.uce.meta.txt")" ]]
|
||||||
|
|
||||||
|
assert_timeout '/driver.uce?dynamic=1' "$site/components/slow-component.uce"
|
||||||
|
assert_timeout /cold-entry.uce "$site/cold-entry.uce"
|
||||||
|
|
||||||
|
printf '%s\n' "$site/slow.uce" >"$root/delay"
|
||||||
|
timeout 25s env UCE_PRECOMPILE_FILES_IN="$site" UCE_PRECOMPILE_BIN_DIRECTORY="$work" UCE_PRECOMPILE_JOBS=1 bin/uce_fastcgi.linux.bin --precompile >/dev/null
|
||||||
|
rm -f "$root/delay"
|
||||||
|
|
||||||
|
exec 8>"$generation/known-uce-files.txt.lock"
|
||||||
|
flock 8
|
||||||
|
started=$(date +%s%N)
|
||||||
|
set +e
|
||||||
|
registry_locked=$(request /driver.uce 2>&1)
|
||||||
|
set -e
|
||||||
|
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||||
|
flock -u 8
|
||||||
|
[[ "$registry_locked" == *UCE_INVOCATION_TIMEOUT:* && "$elapsed" -le 3500 ]]
|
||||||
|
|
||||||
|
exec 9>"$cache/slow.uce.wasm.lock"
|
||||||
|
flock 9
|
||||||
|
started=$(date +%s%N)
|
||||||
|
set +e
|
||||||
|
locked=$(request /driver.uce 2>&1)
|
||||||
|
set -e
|
||||||
|
elapsed=$(( ($(date +%s%N) - started) / 1000000 ))
|
||||||
|
flock -u 9
|
||||||
|
[[ "$locked" == *UCE_INVOCATION_TIMEOUT:* && "$elapsed" -le 3500 ]]
|
||||||
|
|
||||||
|
echo "wasm compile timeout passed (entry, explicit, dynamic, unit/registry locks, error page, process group, staging, native precompile)"
|
||||||
@@ -113,7 +113,7 @@ for unit in hostcall-loop.uce legacy-shell.uce sleep.uce; do
|
|||||||
expected_ms=5000
|
expected_ms=5000
|
||||||
fi
|
fi
|
||||||
else
|
else
|
||||||
grep -q 'UCE_INVOCATION_TIMEOUT:' "$body_file" || { echo "$unit lacked invocation-timeout classification" >&2; exit 1; }
|
grep -q 'UCE_INVOCATION_TIMEOUT:' "$body_file" || { echo "$unit lacked invocation-timeout classification: $(cat "$body_file")" >&2; exit 1; }
|
||||||
expected_ms=$invocation_ms
|
expected_ms=$invocation_ms
|
||||||
fi
|
fi
|
||||||
(( elapsed_ms >= expected_ms - 1000 && elapsed_ms <= expected_ms + 3000 )) || { echo "$unit completed in ${elapsed_ms}ms; expected about ${expected_ms}ms" >&2; exit 1; }
|
(( elapsed_ms >= expected_ms - 1000 && elapsed_ms <= expected_ms + 3000 )) || { echo "$unit completed in ${elapsed_ms}ms; expected about ${expected_ms}ms" >&2; exit 1; }
|
||||||
|
|||||||
@@ -567,7 +567,7 @@ String compiler_preprocess_shared_unit_char_wise(Request* context, SharedUnit* s
|
|||||||
String resolved_unit = unit_name;
|
String resolved_unit = unit_name;
|
||||||
if(resolved_unit != "" && resolved_unit[0] != '/')
|
if(resolved_unit != "" && resolved_unit[0] != '/')
|
||||||
resolved_unit = expand_path(resolved_unit, su->src_path);
|
resolved_unit = expand_path(resolved_unit, su->src_path);
|
||||||
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit(context, resolved_unit));
|
SharedUnit* sub_su = (resolved_unit == "" ? 0 : get_shared_unit_for_preprocess(context, resolved_unit));
|
||||||
if(sub_su)
|
if(sub_su)
|
||||||
parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n");
|
parsed_content.append("#include \"" + sub_su->bin_path + "/" + sub_su->pre_file_name + "\"\n");
|
||||||
}
|
}
|
||||||
|
|||||||
+439
-22
@@ -3,6 +3,7 @@
|
|||||||
#include "hash.h"
|
#include "hash.h"
|
||||||
#include "../wasm/abi.h"
|
#include "../wasm/abi.h"
|
||||||
#include <algorithm>
|
#include <algorithm>
|
||||||
|
#include <atomic>
|
||||||
#include <cerrno>
|
#include <cerrno>
|
||||||
#include <chrono>
|
#include <chrono>
|
||||||
#include <cstdlib>
|
#include <cstdlib>
|
||||||
@@ -52,6 +53,49 @@ struct SharedUnitCompileCheck
|
|||||||
bool needs_compile = false;
|
bool needs_compile = false;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
struct CompilerDeadline
|
||||||
|
{
|
||||||
|
using Clock = std::chrono::steady_clock;
|
||||||
|
Clock::time_point expires_at;
|
||||||
|
bool timed_out = false;
|
||||||
|
bool operational_failure = false;
|
||||||
|
String operational_error;
|
||||||
|
bool dependency_failure = false;
|
||||||
|
String dependency_error;
|
||||||
|
|
||||||
|
explicit CompilerDeadline(u64 timeout_ms) : expires_at(Clock::now() + std::chrono::milliseconds(timeout_ms)) {}
|
||||||
|
|
||||||
|
u64 remaining_ms() const
|
||||||
|
{
|
||||||
|
auto now = Clock::now();
|
||||||
|
if(now >= expires_at)
|
||||||
|
return(0);
|
||||||
|
return((u64)std::chrono::duration_cast<std::chrono::milliseconds>(expires_at - now).count());
|
||||||
|
}
|
||||||
|
|
||||||
|
bool expire_if_needed()
|
||||||
|
{
|
||||||
|
if(remaining_ms() > 0)
|
||||||
|
return(false);
|
||||||
|
timed_out = true;
|
||||||
|
return(true);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
static std::atomic<u64> compiler_invocation_stage_counter(0);
|
||||||
|
static thread_local CompilerDeadline* compiler_active_deadline = 0;
|
||||||
|
|
||||||
|
class CompilerDeadlineScope
|
||||||
|
{
|
||||||
|
CompilerDeadline* previous;
|
||||||
|
public:
|
||||||
|
explicit CompilerDeadlineScope(CompilerDeadline* deadline) : previous(compiler_active_deadline)
|
||||||
|
{
|
||||||
|
compiler_active_deadline = deadline;
|
||||||
|
}
|
||||||
|
~CompilerDeadlineScope() { compiler_active_deadline = previous; }
|
||||||
|
};
|
||||||
|
|
||||||
struct UnitSourceSignatureEntry
|
struct UnitSourceSignatureEntry
|
||||||
{
|
{
|
||||||
std::chrono::steady_clock::time_point checked_at;
|
std::chrono::steady_clock::time_point checked_at;
|
||||||
@@ -304,6 +348,92 @@ String compiler_source_map_path(String wasm_path)
|
|||||||
return(wasm_path + ".source-map");
|
return(wasm_path + ".source-map");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static bool compiler_publish_staged_artifacts(SharedUnit* su, String staged_pre_name, String staged_api_name,
|
||||||
|
String staged_wasm_name, String staged_map_name, String staged_meta_name, CompilerDeadline* deadline, String& error)
|
||||||
|
{
|
||||||
|
struct Artifact
|
||||||
|
{
|
||||||
|
String staged;
|
||||||
|
String canonical;
|
||||||
|
String previous;
|
||||||
|
bool remove_only = false;
|
||||||
|
bool existed = false;
|
||||||
|
bool published = false;
|
||||||
|
};
|
||||||
|
std::vector<Artifact> artifacts = {
|
||||||
|
{ staged_pre_name, compiler_generated_cpp_path(su), staged_pre_name + ".previous" },
|
||||||
|
{ staged_api_name, su->api_file_name, staged_api_name + ".previous" },
|
||||||
|
{ staged_map_name, compiler_source_map_path(su->wasm_name), staged_map_name + ".previous" },
|
||||||
|
{ "", compiler_cached_wasm_path(su->wasm_name), staged_wasm_name + ".cached.previous", true },
|
||||||
|
{ staged_wasm_name, su->wasm_name, staged_wasm_name + ".previous" },
|
||||||
|
{ "", su->compile_output_file_name, staged_wasm_name + ".compile.previous", true },
|
||||||
|
{ "", su->wasm_check_file_name, staged_wasm_name + ".check.previous", true },
|
||||||
|
{ staged_meta_name, su->meta_file_name, staged_meta_name + ".previous" }
|
||||||
|
};
|
||||||
|
auto rollback = [&]() {
|
||||||
|
for(auto it = artifacts.rbegin(); it != artifacts.rend(); ++it)
|
||||||
|
{
|
||||||
|
if(it->published)
|
||||||
|
{
|
||||||
|
if(it->existed)
|
||||||
|
{
|
||||||
|
if(rename(it->previous.c_str(), it->canonical.c_str()) != 0)
|
||||||
|
error += "\ncould not restore previous bounded compile artifact " + it->canonical + ": " + String(std::strerror(errno));
|
||||||
|
}
|
||||||
|
else
|
||||||
|
file_unlink(it->canonical);
|
||||||
|
}
|
||||||
|
file_unlink(it->previous);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for(auto& artifact : artifacts)
|
||||||
|
{
|
||||||
|
if(deadline && deadline->expire_if_needed())
|
||||||
|
{
|
||||||
|
error = "bounded compile deadline expired while preserving prior artifacts";
|
||||||
|
for(auto& cleanup : artifacts)
|
||||||
|
file_unlink(cleanup.previous);
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
file_unlink(artifact.previous);
|
||||||
|
artifact.existed = file_exists(artifact.canonical);
|
||||||
|
if(artifact.existed && link(artifact.canonical.c_str(), artifact.previous.c_str()) != 0)
|
||||||
|
{
|
||||||
|
error = "could not preserve previous bounded compile artifacts: " + String(std::strerror(errno));
|
||||||
|
for(auto& cleanup : artifacts)
|
||||||
|
file_unlink(cleanup.previous);
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for(auto& artifact : artifacts)
|
||||||
|
{
|
||||||
|
if(deadline && deadline->expire_if_needed())
|
||||||
|
{
|
||||||
|
error = "bounded compile deadline expired during artifact publication";
|
||||||
|
rollback();
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
if(artifact.remove_only)
|
||||||
|
file_unlink(artifact.canonical);
|
||||||
|
else if(rename(artifact.staged.c_str(), artifact.canonical.c_str()) != 0)
|
||||||
|
{
|
||||||
|
error = "could not publish bounded compile artifacts: " + String(std::strerror(errno));
|
||||||
|
rollback();
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
artifact.published = true;
|
||||||
|
}
|
||||||
|
if(deadline && deadline->expire_if_needed())
|
||||||
|
{
|
||||||
|
error = "bounded compile deadline expired during artifact publication";
|
||||||
|
rollback();
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
for(auto& artifact : artifacts)
|
||||||
|
file_unlink(artifact.previous);
|
||||||
|
return(true);
|
||||||
|
}
|
||||||
|
|
||||||
void compiler_unlink_unit_wasm_artifacts(SharedUnit* su)
|
void compiler_unlink_unit_wasm_artifacts(SharedUnit* su)
|
||||||
{
|
{
|
||||||
file_unlink(su->wasm_name);
|
file_unlink(su->wasm_name);
|
||||||
@@ -437,6 +567,35 @@ int compiler_open_lock_file(String file_name, String purpose, bool nonblocking =
|
|||||||
return(fdlock);
|
return(fdlock);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
int compiler_open_lock_file_bounded(String file_name, String purpose, CompilerDeadline* deadline)
|
||||||
|
{
|
||||||
|
if(!deadline)
|
||||||
|
return(compiler_open_lock_file(file_name, purpose));
|
||||||
|
auto lock_dir = dirname(file_name);
|
||||||
|
if(lock_dir != "")
|
||||||
|
mkdir(lock_dir);
|
||||||
|
int fdlock = open(file_name.c_str(), O_RDWR | O_CREAT, 0666);
|
||||||
|
if(fdlock == -1)
|
||||||
|
return(-1);
|
||||||
|
fcntl(fdlock, F_SETFD, FD_CLOEXEC);
|
||||||
|
while(true)
|
||||||
|
{
|
||||||
|
if(flock(fdlock, LOCK_EX | LOCK_NB) == 0)
|
||||||
|
return(fdlock);
|
||||||
|
if(errno != EWOULDBLOCK && errno != EAGAIN && errno != EINTR)
|
||||||
|
{
|
||||||
|
close(fdlock);
|
||||||
|
return(-1);
|
||||||
|
}
|
||||||
|
if(deadline->expire_if_needed())
|
||||||
|
{
|
||||||
|
close(fdlock);
|
||||||
|
return(-2);
|
||||||
|
}
|
||||||
|
usleep(1000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
void compiler_close_lock_file(int fdlock)
|
void compiler_close_lock_file(int fdlock)
|
||||||
{
|
{
|
||||||
if(fdlock == -1)
|
if(fdlock == -1)
|
||||||
@@ -445,6 +604,18 @@ void compiler_close_lock_file(int fdlock)
|
|||||||
close(fdlock);
|
close(fdlock);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static void compiler_mark_source_generation_nonblocking(Request* context)
|
||||||
|
{
|
||||||
|
if(!context || !context->server)
|
||||||
|
return;
|
||||||
|
String file_name = compiler_source_generation_file_name(context);
|
||||||
|
int fdlock = compiler_open_lock_file(file_name + ".lock", "source-generation", true);
|
||||||
|
if(fdlock < 0)
|
||||||
|
return;
|
||||||
|
file_put_contents(file_name, std::to_string(getpid()) + ":" + std::to_string((u64)(time_precise() * 1000000.0)) + "\n");
|
||||||
|
compiler_close_lock_file(fdlock);
|
||||||
|
}
|
||||||
|
|
||||||
String compiler_normalize_unit_path(Request* context, String file_name)
|
String compiler_normalize_unit_path(Request* context, String file_name)
|
||||||
{
|
{
|
||||||
file_name = trim(file_name);
|
file_name = trim(file_name);
|
||||||
@@ -947,7 +1118,7 @@ String compiler_format_source_read_failure(Request* context, SharedUnit* su, Str
|
|||||||
return(result);
|
return(result);
|
||||||
}
|
}
|
||||||
|
|
||||||
void compile_shared_unit(Request* context, SharedUnit* su)
|
void compile_shared_unit_bounded(Request* context, SharedUnit* su, CompilerDeadline* deadline)
|
||||||
{
|
{
|
||||||
f64 comp_start = time_precise();
|
f64 comp_start = time_precise();
|
||||||
|
|
||||||
@@ -955,9 +1126,13 @@ void compile_shared_unit(Request* context, SharedUnit* su)
|
|||||||
{
|
{
|
||||||
su->compiler_messages = "source file not found (" + su->file_name + ")";
|
su->compiler_messages = "source file not found (" + su->file_name + ")";
|
||||||
file_put_contents(su->compile_output_file_name, su->compiler_messages + "\n");
|
file_put_contents(su->compile_output_file_name, su->compiler_messages + "\n");
|
||||||
compiler_untrack_known_unit(context, su->file_name);
|
if(!deadline)
|
||||||
|
compiler_untrack_known_unit(context, su->file_name);
|
||||||
compiler_record_compile_result(su, time_precise() - comp_start, false, "missing_source", su->compiler_messages);
|
compiler_record_compile_result(su, time_precise() - comp_start, false, "missing_source", su->compiler_messages);
|
||||||
compiler_mark_source_generation(context);
|
if(deadline)
|
||||||
|
compiler_mark_source_generation_nonblocking(context);
|
||||||
|
else
|
||||||
|
compiler_mark_source_generation(context);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
struct stat source_info;
|
struct stat source_info;
|
||||||
@@ -974,39 +1149,180 @@ void compile_shared_unit(Request* context, SharedUnit* su)
|
|||||||
compiler_unlink_unit_wasm_artifacts(su);
|
compiler_unlink_unit_wasm_artifacts(su);
|
||||||
compiler_record_compile_result(su, time_precise() - comp_start, false, "compile_error", su->compiler_messages);
|
compiler_record_compile_result(su, time_precise() - comp_start, false, "compile_error", su->compiler_messages);
|
||||||
printf("%s \n", compiler_format_source_read_failure(context, su, su->compiler_messages).c_str());
|
printf("%s \n", compiler_format_source_read_failure(context, su, su->compiler_messages).c_str());
|
||||||
compiler_mark_source_generation(context);
|
if(deadline)
|
||||||
|
compiler_mark_source_generation_nonblocking(context);
|
||||||
|
else
|
||||||
|
compiler_mark_source_generation(context);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
shell_exec("mkdir -p " + shell_escape(su->pre_path));
|
mkdir(su->pre_path);
|
||||||
String compiled_input_signature;
|
String compiled_input_signature;
|
||||||
|
String staged_wasm_file_name;
|
||||||
|
String staged_wasm_name;
|
||||||
|
String staged_map_name;
|
||||||
|
String staged_pre_file_name;
|
||||||
|
String staged_pre_name;
|
||||||
|
String staged_api_name;
|
||||||
|
String staged_meta_name;
|
||||||
|
bool publication_failed = false;
|
||||||
|
if(deadline)
|
||||||
|
{
|
||||||
|
u64 stage_id = compiler_invocation_stage_counter.fetch_add(1, std::memory_order_relaxed) + 1;
|
||||||
|
String stage_suffix = ".invocation-" + std::to_string((u64)getpid()) + "-" + std::to_string(stage_id);
|
||||||
|
staged_wasm_file_name = su->src_file_name + stage_suffix + ".wasm";
|
||||||
|
staged_wasm_name = su->bin_path + "/" + staged_wasm_file_name;
|
||||||
|
staged_map_name = compiler_source_map_path(staged_wasm_name);
|
||||||
|
staged_pre_file_name = su->src_file_name + stage_suffix + ".cpp";
|
||||||
|
staged_pre_name = su->pre_path + "/" + staged_pre_file_name;
|
||||||
|
staged_api_name = su->bin_path + "/" + su->src_file_name + stage_suffix + ".exports.txt";
|
||||||
|
staged_meta_name = su->bin_path + "/" + su->src_file_name + stage_suffix + ".meta.txt";
|
||||||
|
file_unlink(staged_wasm_name);
|
||||||
|
file_unlink(staged_map_name);
|
||||||
|
file_unlink(staged_pre_name);
|
||||||
|
file_unlink(staged_api_name);
|
||||||
|
file_unlink(staged_meta_name);
|
||||||
|
}
|
||||||
for(u64 attempt = 0; attempt < 2; attempt++)
|
for(u64 attempt = 0; attempt < 2; attempt++)
|
||||||
{
|
{
|
||||||
|
if(deadline && deadline->expire_if_needed())
|
||||||
|
break;
|
||||||
su->api_declarations.clear();
|
su->api_declarations.clear();
|
||||||
compiled_input_signature = compiler_unit_input_signature(context, su);
|
compiled_input_signature = compiler_unit_input_signature(context, su);
|
||||||
file_put_contents(su->pre_path + "/" + su->pre_file_name, preprocess_shared_unit(context, su));
|
String generated_source;
|
||||||
file_put_contents(su->api_file_name, join(su->api_declarations, "\n"));
|
{
|
||||||
|
CompilerDeadlineScope deadline_scope(deadline);
|
||||||
|
generated_source = preprocess_shared_unit(context, su);
|
||||||
|
}
|
||||||
|
if(deadline && deadline->dependency_failure)
|
||||||
|
{
|
||||||
|
su->compiler_messages = first(deadline->dependency_error, "transitive dependency compilation failed");
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if(deadline && (deadline->timed_out || deadline->operational_failure))
|
||||||
|
break;
|
||||||
|
if(!file_put_contents(deadline ? staged_pre_name : su->pre_path + "/" + su->pre_file_name, generated_source) ||
|
||||||
|
!file_put_contents(deadline ? staged_api_name : su->api_file_name, join(su->api_declarations, "\n")))
|
||||||
|
{
|
||||||
|
su->compiler_messages = "could not write generated bounded compile inputs";
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
su->compiler_messages = trim(shell_exec(shell_escape(compiler_wasm_compile_script(context))+" "+
|
String compile_command = shell_escape(compiler_wasm_compile_script(context))+" "+
|
||||||
shell_escape(su->src_path)+" "+
|
shell_escape(su->src_path)+" "+
|
||||||
shell_escape(su->bin_path)+" "+
|
shell_escape(su->bin_path)+" "+
|
||||||
shell_escape(su->file_name)+" "+
|
shell_escape(su->file_name)+" "+
|
||||||
shell_escape(su->pre_file_name)+" "+
|
shell_escape(deadline ? staged_pre_file_name : su->pre_file_name)+" "+
|
||||||
shell_escape(su->wasm_file_name)+" "+
|
shell_escape(deadline ? staged_wasm_file_name : su->wasm_file_name)+" "+
|
||||||
shell_escape(compiler_unit_bin_directory(context))
|
shell_escape(compiler_unit_bin_directory(context));
|
||||||
));
|
if(deadline)
|
||||||
|
{
|
||||||
|
u64 remaining_ms = deadline->remaining_ms();
|
||||||
|
if(remaining_ms == 0)
|
||||||
|
{
|
||||||
|
deadline->timed_out = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
DValue execution = process_exec(compile_command + " 2>&1", "", StringMap(), remaining_ms, 1024 * 1024);
|
||||||
|
if(execution["timed_out"].to_bool())
|
||||||
|
{
|
||||||
|
deadline->timed_out = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
su->compiler_messages = trim(execution["stdout"].to_string() + execution["stderr"].to_string());
|
||||||
|
if(execution["output_truncated"].to_bool())
|
||||||
|
su->compiler_messages += (su->compiler_messages == "" ? "" : "\n") + String("compiler output truncated at 1048576 bytes");
|
||||||
|
if(execution["exit_code"].to_s64(-1) != 0 && su->compiler_messages == "")
|
||||||
|
su->compiler_messages = "wasm compile script exited with status " + std::to_string(execution["exit_code"].to_s64(-1));
|
||||||
|
}
|
||||||
|
else
|
||||||
|
su->compiler_messages = trim(shell_exec(compile_command));
|
||||||
|
|
||||||
if(su->compiler_messages.length() == 0 && !file_exists(su->wasm_name))
|
String compiled_wasm_name = deadline ? staged_wasm_name : su->wasm_name;
|
||||||
su->compiler_messages = "wasm compile script completed without creating " + su->wasm_name;
|
if(su->compiler_messages.length() == 0 && !file_exists(compiled_wasm_name))
|
||||||
|
su->compiler_messages = "wasm compile script completed without creating " + compiled_wasm_name;
|
||||||
if(su->compiler_messages.length() > 0)
|
if(su->compiler_messages.length() > 0)
|
||||||
break;
|
break;
|
||||||
String current_input_signature = compiler_unit_input_signature(context, su);
|
String current_input_signature = compiler_unit_input_signature(context, su);
|
||||||
if(current_input_signature == compiled_input_signature)
|
if(current_input_signature == compiled_input_signature)
|
||||||
|
{
|
||||||
|
if(deadline)
|
||||||
|
{
|
||||||
|
if(deadline->expire_if_needed())
|
||||||
|
break;
|
||||||
|
String source_map = file_get_contents(staged_map_name);
|
||||||
|
if(source_map == "")
|
||||||
|
su->compiler_messages = "bounded wasm compile did not create a source map";
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if(!file_put_contents(staged_map_name, replace(source_map, staged_pre_name, compiler_generated_cpp_path(su))) ||
|
||||||
|
!file_put_contents(staged_meta_name, compiler_unit_metadata_text(context, su, compiled_input_signature)))
|
||||||
|
{
|
||||||
|
su->compiler_messages = "could not stage bounded compile metadata";
|
||||||
|
publication_failed = true;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
publication_failed = !compiler_publish_staged_artifacts(su, staged_pre_name, staged_api_name,
|
||||||
|
staged_wasm_name, staged_map_name, staged_meta_name, deadline, su->compiler_messages);
|
||||||
|
}
|
||||||
|
}
|
||||||
break;
|
break;
|
||||||
compiler_unlink_unit_wasm_artifacts(su);
|
}
|
||||||
|
if(deadline)
|
||||||
|
{
|
||||||
|
file_unlink(staged_wasm_name);
|
||||||
|
file_unlink(staged_map_name);
|
||||||
|
file_unlink(staged_pre_name);
|
||||||
|
file_unlink(staged_api_name);
|
||||||
|
file_unlink(staged_meta_name);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
compiler_unlink_unit_wasm_artifacts(su);
|
||||||
if(attempt == 1)
|
if(attempt == 1)
|
||||||
su->compiler_messages = "source changed during wasm compile; retry required";
|
su->compiler_messages = "source changed during wasm compile; retry required";
|
||||||
}
|
}
|
||||||
|
if(deadline && deadline->timed_out)
|
||||||
|
{
|
||||||
|
file_unlink(staged_wasm_name);
|
||||||
|
file_unlink(staged_map_name);
|
||||||
|
file_unlink(staged_pre_name);
|
||||||
|
file_unlink(staged_api_name);
|
||||||
|
file_unlink(staged_meta_name);
|
||||||
|
su->compiler_messages = "UCE_INVOCATION_TIMEOUT: unit compilation exceeded the invocation deadline";
|
||||||
|
compiler_record_compile_result(su, time_precise() - comp_start, false, "compile_timeout", su->compiler_messages);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if(deadline && deadline->operational_failure)
|
||||||
|
{
|
||||||
|
file_unlink(staged_wasm_name);
|
||||||
|
file_unlink(staged_map_name);
|
||||||
|
file_unlink(staged_pre_name);
|
||||||
|
file_unlink(staged_api_name);
|
||||||
|
file_unlink(staged_meta_name);
|
||||||
|
su->compiler_messages = first(deadline->operational_error, "transitive dependency compilation failed");
|
||||||
|
compiler_record_compile_result(su, time_precise() - comp_start, false, "dependency_error", su->compiler_messages);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if(publication_failed)
|
||||||
|
{
|
||||||
|
file_unlink(staged_wasm_name);
|
||||||
|
file_unlink(staged_map_name);
|
||||||
|
file_unlink(staged_pre_name);
|
||||||
|
file_unlink(staged_api_name);
|
||||||
|
file_unlink(staged_meta_name);
|
||||||
|
compiler_record_compile_result(su, time_precise() - comp_start, false, "publish_error", su->compiler_messages);
|
||||||
|
deadline->operational_failure = true;
|
||||||
|
deadline->operational_error = su->compiler_messages;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if(deadline)
|
||||||
|
{
|
||||||
|
file_unlink(staged_wasm_name);
|
||||||
|
file_unlink(staged_map_name);
|
||||||
|
file_unlink(staged_pre_name);
|
||||||
|
file_unlink(staged_api_name);
|
||||||
|
file_unlink(staged_meta_name);
|
||||||
|
}
|
||||||
|
|
||||||
if(su->compiler_messages.length() > 0)
|
if(su->compiler_messages.length() > 0)
|
||||||
{
|
{
|
||||||
@@ -1014,15 +1330,19 @@ void compile_shared_unit(Request* context, SharedUnit* su)
|
|||||||
file_put_contents(su->compile_output_file_name, raw_messages + "\n");
|
file_put_contents(su->compile_output_file_name, raw_messages + "\n");
|
||||||
file_put_contents(su->wasm_check_file_name, raw_messages + "\n");
|
file_put_contents(su->wasm_check_file_name, raw_messages + "\n");
|
||||||
file_put_contents(su->meta_file_name, compiler_unit_metadata_text(context, su, compiled_input_signature));
|
file_put_contents(su->meta_file_name, compiler_unit_metadata_text(context, su, compiled_input_signature));
|
||||||
compiler_unlink_unit_wasm_artifacts(su);
|
if(!publication_failed)
|
||||||
|
compiler_unlink_unit_wasm_artifacts(su);
|
||||||
compiler_record_compile_result(su, time_precise() - comp_start, false, "compile_error", raw_messages);
|
compiler_record_compile_result(su, time_precise() - comp_start, false, "compile_error", raw_messages);
|
||||||
printf("%s \n", compiler_format_compile_failure(context, su, raw_messages).c_str());
|
printf("%s \n", compiler_format_compile_failure(context, su, raw_messages).c_str());
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
su->last_compiled = file_mtime(su->wasm_name);
|
su->last_compiled = file_mtime(su->wasm_name);
|
||||||
file_unlink(compiler_cached_wasm_path(su->wasm_name));
|
if(!deadline)
|
||||||
file_put_contents(su->meta_file_name, compiler_unit_metadata_text(context, su, compiled_input_signature));
|
{
|
||||||
|
file_unlink(compiler_cached_wasm_path(su->wasm_name));
|
||||||
|
file_put_contents(su->meta_file_name, compiler_unit_metadata_text(context, su, compiled_input_signature));
|
||||||
|
}
|
||||||
file_unlink(su->compile_output_file_name);
|
file_unlink(su->compile_output_file_name);
|
||||||
file_unlink(su->wasm_check_file_name);
|
file_unlink(su->wasm_check_file_name);
|
||||||
compiler_record_compile_result(
|
compiler_record_compile_result(
|
||||||
@@ -1036,10 +1356,18 @@ void compile_shared_unit(Request* context, SharedUnit* su)
|
|||||||
(su->pre_path + "/" + su->pre_file_name).c_str(),
|
(su->pre_path + "/" + su->pre_file_name).c_str(),
|
||||||
time_precise() - comp_start);
|
time_precise() - comp_start);
|
||||||
}
|
}
|
||||||
compiler_mark_source_generation(context);
|
if(deadline)
|
||||||
|
compiler_mark_source_generation_nonblocking(context);
|
||||||
|
else
|
||||||
|
compiler_mark_source_generation(context);
|
||||||
}
|
}
|
||||||
|
|
||||||
SharedUnit* compiler_get_shared_unit_internal(Request* context, String file_name, bool force_recompile, bool retry_current_failure = false)
|
void compile_shared_unit(Request* context, SharedUnit* su)
|
||||||
|
{
|
||||||
|
compile_shared_unit_bounded(context, su, 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
SharedUnit* compiler_get_shared_unit_internal(Request* context, String file_name, bool force_recompile, bool retry_current_failure = false, CompilerDeadline* deadline = 0)
|
||||||
{
|
{
|
||||||
file_name = compiler_normalize_unit_path(context, file_name);
|
file_name = compiler_normalize_unit_path(context, file_name);
|
||||||
bool bypass_cached_result = force_recompile || retry_current_failure;
|
bool bypass_cached_result = force_recompile || retry_current_failure;
|
||||||
@@ -1054,7 +1382,13 @@ SharedUnit* compiler_get_shared_unit_internal(Request* context, String file_name
|
|||||||
setup_unit_paths(context, su, file_name);
|
setup_unit_paths(context, su, file_name);
|
||||||
|
|
||||||
bool can_serve_stale = !bypass_cached_result && compiler_unit_can_serve_stale_artifact(context, file_name);
|
bool can_serve_stale = !bypass_cached_result && compiler_unit_can_serve_stale_artifact(context, file_name);
|
||||||
int fdlock = compiler_open_lock_file(su->wasm_name + ".lock", "shared-unit:" + file_name, can_serve_stale);
|
int fdlock = deadline ? compiler_open_lock_file_bounded(su->wasm_name + ".lock", "shared-unit:" + file_name, deadline) :
|
||||||
|
compiler_open_lock_file(su->wasm_name + ".lock", "shared-unit:" + file_name, can_serve_stale);
|
||||||
|
if(deadline && fdlock == -2)
|
||||||
|
{
|
||||||
|
delete su;
|
||||||
|
return(0);
|
||||||
|
}
|
||||||
if(fdlock == -2 && file_exists(su->wasm_name))
|
if(fdlock == -2 && file_exists(su->wasm_name))
|
||||||
{
|
{
|
||||||
auto state = inspect_shared_unit_filesystem(context, su);
|
auto state = inspect_shared_unit_filesystem(context, su);
|
||||||
@@ -1106,7 +1440,7 @@ SharedUnit* compiler_get_shared_unit_internal(Request* context, String file_name
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
compile_shared_unit(context, su);
|
compile_shared_unit_bounded(context, su, deadline);
|
||||||
}
|
}
|
||||||
else
|
else
|
||||||
{
|
{
|
||||||
@@ -1116,6 +1450,13 @@ SharedUnit* compiler_get_shared_unit_internal(Request* context, String file_name
|
|||||||
su->compile_error_status = "";
|
su->compile_error_status = "";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if(deadline && (deadline->timed_out || deadline->operational_failure))
|
||||||
|
{
|
||||||
|
compiler_close_lock_file(fdlock);
|
||||||
|
delete su;
|
||||||
|
return(0);
|
||||||
|
}
|
||||||
|
|
||||||
auto observed_state = inspect_shared_unit_filesystem(context, su);
|
auto observed_state = inspect_shared_unit_filesystem(context, su);
|
||||||
compiler_record_observed_filesystem_state(su, observed_state);
|
compiler_record_observed_filesystem_state(su, observed_state);
|
||||||
|
|
||||||
@@ -1130,6 +1471,37 @@ SharedUnit* get_shared_unit(Request* context, String file_name)
|
|||||||
return(compiler_get_shared_unit_internal(context, file_name, false));
|
return(compiler_get_shared_unit_internal(context, file_name, false));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
SharedUnit* get_shared_unit_for_preprocess(Request* context, String file_name)
|
||||||
|
{
|
||||||
|
if(!compiler_active_deadline)
|
||||||
|
return(get_shared_unit(context, file_name));
|
||||||
|
auto result = compiler_get_shared_unit_internal(context, file_name, false, false, compiler_active_deadline);
|
||||||
|
if(result && trim(result->compiler_messages) != "")
|
||||||
|
{
|
||||||
|
compiler_active_deadline->dependency_failure = true;
|
||||||
|
compiler_active_deadline->dependency_error = "transitive dependency compilation failed: " + file_name + "\n" + trim(result->compiler_messages);
|
||||||
|
return(0);
|
||||||
|
}
|
||||||
|
if(!result)
|
||||||
|
return(0);
|
||||||
|
if(!file_exists(compiler_generated_cpp_path(result)))
|
||||||
|
{
|
||||||
|
compiler_active_deadline->dependency_failure = true;
|
||||||
|
compiler_active_deadline->dependency_error = "transitive dependency generated source unavailable: " + file_name;
|
||||||
|
return(0);
|
||||||
|
}
|
||||||
|
return(result);
|
||||||
|
}
|
||||||
|
|
||||||
|
SharedUnit* get_shared_unit_bounded(Request* context, String file_name, u64 timeout_ms, bool* timed_out)
|
||||||
|
{
|
||||||
|
CompilerDeadline deadline(timeout_ms);
|
||||||
|
auto result = compiler_get_shared_unit_internal(context, file_name, false, false, &deadline);
|
||||||
|
if(timed_out)
|
||||||
|
*timed_out = deadline.timed_out;
|
||||||
|
return(result);
|
||||||
|
}
|
||||||
|
|
||||||
String compiler_source_generation(Request* context)
|
String compiler_source_generation(Request* context)
|
||||||
{
|
{
|
||||||
if(!context || !context->server)
|
if(!context || !context->server)
|
||||||
@@ -1435,6 +1807,25 @@ void compiler_track_known_unit(Request* context, String file_name)
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static bool compiler_track_known_unit_bounded(Request* context, String file_name, CompilerDeadline* deadline)
|
||||||
|
{
|
||||||
|
file_name = compiler_normalize_unit_path(context, file_name);
|
||||||
|
if(file_name == "" || !compiler_is_known_unit_file(file_name))
|
||||||
|
return(true);
|
||||||
|
String lock_file_name = compiler_registry_lock_file_name(context);
|
||||||
|
int fdlock = compiler_open_lock_file_bounded(lock_file_name, "compiler-registry", deadline);
|
||||||
|
if(fdlock < 0)
|
||||||
|
return(false);
|
||||||
|
auto files = compiler_read_known_units_unlocked(context);
|
||||||
|
if(std::find(files.begin(), files.end(), file_name) == files.end())
|
||||||
|
{
|
||||||
|
files.push_back(file_name);
|
||||||
|
compiler_write_known_units_unlocked(context, files);
|
||||||
|
}
|
||||||
|
compiler_close_lock_file(fdlock);
|
||||||
|
return(true);
|
||||||
|
}
|
||||||
|
|
||||||
void compiler_untrack_known_unit(Request* context, String file_name)
|
void compiler_untrack_known_unit(Request* context, String file_name)
|
||||||
{
|
{
|
||||||
file_name = compiler_normalize_unit_path(context, file_name);
|
file_name = compiler_normalize_unit_path(context, file_name);
|
||||||
@@ -1591,3 +1982,29 @@ bool unit_compile(String path)
|
|||||||
auto su = compiler_get_shared_unit_internal(context, resolved_path, true);
|
auto su = compiler_get_shared_unit_internal(context, resolved_path, true);
|
||||||
return(su && trim(su->compiler_messages) == "" && file_exists(su->wasm_name));
|
return(su && trim(su->compiler_messages) == "" && file_exists(su->wasm_name));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
bool unit_compile_bounded(Request* request, String path, u64 timeout_ms, bool* timed_out)
|
||||||
|
{
|
||||||
|
if(timed_out)
|
||||||
|
*timed_out = false;
|
||||||
|
if(!request || timeout_ms == 0)
|
||||||
|
{
|
||||||
|
if(timed_out)
|
||||||
|
*timed_out = timeout_ms == 0;
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
CompilerDeadline deadline(timeout_ms);
|
||||||
|
String resolved_path = compiler_resolve_unit_path(request, path);
|
||||||
|
if(resolved_path == "")
|
||||||
|
return(false);
|
||||||
|
if(!compiler_track_known_unit_bounded(request, resolved_path, &deadline))
|
||||||
|
{
|
||||||
|
if(timed_out)
|
||||||
|
*timed_out = deadline.timed_out;
|
||||||
|
return(false);
|
||||||
|
}
|
||||||
|
auto su = compiler_get_shared_unit_internal(request, resolved_path, true, false, &deadline);
|
||||||
|
if(timed_out)
|
||||||
|
*timed_out = deadline.timed_out;
|
||||||
|
return(su && trim(su->compiler_messages) == "" && file_exists(su->wasm_name));
|
||||||
|
}
|
||||||
|
|||||||
@@ -25,6 +25,11 @@ String compiler_unit_wasm_path(Request* context, String source_file);
|
|||||||
void setup_unit_paths(Request* context, SharedUnit* su, String file_name);
|
void setup_unit_paths(Request* context, SharedUnit* su, String file_name);
|
||||||
void compile_shared_unit(Request* context, SharedUnit* su);
|
void compile_shared_unit(Request* context, SharedUnit* su);
|
||||||
SharedUnit* get_shared_unit(Request* context, String file_name);
|
SharedUnit* get_shared_unit(Request* context, String file_name);
|
||||||
|
#ifndef __UCE_WASM_UNIT__
|
||||||
|
SharedUnit* get_shared_unit_for_preprocess(Request* context, String file_name);
|
||||||
|
SharedUnit* get_shared_unit_bounded(Request* context, String file_name, u64 timeout_ms, bool* timed_out);
|
||||||
|
bool unit_compile_bounded(Request* context, String path, u64 timeout_ms, bool* timed_out);
|
||||||
|
#endif
|
||||||
String compiler_error_page_unit(Request* context, String config_key);
|
String compiler_error_page_unit(Request* context, String config_key);
|
||||||
bool compiler_unit_compile_pending(Request* context, String file_name);
|
bool compiler_unit_compile_pending(Request* context, String file_name);
|
||||||
bool compiler_unit_compile_in_progress(Request* context, String file_name);
|
bool compiler_unit_compile_in_progress(Request* context, String file_name);
|
||||||
|
|||||||
+179
@@ -1438,6 +1438,185 @@ bool child_exit_status_take(pid_t pid, int& status, unsigned int since)
|
|||||||
return(false);
|
return(false);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
class ProcessSigchldBlock
|
||||||
|
{
|
||||||
|
sigset_t previous;
|
||||||
|
bool blocked = false;
|
||||||
|
public:
|
||||||
|
ProcessSigchldBlock()
|
||||||
|
{
|
||||||
|
sigset_t mask;
|
||||||
|
sigemptyset(&mask);
|
||||||
|
sigaddset(&mask, SIGCHLD);
|
||||||
|
blocked = pthread_sigmask(SIG_BLOCK, &mask, &previous) == 0;
|
||||||
|
}
|
||||||
|
void restore()
|
||||||
|
{
|
||||||
|
if(blocked)
|
||||||
|
{
|
||||||
|
pthread_sigmask(SIG_SETMASK, &previous, 0);
|
||||||
|
blocked = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
~ProcessSigchldBlock() { restore(); }
|
||||||
|
};
|
||||||
|
|
||||||
|
}
|
||||||
|
|
||||||
|
DValue process_exec(String cmd, String input, StringMap env, u64 timeout_ms, u64 output_limit)
|
||||||
|
{
|
||||||
|
DValue result;
|
||||||
|
result["exit_code"] = (f64)-1;
|
||||||
|
result["stdout"] = "";
|
||||||
|
result["stderr"] = "";
|
||||||
|
result["timed_out"].set_bool(false);
|
||||||
|
result["output_truncated"].set_bool(false);
|
||||||
|
if(timeout_ms == 0)
|
||||||
|
timeout_ms = 5000;
|
||||||
|
int inpipe[2] = {-1, -1}, outpipe[2] = {-1, -1}, errpipe[2] = {-1, -1};
|
||||||
|
if(pipe(inpipe) != 0 || pipe(outpipe) != 0 || pipe(errpipe) != 0)
|
||||||
|
{
|
||||||
|
for(int fd : {inpipe[0], inpipe[1], outpipe[0], outpipe[1], errpipe[0], errpipe[1]})
|
||||||
|
if(fd >= 0)
|
||||||
|
close(fd);
|
||||||
|
result["stderr"] = "pipe failed";
|
||||||
|
return(result);
|
||||||
|
}
|
||||||
|
ProcessSigchldBlock sigchld;
|
||||||
|
unsigned int child_status_snapshot_value = child_exit_status_snapshot();
|
||||||
|
pid_t pid = fork();
|
||||||
|
if(pid < 0)
|
||||||
|
{
|
||||||
|
close(inpipe[0]); close(inpipe[1]); close(outpipe[0]); close(outpipe[1]); close(errpipe[0]); close(errpipe[1]);
|
||||||
|
result["stderr"] = "fork failed";
|
||||||
|
return(result);
|
||||||
|
}
|
||||||
|
if(pid == 0)
|
||||||
|
{
|
||||||
|
sigchld.restore();
|
||||||
|
setpgid(0, 0);
|
||||||
|
pid_t expected_parent = getppid();
|
||||||
|
prctl(PR_SET_PDEATHSIG, SIGKILL);
|
||||||
|
if(getppid() != expected_parent)
|
||||||
|
_exit(127);
|
||||||
|
dup2(inpipe[0], 0); dup2(outpipe[1], 1); dup2(errpipe[1], 2);
|
||||||
|
close(inpipe[0]); close(inpipe[1]); close(outpipe[0]); close(outpipe[1]); close(errpipe[0]); close(errpipe[1]);
|
||||||
|
for(auto& value : env)
|
||||||
|
setenv(value.first.c_str(), value.second.c_str(), 1);
|
||||||
|
execl("/bin/sh", "sh", "-c", cmd.c_str(), (char*)0);
|
||||||
|
_exit(127);
|
||||||
|
}
|
||||||
|
setpgid(pid, pid);
|
||||||
|
close(inpipe[0]); close(outpipe[1]); close(errpipe[1]);
|
||||||
|
fcntl(inpipe[1], F_SETFL, fcntl(inpipe[1], F_GETFL, 0) | O_NONBLOCK);
|
||||||
|
fcntl(outpipe[0], F_SETFL, fcntl(outpipe[0], F_GETFL, 0) | O_NONBLOCK);
|
||||||
|
fcntl(errpipe[0], F_SETFL, fcntl(errpipe[0], F_GETFL, 0) | O_NONBLOCK);
|
||||||
|
size_t input_offset = 0;
|
||||||
|
bool input_open = true, output_open = true, error_open = true, exited = false, status_valid = false;
|
||||||
|
int status = 0;
|
||||||
|
u64 now_ms = monotonic_ms();
|
||||||
|
u64 deadline = timeout_ms > UINT64_MAX - now_ms ? UINT64_MAX : now_ms + timeout_ms;
|
||||||
|
auto append_output = [&](String key, const char* data, size_t length) {
|
||||||
|
String current = result[key].to_string();
|
||||||
|
u64 captured = result["stdout"].to_string().size() + result["stderr"].to_string().size();
|
||||||
|
size_t accepted = length;
|
||||||
|
if(output_limit > 0)
|
||||||
|
{
|
||||||
|
u64 available = captured < output_limit ? output_limit - captured : 0;
|
||||||
|
accepted = std::min<size_t>(accepted, (size_t)available);
|
||||||
|
if(accepted < length)
|
||||||
|
result["output_truncated"].set_bool(true);
|
||||||
|
}
|
||||||
|
if(accepted > 0)
|
||||||
|
result[key] = current + String(data, accepted);
|
||||||
|
};
|
||||||
|
while(output_open || error_open || !exited)
|
||||||
|
{
|
||||||
|
if(!exited)
|
||||||
|
{
|
||||||
|
pid_t waited = waitpid(pid, &status, WNOHANG);
|
||||||
|
if(waited == pid)
|
||||||
|
{
|
||||||
|
exited = true;
|
||||||
|
status_valid = true;
|
||||||
|
}
|
||||||
|
else if(waited < 0 && errno == ECHILD)
|
||||||
|
{
|
||||||
|
u64 transfer_deadline = monotonic_ms() + 50;
|
||||||
|
do
|
||||||
|
{
|
||||||
|
status_valid = child_exit_status_take(pid, status, child_status_snapshot_value);
|
||||||
|
if(!status_valid)
|
||||||
|
sched_yield();
|
||||||
|
} while(!status_valid && monotonic_ms() < transfer_deadline);
|
||||||
|
exited = true;
|
||||||
|
if(!status_valid)
|
||||||
|
result["stderr"] = result["stderr"].to_string() + "lost child exit status";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if(input_open)
|
||||||
|
{
|
||||||
|
if(input_offset < input.size())
|
||||||
|
{
|
||||||
|
ssize_t written = write(inpipe[1], input.data() + input_offset, input.size() - input_offset);
|
||||||
|
if(written > 0)
|
||||||
|
input_offset += (size_t)written;
|
||||||
|
else if(written < 0 && errno != EINTR && errno != EAGAIN && errno != EWOULDBLOCK)
|
||||||
|
{
|
||||||
|
close(inpipe[1]);
|
||||||
|
input_open = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
close(inpipe[1]);
|
||||||
|
input_open = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
char buffer[4096];
|
||||||
|
ssize_t length;
|
||||||
|
while((length = read(outpipe[0], buffer, sizeof(buffer))) > 0)
|
||||||
|
append_output("stdout", buffer, (size_t)length);
|
||||||
|
if(length == 0 && output_open)
|
||||||
|
{
|
||||||
|
close(outpipe[0]);
|
||||||
|
output_open = false;
|
||||||
|
}
|
||||||
|
while((length = read(errpipe[0], buffer, sizeof(buffer))) > 0)
|
||||||
|
append_output("stderr", buffer, (size_t)length);
|
||||||
|
if(length == 0 && error_open)
|
||||||
|
{
|
||||||
|
close(errpipe[0]);
|
||||||
|
error_open = false;
|
||||||
|
}
|
||||||
|
if((output_open || error_open || !exited) && monotonic_ms() >= deadline)
|
||||||
|
{
|
||||||
|
result["timed_out"].set_bool(true);
|
||||||
|
kill(-pid, SIGKILL);
|
||||||
|
kill(pid, SIGKILL);
|
||||||
|
if(!exited)
|
||||||
|
status_valid = waitpid(pid, &status, 0) == pid;
|
||||||
|
exited = true;
|
||||||
|
if(input_open) { close(inpipe[1]); input_open = false; }
|
||||||
|
if(output_open) { close(outpipe[0]); output_open = false; }
|
||||||
|
if(error_open) { close(errpipe[0]); error_open = false; }
|
||||||
|
}
|
||||||
|
if(output_open || error_open || !exited)
|
||||||
|
{
|
||||||
|
u64 remaining_ms = deadline > monotonic_ms() ? deadline - monotonic_ms() : 0;
|
||||||
|
if(remaining_ms > 0)
|
||||||
|
usleep((useconds_t)std::min<u64>(1000, remaining_ms * 1000));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if(status_valid && WIFEXITED(status))
|
||||||
|
result["exit_code"] = (f64)WEXITSTATUS(status);
|
||||||
|
else if(status_valid && WIFSIGNALED(status))
|
||||||
|
result["exit_code"] = (f64)(128 + WTERMSIG(status));
|
||||||
|
return(result);
|
||||||
|
}
|
||||||
|
|
||||||
pid_t spawn_subprocess(std::function<void()> exec_after_spawn)
|
pid_t spawn_subprocess(std::function<void()> exec_after_spawn)
|
||||||
{
|
{
|
||||||
parent_pid = getpid();
|
parent_pid = getpid();
|
||||||
|
|||||||
@@ -30,6 +30,9 @@ bool child_exit_status_take(pid_t pid, int& status, unsigned int since);
|
|||||||
struct DValue;
|
struct DValue;
|
||||||
|
|
||||||
String shell_exec(String cmd);
|
String shell_exec(String cmd);
|
||||||
|
#if !defined(__UCE_WASM_CORE__) && !defined(__UCE_WASM_UNIT__)
|
||||||
|
DValue process_exec(String cmd, String input, StringMap env, u64 timeout_ms, u64 output_limit = 0);
|
||||||
|
#endif
|
||||||
DValue http_request(DValue req);
|
DValue http_request(DValue req);
|
||||||
u64 http_request_async(DValue req);
|
u64 http_request_async(DValue req);
|
||||||
DValue shell_exec(DValue spec);
|
DValue shell_exec(DValue spec);
|
||||||
|
|||||||
+69
-16
@@ -6,6 +6,7 @@
|
|||||||
// Minimal FastCGI client: connection brokers forward to a clean-engine worker.
|
// Minimal FastCGI client: connection brokers forward to a clean-engine worker.
|
||||||
#include "lib/fcgi_forward.h"
|
#include "lib/fcgi_forward.h"
|
||||||
#include <csetjmp>
|
#include <csetjmp>
|
||||||
|
#include <chrono>
|
||||||
#include <errno.h>
|
#include <errno.h>
|
||||||
#include <fcntl.h>
|
#include <fcntl.h>
|
||||||
#include <sys/socket.h>
|
#include <sys/socket.h>
|
||||||
@@ -43,6 +44,16 @@ static volatile sig_atomic_t request_fault_frame_count = 0;
|
|||||||
|
|
||||||
void close_inherited_server_sockets();
|
void close_inherited_server_sockets();
|
||||||
u64 request_seed_from_time(f64 time_value);
|
u64 request_seed_from_time(f64 time_value);
|
||||||
|
|
||||||
|
using WasmInvocationClock = std::chrono::steady_clock;
|
||||||
|
u64 wasm_invocation_remaining_ms(WasmInvocationClock::time_point deadline)
|
||||||
|
{
|
||||||
|
auto now = WasmInvocationClock::now();
|
||||||
|
if(now >= deadline)
|
||||||
|
return(0);
|
||||||
|
return((u64)std::chrono::duration_cast<std::chrono::milliseconds>(deadline - now).count());
|
||||||
|
}
|
||||||
|
|
||||||
void prepare_request_body_maps(Request& request);
|
void prepare_request_body_maps(Request& request);
|
||||||
|
|
||||||
Request* set_active_request(Request& request)
|
Request* set_active_request(Request& request)
|
||||||
@@ -73,7 +84,8 @@ void clear_request_output(Request& request)
|
|||||||
request.ob_start();
|
request.ob_start();
|
||||||
}
|
}
|
||||||
|
|
||||||
bool render_wasm_error_page(Request& request, String config_key, s32 status_code, String status_reason, DValue error_info)
|
bool render_wasm_error_page(Request& request, String config_key, s32 status_code, String status_reason, DValue error_info,
|
||||||
|
WasmInvocationClock::time_point invocation_deadline, u64 invocation_budget_ms)
|
||||||
{
|
{
|
||||||
if(!(request.params["REQUEST_METHOD"] != "" && request.ob && request.ob_stack.size() > 0))
|
if(!(request.params["REQUEST_METHOD"] != "" && request.ob && request.ob_stack.size() > 0))
|
||||||
return(false);
|
return(false);
|
||||||
@@ -93,13 +105,22 @@ bool render_wasm_error_page(Request& request, String config_key, s32 status_code
|
|||||||
request.header["Content-Type"] = first(request.server->config["CONTENT_TYPE"], "text/html; charset=utf-8");
|
request.header["Content-Type"] = first(request.server->config["CONTENT_TYPE"], "text/html; charset=utf-8");
|
||||||
|
|
||||||
String unit = compiler_normalize_unit_path(&request, unit_file);
|
String unit = compiler_normalize_unit_path(&request, unit_file);
|
||||||
|
bool compile_timed_out = false;
|
||||||
if(!wasm_backend_should_handle(request, unit))
|
if(!wasm_backend_should_handle(request, unit))
|
||||||
get_shared_unit(&request, unit);
|
{
|
||||||
|
u64 remaining_ms = wasm_invocation_remaining_ms(invocation_deadline);
|
||||||
|
if(remaining_ms == 0)
|
||||||
|
compile_timed_out = true;
|
||||||
|
else
|
||||||
|
get_shared_unit_bounded(&request, unit, remaining_ms, &compile_timed_out);
|
||||||
|
}
|
||||||
|
|
||||||
ob_start();
|
ob_start();
|
||||||
String wasm_error = "";
|
String wasm_error = "";
|
||||||
if(wasm_backend_should_handle(request, unit))
|
if(compile_timed_out)
|
||||||
wasm_error = wasm_backend_serve(request, unit, "render");
|
wasm_error = "UCE_INVOCATION_TIMEOUT: wasm invocation exceeded " + std::to_string(invocation_budget_ms) + " ms";
|
||||||
|
else if(wasm_backend_should_handle(request, unit))
|
||||||
|
wasm_error = wasm_backend_serve(request, unit, "render", wasm_invocation_remaining_ms(invocation_deadline));
|
||||||
else
|
else
|
||||||
wasm_error = "error page wasm unit unavailable after compile: " + unit_file;
|
wasm_error = "error page wasm unit unavailable after compile: " + unit_file;
|
||||||
String html = ob_get_close();
|
String html = ob_get_close();
|
||||||
@@ -122,7 +143,8 @@ bool render_wasm_error_page(Request& request, String config_key, s32 status_code
|
|||||||
return(true);
|
return(true);
|
||||||
}
|
}
|
||||||
|
|
||||||
void render_request_failure(Request& request, String title, String details, String trace, int status_code = 500)
|
void render_request_failure(Request& request, String title, String details, String trace,
|
||||||
|
WasmInvocationClock::time_point invocation_deadline, u64 invocation_budget_ms, int status_code = 500)
|
||||||
{
|
{
|
||||||
request.response_code = request_status_line(request, status_code, "Internal Server Error");
|
request.response_code = request_status_line(request, status_code, "Internal Server Error");
|
||||||
request.header.clear();
|
request.header.clear();
|
||||||
@@ -149,7 +171,8 @@ void render_request_failure(Request& request, String title, String details, Stri
|
|||||||
error_info["signal_name"] = signal_name((int)request_fault_signal);
|
error_info["signal_name"] = signal_name((int)request_fault_signal);
|
||||||
}
|
}
|
||||||
error_info["trace"] = trace;
|
error_info["trace"] = trace;
|
||||||
if(render_wasm_error_page(request, "page_runtime_error", status_code, "Internal Server Error", error_info))
|
if(render_wasm_error_page(request, "page_runtime_error", status_code, "Internal Server Error", error_info,
|
||||||
|
invocation_deadline, invocation_budget_ms))
|
||||||
{
|
{
|
||||||
request.err = "UCE runtime error: " + title + (details != "" ? " (" + details + ")" : "");
|
request.err = "UCE runtime error: " + title + (details != "" ? " (" + details + ")" : "");
|
||||||
restore_active_request(previous_context);
|
restore_active_request(previous_context);
|
||||||
@@ -388,6 +411,8 @@ int handle_cli_complete(FastCGIRequest& request)
|
|||||||
request.random_index = 0;
|
request.random_index = 0;
|
||||||
request.random_seed = request_seed_from_time(request.stats.time_start);
|
request.random_seed = request_seed_from_time(request.stats.time_start);
|
||||||
request.ob_start();
|
request.ob_start();
|
||||||
|
u64 invocation_budget_ms = wasm_backend_invocation_timeout_ms(request);
|
||||||
|
auto invocation_deadline = WasmInvocationClock::now() + std::chrono::milliseconds(invocation_budget_ms);
|
||||||
|
|
||||||
String method = trim(request.params["REQUEST_METHOD"]);
|
String method = trim(request.params["REQUEST_METHOD"]);
|
||||||
String command = trim(first(request.params["DOCUMENT_URI"], request.params["REQUEST_URI"]));
|
String command = trim(first(request.params["DOCUMENT_URI"], request.params["REQUEST_URI"]));
|
||||||
@@ -441,15 +466,26 @@ int handle_cli_complete(FastCGIRequest& request)
|
|||||||
// been removed, so a unit that still cannot be served by wasm is a
|
// been removed, so a unit that still cannot be served by wasm is a
|
||||||
// request failure instead of a fallback path.
|
// request failure instead of a fallback path.
|
||||||
SharedUnit* cli_compile_state = 0;
|
SharedUnit* cli_compile_state = 0;
|
||||||
|
bool cli_compile_timed_out = false;
|
||||||
bool cli_wasm_ready = wasm_backend_should_handle(request, cli_unit);
|
bool cli_wasm_ready = wasm_backend_should_handle(request, cli_unit);
|
||||||
if(!cli_wasm_ready)
|
if(!cli_wasm_ready)
|
||||||
{
|
{
|
||||||
cli_compile_state = get_shared_unit(&request, cli_unit);
|
u64 remaining_ms = wasm_invocation_remaining_ms(invocation_deadline);
|
||||||
cli_wasm_ready = wasm_backend_should_handle(request, cli_unit);
|
if(remaining_ms == 0)
|
||||||
|
cli_compile_timed_out = true;
|
||||||
|
else
|
||||||
|
cli_compile_state = get_shared_unit_bounded(&request, cli_unit, remaining_ms, &cli_compile_timed_out);
|
||||||
|
if(!cli_compile_timed_out)
|
||||||
|
cli_wasm_ready = wasm_backend_should_handle(request, cli_unit);
|
||||||
}
|
}
|
||||||
if(cli_wasm_ready)
|
if(cli_compile_timed_out)
|
||||||
{
|
{
|
||||||
String wasm_error = wasm_backend_serve(request, cli_unit, "cli");
|
request.set_status(500, "Internal Server Error");
|
||||||
|
print("UCE_INVOCATION_TIMEOUT: wasm invocation exceeded ", invocation_budget_ms, " ms\n");
|
||||||
|
}
|
||||||
|
else if(cli_wasm_ready)
|
||||||
|
{
|
||||||
|
String wasm_error = wasm_backend_serve(request, cli_unit, "cli", wasm_invocation_remaining_ms(invocation_deadline));
|
||||||
if(wasm_error != "")
|
if(wasm_error != "")
|
||||||
{
|
{
|
||||||
request.set_status(500, "Internal Server Error");
|
request.set_status(500, "Internal Server Error");
|
||||||
@@ -476,11 +512,13 @@ int handle_cli_complete(FastCGIRequest& request)
|
|||||||
}
|
}
|
||||||
catch(const std::exception& e)
|
catch(const std::exception& e)
|
||||||
{
|
{
|
||||||
render_request_failure(request, "uncaught exception during CLI request", e.what(), backtrace_capture(32, 1), 500);
|
render_request_failure(request, "uncaught exception during CLI request", e.what(), backtrace_capture(32, 1),
|
||||||
|
invocation_deadline, invocation_budget_ms, 500);
|
||||||
}
|
}
|
||||||
catch(...)
|
catch(...)
|
||||||
{
|
{
|
||||||
render_request_failure(request, "unknown uncaught exception during CLI request", "", backtrace_capture(32, 1), 500);
|
render_request_failure(request, "unknown uncaught exception during CLI request", "", backtrace_capture(32, 1),
|
||||||
|
invocation_deadline, invocation_budget_ms, 500);
|
||||||
}
|
}
|
||||||
|
|
||||||
for(auto &f : request.uploaded_files)
|
for(auto &f : request.uploaded_files)
|
||||||
@@ -522,6 +560,8 @@ int handle_complete(FastCGIRequest& request) {
|
|||||||
request.random_index = 0;
|
request.random_index = 0;
|
||||||
request.random_seed = request_seed_from_time(request.stats.time_start);
|
request.random_seed = request_seed_from_time(request.stats.time_start);
|
||||||
request.ob_start();
|
request.ob_start();
|
||||||
|
u64 invocation_budget_ms = wasm_backend_invocation_timeout_ms(request);
|
||||||
|
auto invocation_deadline = WasmInvocationClock::now() + std::chrono::milliseconds(invocation_budget_ms);
|
||||||
request_fault_request = &request;
|
request_fault_request = &request;
|
||||||
request_fault_active = 1;
|
request_fault_active = 1;
|
||||||
request_fault_signal = 0;
|
request_fault_signal = 0;
|
||||||
@@ -557,7 +597,7 @@ int handle_complete(FastCGIRequest& request) {
|
|||||||
request_fault_active = 0;
|
request_fault_active = 0;
|
||||||
restore_request_fault_handlers();
|
restore_request_fault_handlers();
|
||||||
request.stats.wasm_backend_started = time_precise();
|
request.stats.wasm_backend_started = time_precise();
|
||||||
String wasm_error = wasm_backend_serve(request, entry_unit, handler);
|
String wasm_error = wasm_backend_serve(request, entry_unit, handler, wasm_invocation_remaining_ms(invocation_deadline));
|
||||||
request.stats.wasm_backend_finished = time_precise();
|
request.stats.wasm_backend_finished = time_precise();
|
||||||
install_request_fault_handlers();
|
install_request_fault_handlers();
|
||||||
request_fault_active = 1;
|
request_fault_active = 1;
|
||||||
@@ -605,13 +645,25 @@ int handle_complete(FastCGIRequest& request) {
|
|||||||
// recheck. Native execution has been removed, so a unit that still cannot
|
// recheck. Native execution has been removed, so a unit that still cannot
|
||||||
// be served by wasm becomes a clean 500 request failure.
|
// be served by wasm becomes a clean 500 request failure.
|
||||||
SharedUnit* entry_compile_state = 0;
|
SharedUnit* entry_compile_state = 0;
|
||||||
|
bool entry_compile_timed_out = false;
|
||||||
auto wasm_ready = [&](const String& unit) -> bool {
|
auto wasm_ready = [&](const String& unit) -> bool {
|
||||||
if(wasm_backend_should_handle(request, unit))
|
if(wasm_backend_should_handle(request, unit))
|
||||||
return(true);
|
return(true);
|
||||||
entry_compile_state = get_shared_unit(&request, unit);
|
u64 remaining_ms = wasm_invocation_remaining_ms(invocation_deadline);
|
||||||
return(wasm_backend_should_handle(request, unit));
|
if(remaining_ms == 0)
|
||||||
|
entry_compile_timed_out = true;
|
||||||
|
else
|
||||||
|
entry_compile_state = get_shared_unit_bounded(&request, unit, remaining_ms, &entry_compile_timed_out);
|
||||||
|
return(!entry_compile_timed_out && wasm_backend_should_handle(request, unit));
|
||||||
};
|
};
|
||||||
auto fail_wasm_unavailable = [&](const String& handler) {
|
auto fail_wasm_unavailable = [&](const String& handler) {
|
||||||
|
if(entry_compile_timed_out)
|
||||||
|
{
|
||||||
|
failure_title = "wasm invocation timeout";
|
||||||
|
failure_details = "";
|
||||||
|
failure_trace = "UCE_INVOCATION_TIMEOUT: wasm invocation exceeded " + std::to_string(invocation_budget_ms) + " ms";
|
||||||
|
return;
|
||||||
|
}
|
||||||
failure_title = "wasm unit unavailable after compile";
|
failure_title = "wasm unit unavailable after compile";
|
||||||
String compile_error = entry_compile_state ? first(entry_compile_state->compiler_messages, entry_compile_state->compile_error_status) : String("");
|
String compile_error = entry_compile_state ? first(entry_compile_state->compiler_messages, entry_compile_state->compile_error_status) : String("");
|
||||||
failure_details = compile_error != "" ? compile_error : handler + " handler could not be served by wasm";
|
failure_details = compile_error != "" ? compile_error : handler + " handler could not be served by wasm";
|
||||||
@@ -691,7 +743,8 @@ int handle_complete(FastCGIRequest& request) {
|
|||||||
restore_request_fault_handlers();
|
restore_request_fault_handlers();
|
||||||
|
|
||||||
if(failure_title != "")
|
if(failure_title != "")
|
||||||
render_request_failure(request, failure_title, failure_details, failure_trace, 500);
|
render_request_failure(request, failure_title, failure_details, failure_trace,
|
||||||
|
invocation_deadline, invocation_budget_ms, 500);
|
||||||
|
|
||||||
for( auto &f : request.uploaded_files)
|
for( auto &f : request.uploaded_files)
|
||||||
{
|
{
|
||||||
|
|||||||
+12
-2
@@ -178,15 +178,25 @@ bool wasm_backend_should_handle(Request& request, const String& entry_unit)
|
|||||||
return(true);
|
return(true);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
u64 wasm_backend_invocation_timeout_ms(Request& request)
|
||||||
|
{
|
||||||
|
if(!request.server)
|
||||||
|
return(0);
|
||||||
|
u64 timeout_ms = to_u64(first(request.server->config["WASM_INVOCATION_TIMEOUT_MS"], "30000"), 0);
|
||||||
|
return(timeout_ms > 0 && timeout_ms <= 86400000 ? timeout_ms : 0);
|
||||||
|
}
|
||||||
|
|
||||||
// Serve a request through a wasm workspace using the unit handler selected by
|
// Serve a request through a wasm workspace using the unit handler selected by
|
||||||
// `kind` (page render / cli / serve_http, with an optional named serve_http
|
// `kind` (page render / cli / serve_http, with an optional named serve_http
|
||||||
// handler). Populates the native Request (status/headers/cookies/session/body)
|
// handler). Populates the native Request (status/headers/cookies/session/body)
|
||||||
// so the existing transport writes the response unchanged. Returns "" on
|
// so the existing transport writes the response unchanged. Returns "" on
|
||||||
// success, or a collapsed error/trace string for the caller to route into the
|
// success, or a collapsed error/trace string for the caller to route into the
|
||||||
// configured error page.
|
// configured error page.
|
||||||
String wasm_backend_serve(Request& request, const String& entry_unit, const String& handler = "render")
|
String wasm_backend_serve(Request& request, const String& entry_unit, const String& handler, u64 timeout_cap_ms)
|
||||||
{
|
{
|
||||||
WasmResponse response = wasm_worker_serve(*g_wasm_worker, request, entry_unit, handler);
|
if(timeout_cap_ms == 0)
|
||||||
|
return("UCE_INVOCATION_TIMEOUT: wasm invocation exceeded " + std::to_string(wasm_backend_invocation_timeout_ms(request)) + " ms");
|
||||||
|
WasmResponse response = wasm_worker_serve(*g_wasm_worker, request, entry_unit, handler, timeout_cap_ms);
|
||||||
request.stats.wasm_dispatch_us = response.dispatch_us;
|
request.stats.wasm_dispatch_us = response.dispatch_us;
|
||||||
request.stats.wasm_workspace_complete_us = response.workspace_complete_us;
|
request.stats.wasm_workspace_complete_us = response.workspace_complete_us;
|
||||||
request.stats.wasm_entry_invoke_us = response.entry_invoke_us;
|
request.stats.wasm_entry_invoke_us = response.entry_invoke_us;
|
||||||
|
|||||||
+2
-1
@@ -9,6 +9,7 @@
|
|||||||
// True if this request can be served by the wasm backend for the named unit
|
// True if this request can be served by the wasm backend for the named unit
|
||||||
// artifact; handler-agnostic, the caller names the handler.
|
// artifact; handler-agnostic, the caller names the handler.
|
||||||
bool wasm_backend_should_handle(Request& request, const String& entry_unit);
|
bool wasm_backend_should_handle(Request& request, const String& entry_unit);
|
||||||
|
u64 wasm_backend_invocation_timeout_ms(Request& request);
|
||||||
|
|
||||||
// Initialize the process-local engine after fork and before the worker accepts
|
// Initialize the process-local engine after fork and before the worker accepts
|
||||||
// requests. Returns "" on success or the retained backend initialization error.
|
// requests. Returns "" on success or the retained backend initialization error.
|
||||||
@@ -23,7 +24,7 @@ String wasm_serialize_module_artifact(const String& wasm_path);
|
|||||||
// "render", "cli", "websocket", "serve_http", "serve_http:named" — and populate
|
// "render", "cli", "websocket", "serve_http", "serve_http:named" — and populate
|
||||||
// the native Request. Returns "" on success or a collapsed error. The handler is
|
// the native Request. Returns "" on success or a collapsed error. The handler is
|
||||||
// just an export name; there is no per-mode machinery.
|
// just an export name; there is no per-mode machinery.
|
||||||
String wasm_backend_serve(Request& request, const String& entry_unit, const String& handler = "render");
|
String wasm_backend_serve(Request& request, const String& entry_unit, const String& handler = "render", u64 timeout_cap_ms = UINT64_MAX);
|
||||||
|
|
||||||
// Join the per-process epoch ticker before the worker process exits.
|
// Join the per-process epoch ticker before the worker process exits.
|
||||||
void wasm_backend_shutdown();
|
void wasm_backend_shutdown();
|
||||||
|
|||||||
+50
-96
@@ -542,90 +542,9 @@ static void uce_job_reap()
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
static DValue uce_process_exec(String cmd, String input, StringMap env, u64 timeout_ms)
|
|
||||||
{
|
|
||||||
DValue r;
|
|
||||||
r["exit_code"] = (f64)-1;
|
|
||||||
r["stdout"] = "";
|
|
||||||
r["stderr"] = "";
|
|
||||||
r["timed_out"].set_bool(false);
|
|
||||||
if(timeout_ms == 0) timeout_ms = 5000;
|
|
||||||
int inpipe[2], outpipe[2], errpipe[2];
|
|
||||||
if(pipe(inpipe) || pipe(outpipe) || pipe(errpipe)) { r["stderr"]="pipe failed"; return(r); }
|
|
||||||
// The process-wide handler reaps background children; this caller owns this child's status.
|
|
||||||
WasmSigchldBlock sigchld;
|
|
||||||
unsigned int child_status_snapshot = child_exit_status_snapshot();
|
|
||||||
pid_t pid = fork();
|
|
||||||
if(pid < 0)
|
|
||||||
{
|
|
||||||
close(inpipe[0]); close(inpipe[1]); close(outpipe[0]); close(outpipe[1]); close(errpipe[0]); close(errpipe[1]);
|
|
||||||
r["stderr"] = "fork failed";
|
|
||||||
return(r);
|
|
||||||
}
|
|
||||||
if(pid == 0)
|
|
||||||
{
|
|
||||||
sigchld.restore();
|
|
||||||
setpgid(0, 0);
|
|
||||||
dup2(inpipe[0], 0); dup2(outpipe[1], 1); dup2(errpipe[1], 2);
|
|
||||||
close(inpipe[0]); close(inpipe[1]); close(outpipe[0]); close(outpipe[1]); close(errpipe[0]); close(errpipe[1]);
|
|
||||||
for(auto& kv : env) setenv(kv.first.c_str(), kv.second.c_str(), 1);
|
|
||||||
execl("/bin/sh", "sh", "-c", cmd.c_str(), (char*)0);
|
|
||||||
_exit(127);
|
|
||||||
}
|
|
||||||
setpgid(pid, pid);
|
|
||||||
close(inpipe[0]); close(outpipe[1]); close(errpipe[1]);
|
|
||||||
fcntl(inpipe[1], F_SETFL, fcntl(inpipe[1], F_GETFL, 0) | O_NONBLOCK);
|
|
||||||
fcntl(outpipe[0], F_SETFL, fcntl(outpipe[0], F_GETFL, 0) | O_NONBLOCK);
|
|
||||||
fcntl(errpipe[0], F_SETFL, fcntl(errpipe[0], F_GETFL, 0) | O_NONBLOCK);
|
|
||||||
size_t input_off = 0; bool in_open = true, out_open = true, err_open = true; int status = 0; bool exited = false, status_valid = false;
|
|
||||||
u64 deadline = wasm_monotonic_ms() + timeout_ms;
|
|
||||||
while(out_open || err_open || !exited)
|
|
||||||
{
|
|
||||||
if(!exited)
|
|
||||||
{
|
|
||||||
pid_t w = waitpid(pid, &status, WNOHANG);
|
|
||||||
if(w == pid) { exited = true; status_valid = true; }
|
|
||||||
else if(w < 0 && errno == ECHILD)
|
|
||||||
{
|
|
||||||
u64 transfer_deadline = wasm_deadline_after_ms(50);
|
|
||||||
do { status_valid = child_exit_status_take(pid, status, child_status_snapshot); if(!status_valid) sched_yield(); } while(!status_valid && wasm_monotonic_ms() < transfer_deadline);
|
|
||||||
exited = true;
|
|
||||||
if(!status_valid) r["stderr"] = r["stderr"].to_string() + "lost child exit status";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if(in_open)
|
|
||||||
{
|
|
||||||
if(input_off < input.size()) { ssize_t n=write(inpipe[1], input.data()+input_off, input.size()-input_off); if(n>0) input_off += (size_t)n; else if(n<0 && errno!=EINTR && errno!=EAGAIN && errno!=EWOULDBLOCK) { close(inpipe[1]); in_open=false; } }
|
|
||||||
else { close(inpipe[1]); in_open=false; }
|
|
||||||
}
|
|
||||||
char buf[4096];
|
|
||||||
ssize_t n;
|
|
||||||
while((n=read(outpipe[0], buf, sizeof(buf))) > 0) r["stdout"] = r["stdout"].to_string() + String(buf, n);
|
|
||||||
if(n == 0 && out_open) { close(outpipe[0]); out_open=false; }
|
|
||||||
while((n=read(errpipe[0], buf, sizeof(buf))) > 0) r["stderr"] = r["stderr"].to_string() + String(buf, n);
|
|
||||||
if(n == 0 && err_open) { close(errpipe[0]); err_open=false; }
|
|
||||||
if((out_open || err_open || !exited) && wasm_monotonic_ms() >= deadline)
|
|
||||||
{
|
|
||||||
r["timed_out"].set_bool(true);
|
|
||||||
kill(-pid, SIGKILL);
|
|
||||||
kill(pid, SIGKILL);
|
|
||||||
if(!exited)
|
|
||||||
status_valid = waitpid(pid, &status, 0) == pid;
|
|
||||||
exited = true;
|
|
||||||
if(in_open) { close(inpipe[1]); in_open=false; }
|
|
||||||
if(out_open) { close(outpipe[0]); out_open=false; }
|
|
||||||
if(err_open) { close(errpipe[0]); err_open=false; }
|
|
||||||
}
|
|
||||||
if((out_open || err_open || !exited)) usleep(10000);
|
|
||||||
}
|
|
||||||
if(status_valid && WIFEXITED(status)) r["exit_code"] = (f64)WEXITSTATUS(status);
|
|
||||||
else if(status_valid && WIFSIGNALED(status)) r["exit_code"] = (f64)(128 + WTERMSIG(status));
|
|
||||||
return(r);
|
|
||||||
}
|
|
||||||
|
|
||||||
static DValue uce_shell_exec_spec(const DValue& spec)
|
static DValue uce_shell_exec_spec(const DValue& spec)
|
||||||
{
|
{
|
||||||
return(uce_process_exec(spec.key("cmd") ? spec.key("cmd")->to_string() : String(""), spec.key("stdin") ? spec.key("stdin")->to_string() : String(""), spec.key("env") ? spec.key("env")->to_stringmap() : StringMap(), spec.key("timeout_ms") ? spec.key("timeout_ms")->to_u64(5000) : 5000));
|
return(process_exec(spec.key("cmd") ? spec.key("cmd")->to_string() : String(""), spec.key("stdin") ? spec.key("stdin")->to_string() : String(""), spec.key("env") ? spec.key("env")->to_stringmap() : StringMap(), spec.key("timeout_ms") ? spec.key("timeout_ms")->to_u64(5000) : 5000));
|
||||||
}
|
}
|
||||||
|
|
||||||
static void uce_job_finish(u64 id, DValue result, String final_state="done")
|
static void uce_job_finish(u64 id, DValue result, String final_state="done")
|
||||||
@@ -1606,8 +1525,7 @@ public:
|
|||||||
return(UINT64_MAX);
|
return(UINT64_MAX);
|
||||||
if(now >= invocation_deadline)
|
if(now >= invocation_deadline)
|
||||||
return(0);
|
return(0);
|
||||||
u64 remaining_us = (u64)std::chrono::duration_cast<std::chrono::microseconds>(invocation_deadline - now).count();
|
return((u64)std::chrono::duration_cast<std::chrono::milliseconds>(invocation_deadline - now).count());
|
||||||
return(remaining_us / 1000 + (remaining_us % 1000 != 0));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
bool invocation_expired(InvocationClock::time_point now = InvocationClock::now()) const
|
bool invocation_expired(InvocationClock::time_point now = InvocationClock::now()) const
|
||||||
@@ -1659,7 +1577,7 @@ public:
|
|||||||
InvocationClock::time_point previous_deadline;
|
InvocationClock::time_point previous_deadline;
|
||||||
u64 previous_budget_ms = 0;
|
u64 previous_budget_ms = 0;
|
||||||
public:
|
public:
|
||||||
InvocationScope(WasmWorkspace& workspace, u64 timeout_cap_ms = 0, bool force_new = false) : workspace(workspace)
|
InvocationScope(WasmWorkspace& workspace, u64 timeout_cap_ms = 0, bool force_new = false, u64 reported_budget_ms = 0) : workspace(workspace)
|
||||||
{
|
{
|
||||||
if(!workspace.invocation_active || force_new)
|
if(!workspace.invocation_active || force_new)
|
||||||
{
|
{
|
||||||
@@ -1668,10 +1586,10 @@ public:
|
|||||||
previous_deadline = workspace.invocation_deadline;
|
previous_deadline = workspace.invocation_deadline;
|
||||||
previous_budget_ms = workspace.invocation_budget_ms;
|
previous_budget_ms = workspace.invocation_budget_ms;
|
||||||
u64 budget_ms = workspace.worker.cfg.invocation_timeout_ms;
|
u64 budget_ms = workspace.worker.cfg.invocation_timeout_ms;
|
||||||
if(timeout_cap_ms > 0)
|
if(timeout_cap_ms != UINT64_MAX)
|
||||||
budget_ms = std::min(budget_ms, timeout_cap_ms);
|
budget_ms = std::min(budget_ms, timeout_cap_ms);
|
||||||
workspace.invocation_active = true;
|
workspace.invocation_active = true;
|
||||||
workspace.invocation_budget_ms = budget_ms;
|
workspace.invocation_budget_ms = reported_budget_ms > 0 ? reported_budget_ms : budget_ms;
|
||||||
workspace.invocation_deadline = InvocationClock::now() + std::chrono::milliseconds(budget_ms);
|
workspace.invocation_deadline = InvocationClock::now() + std::chrono::milliseconds(budget_ms);
|
||||||
}
|
}
|
||||||
workspace.arm_guest_deadline(workspace.ctx());
|
workspace.arm_guest_deadline(workspace.ctx());
|
||||||
@@ -1786,6 +1704,8 @@ public:
|
|||||||
phase_start = now;
|
phase_start = now;
|
||||||
return(elapsed);
|
return(elapsed);
|
||||||
};
|
};
|
||||||
|
if(invocation_expired())
|
||||||
|
return(invocation_timeout_error());
|
||||||
auto cx = ctx();
|
auto cx = ctx();
|
||||||
store.limiter(worker.cfg.memory_limit, -1, -1, -1, -1);
|
store.limiter(worker.cfg.memory_limit, -1, -1, -1, -1);
|
||||||
arm_guest_deadline(cx);
|
arm_guest_deadline(cx);
|
||||||
@@ -2096,7 +2016,9 @@ private:
|
|||||||
String trap_text(const wasmtime::TrapError& error)
|
String trap_text(const wasmtime::TrapError& error)
|
||||||
{
|
{
|
||||||
String result = wasm_trace_collapse(String(error.message()));
|
String result = wasm_trace_collapse(String(error.message()));
|
||||||
if(invocation_expired() && result.find("interrupt") != String::npos && result.find("UCE_INVOCATION_TIMEOUT:") == String::npos)
|
bool invocation_deadline_interrupt = invocation_active &&
|
||||||
|
(invocation_expired() || invocation_remaining_ms() <= worker.cfg.epoch_period_ms);
|
||||||
|
if(invocation_deadline_interrupt && result.find("interrupt") != String::npos && result.find("UCE_INVOCATION_TIMEOUT:") == String::npos)
|
||||||
result = invocation_timeout_error() + "\n" + result;
|
result = invocation_timeout_error() + "\n" + result;
|
||||||
struct Frame
|
struct Frame
|
||||||
{
|
{
|
||||||
@@ -2795,10 +2717,12 @@ private:
|
|||||||
// `handler` names the export ("render", "component:CARD", "cli",
|
// `handler` names the export ("render", "component:CARD", "cli",
|
||||||
// "serve_http:named", "once") or is "exists" (probe only, loads nothing).
|
// "serve_http:named", "once") or is "exists" (probe only, loads nothing).
|
||||||
int32_t component_resolve(const String& target, const String& handler, const String& current_unit,
|
int32_t component_resolve(const String& target, const String& handler, const String& current_unit,
|
||||||
String& resolved_out, int32_t* once_slot_out = 0)
|
String& resolved_out, int32_t* once_slot_out = 0, bool* compile_timed_out = 0)
|
||||||
{
|
{
|
||||||
if(once_slot_out)
|
if(once_slot_out)
|
||||||
*once_slot_out = 0;
|
*once_slot_out = 0;
|
||||||
|
if(compile_timed_out)
|
||||||
|
*compile_timed_out = false;
|
||||||
auto probe_start = std::chrono::steady_clock::now();
|
auto probe_start = std::chrono::steady_clock::now();
|
||||||
auto record_probe = [&]() {
|
auto record_probe = [&]() {
|
||||||
component_resolve_count += 1;
|
component_resolve_count += 1;
|
||||||
@@ -2925,7 +2849,21 @@ private:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if(!artifact_exists || (stale && !can_serve_stale))
|
if(!artifact_exists || (stale && !can_serve_stale))
|
||||||
get_shared_unit(context, resolved);
|
{
|
||||||
|
u64 remaining_ms = invocation_remaining_ms();
|
||||||
|
bool timed_out = false;
|
||||||
|
if(remaining_ms == 0)
|
||||||
|
timed_out = true;
|
||||||
|
else
|
||||||
|
get_shared_unit_bounded(context, resolved, remaining_ms, &timed_out);
|
||||||
|
if(timed_out)
|
||||||
|
{
|
||||||
|
if(compile_timed_out)
|
||||||
|
*compile_timed_out = true;
|
||||||
|
record_probe();
|
||||||
|
return(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
component_artifact_total_us += (u64)std::chrono::duration_cast<std::chrono::microseconds>(
|
component_artifact_total_us += (u64)std::chrono::duration_cast<std::chrono::microseconds>(
|
||||||
std::chrono::steady_clock::now() - artifact_start).count();
|
std::chrono::steady_clock::now() - artifact_start).count();
|
||||||
|
|
||||||
@@ -3342,7 +3280,7 @@ private:
|
|||||||
{
|
{
|
||||||
u64 remaining_ms = self->invocation_remaining_ms();
|
u64 remaining_ms = self->invocation_remaining_ms();
|
||||||
u64 timeout_ms = std::max<u64>(1, self->bounded_hostcall_timeout_ms(5000));
|
u64 timeout_ms = std::max<u64>(1, self->bounded_hostcall_timeout_ms(5000));
|
||||||
DValue execution = uce_process_exec(cmd + " 2>&1", "", StringMap(), timeout_ms);
|
DValue execution = process_exec(cmd + " 2>&1", "", StringMap(), timeout_ms);
|
||||||
if(execution["timed_out"].to_bool())
|
if(execution["timed_out"].to_bool())
|
||||||
{
|
{
|
||||||
String kind = self->invocation_expired() || remaining_ms <= 5000 ?
|
String kind = self->invocation_expired() || remaining_ms <= 5000 ?
|
||||||
@@ -3850,7 +3788,14 @@ private:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
else if(op == "compile")
|
else if(op == "compile")
|
||||||
response["ok"].set_bool(unit_compile(request["path"].to_string()));
|
{
|
||||||
|
u64 remaining_ms = self->invocation_remaining_ms();
|
||||||
|
bool timed_out = false;
|
||||||
|
bool ok = remaining_ms > 0 && unit_compile_bounded(context, request["path"].to_string(), remaining_ms, &timed_out);
|
||||||
|
if(timed_out || remaining_ms == 0)
|
||||||
|
return(Trap(self->invocation_timeout_error()));
|
||||||
|
response["ok"].set_bool(ok);
|
||||||
|
}
|
||||||
else if(op == "call")
|
else if(op == "call")
|
||||||
{
|
{
|
||||||
DValue* param = request.key("param");
|
DValue* param = request.key("param");
|
||||||
@@ -4292,7 +4237,10 @@ private:
|
|||||||
self->hostcall_read(args[2].i32(), args[3].i32(), handler);
|
self->hostcall_read(args[2].i32(), args[3].i32(), handler);
|
||||||
self->hostcall_read(args[4].i32(), args[5].i32(), current);
|
self->hostcall_read(args[4].i32(), args[5].i32(), current);
|
||||||
int32_t once_slot = 0;
|
int32_t once_slot = 0;
|
||||||
int32_t slot = self->component_resolve(target, handler, current, resolved, &once_slot);
|
bool compile_timed_out = false;
|
||||||
|
int32_t slot = self->component_resolve(target, handler, current, resolved, &once_slot, &compile_timed_out);
|
||||||
|
if(compile_timed_out)
|
||||||
|
return(Trap(self->invocation_timeout_error()));
|
||||||
u32 cap = (u32)args[7].i32();
|
u32 cap = (u32)args[7].i32();
|
||||||
if(cap > 0)
|
if(cap > 0)
|
||||||
{
|
{
|
||||||
@@ -4335,7 +4283,7 @@ inline String wasm_worker_prepare(WasmWorker& worker)
|
|||||||
// ---- public entry: one request through one workspace -----------------------
|
// ---- public entry: one request through one workspace -----------------------
|
||||||
|
|
||||||
inline WasmResponse wasm_worker_serve(WasmWorker& worker, const Request& request, const String& entry_source_path,
|
inline WasmResponse wasm_worker_serve(WasmWorker& worker, const Request& request, const String& entry_source_path,
|
||||||
const String& handler = "render")
|
const String& handler = "render", u64 timeout_cap_ms = UINT64_MAX)
|
||||||
{
|
{
|
||||||
WasmResponse response;
|
WasmResponse response;
|
||||||
f64 serve_started = time_precise();
|
f64 serve_started = time_precise();
|
||||||
@@ -4345,9 +4293,15 @@ inline WasmResponse wasm_worker_serve(WasmWorker& worker, const Request& request
|
|||||||
bool thread_runtime_profiled = worker.cfg.profile_thread_runtime && getrusage(RUSAGE_THREAD, &thread_runtime_start) == 0;
|
bool thread_runtime_profiled = worker.cfg.profile_thread_runtime && getrusage(RUSAGE_THREAD, &thread_runtime_start) == 0;
|
||||||
int thread_cpu_start = thread_runtime_profiled ? sched_getcpu() : -1;
|
int thread_cpu_start = thread_runtime_profiled ? sched_getcpu() : -1;
|
||||||
WasmWorkspace workspace(worker);
|
WasmWorkspace workspace(worker);
|
||||||
f64 setup_cpu_finished = wasm_thread_cpu_time();
|
u64 workspace_setup_us = (u64)std::chrono::duration_cast<std::chrono::microseconds>(
|
||||||
workspace.workspace_setup_us = (u64)std::chrono::duration_cast<std::chrono::microseconds>(
|
|
||||||
std::chrono::steady_clock::now() - workspace_start).count();
|
std::chrono::steady_clock::now() - workspace_start).count();
|
||||||
|
u64 workspace_setup_ms = workspace_setup_us / 1000 + (workspace_setup_us % 1000 != 0);
|
||||||
|
u64 workspace_budget_ms = timeout_cap_ms;
|
||||||
|
if(workspace_budget_ms != UINT64_MAX)
|
||||||
|
workspace_budget_ms = workspace_budget_ms > workspace_setup_ms ? workspace_budget_ms - workspace_setup_ms : 0;
|
||||||
|
WasmWorkspace::InvocationScope invocation(workspace, workspace_budget_ms, false, worker.cfg.invocation_timeout_ms);
|
||||||
|
f64 setup_cpu_finished = wasm_thread_cpu_time();
|
||||||
|
workspace.workspace_setup_us = workspace_setup_us;
|
||||||
workspace.workspace_setup_cpu_us = cpu_started > 0 && setup_cpu_finished > cpu_started ?
|
workspace.workspace_setup_cpu_us = cpu_started > 0 && setup_cpu_finished > cpu_started ?
|
||||||
(u64)((setup_cpu_finished - cpu_started) * 1000000.0) : 0;
|
(u64)((setup_cpu_finished - cpu_started) * 1000000.0) : 0;
|
||||||
workspace.set_perf_snapshot(my_pid, (u64)parent_pid, request.server ? request.server->request_count : 0,
|
workspace.set_perf_snapshot(my_pid, (u64)parent_pid, request.server ? request.server->request_count : 0,
|
||||||
|
|||||||
Reference in New Issue
Block a user