Parallelize proactive unit compilation

docs/wasm-runtime-architecture.md

@@ -246,14 +246,16 @@ publication and the shared generation PCH use separate advisory locks, and any
 worker/reporting failure rejects the candidate generation.
 
 The proactive compiler and request workers coordinate through per-unit file
-locks and a lock-protected demand-priority queue under `BIN_DIRECTORY`. A small
-priority-only compiler process drains that queue independently of the full-site
-scanner. This matters when the scanner is already inside a long transitive C++
-compile: a requested stale component can rebuild immediately instead of waiting
-for that unrelated compile to finish. Both compiler processes use the same
-per-unit lock, so concurrent demand and scan discovery cannot publish duplicate
-artifacts. The priority worker is idle when there is no demand and never scans
-the site on its own. Unit
+locks and a lock-protected demand-priority queue under `BIN_DIRECTORY`. The
+full-site scan uses two low-priority processes by default, bounded from 1–16 by
+`PROACTIVE_COMPILE_JOBS`; a stable canonical-path partition gives every unit one
+scanner owner and keeps retry/backoff state local. A separate higher-priority
+compiler process exclusively drains the demand queue. This matters when the
+scanners are already inside long transitive C++ compiles: a requested stale
+component can rebuild immediately instead of waiting for an unrelated compile.
+All compiler processes use the same per-unit lock, so concurrent demand and scan
+discovery cannot publish duplicate artifacts. The priority worker is idle when
+there is no demand and never scans the site on its own. Unit
 compilation writes and validates a process-unique temporary wasm file, then
 publishes it with an atomic rename. When proactive compilation is enabled,
 read-only HTTP requests keep using the last complete artifact while requesting