# Tests

Run the compiler checks with `scripts/test_capy_native.sh`. The suite reports elapsed time for each stage. It runs all assertions and compiles every tracked site fixture on each invocation.

The suite caches native test builds under `tmp/`. The build cache checks dependency contents, including system headers, compiler identity, arguments, environment, and executable contents. It never caches test results. Failed builds remove the cache manifest. Per-output locks protect concurrent builds. The native suite also locks its artifact directory.

Test binaries use `-O1` without `NDEBUG`. Assertions remain enabled. The suite uses a release compiler for fixture compilation. The first build costs more than a repeat run.

Run scaling benchmarks with `scripts/test_capy_native.sh --benchmarks`, or run `scripts/benchmark_capy_compiler_scaling.sh` directly. Routine correctness checks do not run benchmarks.

Run `python3 scripts/test_native_build_cache.py` to check cache invalidation, failed builds, concurrent builds, and executable corruption.

Run `scripts/test_capy_artifact_golden.sh` to compare Wasm and source-map goldens. It compiles with at most four workers. Each invocation owns a temporary directory. The golden check remains independent of the native suite's compiler.

Run `python3 scripts/test_capy_format.py site --capyc bin/capyc --allow-backtrace` to verify formatting. It checks idempotence for every file. Changed files require original and formatted compilations. Unchanged files require only one compilation. The only allowed Wasm difference is the approved backtrace fixture's embedded source locations.

## Offline editor packaging

Run these on the native Linux build host after building `bin/capyc`:

```bash
python3 scripts/test_package_editor_tools.py
python3 scripts/package_editor_tools.py --test-dependencies editors/vscode/node_modules
(cd editors/vscode && npm test)
```

The dependency extraction command requires a new node_modules directory; omit
it if the reviewed pinned dependencies are already installed. The packaging
gate checks dependency integrity/licenses/hooks, duplicate-free VSIX layout,
manifest metadata, executable modes, complete runtime imports with a VS Code
API stub, untrusted activation, deterministic archives, hash receipts, native
`--help`, and real initialize/shutdown over stdio from the extracted compiler.
Path tests cover stable home discovery, bundled platform selection, explicit
failure, and managed lifecycle replacements. These gates do not replace a real
VS Code local/Remote SSH session, unsaved document recovery, or target-host
compiler smoke tests. Build commands and platform limits are in the
[editor README](../editors/vscode/README.md).

For an isolated real VS Code installation gate, set `CAPY_VSCODE_CLI` to an
existing VS Code Server `server/bin/code-server` executable and run
`python3 scripts/test_package_editor_tools.py`. The test uses private temporary
extension/user-data directories, installs only the local VSIX, lists the
installed extension, and runs the installed compiler and Node entrypoint.
It does not start a listener or change the user's normal extension installation.
This still does not exercise the graphical editor or a live Remote SSH session.

Set `CAPY_EDITOR_VSIX` to an existing final release VSIX to run archive/runtime
and optional CLI-install checks against that exact file, including all bundled
platforms. The deterministic build assertions still build isolated test copies.

For the static release gate, set `CAPY_TEST_COMPILER` to
`$PWD/tmp/editor-tools-release/linux-x64/capyc` and `CAPY_EXPECT_STATIC=1`.
The gate rejects an ELF interpreter and requires `ldd` to report no dynamic
executable. Static packaging also requires matching toolchain evidence from
`scripts/record_editor_toolchain.py` and the matching glibc source
materials described in [the source notes](editor-glibc-source.md).
`CAPY_TOOLCHAIN_DIR` and `CAPY_GLIBC_SOURCE` select isolated release inputs.
The source gate verifies the source manifest inside the VSIX, the separately
downloadable archive hash, and every source file inside the offline bundle.
Missing corresponding source must fail before a distributable is written.

The final bundle smoke invokes its own
installer with private destinations, verifies a dry-run makes no directory,
then checks fresh installation, repeat installation/backup and compiler help.
No normal home-directory compiler is changed.

`python3 scripts/test_editor_glibc_source.py` exercises the small source
collector's index parsing and package-file selection. There are no signing
fixtures, native receipts or separate release-control suites.
[Publishing](editor-release-pipeline.md) uses the normal build/test/package/upload
steps.

### Real VS Code editor and replacement gate

`editors/vscode/test/editor-host.cjs` runs inside an existing graphical VS Code
installation. It verifies bundled compiler discovery without home/PATH compiler
copies, file recognition, diagnostics and Outline. It then atomically replaces
the compiler **in a disposable installed extension**, verifies a different LSP
child PID and checks re-synchronization of an unsaved document. The test refuses
to replace a compiler outside its private root. It deliberately waits for
provider registration; diagnostics alone do not imply providers are registered.

On a supported Mac/Linux platform, use an existing VS Code CLI:

```bash
sandbox="$PWD/tmp/editor-host-acceptance"
mkdir -p "$sandbox/workspace"
code --user-data-dir "$sandbox/user-data" --extensions-dir "$sandbox/extensions" \
  --install-extension "$PWD/dist/editor-tools/capy-language-0.3.0.vsix" --force
CAPY_EDITOR_TEST_ROOT="$sandbox" code \
  --user-data-dir "$sandbox/user-data" --extensions-dir "$sandbox/extensions" \
  --extensionDevelopmentPath="$sandbox/extensions/udo.capy-language-0.3.0" \
  --extensionTestsPath="$PWD/editors/vscode/test/editor-host.cjs" \
  --disable-workspace-trust --skip-welcome --skip-release-notes \
  --new-window "$sandbox/workspace"
```

Inspect `$sandbox/editor-host-result.json`; a failed test writes
`editor-host-error.txt`. Normal editor settings and extension installations are
not changed. This gate disables trust prompts in its isolated host; it does
not claim interactive trust-dialog acceptance or a graphical Remote SSH test.