# Test suite This directory holds the `.ssa` inputs used by `compare.py` to verify the MoonBit QBE implementation against the vendored reference C QBE. All 409 non-underscore files must compile identically on both implementations for the default `amd64_sysv` target. ## Reference binary `compare.py` locates the reference binary in this order: 1. the `QBE_REF` environment variable; 2. `vendor/qbe/qbe(.exe)`, built with `make -C vendor/qbe`. The vendored submodule is the only reference implementation used by this repository. If its output differs from historical snapshots, update the expected baseline rather than maintaining a second reference tree. ## Cross-target verification `compare.py` compares any of the three reference-backed backends against the vendored reference: python compare.py --target arm64 # arm64 debug dumps python compare.py --target rv64 --asm # rv64 assembly The port is byte-for-byte identical to the reference on all three targets (amd64_sysv, arm64, rv64), for both the debug-dump differential suite (`compare.py`, every `-d` flag) and the emitted assembly (`compare.py --asm`) on all 409 tests (409/409 per target). The two extra backends have no reference target, so they are gated on independent validity tools instead: python tools/check_la64_asm.py # clang's LoongArch64 assembler python tools/check_wasm.py # moon-wasm-opt (wat2wasm + validator) `tools/check_all_backends.py` runs all five gates in one pass (clang for amd64/arm64/rv64/la64, moon-wasm-opt for wasm) and skips inputs the reference itself rejects. All 339 reference-compilable inputs pass on every backend. ## Layout - `*.ssa` — the 32 upstream QBE examples at the repository root. - `core/` — generated arithmetic/compare/memory/conversion/constant tests (`arith` 86, `compare` 56, `mem` 2, `conv` 14, `const` 24). - `fold/` — constant-folding / dead-code tests (23). - `abi/` — argument/return ABI and call conventions (48). - `isel/` — instruction-selection shapes (34). - `regalloc/` — register allocation / spilling stress (17). - `emit/` — output emission shapes, incl. float constants (15). - `programs/` — 55 hand-written realistic programs (number theory, sorting, strings, floats, recursion, nested loops). - `dbg/` — `dbgfile`/`dbgloc` line-table inputs (2). - `stress/` — the scale series (11 exact + 3 `_`-prefixed), generated by `tools/gen_stress.py`. Where `core/` pins *which* operation runs, this series pins *how much*: deep dependency chains, wide live ranges, long CFG chains, diamond joins, a large fan-in phi, many stack arguments, many locals, call-heavy blocks, a compare/jump ladder, a large data section, float round trips, extension chains and a three-deep loop nest. Sizes are chosen to stay in the tens of kilobytes so the directory does not dominate a run. ## Running From the repository root: python compare.py # all debug flags x all tests python compare.py --cat programs # only one category python compare.py --jobs 1 # single-worker (default 4) python compare.py test\programs\003_arr_max.ssa # one file, all flags python compare.py --cat stress # the scale series `tools/bench.py` answers the other half of the question — how long each case takes, and how that compares with the reference: python tools/bench.py # whole corpus, -dR python tools/bench.py --cat stress # the scale series python tools/bench.py --by-flag # one row per debug stage python tools/bench.py --all # include the _-prefixed inputs python tools/bench.py --save bench.json python tools/bench.py --baseline bench.json # exit 1 on a >25% regression Every case is timed through both binaries, so the reported `ratio` (mine over reference) is machine independent. Expected baseline: every case passes for the default debug differential suite. `compare.py --asm` must also be byte-identical for each target. ## Regenerating generated tests python tools\gen_tests.py python tools\gen_programs.py python tools\gen_stress.py `gen_programs.py` applies an internal SSA fix-up (renames duplicate definitions, corrects phi predecessor labels incl. fall-through edges) so the emitted files are valid strict SSA out of the box. ## Constraints honored by the generators The reference binary rejects some inputs, so generated tests avoid: - `sltof` with a word operand (use `swtof`; `sltof` needs a long). - unsigned comparisons spelled `cs...` (use `cu...`, e.g. `cugew`). - decimal-point literals in `data` (e.g. `d 1.5`); use raw integer bits. - duplicate definitions and mislabeled phi predecessors (both reject at `-dA` ssacheck). `gen_stress.py` additionally relies on the rule that a phi argument list follows the *block layout order* of its predecessors: `fillpreds` walks `fn->start` to `fn->link` and appends in that order, so the fan-in test puts the block laid out first at the head of its phi. ## Inputs excluded from the exact differential A file whose name starts with `_` is skipped by `compare.py`: the upstream benchmarks (`_bf99`, `_bfmandel`, `_chacha20`, `_dragon`), the internal pass helpers (`_live`, `_rpo`, `_spill*`) and the three `stress/` inputs that still diverge from the reference (see `TODO.md`, "Known divergences"). `tools/bench.py --all` includes them, so they keep being exercised for timing even while they are out of the byte-exact gate. The three `stress/` entries are there for a different reason than the benchmarks: they are the inputs on which the reference disagrees with itself. `spill.c`'s `limit` sorts tied spill costs with `qsort`, whose order for equal keys is implementation defined, so a build linked against Apple's `qsort` picks a different temporary to spill than one linked against glibc's mergesort. On those inputs the port agrees with a *stable* `qsort`; to reproduce that locally, copy `vendor/qbe`, replace the two `qsort` calls in `spill.c` with a stable sort, build, and point `QBE_REF` at it: QBE_REF=/path/to/stable/qbe python compare.py --cat stress Every other input in this directory is byte-identical to the vendored reference on all three targets.