# Reproducible performance measurements

The checked-in [baseline raw report](performance-data/baseline-linux-x86_64.json) was collected on 2026-10-25 with the pinned 0.10.15+943464bfa Linux Native release toolchain, before changing the resolver/index. Its Git revision points to the pre-harness library baseline and `working_tree: true` records the added harness; the core resolver/types were unchanged. The full 36-case matrix and canonical evidence completed successfully. The harness and raw baseline are committed together so a later checkout can reproduce the pre-optimization workload. Source files and binary identity, not only reported timings, must be considered when comparing runs.

`cmd/benchmark` uses the verified core `bench.monotonic_clock_start/end` microsecond clock. It accepts `PHASE SHAPE SCANS SAMPLES`: phases `build`, `audit`, `compare`, `serialize`, or untimed `evidence`; shapes `sparse`, `scoped`, `root-heavy`; scans 1–5000; samples 3–31. Workloads are deterministic, synthetic and within the unchanged snapshot limit. Each timing process performs one untimed warmup followed by the requested individual samples, consumes/checks results and emits the raw samples and median. Fixture generation, initial snapshots, prepared audit for serialization, and final JSON output are outside the timed region. Build includes existing full-snapshot validation and `data_paths`; audit/compare include error/count checks; serialization is separate.

The default matrix uses 100/1,000/5,000 scans and seven samples. Sparse has one root source; scoped adds session overrides and unrelated anatomical sources; root-heavy adds one nonmatching root BOLD selector per scan. The last distribution deliberately exposes a case where ancestor/suffix lookup alone cannot prune candidates. Root RT changes in the comparison snapshot, affecting every scan. Unknown values include null, negative zero, a large integer and an overflowing exponent.

Build and collect on Linux (a Native C toolchain and `/usr/bin/time` are required):

```sh
moon build --target native --release cmd/benchmark --deny-warn
moon run scripts/benchmark.mbtx -- _build/native/release/build/cmd/benchmark/benchmark.exe _build/performance/baseline
```

Use the executable path printed/generated by your toolchain if it differs. `--quick` after the output directory collects only 100-scan cases. Wasm, Wasm-GC and JS can run the timing program directly with `moon run --release --target TARGET cmd/benchmark -- audit scoped 1000 7`; Linux RSS collection is specifically for the supplied native executable.

Each case launches a separate process. Linux `time` reports whole-process peak resident memory in KiB: this includes fixture generation, initial/prepared snapshots, warmup, live results and allocator/runtime behavior. It is not incremental index memory, a strict memory bound, or a cross-backend memory comparison. Compiler and `moon run` startup are outside collected native process measurements. The report records toolchain, CPU, platform, binary digest, Git revision and dirty status; the caller must supply a release binary. Avoid concurrent benchmark processes or other heavy workloads.

The collector also saves complete canonical manifest/audit/impact evidence per shape and size. Compare baseline and optimized evidence byte-for-byte, alongside unit tests; timing equality or matching counts alone cannot establish preservation. Keep old/new executables in separate directories and alternate repeated runs to assess noise. Raw measurements are machine-specific observations, not universal speed guarantees or comparisons with other libraries. Publication is not part of this workflow.

## Indexed lookup and paired results — 2026-10-26

The index now keeps exact scan-path positions and source buckets by ancestor directory and suffix. Buckets with at least eight sources optionally subdivide by one existing filename entity; the full original `applies` predicate still checks every remaining constraint. Sources occur in only one sub-bucket. Candidate positions are sorted back into the original depth/ordinal-path order, preserving histories and ambiguity diagnostics. At most eight total sources use the original linear candidate filter, avoiding ancestor/map overhead for tiny snapshots. Small scope buckets keep linear lists instead of allocating entity/value maps. These thresholds are internal implementation choices, not public semantics or a claim of universally optimal tuning.

Construction now allocates lookup structures; resolution still depends on the number of remaining candidates and metadata size, and sorting costs depend on candidate count. No resolved-metadata cache is shared with callers. Input limits, source preservation, errors and public interfaces are unchanged. Worst-case buckets with many applicable sources can still require substantial work and complete diagnostic evidence.

The [paired raw report](performance-data/indexed-paired-linux-x86_64.json) records all 36 current and 36 baseline cases, binary digests and **nine byte-identical complete manifest/audit/impact output pairs**. Each case alternates which executable runs first. The baseline executable is the frozen pre-index binary from the foundation experiment; both binaries use the same pinned Linux Native release toolchain. The current report points to parent `6f583d1` with `working_tree: true`; the tested resolver/index changes and report are committed together after collection. Hardware: Intel i9-14900HX, WSL2 Linux x86_64. Each median is seven samples after one warmup, not an end-to-end CLI startup measurement.

5,000-scan observations (milliseconds; RSS is whole-process peak in KiB):

| Sources | Phase | Before ms | Indexed ms | Before RSS → indexed RSS |
| --- | --- | ---: | ---: | ---: |
| Sparse | Build | 14.575 | 15.630 | 12388 → 12808 |
| Sparse | Audit | 19.009 | 3.527 | 11884 → 12300 |
| Sparse | Compare | 64.211 | 29.304 | 24380 → 25024 |
| Sparse | Serialize | 34.147 | 33.576 | 33860 → 34416 |
| Scoped | Build | 31.266 | 33.906 | 24668 → 28352 |
| Scoped | Audit | 355.479 | 10.349 | 18488 → 20340 |
| Scoped | Compare | 775.083 | 64.655 | 37096 → 41304 |
| Scoped | Serialize | 55.742 | 57.152 | 53656 → 55272 |
| Root-heavy | Build | 33.764 | 32.633 | 22868 → 23208 |
| Root-heavy | Audit | 1414.162 | 6.128 | 16480 → 17932 |
| Root-heavy | Compare | 2933.409 | 38.720 | 33468 → 34568 |
| Root-heavy | Serialize | 37.435 | 37.486 | 38636 → 42912 |

This run observed roughly 34.3×/230.8× audit reductions and 12.0×/75.8× comparison reductions for scoped/root-heavy workloads. The tradeoff is visible: scoped construction was about 8.4% slower with about 14.9% higher peak RSS; memory generally increased. Serialization was not optimized. Small/sub-millisecond results fluctuate substantially (including the unchanged serialization control), so they do not justify precise speed claims. An initial always-indexed prototype used more memory and slowed small sparse audits; the final small-set/bucket policy reduced those costs. The raw report includes every scale and phase, including regressions.

To reproduce an interleaved comparison, preserve a native release executable built from the unoptimized foundation commit `6f583d1` in a separate checkout/build directory. Build the current executable separately with the same toolchain, then:

```sh
moon run scripts/benchmark.mbtx -- PATH_TO_CURRENT_BINARY _build/performance/paired --compare-to PATH_TO_BASELINE_BINARY
```

Append `--quick` for the 100-scan subset. The collector rejects byte differences in complete canonical evidence and writes no successful summary on a mismatching run. A 48-layout linear-resolver oracle suite separately covers normalized entities, scope prefixes, errors and ambiguity ordering; additional tests cover all nine entity discriminators, absent optional entities, snapshot rebuilding and small-set transition boundaries. These preservation checks support this optimization; they do not certify full BIDS conformance or remove the documented snapshot limit.

The portable timing executable normalizes Node.js's extra runtime entry in `core/env.args()`; native/WASI argument handling stays unchanged. CI actually launches Release benchmarks and verifies invalid-argument diagnostics on Wasm, Wasm-GC and JavaScript, in addition to Native test/build coverage. The paired performance report was collected from the frozen Native binaries before this JS-only argument adapter; the timed operations and lookup implementation are unchanged.
## Dependency-driven incremental measurements

The historical lookup-index measurements above are retained unchanged. New
full/cold/warm graph measurements on stable 0.10.17, including initial costs,
regressions, RSS and complete-report equality, are recorded separately in
[incremental impact performance](incremental-performance.md).