# Development This document is for maintainers. User-facing quick start material belongs in the root README; agent-specific workflow belongs in `AGENTS.md`. ## MoonBit toolchain Local development and CI use the official stable `latest` channel. Update the whole toolchain (compiler, standard library, and formatter) together: ```bash curl -fsSL https://cli.moonbitlang.com/install/unix.sh | bash -s latest moon version --all moon update moon check --target all --deny-warn moon test --deny-warn ``` Restart the editor after updating so its language server uses the new toolchain. Run `moon fmt` with this same toolchain; older formatters can produce different trailing commas. CI installs the stable `latest` channel for every job and logs `moon version --all`. Dependency and build caches include both the build-tool and compiler versions. The toolchain itself is not restored from cache, so it cannot silently remain on an old stable release. This tracks upstream updates; it does not pin a reproducible historical release. When an upstream update breaks CI, use the logged versions to reproduce and fix it. Run `bash scripts/test_moonbit_toolchain_contract.sh` after workflow changes. ## Package Map Run the live package overview before broad changes: ```bash scripts/package-overview.sh ``` Current package responsibilities, verified from `moon.pkg` files and generated interfaces: | Package | Responsibility | |---|---| | `.` | User-facing facade over parse, interpreter execution, module execution, and the opt-in compiled path. | | `token` | Token kinds, source locations, and literal provenance tags. | | `errors` | JavaScript error variants and formatting helpers. | | `ast` | Public AST node definitions consumed by parser, static semantics, compiler, and runtime. | | `lexer` | Source text to token stream. | | `parser` | Token stream to AST. | | `static_semantics` | Early-error checks and declaration-fact analysis. | | `compiler` | Default bytecode candidate pipeline plus legacy closure-conversion experiments. | | `interpreter` | Wiring layer that creates a runtime interpreter with stdlib hooks. | | `interpreter/runtime` | Tree-walking evaluator, value model, environments, property dispatch, modules, event-loop state. | | `interpreter/stdlib` | JavaScript built-ins and stdlib/runtime hook implementations. | | `benchmarks` | Benchmark workloads, benchmark tests, and benchmark CLI. | | `cmd/main` | User-facing CLI executable. | | `cmd/test262_runner` | Authoritative native Test262 runner. | | `cmd/report_test262` | CI-artifact conformance report generator. | | `cmd/test262_analyze`, `cmd/test262_validate_skips`, `cmd/classify_by_edition` | Test262 metadata, skip validation, and reporting helpers. | | `cmd/architecture_*`, `tooling/architecture_*` | Architecture state/boundary audit CLIs and libraries. | | `cmd/bench_focus` | Native repeated-benchmark helper for lower-noise local timing. | | `tooling/test262_*`, `tooling/subprocess_helpers` | Shared native tooling libraries for Test262 and subprocess handling. | Use `moon ide outline ` or `pkg.generated.mbti` for the current public API. Do not infer public API from file names. ## Routine Commands ```bash moon check moon test moon info moon fmt moon build ``` MoonBit coding conventions (including arrow functions for higher-order callbacks) live in `AGENTS.md` / `CLAUDE.md` — update those files when adding project-wide style rules. `moon info` regenerates `pkg.generated.mbti` files. Review those diffs as API changes, especially for `@js_engine`, `@js_engine/token`, `@js_engine/ast`, and `@js_engine/interpreter/runtime`. ## Focused Stack-Safety Gate Run the permanent engine and stable-root-facade stack-safety suites for one target and profile with: ```bash make stack-safety-test TARGET= PROFILE= ``` The target and profile are required command-line arguments; omission or any other value fails before MoonBit is invoked. The adoption workflow runs both profiles for every supported target, without host stack-size overrides. Its eight matrix cells feed the stable `stack-safety-required` check, which fails unless the matrix result is exactly `success`. Validate that the Make target, workflow matrix, selected suites, required direct-comma workload, and the exact #790 retained-argument mixed-call result remain wired as intended with: ```bash make stack-safety-validate ``` ## Test262 The authoritative full-suite workflow is `.github/workflows/test262.yml`. It builds the JS target with `moon build --target js --release` and runs the native MoonBit runner (`cmd/test262_runner`) in a matrix of two modes (`strict`, `non-strict`) × four shards. Each shard uses 4 threads and a 5-second per-test timeout. The GitHub Actions job timeout is set in the workflow file. Generate release-grade conformance text from CI artifacts: ```bash make test262-report # includes edition tables by default make test262-report ARGS="--format=changelog" ``` Do not hand-copy headline conformance numbers between docs. If a table needs refreshing, regenerate it with `make test262-report` (native) or state clearly that it is a dated snapshot. For local focused runs: ```bash make test262-quick make test262-filter FILTER=language/expressions make test262-filter FILTER=built-ins/Promise ``` The native MoonBit runner (`cmd/test262_runner`) is authoritative for execution and skip decisions. Shared skip metadata lives in `scripts/test262_skip_metadata.json` (the `.py` alongside it is a shared reader/classifier, not the data) to keep runner and analyzer classifications from drifting. The static analyzer is still only a rough metadata census; it does not execute tests and must not be treated as conformance data or the skip-list source of truth. After editing shared skip metadata, run: ```bash make test262-validate-skips ``` This target checks that skip features, flags, and path suffixes still match the checked-out Test262 suite. It does not run tests or produce conformance numbers. When removing a blanket feature skip, also update the active intent docs listed in `scripts/docs_skip_policy_manifest.json` and add the feature to `graduated_features` there. Then run: ```bash make validate-docs-skip-policy ``` This fast check ensures active docs do not still claim the feature is blanket- skipped or wholly unimplemented. CI runs it on docs/metadata changes via `.github/workflows/docs-skip-policy.yml` and on every main/PR unit-test job when skip metadata or tooling changes. ## Benchmarks CI runs benchmark tests on the JS target: ```bash moon test benchmarks/ --target js ``` The benchmark CLI is also the `benchmarks` package: ```bash moon run benchmarks --target js -- --list moon run benchmarks --target js -- --all --csv ``` Timing is meaningful only on the JS target; the WASM and WASM-GC timer files return `0.0` by design. To inspect the verified candidate route without executing a script, run the native profiling CLI: ```bash moon run cmd/candidate_route_probe --target native -- path/to/script.js ``` Successful output is JSON on stdout. Usage, file, parse, and preparation errors go to stderr with a non-zero exit status. The report is unstable measurement data, not JavaScript semantics or a public compatibility contract. The pinned official JetStream 3 shell runner is available as a compatibility admission diagnostic: ```bash make jetstream3-admission-test # local contract tests; no network access make jetstream3-admission # acquire the pinned source and run admitted workloads ``` The second command builds the native release CLI and runs fixed two-iteration `raytrace` and `navier-stokes` admissions. Each workload checks discovery, execution, exact selected-workload result validation, and JSON serialization, then writes `jetstream3-admission.json` or `jetstream3-admission-navier-stokes.json`. The reports also record the MoonBit version, native/release profile, OS/architecture, engine commit, and working-tree state. They are not a full-suite JetStream score, a stable performance baseline, or evidence for an optimization. The scheduled workflow archives both reports without making this diagnostic a pull-request gate. Cross-engine reference work uses separate, generation-pinned candidate probes. Run their deterministic contracts locally with: ```bash make jetstream3-quickjs-probe-test make jetstream3-javascriptcore-probe-test make jetstream3-v8-probe-test make jetstream3-spidermonkey-probe-test ``` A manual dispatch of the `JetStream 3 admission` workflow runs independent QuickJS-ng, JavaScriptCore, V8, and SpiderMonkey compatibility jobs. Each acquires the exact engine generation through the recorded `jsvu` version, verifies the installed payload fingerprint, and runs only `navier-stokes` against the pinned JetStream revision. The JavaScriptCore, V8, and SpiderMonkey probes execute the standard shell from their official Linux payloads; they do not add repository-owned shell adapters. The SpiderMonkey probe also selects the fingerprinted NSPR libraries shipped in the same payload. These JSON artifacts are preparation evidence only. They do not define a Reference Cohort Lock, cross-engine score, regression threshold, or publication baseline. QuickJS-ng generation 1 is incompatible because the standard `qjs` shell does not expose the isolated global required by JetStream's `runString` contract. JavaScriptCore generation 1 is compatible with the admitted slice. V8 and SpiderMonkey generation 1 are also compatible. A launch error, timeout, or signal termination is recorded as `probe_failed`; the diagnostic artifact is preserved and the corresponding workflow job fails. To reproduce the Stage 1 stable-embedding usage baselines, run: ```bash make embedding-baseline ``` The command prints the MoonBit version, target, release mode, and sampling policy before running two fixed scenarios: - `embedding/one_shot/run_rule` includes fresh runtime construction, builtin setup, parsing, evaluation, and display-string conversion. It excludes process startup. - `embedding/persistent/call_json_rule` starts after `Engine` construction and rule-source evaluation. Each measured operation includes argument conversion, a synchronous call that mutates Engine-owned state, and result conversion through the strict JSON bridge. Both use 10 outer runs with auto-calibrated inner runs. They are different usage envelopes, so do not divide their results to claim that one API is faster than the other. Keep source, input data, target, mode, and sampling policy fixed when recording a new baseline. Optimization work still requires a separate, isolated benchmark that reproduces a concrete bottleneck. Benchmark output has both a category and a stage. Category answers when the benchmark should run (`regression`, `component`, `workflow`); stage answers what part of the engine it measures (`startup`, `frontend`, `execution`). Keep those separate when interpreting results: a frontend lexer regression and an execution property-lookup regression need different follow-up work. The scheduled and manual benchmark workflow uploads the raw CSV, writes a GitHub Actions summary with a full table, per-stage totals, a log-scale text chart, and closure-conversion comparisons, and publishes historical trend data with `benchmark-action/github-action-benchmark` on the `gh-pages` branch. The published `benchmarks/data.js` file stays generated by benchmark-action; after that update, the workflow replaces `benchmarks/index.html` with the source-controlled responsive dashboard template at `benchmarks/dashboard/index.html`. The CSV-to-Markdown renderer lives in `scripts/render-benchmark-summary.py` so the summary and PR comment share one formatter. Same-repository PRs additionally run the CSV benchmark CLI at the PR base SHA and head SHA on the same runner, upload both raw CSVs, and render reporting-only base-vs-head tables with base mean, PR mean, delta percent, PR/base ratio, CV, and noisy flags. These PR comparisons do not gate the workflow and do not update the historical baseline; fork PRs skip write-token benchmark reporting for safety. PR comment Markdown is rendered in the read-only benchmark execution job before the publish job posts it. Repository-write permissions are scoped to the publish job; the benchmark execution job runs with read-only repository contents permission. For less-noisy local timing, execute only the selected rows repeatedly and compare medians instead of relying on a single process run: ```bash make bench-focus ARGS="--runs 5" make bench-focus ARGS="--runs 3 --rows isolate/bytecode/property_get,pipeline/bytecode/evaluate" ``` (`make bench-focus` runs the native `cmd/bench_focus`.) With its default command, the helper runs the release benchmark separately for each selected row. This prevents an earlier row in the same JavaScript process from changing a later row's timing. It combines those isolated results into one CSV per run under `_build/bench-focus//` and reports median/mean/range across runs plus the median in-run CV. A custom `--command` is run once per run and must select and emit its own rows. Use this helper for post-merge baselines or exploratory follow-up work; use paired, same-runner base/head measurements for optimization claims. The `startup/tiny_program` benchmark is the low-noise guardrail for interpreter startup and built-in installation. It intentionally measures `run("1 + 1")` in process so CI trend data is not dominated by Node process spawn time. To split that in-process path into independently measured phases on the JS target, run: ```bash moon run --target js --release benchmarks -- --startup-phases --csv ``` The phase breakdown reports the full `startup/tiny_program` workload plus tiny parse, `new_interpreter`, already-parsed execution, empty event-loop drain, and result stringification/output handling. Treat the phase timings as separate microbenchmarks, not as an additive profile. To split `new_interpreter` itself into focused JS-target subphases before designing lazy or cached runtime initialization, run: ```bash moon run --target js --release benchmarks -- --startup-new-interpreter-subphases --csv ``` Those rows isolate realm/symbol setup, global environment/object setup, stdlib hook and host setup, `setup_builtins_with_realm_state`, its builtin-family subphases, global mirroring, generator/async constructors, test262 harness setup, and the harness's own print/agent/$262/stamping slices. They are also separate microbenchmarks, not an additive profile. Benchmark-only public profiling hooks are allowed only when a separate package (such as `benchmarks`) cannot call a private helper. Name them with a `profile_*` prefix, keep them in `*_profile.mbt`, and document that they return measurement data rather than JavaScript semantics. The startup subphase mirror intentionally duplicates the production final realm-stamp traversal; when that production traversal changes, update the mirror and its comments in the same patch. Engine-private hidden metadata uses negative symbol IDs in `PropertyBag` symbol maps. Reserve IDs in code comments before adding new hidden slots: - `-1..-2`: Array exotic length/prototype override slots in `interpreter/runtime/value.mbt`. - `-3..-5`: Array iterator next-index, iterated-object, and kind slots in `interpreter/runtime/iterators.mbt`. - `-101`: packed function metadata slot containing home-realm intrinsic prototypes, the realm's intrinsic-constructor prototype registry, and the optional host-supplied source identity in `interpreter/runtime/factories.mbt`. - `-111`: final realm-stamp traversal marker in `interpreter/stdlib/builtins.mbt`, mirrored by `benchmarks/startup_new_interpreter_subphases.mbt`. - `-112`: realm-owned `%ArrayProto_values%` intrinsic cache on `Array.prototype` in `interpreter/runtime/iterators.mbt`. - `-120..-122`: RegExp original source/flags and intrinsic constructor slots in `interpreter/stdlib/builtins_regex.mbt`. - `-123..-128`: RegExp String Iterator brand, regexp, string, global, unicode, and done slots in `interpreter/stdlib/builtins_regex.mbt`. - `-130..-132`: Map/Set iterator next-index, iterated collection, and kind slots in `interpreter/stdlib/builtins_map_set.mbt`. Runtime-created user and well-known symbols must stay non-negative. Traversal code treats negative symbol IDs as engine-private metadata and must not expose them as ordinary JavaScript symbol properties. For hosted process-level startup snapshots, use the manual-only Startup Hyperfine workflow (`.github/workflows/startup-hyperfine.yml`). It builds the JS release CLI, times repeated invocations of `node _build/js/release/build/cmd/main/main.js -e "1 + 1"` against Node.js and Bun, uploads the raw Hyperfine artifacts, and writes a reporting-only job summary. It does not publish `gh-pages`, comment on PRs, or enforce thresholds. For reproducible local startup decomposition, use the checked-in helper: ```bash scripts/startup-hyperfine-decompose.sh --warmup 10 --min-runs 50 # Fast smoke test before sending changes: scripts/startup-hyperfine-decompose.sh --warmup 1 --min-runs 2 ``` The helper builds `moon build cmd/main --target js --release`, captures the expected expression stdout from `node -p ` (default source: `1 + 1`), verifies that `js_engine` and any included Bun expression probes match that stdout, records the release bundle byte size and line count, and writes Markdown/JSON artifacts under `_build/startup-hyperfine-decompose//` by default. Its probes separate empty host startup, native host expression evaluation, `js_engine` load/no-source, full `js_engine` expression evaluation, and Bun-hosted `js_engine` when Bun is available (or required with `--include-bun`). Like the hosted workflow, it is reporting-only: no thresholds, publishing, or PR comments. ## Release Workflow Use [RELEASING.md](RELEASING.md). The release checklist makes CI artifacts the source of truth for conformance numbers and explains how to handle tag-vs-file drift after the release tip runs.