# Ghostty Parser Architecture

## Scope

This document covers the upstream Ghostty parser subsystem under
`upstream/ghostty`, limited to the files that define the VT parser pipeline and
its runtime integration:

- `src/terminal/Parser.zig`
- `src/terminal/parse_table.zig`
- `src/terminal/stream.zig`
- `src/terminal/stream_terminal.zig`
- `src/terminal/osc.zig`
- `src/terminal/dcs.zig`
- `src/terminal/sgr.zig`
- `src/terminal/UTF8Decoder.zig`
- `src/termio/Termio.zig`

The goal is a faithful MoonBit translation of the parser stack. This means
preserving module boundaries, state machines, ordered side effects, and the
data contracts between layers, not just reproducing the external behavior.

## Source Map

The parser is not one parser. Upstream splits it into four layers:

1. `Parser.zig`: byte-oriented DEC ANSI state machine.
2. `stream.zig`: bridge from parser actions to typed terminal actions.
3. `osc.zig`, `dcs.zig`, `sgr.zig`: sequence-specific semantic decoders.
4. `stream_terminal.zig` and `termio/Termio.zig`: action application and host
   integration.

`UTF8Decoder.zig` sits beside the VT parser. It is used only when the stream is
in ground state.

## End-to-End Data Flow

The live application path is:

1. `Termio.processOutputLocked(buf)` receives PTY bytes.
2. It schedules renderer work, handles cursor blink reset throttling, and then
   forwards bytes into `terminal_stream`.
3. In normal mode it calls `terminal_stream.nextSlice(buf)`.
4. In inspector mode it falls back to byte-at-a-time `terminal_stream.next(b)`.
5. `stream.zig` decides whether bytes stay on the UTF-8 fast path or must enter
   the VT control parser.
6. `Parser.next(byte)` emits up to three ordered actions for one byte:
   exit action, transition action, entry action.
7. `stream.zig` interprets those parser actions into `Stream.Action` values and
   immediately calls the handler's `vt` method.
8. A handler applies those actions either to a `Terminal` model
   (`stream_terminal.zig`) or to the full application runtime
   (`termio/stream_handler.zig`, instantiated by `Termio`).

The important architectural point is that Ghostty keeps these concerns
separate:

- byte classification and state transition
- escape-sequence semantic decoding
- terminal-state mutation
- host side effects such as mailbox messages, PTY writes, clipboard, and title
  updates

## Layer 1: DEC ANSI State Machine

### `terminal/Parser.zig`

`Parser` is a compact stateful machine modeled directly on
<https://vt100.net/emu/dec_ansi_parser>.

Its persistent fields are:

- `state: State`
- `intermediates[4]` plus `intermediates_idx`
- `params[24]`, `params_sep`, `params_idx`
- `param_acc` and `param_acc_idx`
- embedded `osc_parser: osc.Parser`

The state enum is the classic DEC-style split:

- `ground`
- `escape`
- `escape_intermediate`
- `csi_entry`, `csi_param`, `csi_intermediate`, `csi_ignore`
- `dcs_entry`, `dcs_param`, `dcs_intermediate`, `dcs_passthrough`, `dcs_ignore`
- `osc_string`
- `sos_pm_apc_string`

`Parser.Action` is the stable output contract of this layer. It does not know
about `Terminal`; it only emits structural events:

- `print`
- `execute`
- `esc_dispatch`
- `csi_dispatch`
- `osc_dispatch`
- `dcs_hook`, `dcs_put`, `dcs_unhook`
- `apc_start`, `apc_put`, `apc_end`

The critical function is `Parser.next(c: u8) -> [3]?Action`. For a single byte
it can emit:

1. an exit action from the old state
2. the transition action for the byte
3. an entry action for the new state

That ordering is semantically important. Examples:

- leaving `osc_string` runs `osc_parser.end(c)` and may emit `osc_dispatch`
- entering `dcs_passthrough` emits `dcs_hook`
- leaving `dcs_passthrough` emits `dcs_unhook`
- entering `sos_pm_apc_string` emits `apc_start`
- leaving `sos_pm_apc_string` emits `apc_end`

`Parser.doAction` is where the machine mutates temporary parse buffers:

- `collect` appends intermediate bytes
- `param` accumulates numeric parameters and records `;` vs `:`
- `osc_put` forwards bytes into `osc.Parser.next`
- `csi_dispatch` finalizes the current parameter list

Two invariants matter for translation:

- colon separators are represented explicitly in `params_sep`
- colon separators are rejected for non-`m` CSI finals

### `terminal/parse_table.zig`

`parse_table.zig` generates the full transition table at compile time. It is
not a lookup cache over handwritten logic; it is the logic.

Each table cell is:

- `state: State`
- `action: Parser.TransitionAction`

The generated table covers:

- global "anywhere" transitions for ESC, C1 controls, OSC, CSI, DCS, APC/SOS/PM
- per-state byte ranges
- state-specific transition actions like `collect`, `param`, `put`, `osc_put`

Ghostty keeps one deliberate grammar extension here: `csi_param` accepts `:`
because SGR supports colon-separated subparameters.

For a line-by-line MoonBit port, the table should remain data-driven. Replacing
it with a single large `switch` would erase one of the core structural choices
of the upstream code.

## Layer 2: Ground-State UTF-8 and Parser Bridging

### `terminal/UTF8Decoder.zig`

`UTF8Decoder` is a DFA decoder with replacement semantics and one nontrivial
API guarantee:

- `next(byte)` returns `{ ?u21, bool }`
- the boolean says whether the byte was consumed

On malformed UTF-8, the decoder can emit U+FFFD and request that the same byte
be retried. `stream.zig` depends on that behavior.

### `terminal/stream.zig`

`Stream(H)` is the core orchestration layer. It owns:

- `handler: H`
- `parser: Parser`
- `utf8decoder: UTF8Decoder`

`Stream.Action` is the semantic action surface of the parser subsystem. It is
much richer than `Parser.Action`: cursor movement, erase commands, modes,
attributes, title changes, color operations, DCS/APC passthrough events, and
more.

The important control-flow functions are:

- `nextSlice`
- `nextSliceCapped`
- `consumeAllEscapes`
- `consumeUntilGround`
- `next`
- `nextUtf8`
- `handleCodepoint`
- `nextNonUtf8`

The fast path in `nextSlice` is tightly structured:

1. finish any partial scalar UTF-8 decode
2. finish any in-progress control sequence until parser returns to `ground`
3. consume runs of back-to-back `ESC` sequences
4. while in `ground`, use SIMD UTF-8 decoding until a control byte appears
5. fall back to scalar handling only when needed

`nextNonUtf8` has two extra fast paths before calling `Parser.next`:

- `ESC [` jumps directly to `csi_entry`
- dense `csi_param` bytes update parser buffers without table lookup

Once `Parser.next` returns its three possible actions, `stream.zig` replays
them in order and maps them as follows:

- `print` -> handler `vt(.print, ...)`
- `execute` -> C0/C1 handling in `execute`
- `csi_dispatch` -> `csiDispatch`
- `esc_dispatch` -> `escDispatch`
- `osc_dispatch` -> `oscDispatch`
- DCS/APC lifecycle events -> direct handler calls

There is also an optional raw interception hook: if the handler defines
`vtRaw`, it can observe and suppress processed parser actions before
`stream.zig` decodes them further.

## Layer 3: Sequence-Specific Decoders

### CSI decoding in `stream.zig`

`Parser.Action.csi_dispatch` contains only:

- `intermediates`
- `params`
- `params_sep`
- `final`

`stream.zig` owns the real CSI semantics. `csiDispatch` is a large final-byte
dispatcher that:

- validates intermediate bytes and parameter count
- fills in VT defaults
- emits typed `Stream.Action` values
- delegates SGR payload interpretation to `sgr.Parser`

This is where sequences like cursor motion, margins, mode changes, reports,
title stack commands, tab control, and kitty keyboard controls become terminal
actions.

### `terminal/sgr.zig`

SGR is pulled out into its own parser instead of being inlined into the CSI
dispatcher.

`sgr.Parser` iterates over the finalized CSI parameter array using:

- `params`
- `params_sep`
- `idx`

Its `next()` method returns one typed `Attribute` at a time. It understands:

- empty SGR as `unset`
- classic numeric attributes
- underline style subparameters
- 8-color, bright-color, 256-color, and direct RGB color forms
- colon and semicolon variants for direct color and underline color

Unknown forms are not silently dropped. They return `.unknown` with both:

- the full original SGR parameter slice
- the unresolved remaining suffix

That is a fidelity detail worth keeping in MoonBit.

### `terminal/osc.zig`

OSC uses a dedicated parser because it is not parameter-array based.

`osc.Parser` contains:

- `state: State`
- fixed `buffer[2048]`
- optional `capture`
- optional allocator for oversized commands
- `command: Command`

The key design choice is that `state` is a prefix trie encoded as an enum, not
an integer accumulator. States include `@"0"`, `@"2"`, `@"52"`, `@"133"`,
`@"777"`, `@"5522"`, and similar bridging states.

`next(c)` advances through that prefix trie until it sees `;`, then starts
capturing trailing payload bytes either:

- in the fixed buffer
- in an allocating writer for large protocols like OSC 52 and kitty clipboard

`end(terminator_ch)` dispatches to specialized parsers under
`terminal/osc/parsers.zig` and returns a typed `osc.Command`.

`stream.zig.oscDispatch` then maps those commands into terminal actions such as:

- `window_title`
- `start_hyperlink` / `end_hyperlink`
- `report_pwd`
- `clipboard_contents`
- `mouse_shape`
- `color_operation`
- `semantic_prompt`

Important fidelity points:

- OSC title changes are validated as UTF-8 before being emitted
- response-capable OSC commands preserve which terminator was used

### `terminal/dcs.zig`

DCS is stateful and streaming, so Ghostty does not parse it as a single
captured blob.

`dcs.Handler` maintains a `state` union with modes for:

- inactive
- ignore
- tmux control mode
- XTGETTCAP
- DECRQSS

The parser interface matches the lifecycle from `Parser.zig`:

- `hook(dcs_metadata)`
- repeated `put(byte)`
- `unhook()`

Recognized hooks are selected from the `DCS` metadata:

- `ESC P 1000 p` -> tmux control mode
- `ESC P + q` -> XTGETTCAP
- `ESC P $ q` -> DECRQSS

This is an incremental protocol parser, not buffered DCS text parsing. The
MoonBit port should preserve that split.

### APC in the stream layer

APC itself is not in the requested file list, but the listed files expose its
shape clearly:

- `Parser.zig` emits `apc_start`, `apc_put`, `apc_end`
- `stream.zig` forwards those events
- `stream_terminal.zig` feeds them into `apc.Handler`

Architecturally, APC behaves like DCS passthrough rather than like OSC.

## Layer 4: Applying Actions to a Terminal

### `terminal/stream_terminal.zig`

`stream_terminal.zig` is the reference terminal-mutating handler. It instantiates
`stream.Stream(Handler)` and gives a concrete meaning to `Stream.Action`.

`Handler` owns:

- `terminal: *Terminal`
- optional `effects` callbacks
- `apc_handler`

Its main dispatch function is `vtFallible(action, value)`. That function
applies:

- print and movement actions to `Terminal`
- attribute changes to `Terminal.setAttribute`
- mode toggles to `setMode`
- OSC color actions to `colorOperation` / `kittyColorOperation`
- APC bytes to `apc_handler`
- query responses through effect callbacks such as `write_pty`

Notable boundaries:

- DCS actions are ignored here; this handler is the minimal terminal-state
  implementation
- query-style actions such as device attributes and size reports are routed
  through effect callbacks instead of mutating terminal state directly
- title updates go through `windowTitle`, which sets terminal title and then
  triggers `title_changed`

For translation work, `stream_terminal.zig` is the cleanest place to understand
the semantic meaning of each `Stream.Action`.

## Runtime Integration

### `termio/Termio.zig`

`Termio` owns the live parser instance as:

- `terminal_stream: StreamHandler.Stream`

That handler type is not `stream_terminal.Handler`; the application uses a
richer wrapper in `termio/stream_handler.zig` that combines terminal mutation
with mailbox, renderer, clipboard, DCS, APC, and PTY-response behavior.

Within the requested file set, the important integration points are:

- `init`: builds the `Terminal`, constructs the stream handler, and creates
  `terminal_stream` with `initAlloc`
- `processOutputLocked`: the input boundary from PTY bytes into the parser
- `changeConfig`: pushes runtime config changes into the handler
- `resetSynchronizedOutput`: shows that parser-controlled modes affect renderer
  scheduling outside the parser itself

`processOutputLocked` does more than feed bytes:

- queues a render before parsing
- rate-limits cursor blink reset notifications
- switches to byte-at-a-time parsing when the inspector is active
- otherwise uses `nextSlice` for throughput
- notifies the termio mailbox if handler actions queued work

So the production integration path is:

`Termio` -> `StreamHandler.Stream` -> `stream.zig` -> `Parser/OSC/DCS/SGR` ->
handler actions -> terminal state plus host side effects.

## Translation Constraints

A faithful MoonBit translation should preserve these properties:

- `Parser.next` must keep the three-slot ordered action contract.
- `parse_table` should stay generated or table-driven.
- UTF-8 decoding must keep the retry-on-error behavior.
- `stream.zig` should preserve the split between UTF-8 ground-state fast path
  and control-sequence parsing.
- CSI semantics should remain in `stream`, with SGR in its own parser.
- OSC should keep its explicit prefix-state machine plus fixed/allocating
  capture split.
- DCS should remain incremental with `hook` / `put` / `unhook`.
- terminal mutation should stay separate from host effects.
- `Termio` integration should continue to accept slices of bytes and feed the
  same layered parser pipeline.

## Current MoonBit Implementation

The translated parser layers now live in separate packages:

| Upstream layer | MoonBit location |
|---|---|
| `Parser.zig` and `parse_table.zig` | [`terminal/parser/`](../terminal/parser/) |
| `UTF8Decoder.zig` | [`terminal/utf8_decoder/`](../terminal/utf8_decoder/) |
| `stream.zig` | [`terminal/stream/`](../terminal/stream/) |
| `osc.zig` and its decoders | [`terminal/osc/`](../terminal/osc/) |
| `sgr.zig` | [`terminal/sgr/`](../terminal/sgr/) |
| Terminal facade and action application | [`stream_terminal.mbt`](../terminal/stream_terminal.mbt) and [`stream_terminal_bridge.mbt`](../terminal/stream_terminal_bridge.mbt) |

DCS lifecycle events pass through the parser and stream packages. APC handling
and Kitty graphics integration remain in the `terminal` package. Host effects
are supplied through the facade's callbacks; the upstream `Termio` runtime
described above is not a translated module in this repository.

See the [terminal package navigation map](terminal-package.md) for the current
state-model, encoder, and output packages, and the [README](../README.md#scope)
for project scope.
