# Schema readiness Generated by `python3 code/tests/eval/run_readiness.py --cli `; do not edit by hand. Three different questions, measured on the same stream (FR-36, FR-39): - **syntax ready** — the first input fraction at which the value JSONMend has *committed* to is the value the document ends with. A member is committed once the token after it settles it: a comma, or the closing brace. This number is close to 100% for any document, which is why valid JSON is the wrong readiness signal. - **settled** — the first input fraction at which every argument the caller *declared* already holds the value it will end up with, **whether or not that value has the declared kind**. This is the number a runtime acts on: it says when the answer — yes or no — arrived. On a long stream it is the moment the caller may cancel the generation. - **schema ready** — the first input fraction at which every declared argument is settled **and** the skeleton is satisfied. It answers “may I execute”, not “do I know yet”: a document that is decidable and invalid never fires it, so it reads `1000‰` — the same value as “not before the end” — even though the refusal was final long before. `settled` is the column that tells those two apart, and it was added in 0.6.0 for exactly that reason. `step` is the sampling granularity (one code unit here, `auto_step` in the library). Fractions are per mille of the input. | case | bytes | syntax ready | settled | schema ready | declared | ok | executable | what it shows | |---|---:|---:|---:|---:|---:|:--:|:--:|---| | search, long tail | 247 | 995‰ | 121‰ | 121‰ | 2/2 | yes | yes | the two arguments are done a fifth of the way in | | search, tight | 30 | 1000‰ | 1000‰ | 1000‰ | 2/2 | yes | yes | nothing after the last argument, so there is nothing to save | | required argument last | 266 | 996‰ | 996‰ | 996‰ | 2/2 | yes | yes | the arguments a gate needs are the last thing the model writes | | three arguments | 272 | 996‰ | 213‰ | 213‰ | 3/3 | yes | yes | the third argument is long and optional | | nested arguments | 315 | 996‰ | 200‰ | 200‰ | 4/4 | yes | yes | a nested object is only ready when its own closer arrives | | truncated by max_tokens | 52 | 1000‰ | 1000‰ | 1000‰ | 3/3 | yes | no | the stream stops before the value is closed | | fenced and prefixed | 223 | 977‰ | 264‰ | 264‰ | 2/2 | yes | yes | a chatty model: a sentence, then a fence | | declared member, wrong kind | 249 | 995‰ | 128‰ | 1000‰ | 1/2 | no | no | `limit` is final at 128‰ and it is a string where an integer was declared, so `schema ready` never fires and stays at 1000‰ — which reads as “not before the end” when the truth is “decided, and the answer is no”. The runtime number is `settled`: the answer arrived at 128‰, the answer is no, and a streaming caller can cancel there instead of reading the 249-byte tail. This is the case 0.6.0 was written for (FR-39). | | declared member, absent in a container | 142 | 992‰ | 84‰ | 1000‰ | 2/3 | no | no | `a.c` is never written, but the closer of `a` decides what is inside it, so the refusal is final at 84‰ — while `schema ready` stays 1000‰ because the skeleton is never satisfied. Together with the wrong-kind row and the absent-at-the-root row this is a three-way contract: a wrong kind is decided by its own value, an absent member inside a container by that container's closer, and an absent root member only by the end of the input. | | declared member, absent at the root | 235 | 995‰ | 1000‰ | 1000‰ | 1/2 | no | no | `limit` never appears, and that is *not* decidable early: an open stream cannot rule out the model writing it, so `settled` is 1000‰ too and both numbers agree. Unlike `a.c` above there is no container closer to decide it — only the end of the input does. The three rows together are the contract: a runtime that treats them the same is either cancelling too late or too early. | ## What the numbers say - Where the declared arguments are written early and the model keeps talking, a gate can act after a small fraction of the stream: the `search, long tail` case is settled at a fraction of the syntax fraction, and the syntax fraction stays above 90%. - Where the last declared argument is the last thing written (`search, tight`, `required argument last`), the numbers meet at the end. Nothing can be saved, and the table says so instead of promising an early stop that is not there. - The three 0.6.0 rows are the reason `settled` exists, and together they read as one three-way contract on the same stream. In all three `schema ready` is 1000‰, so the batch number alone cannot tell them apart; `settled` says when the answer arrived, and the answer is no. | declared member | decided by | `settled` | | --- | --- | ---: | | present, wrong kind | its own value | 128‰ | | absent inside `a` | the closer of `a` | 84‰ | | absent at the root | the end of the input | 1000‰ | A streaming caller cancels at `settled` in the first two cases; in the third there is nothing to cancel early, because an open root may still contain the member the model has not written yet. Treating the three the same is either cancelling too late or too early. - A stream that never closes (`truncated by max_tokens`) is still schema ready — if its declared arguments arrived — but not executable: the JSON itself is not valid, so the gate refuses with `E010`. Readiness is about *when*, the gate is about *whether*. The measurement is a property of the input and the declaration, not of a model: it says how early a gate *could* decide, not how early a particular provider would have emitted the bytes.