# Ion Schema (`moonrockz/ion/schema`)

Loads [Ion Schema](https://amazon-ion.github.io/ion-schema/) 1.0 and 2.0
schemas and validates Ion values and documents against their types. It
implements every constraint of both versions, and schema imports. The loaded
schema holds ISL 2.0 constraints: the loader translates each ISL 1.0
construct into ISL 2.0 constraints with the same meaning (see
[ISL 1.0](#isl-10)).

Constraints: `type`, `all_of`, `any_of`, `one_of`, `not`, `valid_values`,
`element` (with `distinct::`), `ordered_elements`, `contains`, `fields` (with
`occurs` and `closed::`), `field_names` (with `distinct::`), `annotations`
(standard, and simplified with `required::` and `closed::`),
`container_length`, `codepoint_length`, `byte_length`, `utf8_byte_length`,
`precision`, `exponent`, `regex`, `timestamp_precision`, `timestamp_offset`,
and `ieee754_float`.

A length is a non-negative integer or a `range::[lower, upper]` of integers,
and so are `precision` (at least 1) and `exponent` (any integer).
`timestamp_precision` takes the precision names, from `year` to
`nanosecond`.
`valid_values` takes a list of values and ranges, or one range. A range there
is of numbers or of timestamps. Numbers compare by their exact value, whatever
their Ion type, so `1`, `1.0`, and `1e0` are all in `range::[1, 1]`.
Timestamps compare by the instant they name. In any range, a bound can be
`exclusive::`, and `min` or `max` leaves that end open.

`regex` takes the ISL 2.0 subset of ECMA 262 regular expressions, and a
string or symbol is valid when the expression matches any part of it. `\d`,
`\s`, and `\w` are the ASCII classes `[0-9]`, `[ \f\n\r\t]`, and
`[A-Za-z0-9_]`. The flag `i::` ignores case for ASCII letters only, and `m::`
makes `^` and `$` match at line breaks. A construct outside the subset, such
as a backreference or a lazy quantifier, is an error. Matching runs a Pike VM,
which takes time linear in the input, so no expression makes it backtrack
without end. `Regex` is public, so the same expressions can be used on their
own.

In ISL 2.0, a core type such as `int` or `struct` matches only the
non-null values of its Ion type, and `$int` or `$struct` also matches its
typed null. `text`, `lob`, `number`, and `any` cover several Ion types, and
`$text`, `$lob`, `$number`, and `$any` add their nulls. `$null` matches
`null.null`, `nothing` matches no value, and `$null_or::T` matches `null.null`
or a value of `T`. `type` takes a type name, an inline type, or an inline
import. `occurs` is allowed only in `fields` and `ordered_elements`.
`element` constrains the elements of a list or S-expression, or the field
values of a struct.
`valid_values` compares a value without its annotations. `ordered_elements`
matches the elements in order, each type as many times as its `occurs`
allows; an ambiguous match is found without backtracking. `annotations`
validates the value's annotations as a list of symbols.

A document is a stream of top-level values. `validate_document` and
`is_valid_document` check one: `element`, `ordered_elements`,
`container_length`, and `contains` apply to its values, and the built-in type
`document` matches it. No other constraint applies to a document.

A schema that its version of ISL does not allow raises `IonError::DataModel`
when it loads, as the ion-schema-tests suite's invalid schemas and types require.

Named types may recurse through container children. If validation revisits a
named type on the same value without consuming a child, it returns a
`cyclic schema type reference` violation instead of overflowing the stack.
This also applies to documents, imported aliases, and annotation values. A
finite successful `any_of` branch still proves validity; a cycle alone cannot
become valid through `not` or count as a match for `one_of`. These checks run
during validation, so loading a cyclic schema remains supported.

## Schema documents

An ISL 2.0 schema document has the version marker `$ion_schema_2_0`, an optional
`schema_header::{...}`, the `type::{...}` definitions, and an optional
`schema_footer::{...}`, after which nothing counts. Other top-level values are
open content, which the loader ignores, unless they are annotated with a
reserved symbol: `$ion_schema`, a symbol that starts with `$ion_schema_`, or a
symbol in lower snake case. The header, the footer, and each type may have
fields whose names are not reserved. The header's `user_reserved_fields`
declares reserved symbols, other than ISL keywords, as more such fields.
A document with the version marker `$ion_schema_1_0`, or with no version
marker, is ISL 1.0.

A type reference must name a type that the schema defines or imports.
Imports name other schemas by an `id`, which the `resolver` given to
`Schema::load` turns into that schema's values, for example by reading a file
under a base directory. A header import brings in all the types of a schema,
or one type, optionally under another name with `as`; an inline import
`{ id: ..., type: ... }` refers to one type without adding it to the scope.
Types that an imported schema imports in turn are not in scope. Schemas may
import each other in a cycle, but not themselves. Without a resolver, an
import raises `Unsupported`. `types()` lists an imported type as
`<id>#<name>`. An ISL 1.0 schema and an ISL 2.0 schema may import each
other, and each type keeps the meaning of its own version.

## ISL 1.0

ISL 1.0 gives some of the same names other meanings, so the version marker
decides how a document loads. The differences from ISL 2.0 are:

- A type with no `type` constraint has `type: any`, so it matches no null.
  `any` also matches a document. In ISL 2.0, such a type matches any value.
- `nullable::T` matches `null.null`, the typed nulls of the Ion types of `T`'s
  base type, and the values of `T`. The base type is the type that `T`'s
  `type` constraint names, or `any`. For example, `nullable::int` matches
  `null`, `null.int`, and `5`, but not `null.string`. There is no
  `$null_or::`.
- `content: closed` allows only the fields in the type's `fields`, which
  have no `closed::`.
- `scale` is the number of digits after the decimal point: `scale: 2` is
  `exponent: -2`. There is no `exponent`, `field_names`, or
  `ieee754_float`.
- `annotations` takes only a list of symbols, annotated with any of
  `required::`, `ordered::`, and `closed::`. Each symbol can be
  `required::` or `optional::`. With `ordered`, the required symbols must
  appear in order; with `closed` too, the value's annotations must be the
  symbols in order, the optional ones at most once.
- A valid `occurs` is allowed in any type, and has no effect outside
  `fields` and `ordered_elements`. `element` takes no `distinct::`.
- An inline type may be annotated `type::`, and an inline import may have an
  `as`, which has no effect.
- A bound of a timestamp range in `valid_values` must have a known offset.
- There are no reserved symbols: a top-level value that is not a header, a
  type, or a footer, and a field that is not an ISL 1.0 keyword, are open
  content. The header's `imports` is its only field with a meaning. A
  schema has a header only if it has a footer.

```mbt check
///|
test "load an ISL 1.0 schema" {
  let schema = @schema.Schema::load_from_text(
    (
      #|$ion_schema_1_0
      #|type::{
      #|  name: price,
      #|  content: closed,
      #|  fields: {
      #|    amount: { type: decimal, scale: 2, occurs: required },
      #|    note: nullable::string,
      #|  },
      #|}
    ),
  )
  assert_true(
    schema.is_valid("price", @text.read_ion("{ amount: 9.99, note: null }")),
  )
  // 9.9 has a scale of 1, `tax` is not a field of the type, and the default
  // type `any` has no null.
  assert_true(!schema.is_valid("price", @text.read_ion("{ amount: 9.9 }")))
  assert_true(
    !schema.is_valid("price", @text.read_ion("{ amount: 9.99, tax: 1.00 }")),
  )
  assert_true(!schema.is_valid("price", @text.read_ion("null.struct")))
}
```

## Validating values

`Schema::load_from_text` reads the `type::{...}` definitions of a schema
document. `is_valid` answers yes or no, and `validate` lists each violation
with the path to the value that broke it.

```mbt check
///|
test "validate records against a type" {
  let definition =
    #|$ion_schema_2_0
    #|type::{
    #|  name: person,
    #|  type: struct,
    #|  fields: {
    #|    name: { type: string, occurs: required, codepoint_length: range::[1, 40] },
    #|    role: { valid_values: [admin, member, guest] },
    #|    tags: { type: list, element: symbol },
    #|  },
    #|}
  let schema = @schema.Schema::load_from_text(definition)
  assert_true(
    schema.is_valid("person", @text.read_ion("{ name: \"Ada\", role: admin }")),
  )
  let violations = schema.validate(
    "person",
    @text.read_ion("{ role: owner, tags: [a, \"b\"] }"),
  )
  let report = violations
    .map(violation => violation.path() + ": " + violation.message())
    .join("\n")
  inspect(
    report,
    content=(
      #|$.name: field occurs 0 time(s)
      #|$.role: value is not one of the allowed values
      #|$.tags[1]: expected one of [symbol] but found string
    ),
  )
}
```

## Imports

```mbt check
///|
test "import a type from another schema" {
  let schemas : Map[String, String] = {
    "common.isl": "$ion_schema_2_0 type::{ name: positive_int, type: int, valid_values: range::[1, max] }",
  }
  let resolver = (id : String) => {
    match schemas.get(id) {
      Some(text) => @text.read_ion_datagram(text)
      None => raise @ion.IonError::DataModel("no schema named " + id)
    }
  }
  let schema = @schema.Schema::load_from_text(
    (
      #|$ion_schema_2_0
      #|schema_header::{ imports: [{ id: "common.isl", type: positive_int, as: count }] }
      #|type::{ name: tally, fields: { votes: count } }
    ),
    resolver~,
  )
  assert_true(schema.is_valid("tally", @text.read_ion("{ votes: 3 }")))
  assert_true(!schema.is_valid("tally", @text.read_ion("{ votes: 0 }")))
  // Without a resolver, the import cannot load.
  let message = try
    @schema.Schema::load_from_text(
      "$ion_schema_2_0 schema_header::{ imports: [{ id: \"common.isl\" }] }",
    )
  catch {
    error => error.message()
  } noraise {
    _ => "loaded"
  }
  inspect(
    message,
    content="the schema imports 'common.isl', but no resolver was given to load it",
  )
}
```