# Ion binary (`moonrockz/ion/binary`)

Reads and writes the Ion 1.0 binary encoding: the version marker, type
descriptors, every value type, annotation wrappers, NOP padding, and local
symbol tables. The [ion-tests](https://github.com/amazon-ion/ion-tests) suite
checks it.

| Need | API |
| --- | --- |
| A whole datagram | `read_binary`, `write_binary` |
| One value, as its own datagram | `read_binary_value`, `write_binary_value` |
| One value at a time, for a stream | `decode_value`, `encode_value`, `value_size` |
| A pull stream of events | `BinaryEventReader::new`, `next`, `remaining` |
| The symbol table a writer needs | `symbols_for_values`, `local_symbol_table` |
| Type descriptors | `IonTypeCode`, `type_descriptor`, `descriptor_type_code` |

`moonrockz/ion/binary/stream` runs the same codec over async IO.

## Events

`BinaryEventReader` produces the same `@text.IonEvent` sequence as
`@text.IonReader`: one document boundary pair, annotations before their
value, field names before field values, container boundaries, and scalars.
Typed null containers are scalars. Version markers and local symbol tables
update symbol resolution without emitting events. Pass `catalog` to resolve
shared-table imports.

Construction decodes the whole datagram, as the sync text event reader does.
Use the async `BinaryEventReader` to hold one top-level value at a time.

```mbt check
///|
test "read binary events" {
  let bytes = @binary.write_binary(@text.read_ion_datagram("tag::[1]"))
  let reader = @binary.BinaryEventReader::new(bytes)
  debug_inspect(
    reader.remaining().map(event => event.to_text()),
    content=(
      #|[
      #|  "DocumentStart",
      #|  "Annotation(tag)",
      #|  "ListStart",
      #|  "Scalar(int)",
      #|  "ListEnd",
      #|  "DocumentEnd",
      #|]
    ),
  )
}
```

## Reading and writing

A datagram starts with the four-octet version marker `E0 01 00 EA`. The
writer adds a local symbol table when a value uses a symbol that the system
table does not define.

```mbt check
///|
fn octets(bytes : Bytes) -> String {
  bytes
  .to_array()
  .map(octet => {
    let text = octet.to_int().to_string(radix=16)
    if text.length() == 1 {
      "0" + text
    } else {
      text
    }
  })
  .join(" ")
}

///|
test "write and read a datagram" {
  let values = @text.read_ion_datagram("1 true \"hi\"")
  let bytes = @binary.write_binary(values)
  inspect(octets(bytes), content="e0 01 00 ea 21 01 11 82 68 69")
  inspect(
    @text.write_all(@binary.read_binary(bytes)),
    content=(
      #|1
      #|true
      #|"hi"
    ),
  )
}

///|
test "a symbol outside the system table gets a local symbol table" {
  let bytes = @binary.write_binary_value(@ion.IonValue::symbol("hello"))
  // The marker, `$ion_symbol_table::{symbols: ["hello"]}`, and symbol 10.
  inspect(
    octets(bytes),
    content="e0 01 00 ea eb 81 83 d8 87 b6 85 68 65 6c 6c 6f 71 0a",
  )
  inspect(@text.write_ion(@binary.read_binary_value(bytes)), content="hello")
}
```

Binary is usually smaller than text, and it keeps every value exactly:

```mbt check
///|
test "binary round trips keep precision" {
  let text = "{ price: 19.990, when: 2024-02-29T12:00:00.500+01:00, big: 123456789012345678901234567890 }"
  let values = @text.read_ion_datagram(text)
  let bytes = @binary.write_binary(values)
  assert_true(@binary.read_binary(bytes)[0].equals(values[0]))
  inspect(bytes.length() < text.length(), content="true")
}
```

Malformed input raises an `@ion.IonError`:

```mbt check
///|
test "a truncated value" {
  let bytes = @binary.write_binary_value(@ion.IonValue::string("truncated"))
  let message = try @binary.read_binary(bytes[:bytes.length() - 1]) catch {
    error => error.message()
  } noraise {
    _ => "read"
  }
  inspect(message, content="truncated Ion binary value")
}
```

The binary reader and writer handle unrestricted container nesting with a
bounded machine stack. Pass `fuel=Limited(@ion.Budget(128))` to a reader to
bound simultaneously open containers. Annotations and scalar values spend
no binary-reader fuel. Exhaustion raises `@ion.IonError::FuelExhausted` with
the budget, operation and byte offset. Event and asynchronous reader
constructors accept the same option.

## One value at a time

A streaming writer builds one symbol table for all its values, writes it
after the marker, and then encodes each value with it. A streaming reader
asks `value_size` how many octets the next value needs, and decodes it once
they have arrived.

```mbt check
///|
test "encode and decode value by value" {
  let values = [@ion.IonValue::symbol("red"), @ion.IonValue::symbol("green")]
  let symbols = @binary.symbols_for_values(values)
  let mut bytes = @binary.binary_version_marker() +
    @binary.local_symbol_table(symbols).unwrap()
  for value in values {
    bytes = bytes + @binary.encode_value(value, symbols)
  }
  inspect(
    @text.write_all(@binary.read_binary(bytes)),
    content=(
      #|red
      #|green
    ),
  )
  // Decoding from a view: the table applies to the values after it.
  let table = @ion.SymbolTable::system()
  let mut offset = 4
  let read : Array[String] = []
  while @binary.decode_value(bytes[:], offset, table) is Some((value, used)) {
    offset += used
    if @binary.accept_value(table, value) is Some(value) {
      read.push(@text.write_ion(value))
    }
  }
  debug_inspect(
    read,
    content=(
      #|["red", "green"]
    ),
  )
}
```

## Type descriptors

Each value starts with a descriptor octet: a type code in the high nibble and
a length (or a null marker) in the low nibble.

```mbt check
///|
test "type descriptors" {
  let descriptor = @binary.type_descriptor(
    @binary.IonTypeCode::from_ion_type(String),
    3,
  )
  inspect(descriptor.to_string(radix=16), content="83")
  debug_inspect(
    @binary.descriptor_type_code(0x2F).to_ion_type(),
    content="Some(Int)",
  )
  inspect(@binary.descriptor_is_null(0x2F), content="true")
}
```