Conformance

Conformance means that an implementation meets the declared protocol and profile contract for the functions it supplies. Canonical bytes, current authority checks, refusal behavior and recovery assumptions must be assessed together. Passing a decoder's corpus does not make it a durable authority implementation.

Requirement key words#

NormativeBytes, test vectors and proofs

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in these documents are to be interpreted as described in BCP 14 (RFC 2119 (opens in a new tab) and RFC 8174) when, and only when, they appear in all capitals, as shown here. The same words in lower case have their ordinary English meaning and state no requirement.

Protocol core and reference profile#

NormativeBytes, test vectors and proofs

The protocol core is what every conforming implementation of OIP (opens in a new tab) v0.6.0 implements in the same way: the canonical bytes and field numbers, the operand positions of every operation, the envelope and negotiation signing inputs, the version check, the response codes and their effect, and the product rules stated with key words.

The reference profile is the configuration of the Rust reference implementation in this repository. It covers its Ed25519 (opens in a new tab) session and maintenance tokens, its JSON signing inputs, its configuration values and accepted operations, its storage and control transports and its host adapters.

A statement that names the reference, the reference profile or the reference configuration, and every section whose heading names the reference, describes the reference profile. Another implementation MAY use a different profile when it meets the protocol core. All other key-word statements belong to the protocol core.

Document map#

NormativeBytes, test vectors and proofs

Constraints of this version#

NormativeBytes, test vectors and proofs

• Operands are position-based. The family (opens in a new tab) and opcode of an operation fix the meaning of each word and byte string, as operations (opens in a new tab) lists.
• Deployment, domains, principal, key identifier, authority epoch (opens in a new tab) and route scope are registered 64-bit handles. The core carries no address or chain name text.
• Amounts are 32-byte big-endian unsigned integers on the wire. The product accepts an amount only when its first 24 bytes are zero, so an amount is at most 2^64 - 1 base units. For a token with 18 decimal places this is about 18.4 whole tokens.
• SINGLE (opens in a new tab) with Ed25519 is the only implemented signature scheme. THRESHOLD (opens in a new tab) has a scheme code and no verifier, and the reference refuses it as INVALID_THRESHOLD_SIGNATURE.
• A state-changing request executes only when its signed cut (opens in a new tab) equals the current global revision and authority epoch. A write that commits between signing and execution makes the request Busy without effect, also when the two touch different assets.
• A recorded Busy or Unavailable consumes the call identifier (opens in a new tab). Executing the work again needs a new signed cut, a new call sequence number, a new transport UUID and a new signature.
• The reference profile signs its tokens, policy bindings, authorizations and source attestations as JSON text (JSON encodings (opens in a new tab)).
• The maximum frame size, nesting depth, clock skew allowed for issued times, asset-class TTL (opens in a new tab) floor and maximum delivery TTL are configuration values. This version fixes no numbers for them.
• Channels are set up out of band. The host verifies the signed offers and confirmation and installs the channel, and the reference accepts no HEARTBEAT (opens in a new tab) frame as a delivery.
• Delivery is single hop. No relay that changes processing is defined.
• The reference configuration accepts 25 of the 31 operations. Write, Timeout, authorized epoch installation, Capabilities, Liveness and the inbound correlated response receive UNSUPPORTED_VERSION.
• The reference implementation assumes a 64-bit host. It converts 64-bit wire values to the host's native size type.
• The Rust persistent authority implements the reference execution and configured delivery TTL limit. Its signed-Response generation and signature-checked verification APIs accept decoded Ordinary, Legal and Spend calls with caller-qualified call identifiers. QUERY (opens in a new tab) operations 1 through 8 use the typed read/receipt results of receive_wire.
• The Go package implements the wire, negotiation, transport admission (opens in a new tab) and operand decoding functions, including response operand correlation, but has no persistent authority or signature-checked response-verifier API.
• Ordinary request admission can refuse source registration, delivery-window or cancellation checks before envelope signature verification. A caller observing the local unsigned error can distinguish an unregistered source from an invalid signature for a registered source. The response-generation path verifies with the channel's authenticated peer key before signing a response.
• The configured version 5 compatibility record uses JSON decoding without the canonical protobuf version selector. It does not interoperate with the public v0.5 wire format.

Accepted operations#

The reference configuration's accepted set is one match in Rust and one switch in Go, and both accept the same 25 operations:

fn supported_frame(core: &Core) -> bool {    match core.body.as_ref() {        Some(Body::StateSync(op)) => matches!(op.opcode, Some(1..=6)),        Some(Body::RegulatoryAction(op)) => matches!(op.opcode, Some(1..=3)),        Some(Body::Ack(op)) => op.opcode == Some(1),        Some(Body::Query(op)) => matches!(op.opcode, Some(1..=8)),        Some(Body::LockManagement(op)) => matches!(op.opcode, Some(1..=6)),        Some(Body::Governance(op)) => op.opcode == Some(1),        _ => false,    }}

// ReferenceAccepts reports whether the reference configuration executes the// operation of a decoded Core. It accepts 25 of the 31 operations of version// 6 and refuses Write, Timeout, CorrelatedResponse, Capabilities, Liveness// and InstallEpoch.func ReferenceAccepts(core *wirepb.Core) bool {	family, op := Body(core)	opcode := op.GetOpcode()	switch family {	case 1, 3:		return opcode >= 1 && opcode <= 6	case 2:		return opcode >= 1 && opcode <= 3	case 4:		return opcode >= 1 && opcode <= 8	case 5, 7:		return opcode == 1	default:		return false	}}

Every entry of the persistent authority runs this check on the decoded frame before it checks the frame's source fields or signature, so a refused operation receives UNSUPPORTED_VERSION:

/// Refuse an operation outside the selected version profile of this/// configuration. The check reads only the decoded frame.fn check_operation<S: WireSecurity>(    security: &S,    channel: &Channel,    core: &Core,) -> Result<(), WireError> {    if !security.accepts_operation(channel, core) {        return Err(WireError::UnsupportedVersion);    }    Ok(())}

Host word size#

The crate does not compile for a target narrower than 64 bits:

// Wire revisions, commit positions and counts are 64-bit integers that the// reference decodes into usize. A narrower target would refuse large values// that a 64-bit target accepts, so the reference builds only for 64-bit// targets and keeps one verdict for every frame.#[cfg(not(target_pointer_width = "64"))]compile_error!(    "oip-spec requires a 64-bit target: wire revisions, commit positions and counts decode into usize");

Wire format conditions#

NormativeBytes, test vectors and proofs

Generated bindings#

scripts/generate-wire.py regenerates both bindings from proto/oip.proto with the pinned generator versions, and with --check it stops at the first generated file whose bytes differ:

def main():    parser = argparse.ArgumentParser()    parser.add_argument("--protoc", type=Path)    parser.add_argument("--go-plugin", type=Path)    parser.add_argument("--check", action="store_true")    args = parser.parse_args()    args.protoc = tool(args.protoc, "PROTOC", "protoc")    args.go_plugin = tool(args.go_plugin, "PROTOC_GEN_GO", "protoc-gen-go")    if run([args.protoc, "--version"]) != b"libprotoc 33.5":        raise SystemExit("protoc 33.5 required")    if run([args.go_plugin, "--version"]) not in (b"protoc-gen-go v1.36.8", b"protoc-gen-go.exe v1.36.8"):        raise SystemExit("protoc-gen-go 1.36.8 required")    (ROOT / "target").mkdir(exist_ok=True)    with tempfile.TemporaryDirectory(prefix="wire-generate-", dir=ROOT / "target") as temporary:        target = Path(temporary)        descriptor = target / "schema.pb"        run([args.protoc, f"--descriptor_set_out={descriptor}", SCHEMA])        run([args.protoc, f"--plugin=protoc-gen-go={args.go_plugin}", f"--go_out={target}",             f"--go_opt=module={GO_MODULE_ROOT}", SCHEMA])        files = {RUST: rust_file(descriptor.read_bytes()), GO: (target / GO).read_bytes()}        for relative, content in files.items():            path = ROOT / relative            if args.check:                if path.read_bytes() != content:                    raise SystemExit(f"generated bytes differ: {relative}")            else:                path.parent.mkdir(parents=True, exist_ok=True)                path.write_bytes(content)    print("Rust and Go generated bindings match the pinned schema.")

Shared vectors#

These rows of tests/vectors/wire-conformance.tsv pin the large-integer and deadline conditions. Each row gives an id, the function it exercises, its inputs, the expected observation and the evidence it is derived from:

large-fixed64-word	roundtrip	value=18446744073709551615;wire_type=fixed64	exact_value=18446744073709551615;length=8	get_le_octets_reads_every_representable_word,put_le_octets_has_exact_widthunsigned256-lossless-before-product-range-check	roundtrip	amount_hex=ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff;length=32;product_amount_max=18446744073709551615	wire_roundtrip=true;product_conversion_rejected=true	exact_32_octet_unsigned_amount_has_no_accepted_alias,native_envelope_matches_defthree-deadline-presence-states	roundtrip	cases=absent,indefinite,finite_0	distinct_bytes=true;distinct_semantics=true	exact_deadline_roundtrip,wellformed_wire_body_roundtrip

Reproducing the repository checks#

The repository retains the pinned schema, generated bindings, operation table and language conformance vectors. Its README gives the toolchain versions and the check commands, and CONTRIBUTING.md gives the commands for regenerating the bindings and the operation table.

CONTRIBUTING.md lists these development checks, with the arguments the hosted workflows use:

# Rust reference tests, then a list of the tests that need live servicesrustup toolchain install 1.96.1 --profile minimalcargo +1.96.1 test --lockedcargo +1.96.1 test --locked -- --list --ignored# Go tests, and protoc-gen-go v1.36.8 built from the module requirementscd gogo test -mod=readonly ./...go build -mod=readonly -o /path/to/protoc-gen-go google.golang.org/protobuf/cmd/protoc-gen-gocd ..# Generated bindings and the operation name view match their sourcespython3 scripts/generate-wire.py --check --protoc /path/to/protoc --go-plugin /path/to/protoc-gen-gopython3 scripts/render-operations.py --check# Links, vector checksums, workflow pins and public-text rulesnode .github/scripts/verify-repository-health.mjs

Run the Go package and cross-language corpus checks from the repository's documented environment. Compare the exact same bytes and refusal categories. Do not treat a skipped cross-language corpus as an independently reproduced comparison.

Vector runners#

Both runners load tests/vectors/wire-conformance.tsv, require all 25 rows and compare the observations with the row's expected column. The Go package has no persistent authority, so for the post-commit delivery row its runner checks only the nested receipt operand:

#[test]fn proof_derived_wire_vectors() {    let mut seen = BTreeSet::new();    for row in parse_vectors(VECTORS) {        assert!(seen.insert(row.id));        assert_eq!(observe_row(&row), row.expected, "{}", row.id);    }    assert_eq!(seen.len(), 25);}

func TestFrozenWireVectors(t *testing.T) {	rows := vectors(t)	if len(rows) != 25 {		t.Fatalf("unexpected vector count %d", len(rows))	}	for _, row := range rows {		t.Run(row.id, func(t *testing.T) {			var observed string			switch row.id {			case "canonical-query-core", "exact-wire-message-roundtrip", "large-fixed64-word", "unsigned256-lossless-before-product-range-check",				"three-deadline-presence-states", "overlong-tag-rejected-before-normalization", "duplicate-singular-core-field", "two-oneof-body-members",				"header-family-body-mismatch", "unknown-primary-enum", "unknown-secondary-retains-signed-original":				observed = canonicalVector(t, row)			case "invalid-signature-first-cannot-poison-valid-uuid", "current-source-key-and-epoch-required", "ttl-inclusive-expiry",				"ttl-expired-even-with-retained-uuid", "trusted-ttl-floor-and-checked-arithmetic", "same-uuid-changed-exact-signed-core", "complete-uuid-with-distinct-upper-octets":				observed = admissionVector(t, row)			case "explicit-authenticated-legacy-common-profile", "highest-common-no-authenticated-downgrade", "unauthenticated-offer-and-no-parse-fallback":				observed = negotiationVector(t, row)			case "precommit-cancel-does-not-bind-or-execute", "wire-query-consumes-signed-product-cut", "typed-ordinary-debit-operands":				observed = typedVector(t, row)			case "postcommit-delivery-loss-recovers-exact-result":				// This consumer validates the fresh nested receipt operand. The cross				// harness sends these bytes through the real Rust persistence consumer.				original := debitFixture()				raw, err := EncodeCore(original)				if err != nil {					t.Fatal(err)				}				receipt := coreFixture(0, 4, 3)				receipt.Body = &wirepb.Core_Query{Query: &wirepb.Operation{Opcode: proto.Uint64(8), Words: []uint64{2}, Blobs: [][]byte{raw}}}				packet, err := DecodePacket(signFixture(t, receipt), 1024*1024)				if err != nil {					t.Fatal(err)				}				typed, err := DecodeOperands(packet.Core(), 4)				if err != nil || !EqualCore(typed.Original, original) {					t.Fatal("nested original changed", err)				}				return			default:				t.Fatal(row.id)			}			if observed != row.expected {				t.Fatalf("observed %q, expected %q", observed, row.expected)			}		})	}}

Decode mutation corpus#

tests/interop/wire-decode-mutations.tsv holds one valid Core for each of the 31 operations and deterministic mutations of it. The Rust test regenerates every row from the valid Cores and requires the committed file, and the Go test requires the same verdict for every row:

#[test]fn reference_decode_corpus_is_reproducible_and_holds_its_verdicts() {    let rows = rows(CORPUS);    let bases: Vec<_> = rows.into_iter().filter(|row| row.id.ends_with("/valid")).collect();    assert_eq!(bases.len(), 31);    assert!(bases.iter().all(|base| base.expect == "OK"), "every valid Core decodes");    let rendered = render(&bases);    if std::env::var_os("OIP_WRITE_DECODE_CORPUS").is_some() {        let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/interop/wire-decode-mutations.tsv");        std::fs::write(path, &rendered).unwrap();        return;    }    assert_eq!(rendered, CORPUS.replace("\r\n", "\n"));    let counts = rendered.lines().skip(1).fold(std::collections::BTreeMap::new(), |mut counts, line| {        *counts.entry(line.rsplit('\t').next().unwrap().to_string()).or_insert(0) += 1;        counts    });    // The corpus must reach the operand decoder, not only the canonical one.    assert!(counts.get("INVALID_SOURCE").copied().unwrap_or(0) > 0, "{counts:?}");    assert!(counts.get("OK").copied().unwrap_or(0) > 31, "{counts:?}");}

// TestReferenceDecodeCorpus checks the verdict of every row of the decode// corpus that the Rust tests regenerate, including which error a Core with// several defects receives.func TestReferenceDecodeCorpus(t *testing.T) {	data, err := os.ReadFile(filepath.Join("..", "..", "tests", "interop", "wire-decode-mutations.tsv"))	if err != nil {		t.Fatal(err)	}	lines := strings.Split(strings.TrimSpace(strings.ReplaceAll(string(data), "\r\n", "\n")), "\n")	if lines[0] != "id\tnesting\tcore\texpect" {		t.Fatalf("corpus header %q", lines[0])	}	counts := map[string]int{}	for _, line := range lines[1:] {		columns := strings.Split(line, "\t")		if len(columns) != 4 {			t.Fatalf("corpus row %q", line)		}		nesting, err := strconv.Atoi(columns[1])		if err != nil {			t.Fatal(err)		}		raw, err := hex.DecodeString(columns[2])		if err != nil {			t.Fatal(err)		}		got := "OK"		core, err := DecodeCore(raw)		if err == nil {			_, err = DecodeOperands(core, nesting)		}		if err != nil {			got = err.Error()		}		if got != columns[3] {			t.Errorf("%s: got %s, want %s", columns[0], got, columns[3])		}		counts[got]++	}	if len(lines)-1 < 31*20 || counts["INVALID_SOURCE"] == 0 {		t.Fatalf("corpus too small or shallow: %v", counts)	}}

Operation conformance#

python3 scripts/check-operations.py builds frames from the operation table and seeded transport mutations, signs them and requires them to equal the committed tests/vectors/wire-operations.jsonl. It runs the Rust and Go receivers on the same frames and requires equal results:

def evaluate(rows, cargo, go):    input_path = OUT / "input.jsonl"    input_path.write_bytes(encode_rows(rows))    env = os.environ.copy()    env["OIP_WIRE_OPERATIONS_INPUT"] = str(input_path)    rust_path, go_path = OUT / "rust.jsonl", OUT / "go.jsonl"    env["OIP_WIRE_OPERATIONS_OUTPUT"] = str(rust_path)    run([cargo, "test", "--offline", "--locked", "--lib", RUST_TEST, "--", "--exact"], env=env)    env["OIP_WIRE_OPERATIONS_OUTPUT"] = str(go_path)    run([go, "test", "-count=1", "-mod=readonly", "./wire", "-run", "^" + GO_TEST + "$"], env=env, cwd=ROOT / "go")    rust, other = [list(map(json.loads, path.read_bytes().splitlines())) for path in (rust_path, go_path)]    ids=[row["id"] for row in rows]    definition=value_schema()    validate_results(rust,ids,definition);validate_results(other,ids,definition)    require_equal(rust,other)    return rust

Fixed cases must keep their stated verdicts, and unless a diagnostic option is given, the independently pinned executable model must return the same results:

def main():    parser = argparse.ArgumentParser(description=__doc__)    parser.add_argument("--write", action="store_true", help="deliberately regenerate committed test vectors")    parser.add_argument("--cargo")    parser.add_argument("--go")    parser.add_argument("--random-count", type=int, default=512)    parser.add_argument("--mirror-command", nargs="+", help="Diagnostic custom model command; not accepted by the required CI gate")    parser.add_argument("--require-mirror", action="store_true", help="Require the independently pinned default model")    parser.add_argument("--two-language-only", action="store_true", help="Diagnostic only; does not satisfy model conformance")    parser.add_argument("--model-artifact-root", help="Local verified model/driver/origin directory")    parser.add_argument("--polyml", help="Explicit standalone Poly/ML executable")    parser.add_argument("--model-heap-mb", type=int, default=1024)    args = parser.parse_args()    if args.two_language_only and (args.require_mirror or args.mirror_command):        raise SystemExit("Two-language diagnostics cannot satisfy or replace the required model gate")    if args.require_mirror and args.mirror_command:        raise SystemExit("The required gate uses only the independently pinned model loader")    if not args.two_language_only and not args.mirror_command:        args.require_mirror = True        args.mirror_command = [sys.executable, str(ROOT / "scripts/run-wire-model.py")]        if args.model_artifact_root:            args.mirror_command += ["--artifact-root", args.model_artifact_root]        if args.polyml:            args.mirror_command += ["--polyml", args.polyml]        args.mirror_command += ["--heap-mb", str(args.model_heap_mb)]    if args.random_count < 256:        raise SystemExit("at least 256 random transport cases are required")    OUT.mkdir(parents=True, exist_ok=True)    cargo, go = tool(args.cargo, "CARGO", "cargo"), tool(args.go, "GO", "go")    data, operations = table()    inputs, known = signed_rows(cases(bases(data, operations), args.random_count), cargo)    if args.write:        # The test modules compile in the newly generated corpus on their first build.        CORPUS.write_bytes(encode_rows(inputs))    else:        saved = list(map(json.loads, CORPUS.read_bytes().splitlines()))        if [{k: v for k, v in row.items() if k != "expected"} for row in saved] != inputs:            raise SystemExit("operation vector inputs differ; regenerate deliberately with --write")        inputs = saved    results = evaluate(inputs, cargo, go)    for row in results:        if row["id"] in known and row["verdict"] != known[row["id"]]:            raise SystemExit(dump({"id": row["id"], "expected_from_case": known[row["id"]], "actual": row}))    if args.mirror_command:        raw = run(args.mirror_command, data=encode_rows(inputs))        mirror = list(map(json.loads, raw.splitlines()))        validate_results(mirror,[row["id"] for row in inputs],value_schema())        require_equal(mirror,results)

The model runs on a pinned Poly/ML runtime for Ubuntu 24.04. The repository prepares that runtime and then requires the model:

python3 scripts/prepare-polyml-userspace.pytarget/polyml-userspace/bin/poly --versionpython3 scripts/check-operations.py --require-mirror \  --polyml target/polyml-userspace/bin/poly

Cross-language corpus#

python3 scripts/test-wire-interop.py generates the signed corpus with both implementations and stops when the bytes differ. It then runs the Go tests on the Rust corpus and the Rust persistent authority on the Go corpus:

def main():    parser = argparse.ArgumentParser()    parser.add_argument("--go")    parser.add_argument("--cargo")    args = parser.parse_args()    go_tool = tool(args.go, "GO", "go")    cargo = tool(args.cargo, "CARGO", "cargo")    OUT.mkdir(parents=True, exist_ok=True)    rust = normalized(run([cargo, "run", "--offline", "--locked", "--quiet", "--example", "wire_corpus"], ROOT, True))    go = normalized(run([go_tool, "run", "-mod=readonly", "./cmd/wire-corpus"], ROOT / "go", True))    if rust != go:        raise SystemExit("Rust/Go generated signed corpus differs")    rust_path, go_path = OUT / "rust.txt", OUT / "go.txt"    rust_path.write_bytes(rust)    go_path.write_bytes(go)    ENV["OIP_RUST_WIRE_CORPUS"] = str(rust_path)    ENV["OIP_GO_WIRE_CORPUS"] = str(go_path)    run([go_tool, "test", "-mod=readonly", "./..."], ROOT / "go")    run([cargo, "test", "--offline", "--locked", "--lib",         "authority::recovery::wire::vector_tests::go_generated_frames_execute_on_the_same_persistent_authority",         "--", "--exact"], ROOT)    print("Rust and Go exchanged the signed corpus.")

Without the corpus files that the script provides, the Go signed-corpus test skips itself and the Rust test runs on frames that it generates itself, so only the script exchanges frames between the two implementations:

#[test]fn go_generated_frames_execute_on_the_same_persistent_authority() {    let original = debit_core(2);    let receipt = core_with(4, 8, 0, 3, vec![2], vec![encode_core(&original).unwrap()]);    let corpus = if let Some(path) = std::env::var_os("OIP_GO_WIRE_CORPUS") {        std::fs::read_to_string(path).unwrap()    } else {        format!(            "query={}\ndebit={}\nreceipt={}\nlegacy_query={}\nlegacy_heartbeat={}\n",            hex(&signed(canonical())),            hex(&signed(original.clone())),            hex(&signed(receipt.clone())),            hex(&legacy_packet(4, 1, 5, vec![1, 0, 1, 17])),            hex(&legacy_packet(6, 2, 7, vec![3, 190])),        )    };

func TestRustSignedCorpus(t *testing.T) {	path := os.Getenv("OIP_RUST_WIRE_CORPUS")	if path == "" {		t.Skip("cross corpus is run by scripts/test-wire-interop.py")	}	raw, err := os.ReadFile(path)	if err != nil {		t.Fatal(err)	}

What verification establishes#

The formal verification artifacts (opens in a new tab) establish the stated state-machine and trace properties under their named assumptions. Their correspondence (opens in a new tab) to the selected reference source is described by the artifact mapping. Model proofs, executable vectors and the code correspondence each provide different evidence.

Cryptographic implementations, the compiler, operating system, authenticated storage and external legal/source adapters remain part of the stated trust boundary. A bounded randomized corpus does not prove all possible byte strings correct. The reference is a prototype, and this edition makes no throughput or production-security guarantee.

Design-only surfaces#

DesignCarried from v0.5, without bytes or test vectors in this version

Architecture-level concepts from v0.5 are marked separately in this edition. Multi-hop processing, a general condition DSL (opens in a new tab) and the complete OSS (opens in a new tab) deployment have no executable profile here. They are not required code paths hidden behind an otherwise successful conformance result.

Related Pages#

• Wire Format
• Validation
• Open Issues

Edition history#

• 0.6.0: Added this edition. The v0.5 specification remains available at its original addresses.