GitHub

ADR-0041: Gate B End-to-End Standalone Measurement

Status: accepted and implemented as the Gate B measurement freeze; policy 1.3 local observation is about 5.87x and rejected; no claim accepted

Date: 2026-07-30

Requires: finite-accepted R1-S8 and ADR-0040

Normative contract: gate_b_standalone_measurement_contract.md

Context

R1-S8 proves that the generated BranchMix residual runs as a direct, interpreter-free ELF process. Gate B additionally requires that this same standalone path is no slower than 2.0x a hand-specialized implementation.

That ratio is meaningless until both sides have the same process shape and the timing boundary is fixed. Comparing a raw function call on one side with process creation on the other is unfair. Conversely, timing only the generated kernel would hide the startup, protocol, mapping, and teardown costs that the P1 lighthouse explicitly requires Gate B to report.

Decision

1. Structurally matched baseline

The primary Gate B baseline is a separately identified, hand-specialized Linux x86-64/SSE2 target body packaged with:

  • the exact BranchMix R1-S8 startup bytes;
  • the exact R1-S8 input/output protocol;
  • the same direct ET_EXEC ELF grammar and fixed base;
  • the same one-input-mapping RW→R→unmapped lifecycle;
  • the same one-process-per-invocation host harness;
  • no libc, dynamic loader, system linker, interpreter, JIT, or worker.

The baseline target is not derived from the generated R1-S7a target and cannot mint NAUX artifact authority. It has a separate schema, policy, domains, byte identity, verifier, and semantic correspondence test.

The frozen v1 hand target is a 158-byte, syscall-free target body. It saves and restores MXCSR around strict SSE2 arithmetic, retains a verifier-owned full byte oracle, and must pass all 46 BranchMix cases as fresh direct processes. The five Gate A Bounds cases belong to a different workload/profile and are not misreported as baseline coverage.

A C/Rust comparison row may remain supporting evidence, but it is not the primary 2.0x denominator unless it implements this exact process and protocol boundary.

2. Claim workload

The first fixed Gate B observation uses:

profile                 BranchMixF64R1
array elements          65,536
repetitions             64
kernel element-visits   4,194,304
input generated         deterministically before timing
input frame encoded     before timing

The input generator and exact frame hash are part of the evidence. Neither engine may embed or precompute the dynamic input or result.

3. Timing boundary

Executable construction, authority replay, verification, and temporary-file materialization are compile/deployment costs. They are recorded separately and excluded from the primary runtime sample.

Each end-to-end sample begins immediately before spawn and ends only after:

  1. process creation and kernel ELF loading;
  2. complete canonical input write and stdin close;
  3. startup parse, allocation, endian conversion, read-only protection;
  4. target execution;
  5. unmap and complete canonical stdout write;
  6. exact stdout/stderr capture;
  7. normal child reap and process-group containment completion.

Host-side output decoding and statistical aggregation occur after the end timestamp. Every output is still decoded and checked before the sample is admitted.

The primary metric is end-to-end ns/invocation. Diagnostic stage timestamps may report spawn, input, and completion portions, but cannot replace the total.

4. Sampling

The v1 policy is exact:

warmup invocations per engine   5
measured samples per engine    30
sample dropping                 none
schedule                        paired alternating order
timeout per process             30,000 ms
primary statistic               median
tail statistic                  p95
maximum admitted CV             5%
Gate B threshold                NAUX median <= 2 * baseline median

For even pairs NAUX runs first; for odd pairs the baseline runs first. Failure of either member rejects the pair and the complete claim. There is no retry, outlier deletion, best-of selection, or substitution.

5. Host and build admission

A claim-bearing run requires:

  • release build with overflow checks and the frozen target policies;
  • Linux x86-64 with SSE2;
  • one explicitly pinned logical CPU;
  • no network dependency;
  • recorded repository revision and dirty flag;
  • CPU model, topology, memory, OS/kernel, compiler, target, and SIMD fingerprint;
  • recorded executable, target, input, output, and measurement-policy hashes;
  • all R1-S8 correctness gates green at the same source revision.

An unpinned, debug, noisy (CV > 5%), incomplete, or dirty run may be retained as a local observation but cannot close Gate B.

6. Statistics

Samples are unsigned integer nanoseconds. Sorting is deterministic. For 30 samples, the median retains the exact numerator sorted[14] + sorted[15] with denominator two, and p95 is position 28 under zero-based indexing.

The verifier recomputes all statistics from raw samples. CV admission uses an exact checked integer comparison equivalent to population standard_deviation / mean <= 0.05; printed floating-point summaries are non-authoritative.

7. Non-claims

Passing this Gate B measurement does not establish Gate C, the mature 1.25x target, the P1-within-10% target, another workload, another machine, cross-platform performance, C/C++ leadership, P1 completion, or Nauxogenesis.

Consequences

  • Process and I/O overhead are paid symmetrically and cannot be hidden.
  • The primary baseline is harder to beat than a runtime-heavy C executable and remains independent of external compiler defaults.
  • A failed local ratio is useful engineering evidence, not permission to change the benchmark after seeing the result.
  • Any change to workload, timing, schedule, sample count, CV, baseline bytes, or threshold requires a new ADR and new domains.