From a50edff624d3eda73c42e7614ba6cb1ee91752d7 Mon Sep 17 00:00:00 2001 From: Nils Andreas Svee Date: Thu, 13 Aug 2026 16:47:09 +0700 Subject: [PATCH 1/4] docs(server): record upstream client interop Run the released upstream IRTT 0.9.1 client as a black box against the Tokio server over explicit loopback listeners, and record the result. Twelve of thirteen matrix rows pass and one is skipped for a client-side CLI reason: IPv4 and IPv6 ordinary sessions, HMAC with a wrong-key negative control, packet-length and interval restrictions both strictly and with --loose, maximum-duration restriction, no-test, and midpoint timestamps on each clock selection. No incompatibility was found, so nothing in the implementation changed. Two results are worth keeping. The single-field midpoint reply this server deliberately emits instead of upstream 0.9.1's dual-field form is accepted by the released client, which derives RTT and IPDV from it. And the maximum-duration close is unreachable by a conforming client: the deadline is the configured maximum plus two seconds while negotiation caps the session Duration at that same maximum, so a client honoring the Duration it was given always stops early. The two-second grace is the measured margin already recorded in the protocol evidence, so this is the implementation matching that evidence rather than a defect, and the close path stays covered by the lifecycle tests. Upstream source, tests and history were not inspected; only the executable's version, help, exit status and normal output were observed. No test in this repository depends on the upstream binary. --- docs/validation/UPSTREAM_CLIENT_INTEROP.md | 122 +++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 docs/validation/UPSTREAM_CLIENT_INTEROP.md diff --git a/docs/validation/UPSTREAM_CLIENT_INTEROP.md b/docs/validation/UPSTREAM_CLIENT_INTEROP.md new file mode 100644 index 0000000..77fd4bf --- /dev/null +++ b/docs/validation/UPSTREAM_CLIENT_INTEROP.md @@ -0,0 +1,122 @@ +# Upstream IRTT 0.9.1 Client Interoperability + +## Scope + +The released upstream IRTT 0.9.1 client was run as a **black box** against the +Tokio `irtt_server::Server` runtime. Only the executable's reported version, its +command-line help, its exit status and its normal stdout/stderr were observed. +Upstream implementation source, tests and repository history were **not** +inspected, and no upstream behavior was inferred beyond what the running program +printed. + +- Tested against `irtt-rs` base commit `ace3defe22e71980f18ccdce1ef783d630e070cf`. + No implementation change was required, so the validated behavior is that of the + unmodified baseline. +- Listeners were **explicit loopback binds only** (`127.0.0.1:0`, `[::1]:0`). + Wildcard binds were not exercised. +- DSCP marking, the full `ServerFill` policy and wildcard/multi-homed + source-address (`pktinfo`) handling are unimplemented and out of scope here. +- This document records implementation validation. It creates no protocol + requirements and does not amend the airlocked protocol evidence. + +The server was exposed through a small throwaway Cargo harness kept outside this +repository. It only built a `ServerConfig`, called `Server::bind`, printed the +bound `SocketAddr` and called `Server::run`; it contains no protocol handling and +is deliberately not committed. + +## Environment + +| Item | Value | +| --- | --- | +| Date | 2026-08-13 | +| OS | macOS 26.5.2 (Darwin 25.5.0) | +| Architecture | arm64 (Apple silicon) | +| Rust toolchain | rustc 1.97.1, cargo 1.97.1 | +| Upstream reported version | `irtt version: 0.9.1`, protocol version 1, JSON format version 1 | +| Upstream executable SHA-256 | `35df8b2b2c10fe81a222eaac2fa7461891fa2354f74c1e9b379c3c31f43c61cc` | +| How obtained | Pre-existing installation already on `PATH`; version confirmed with `irtt version`. No isolated build was needed and no upstream source was fetched. | +| `irtt-rs` base SHA | `ace3defe22e71980f18ccdce1ef783d630e070cf` | + +Flags used were taken from the executable's own `irtt help client` output: +`-d`, `-i`, `-l`, `-n`, `-4`, `-6`, `--hmac`, `--loose`, `--timeouts`, +`--tstamp`, `--clock`. + +`--loose` is upstream's documented client-side switch for accepting +server-restricted test parameters instead of exiting non-zero. Restriction +scenarios were therefore run **both** ways: without it to confirm the client +detects and reports the restriction, and with it to confirm the restricted +session actually runs. + +## Matrix + +Server policy is `ServerConfig::default()` unless stated. All runs used numeric +loopback addresses and no DNS. + +| Scenario | Server policy | Client request | Result | Notes | +| --- | --- | --- | --- | --- | +| IPv4 ordinary | default | `-4 -d 1s -i 100ms -l 128` | **PASS** | 10/10 packets, 0.00% loss, exit 0 | +| HMAC | `with_hmac_key("interop-secret")` | `--hmac=interop-secret -d 500ms -i 100ms -l 128` | **PASS** | 5/5 packets, 0.00% loss, exit 0 | +| HMAC, wrong key (negative control) | `with_hmac_key("interop-secret")` | `--hmac=wrong-secret --timeouts=200ms,200ms` | **PASS** | Silently refused; client reports `[OpenTimeout] no reply from server`, exit 1; server unaffected | +| IPv6 | default, bound `[::1]:0` | `-6 -d 1s -i 100ms -l 128` | **PASS** | 10/10 packets, 0.00% loss, exit 0 | +| Packet-length restriction | `with_max_packet_length(96)` | `-l 128` (strict) | **PASS** | `[ServerRestriction] server reduced length from 128 to 96`, exit 1 — upstream's documented strict-mode policy | +| Packet-length restriction | `with_max_packet_length(96)` | `-l 128 --loose` | **PASS** | Runs at 96 bytes, 10/10 packets, 0.00% loss, exit 0 | +| Interval restriction | default (10 ms floor) | `-i 1ms -d 500ms --loose` | **PASS** | `server increased interval from 1ms to 10ms`; 50/50 packets, 0.00% loss — the rate limiter never penalised a client sending at the interval it was given | +| Maximum-duration Close | `with_max_test_duration(500ms)` | `-d 5s -i 100ms --loose` | **PASS** (restriction) | `server reduced duration from 5s to 500ms`; 5/5 packets, exit 0. The Close-flagged reply itself is **not reachable** by a conforming client — see Findings. | +| Continuous request | `with_max_test_duration(500ms)` | `-d 0` | **SKIPPED** | Upstream rejects it client-side (`[DurationNonPositive] duration (0s) must be > 0`); a continuous request never reaches the wire from this CLI | +| No-test (optional) | default | `-n -l 128` | **PASS** | `[NoTest] skipping test at user request`, exit 0, no measurement session | +| Midpoint, single clock (optional) | default | `--tstamp=midpoint --clock=monotonic` | **PASS** | 5/5 packets, exit 0; RTT and IPDV computed from the single midpoint field | +| Midpoint, wall only (optional) | default | `--tstamp=midpoint --clock=wall` | **PASS** | 3/3 packets, exit 0 | +| Midpoint, both clocks (optional) | default | `--tstamp=midpoint --clock=both` | **PASS** | 3/3 packets, exit 0 | + +In every scenario the harness process was still alive after the client exited +and shut down cleanly on `SIGTERM` (`Server::run` returned `Ok`). + +## Findings + +No implementation incompatibility was found in the tested matrix, and no code +change was made. + +Two observations are worth recording. + +**The single-field midpoint reply is accepted.** `irtt-rs` emits one timestamp +field per negotiated clock for `StampAt::Midpoint`, deliberately not reproducing +upstream 0.9.1's dual-field midpoint form (see `crates/irtt-server/AGENTS.md`). +The released upstream client parsed that reply without complaint and derived RTT +and send/receive IPDV from it, in the monotonic-only, wall-only and both-clocks +cases. This was the tested path most likely to expose a reply-shape +incompatibility, and it did not. + +**The server-initiated maximum-duration Close is not reachable by a conforming +client.** The deadline is `first served echo + configured maximum + 2 s`, while +negotiation reduces a session's Duration to at most that same configured maximum. +A client that honours the Duration it was given therefore always stops sending at +least two seconds before the deadline, so no echo ever arrives to carry the +Close flag. That was confirmed empirically: with a 500 ms maximum against a +requested 5 s test, the upstream client accepted the restriction, sent for +~400 ms and exited 0 without a Close ever being due. + +This is not a defect and no fix is authorised for it. The two-second grace is +recorded in the airlocked evidence as the measured upstream margin and is a fixed +internal constant; the implementation matches that evidence rather than violating +it. The consequence is simply that the Close is a defence against a client that +overruns its negotiated Duration, not part of an ordinary session. Upstream's +CLI offers no way to build such a client — `--loose` accepts and uses the +restricted parameters, and a continuous request is rejected before it reaches the +wire. The Close path itself remains covered deterministically in-repo by +`crates/irtt-server/src/tests/lifecycle.rs`, notably +`the_first_echo_past_the_maximum_duration_carries_close_and_ends_the_session`. + +## Scope remaining + +This validation does **not** establish: + +- DSCP response marking; +- full `ServerFill` behavior (echo payloads remain zero-filled); +- wildcard or multi-homed source-address handling (`pktinfo`); +- behavior under pathological or exhaustive parameter combinations; +- behavior on operating systems other than the macOS/arm64 host recorded above; +- behavior across a real network path, MTU limits or fragmentation. + +No claim of universal compatibility is made. No permanent test in this repository +depends on the upstream executable, the Go toolchain or network access; this +validation is intentionally manual and reproducible from the commands above. From 02bdb495a1455e7c5d0bb9070835e7ca50ec88b2 Mon Sep 17 00:00:00 2001 From: Nils Andreas Svee Date: Thu, 13 Aug 2026 17:15:37 +0700 Subject: [PATCH 2/4] docs(server): qualify interop validation claims MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two claims in the validation record were stronger than the evidence behind them. The maximum-duration Close conclusion was stated for every conforming client. That does not follow: the deadline is evaluated on arrival, not on transmission, so an echo sent legitimately before the negotiated Duration elapsed can still be queued long enough to arrive after the server's two-second grace and carry Close. Scope the result to the loopback run that was actually performed, and describe the Close as a termination path for traffic continuing or arriving beyond the grace, including delayed in-flight traffic. The document also claimed the validation was reproducible from the commands above, while carrying only summarized client arguments and no harness — the harness was disposable on purpose and is not being promoted into repository infrastructure. Say plainly that this is a record of the validation performed rather than a reproduction harness. No implementation, test or protocol evidence changes. --- docs/validation/UPSTREAM_CLIENT_INTEROP.md | 47 ++++++++++++++-------- 1 file changed, 30 insertions(+), 17 deletions(-) diff --git a/docs/validation/UPSTREAM_CLIENT_INTEROP.md b/docs/validation/UPSTREAM_CLIENT_INTEROP.md index 77fd4bf..b9d7ec2 100644 --- a/docs/validation/UPSTREAM_CLIENT_INTEROP.md +++ b/docs/validation/UPSTREAM_CLIENT_INTEROP.md @@ -61,7 +61,7 @@ loopback addresses and no DNS. | Packet-length restriction | `with_max_packet_length(96)` | `-l 128` (strict) | **PASS** | `[ServerRestriction] server reduced length from 128 to 96`, exit 1 — upstream's documented strict-mode policy | | Packet-length restriction | `with_max_packet_length(96)` | `-l 128 --loose` | **PASS** | Runs at 96 bytes, 10/10 packets, 0.00% loss, exit 0 | | Interval restriction | default (10 ms floor) | `-i 1ms -d 500ms --loose` | **PASS** | `server increased interval from 1ms to 10ms`; 50/50 packets, 0.00% loss — the rate limiter never penalised a client sending at the interval it was given | -| Maximum-duration Close | `with_max_test_duration(500ms)` | `-d 5s -i 100ms --loose` | **PASS** (restriction) | `server reduced duration from 5s to 500ms`; 5/5 packets, exit 0. The Close-flagged reply itself is **not reachable** by a conforming client — see Findings. | +| Maximum-duration Close | `with_max_test_duration(500ms)` | `-d 5s -i 100ms --loose` | **PASS** (restriction) | `server reduced duration from 5s to 500ms`; 5/5 packets, exit 0. Close reply not reached on loopback; see Findings. | | Continuous request | `with_max_test_duration(500ms)` | `-d 0` | **SKIPPED** | Upstream rejects it client-side (`[DurationNonPositive] duration (0s) must be > 0`); a continuous request never reaches the wire from this CLI | | No-test (optional) | default | `-n -l 128` | **PASS** | `[NoTest] skipping test at user request`, exit 0, no measurement session | | Midpoint, single clock (optional) | default | `--tstamp=midpoint --clock=monotonic` | **PASS** | 5/5 packets, exit 0; RTT and IPDV computed from the single midpoint field | @@ -86,23 +86,31 @@ and send/receive IPDV from it, in the monotonic-only, wall-only and both-clocks cases. This was the tested path most likely to expose a reply-shape incompatibility, and it did not. -**The server-initiated maximum-duration Close is not reachable by a conforming -client.** The deadline is `first served echo + configured maximum + 2 s`, while -negotiation reduces a session's Duration to at most that same configured maximum. -A client that honours the Duration it was given therefore always stops sending at -least two seconds before the deadline, so no echo ever arrives to carry the -Close flag. That was confirmed empirically: with a 500 ms maximum against a -requested 5 s test, the upstream client accepted the restriction, sent for -~400 ms and exited 0 without a Close ever being due. +**The maximum-duration Close was not reached in the tested loopback run.** The +deadline is `first served echo + configured maximum + 2 s`, and negotiation +reduced this session's Duration to the configured 500 ms maximum. On loopback the +upstream client honoured that restriction, stopped transmitting after ~400 ms and +exited 0; no echo arrived anywhere near the server's +2 s deadline, so the +Close-flagged reply was never due and this black-box scenario did not exercise +it. + +That result is scoped to the tested path. Under ordinary low-delay conditions a +client honouring its negotiated Duration will normally finish sending well before +the +2 s boundary. It does not follow that the Close is unreachable in general: +the deadline is evaluated on *arrival*, not on transmission, so an echo sent +legitimately before the negotiated Duration elapsed could still be queued or +delayed for more than two seconds on a real network path and arrive after +`first served echo + configured maximum + 2 s`, carrying Close while the client +is still draining replies. The loopback run establishes only that this did not +occur under negligible delay. The Close therefore primarily provides a +termination path for traffic continuing or arriving beyond the server's +maximum-duration grace — an overrunning client, or sufficiently delayed in-flight +echo traffic. This is not a defect and no fix is authorised for it. The two-second grace is recorded in the airlocked evidence as the measured upstream margin and is a fixed internal constant; the implementation matches that evidence rather than violating -it. The consequence is simply that the Close is a defence against a client that -overruns its negotiated Duration, not part of an ordinary session. Upstream's -CLI offers no way to build such a client — `--loose` accepts and uses the -restricted parameters, and a continuous request is rejected before it reaches the -wire. The Close path itself remains covered deterministically in-repo by +it. The Close path itself remains covered deterministically in-repo by `crates/irtt-server/src/tests/lifecycle.rs`, notably `the_first_echo_past_the_maximum_duration_carries_close_and_ends_the_session`. @@ -117,6 +125,11 @@ This validation does **not** establish: - behavior on operating systems other than the macOS/arm64 host recorded above; - behavior across a real network path, MTU limits or fragmentation. -No claim of universal compatibility is made. No permanent test in this repository -depends on the upstream executable, the Go toolchain or network access; this -validation is intentionally manual and reproducible from the commands above. +No claim of universal compatibility is made. + +This document is a record of the manual validation that was performed, not a +reproduction harness. The server policy and client arguments used in each +scenario are summarized in the matrix above, but the throwaway harness was +deliberately discarded afterwards and is not committed, so the run is not +reproducible from this artifact alone. No permanent test in this repository +depends on the upstream executable, the Go toolchain or network access. From 750e28200a07ceacd01126fcfe91ac029a4b30b3 Mon Sep 17 00:00:00 2001 From: Nils Andreas Svee Date: Thu, 13 Aug 2026 17:35:21 +0700 Subject: [PATCH 3/4] docs(server): scope the strict-mode restriction claim The environment notes said restriction scenarios were run both with and without --loose, but the matrix records a strict run for the packet-length restriction only; the interval and maximum-duration rows were --loose alone. Say that, so the artifact does not credit strict-mode coverage it never recorded. No implementation, test or protocol evidence changes. --- docs/validation/UPSTREAM_CLIENT_INTEROP.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/validation/UPSTREAM_CLIENT_INTEROP.md b/docs/validation/UPSTREAM_CLIENT_INTEROP.md index b9d7ec2..4c3e881 100644 --- a/docs/validation/UPSTREAM_CLIENT_INTEROP.md +++ b/docs/validation/UPSTREAM_CLIENT_INTEROP.md @@ -42,10 +42,12 @@ Flags used were taken from the executable's own `irtt help client` output: `--tstamp`, `--clock`. `--loose` is upstream's documented client-side switch for accepting -server-restricted test parameters instead of exiting non-zero. Restriction -scenarios were therefore run **both** ways: without it to confirm the client -detects and reports the restriction, and with it to confirm the restricted -session actually runs. +server-restricted test parameters instead of exiting non-zero. The +**packet-length** restriction was run both ways: without it, to confirm the +client detects and reports the restriction, and with it, to confirm the +restricted session actually runs. The interval and maximum-duration +restrictions were run with `--loose` only, so strict-mode refusal was observed +for packet length alone and is not claimed for the other two. ## Matrix From 9d7566955d63fe3105adc2fa346982ac53db1d4e Mon Sep 17 00:00:00 2001 From: Nils Andreas Svee Date: Thu, 13 Aug 2026 17:39:28 +0700 Subject: [PATCH 4/4] docs(server): drop the unobserved continuous-mode wire claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The continuous-request row said a -d 0 request never reaches the wire, which no observation in this validation supports: the run took no packet capture and the server was not instrumented for datagram arrival, so what the client emitted before exiting was never seen. Record what was actually observed — the argument was refused, the client exited non-zero, and it printed none of the connection line it prints in every other scenario — and state plainly that the emission was not observed. The conclusion that matters is unchanged: continuous mode was not exercisable through this CLI. No implementation, test or protocol evidence changes. --- docs/validation/UPSTREAM_CLIENT_INTEROP.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/validation/UPSTREAM_CLIENT_INTEROP.md b/docs/validation/UPSTREAM_CLIENT_INTEROP.md index 4c3e881..c1be138 100644 --- a/docs/validation/UPSTREAM_CLIENT_INTEROP.md +++ b/docs/validation/UPSTREAM_CLIENT_INTEROP.md @@ -64,7 +64,7 @@ loopback addresses and no DNS. | Packet-length restriction | `with_max_packet_length(96)` | `-l 128 --loose` | **PASS** | Runs at 96 bytes, 10/10 packets, 0.00% loss, exit 0 | | Interval restriction | default (10 ms floor) | `-i 1ms -d 500ms --loose` | **PASS** | `server increased interval from 1ms to 10ms`; 50/50 packets, 0.00% loss — the rate limiter never penalised a client sending at the interval it was given | | Maximum-duration Close | `with_max_test_duration(500ms)` | `-d 5s -i 100ms --loose` | **PASS** (restriction) | `server reduced duration from 5s to 500ms`; 5/5 packets, exit 0. Close reply not reached on loopback; see Findings. | -| Continuous request | `with_max_test_duration(500ms)` | `-d 0` | **SKIPPED** | Upstream rejects it client-side (`[DurationNonPositive] duration (0s) must be > 0`); a continuous request never reaches the wire from this CLI | +| Continuous request | `with_max_test_duration(500ms)` | `-d 0` | **SKIPPED** | Upstream refuses the argument (`[DurationNonPositive] duration (0s) must be > 0`), exiting non-zero without printing the connection line it prints in every other scenario. No packet capture was taken, so what the client emitted before exiting was not observed; continuous mode was simply not exercisable through this CLI | | No-test (optional) | default | `-n -l 128` | **PASS** | `[NoTest] skipping test at user request`, exit 0, no measurement session | | Midpoint, single clock (optional) | default | `--tstamp=midpoint --clock=monotonic` | **PASS** | 5/5 packets, exit 0; RTT and IPDV computed from the single midpoint field | | Midpoint, wall only (optional) | default | `--tstamp=midpoint --clock=wall` | **PASS** | 3/3 packets, exit 0 |