Merge pull request #82 from Lochnair/test/upstream-client-interop
test(server): validate upstream 0.9.1 client interop
This commit was merged in pull request #82.
This commit is contained in:
1 file changed
+137
@@ -0,0 +1,137 @@
|
||||
# Upstream IRTT 0.9.1 Client Interoperability
|
||||
|
||||
## Scope
|
||||
|
||||
The released upstream IRTT 0.9.1 client was run as a **black box** against the
|
||||
Tokio `irtt_server::Server` runtime. Only the executable's reported version, its
|
||||
command-line help, its exit status and its normal stdout/stderr were observed.
|
||||
Upstream implementation source, tests and repository history were **not**
|
||||
inspected, and no upstream behavior was inferred beyond what the running program
|
||||
printed.
|
||||
|
||||
- Tested against `irtt-rs` base commit `ace3defe22e71980f18ccdce1ef783d630e070cf`.
|
||||
No implementation change was required, so the validated behavior is that of the
|
||||
unmodified baseline.
|
||||
- Listeners were **explicit loopback binds only** (`127.0.0.1:0`, `[::1]:0`).
|
||||
Wildcard binds were not exercised.
|
||||
- DSCP marking, the full `ServerFill` policy and wildcard/multi-homed
|
||||
source-address (`pktinfo`) handling are unimplemented and out of scope here.
|
||||
- This document records implementation validation. It creates no protocol
|
||||
requirements and does not amend the airlocked protocol evidence.
|
||||
|
||||
The server was exposed through a small throwaway Cargo harness kept outside this
|
||||
repository. It only built a `ServerConfig`, called `Server::bind`, printed the
|
||||
bound `SocketAddr` and called `Server::run`; it contains no protocol handling and
|
||||
is deliberately not committed.
|
||||
|
||||
## Environment
|
||||
|
||||
| Item | Value |
|
||||
| --- | --- |
|
||||
| Date | 2026-08-13 |
|
||||
| OS | macOS 26.5.2 (Darwin 25.5.0) |
|
||||
| Architecture | arm64 (Apple silicon) |
|
||||
| Rust toolchain | rustc 1.97.1, cargo 1.97.1 |
|
||||
| Upstream reported version | `irtt version: 0.9.1`, protocol version 1, JSON format version 1 |
|
||||
| Upstream executable SHA-256 | `35df8b2b2c10fe81a222eaac2fa7461891fa2354f74c1e9b379c3c31f43c61cc` |
|
||||
| How obtained | Pre-existing installation already on `PATH`; version confirmed with `irtt version`. No isolated build was needed and no upstream source was fetched. |
|
||||
| `irtt-rs` base SHA | `ace3defe22e71980f18ccdce1ef783d630e070cf` |
|
||||
|
||||
Flags used were taken from the executable's own `irtt help client` output:
|
||||
`-d`, `-i`, `-l`, `-n`, `-4`, `-6`, `--hmac`, `--loose`, `--timeouts`,
|
||||
`--tstamp`, `--clock`.
|
||||
|
||||
`--loose` is upstream's documented client-side switch for accepting
|
||||
server-restricted test parameters instead of exiting non-zero. The
|
||||
**packet-length** restriction was run both ways: without it, to confirm the
|
||||
client detects and reports the restriction, and with it, to confirm the
|
||||
restricted session actually runs. The interval and maximum-duration
|
||||
restrictions were run with `--loose` only, so strict-mode refusal was observed
|
||||
for packet length alone and is not claimed for the other two.
|
||||
|
||||
## Matrix
|
||||
|
||||
Server policy is `ServerConfig::default()` unless stated. All runs used numeric
|
||||
loopback addresses and no DNS.
|
||||
|
||||
| Scenario | Server policy | Client request | Result | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| IPv4 ordinary | default | `-4 -d 1s -i 100ms -l 128` | **PASS** | 10/10 packets, 0.00% loss, exit 0 |
|
||||
| HMAC | `with_hmac_key("interop-secret")` | `--hmac=interop-secret -d 500ms -i 100ms -l 128` | **PASS** | 5/5 packets, 0.00% loss, exit 0 |
|
||||
| HMAC, wrong key (negative control) | `with_hmac_key("interop-secret")` | `--hmac=wrong-secret --timeouts=200ms,200ms` | **PASS** | Silently refused; client reports `[OpenTimeout] no reply from server`, exit 1; server unaffected |
|
||||
| IPv6 | default, bound `[::1]:0` | `-6 -d 1s -i 100ms -l 128` | **PASS** | 10/10 packets, 0.00% loss, exit 0 |
|
||||
| Packet-length restriction | `with_max_packet_length(96)` | `-l 128` (strict) | **PASS** | `[ServerRestriction] server reduced length from 128 to 96`, exit 1 — upstream's documented strict-mode policy |
|
||||
| Packet-length restriction | `with_max_packet_length(96)` | `-l 128 --loose` | **PASS** | Runs at 96 bytes, 10/10 packets, 0.00% loss, exit 0 |
|
||||
| Interval restriction | default (10 ms floor) | `-i 1ms -d 500ms --loose` | **PASS** | `server increased interval from 1ms to 10ms`; 50/50 packets, 0.00% loss — the rate limiter never penalised a client sending at the interval it was given |
|
||||
| Maximum-duration Close | `with_max_test_duration(500ms)` | `-d 5s -i 100ms --loose` | **PASS** (restriction) | `server reduced duration from 5s to 500ms`; 5/5 packets, exit 0. Close reply not reached on loopback; see Findings. |
|
||||
| Continuous request | `with_max_test_duration(500ms)` | `-d 0` | **SKIPPED** | Upstream refuses the argument (`[DurationNonPositive] duration (0s) must be > 0`), exiting non-zero without printing the connection line it prints in every other scenario. No packet capture was taken, so what the client emitted before exiting was not observed; continuous mode was simply not exercisable through this CLI |
|
||||
| No-test (optional) | default | `-n -l 128` | **PASS** | `[NoTest] skipping test at user request`, exit 0, no measurement session |
|
||||
| Midpoint, single clock (optional) | default | `--tstamp=midpoint --clock=monotonic` | **PASS** | 5/5 packets, exit 0; RTT and IPDV computed from the single midpoint field |
|
||||
| Midpoint, wall only (optional) | default | `--tstamp=midpoint --clock=wall` | **PASS** | 3/3 packets, exit 0 |
|
||||
| Midpoint, both clocks (optional) | default | `--tstamp=midpoint --clock=both` | **PASS** | 3/3 packets, exit 0 |
|
||||
|
||||
In every scenario the harness process was still alive after the client exited
|
||||
and shut down cleanly on `SIGTERM` (`Server::run` returned `Ok`).
|
||||
|
||||
## Findings
|
||||
|
||||
No implementation incompatibility was found in the tested matrix, and no code
|
||||
change was made.
|
||||
|
||||
Two observations are worth recording.
|
||||
|
||||
**The single-field midpoint reply is accepted.** `irtt-rs` emits one timestamp
|
||||
field per negotiated clock for `StampAt::Midpoint`, deliberately not reproducing
|
||||
upstream 0.9.1's dual-field midpoint form (see `crates/irtt-server/AGENTS.md`).
|
||||
The released upstream client parsed that reply without complaint and derived RTT
|
||||
and send/receive IPDV from it, in the monotonic-only, wall-only and both-clocks
|
||||
cases. This was the tested path most likely to expose a reply-shape
|
||||
incompatibility, and it did not.
|
||||
|
||||
**The maximum-duration Close was not reached in the tested loopback run.** The
|
||||
deadline is `first served echo + configured maximum + 2 s`, and negotiation
|
||||
reduced this session's Duration to the configured 500 ms maximum. On loopback the
|
||||
upstream client honoured that restriction, stopped transmitting after ~400 ms and
|
||||
exited 0; no echo arrived anywhere near the server's +2 s deadline, so the
|
||||
Close-flagged reply was never due and this black-box scenario did not exercise
|
||||
it.
|
||||
|
||||
That result is scoped to the tested path. Under ordinary low-delay conditions a
|
||||
client honouring its negotiated Duration will normally finish sending well before
|
||||
the +2 s boundary. It does not follow that the Close is unreachable in general:
|
||||
the deadline is evaluated on *arrival*, not on transmission, so an echo sent
|
||||
legitimately before the negotiated Duration elapsed could still be queued or
|
||||
delayed for more than two seconds on a real network path and arrive after
|
||||
`first served echo + configured maximum + 2 s`, carrying Close while the client
|
||||
is still draining replies. The loopback run establishes only that this did not
|
||||
occur under negligible delay. The Close therefore primarily provides a
|
||||
termination path for traffic continuing or arriving beyond the server's
|
||||
maximum-duration grace — an overrunning client, or sufficiently delayed in-flight
|
||||
echo traffic.
|
||||
|
||||
This is not a defect and no fix is authorised for it. The two-second grace is
|
||||
recorded in the airlocked evidence as the measured upstream margin and is a fixed
|
||||
internal constant; the implementation matches that evidence rather than violating
|
||||
it. The Close path itself remains covered deterministically in-repo by
|
||||
`crates/irtt-server/src/tests/lifecycle.rs`, notably
|
||||
`the_first_echo_past_the_maximum_duration_carries_close_and_ends_the_session`.
|
||||
|
||||
## Scope remaining
|
||||
|
||||
This validation does **not** establish:
|
||||
|
||||
- DSCP response marking;
|
||||
- full `ServerFill` behavior (echo payloads remain zero-filled);
|
||||
- wildcard or multi-homed source-address handling (`pktinfo`);
|
||||
- behavior under pathological or exhaustive parameter combinations;
|
||||
- behavior on operating systems other than the macOS/arm64 host recorded above;
|
||||
- behavior across a real network path, MTU limits or fragmentation.
|
||||
|
||||
No claim of universal compatibility is made.
|
||||
|
||||
This document is a record of the manual validation that was performed, not a
|
||||
reproduction harness. The server policy and client arguments used in each
|
||||
scenario are summarized in the matrix above, but the throwaway harness was
|
||||
deliberately discarded afterwards and is not committed, so the run is not
|
||||
reproducible from this artifact alone. No permanent test in this repository
|
||||
depends on the upstream executable, the Go toolchain or network access.
|
||||
Reference in new issue
Block a user