test(server): validate upstream 0.9.1 client interop #82
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: Lochnair-Labs/irtt-rs#82
Reference in new issue
Block a user
Summary
Ran the released upstream IRTT 0.9.1 client as a black box against the real
Tokio
irtt_server::Serverover explicit loopback listeners, and recorded theresult in
docs/validation/UPSTREAM_CLIENT_INTEROP.md.12 of 13 matrix rows pass; 1 is skipped for a client-side CLI reason. No
implementation incompatibility was found, so no code fix is included — the
only committed file is the validation document.
At a glance: IPv4 ordinary PASS, HMAC PASS (plus a wrong-key negative
control PASS), IPv6 loopback PASS, packet-length restriction PASS
(both strict and
--loose), interval restriction / rate limiter PASS,maximum-duration restriction PASS, no-test PASS, midpoint timestamps
PASS (monotonic-only, wall-only, both clocks). Continuous request
SKIPPED — upstream's CLI refuses
-d 0, so continuous mode was notexercisable through it (no capture was taken, so what the client emitted before
exiting was not observed).
Two findings worth reading in the doc:
irtt-rsdeliberately emits (ratherthan upstream 0.9.1's dual-field form) is accepted by the released client,
which derives RTT and IPDV from it correctly. This was the likeliest
reply-shape incompatibility and it did not materialise.
low-delay loopback run: the upstream client honoured the negotiated Duration
and stopped well before the
+2 sboundary, so no echo arrived to carry Close.This is not a universal claim about every conforming-client/network-path
combination — the deadline is evaluated on arrival, so sufficiently delayed
in-flight traffic could still reach that boundary. Not a defect either: the
two-second grace is the measured margin recorded in the airlocked evidence, so
no change was made, and the Close path stays covered deterministically by the
in-repo tests in
crates/irtt-server/src/tests/lifecycle.rs.Validation boundary
irtt help clientoutput, its exitstatus and its normal stdout/stderr were observed.
there was no module cache to avoid reading in the first place.
docs/protocol/**,docs/clean-room/**) wasnot changed.
that only builds a
ServerConfig, binds, prints the address and runs. It isnot committed, and no CLI server applet was added.
Tested
--looseThe tested client arguments/scenarios, the upstream executable's SHA-256 and
reported version, and the host/toolchain details are recorded in the validation
document.
Remaining scope
Not established by this validation: DSCP response marking; the full
ServerFillpolicy (echo payloads remain zero-filled); wildcard/multi-homed source-address
handling (
pktinfo); exhaustive or pathological parameter combinations; anyoperating system other than the macOS/arm64 host recorded in the document; and
behavior across a real network path. No claim of universal compatibility is made.
Verification
cargo fmt --all -- --checkcargo check --workspace --all-targets --all-features --lockedcargo test --workspace --all-targets --all-features --lockedcargo clippy --workspace --all-targets --all-features --locked -- -D warningsRUSTDOCFLAGS="-D warnings" cargo doc -p irtt-proto -p irtt-client -p irtt-server --all-features --no-depswith every harness process confirmed alive after its client exited and shut
down cleanly afterwards.
No permanent test in this repository depends on the upstream executable, the Go
toolchain or network access.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:
a50edff624ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityThe document claims the validation is reproducible from commands above, but it contains only partial client argument lists: there are no complete
irtt clientinvocations or targets, and the external server harness needed to select configurations, report its ephemeral port, handle shutdown, and runServer::runis deliberately omitted. Anyone attempting to rerun or audit this validation therefore cannot reproduce the recorded matrix from the committed artifact; include the harness and complete commands (or remove the reproducibility claim).Useful? React with 👍 / 👎.
Stopping transmission before the server deadline does not guarantee that no echo arrives after it: on a real or sufficiently queued network path, a request sent within the negotiated Duration can remain in flight for over two seconds and reach the server after
first served echo + maximum + 2 s, at which point it can carry Close while the client is draining replies. The loopback run establishes only that this path was unreachable under negligible delay, so qualify this conclusion rather than stating it for every conforming client.Useful? React with 👍 / 👎.
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityGood catch — fixed in
02bdb495a1by removing the unsupported reproducibility/exact-command claim rather than by adding the missing material.The closing sentence now says this document is a record of the manual validation that was performed, not a reproduction harness: the server policy and client arguments are summarized in the matrix, but the throwaway harness was discarded afterwards and is not committed, so the run is not reproducible from this artifact alone. The disposable harness is deliberately not being promoted into repository infrastructure.
The PR body was corrected the same way ("Exact commands ..." → "The tested client arguments/scenarios ... are recorded in the validation document").
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityCorrect, and the original reasoning was wrong in exactly the way you describe — it argued from transmission time when the deadline is evaluated on arrival. Fixed in
02bdb495a1.The finding is now scoped to the tested low-delay loopback path: negotiation reduced Duration to 500 ms, the client stopped transmitting after ~400 ms, no echo arrived near the +2 s deadline, so the scenario did not exercise the Close. The document now states explicitly that this does not follow for every conforming client — an echo sent legitimately before the negotiated Duration elapsed could still be queued or delayed for more than two seconds and arrive after
first served echo + configured maximum + 2 s, carrying Close while the client drains replies. The conclusion is reframed as the Close providing a termination path for traffic continuing or arriving beyond the grace, including delayed in-flight traffic, and the matrix row now reads "Close reply not reached on loopback".Unchanged: this is not an implementation defect, the +2 s grace matches the airlocked measured behavior, no code fix is justified, and the lifecycle tests still cover the Close path. The PR body carries the same qualification.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:
02bdb495a1ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityThe interval-restriction and maximum-duration rows are recorded only with
--loose, yet this paragraph says restriction scenarios were run both ways. As written, this clean-room validation artifact credits strict-mode coverage that the matrix does not document; either qualify the statement as applying only to packet-length restriction or add the missing strict runs and outcomes.AGENTS.md reference: AGENTS.md:L12-L14
Useful? React with 👍 / 👎.
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityQualified to packet-length restriction only in
750e28200a— the interval and maximum-duration rows are now stated as--looseonly, rather than adding strict runs (that would mean recreating the discarded harness and rerunning the upstream matrix, which is out of scope for this artifact).🤖 Addressed by Claude Code
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:
750e28200aℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityThe recorded observations establish that the client printed
DurationNonPositive, but the stated methodology includes neither a packet capture nor server-side datagram instrumentation, and explicitly says no upstream behavior was inferred beyond program output. Consequently, this skipped run does not establish that the client emitted no datagram before reporting the error; qualify the wire-level conclusion or support it with a black-box packet observation.AGENTS.md reference: AGENTS.md:L12-L14
Useful? React with 👍 / 👎.
@@ -0,0 +1,137 @@# Upstream IRTT 0.9.1 Client InteroperabilityDropped the wire-level claim in
9d7566955d— the row now records only what was observed (argument refused, non-zero exit, none of the connection line printed in every other scenario), states explicitly that no packet capture was taken so the emission was not observed, and keeps the supportable conclusion that continuous mode was not exercisable through this CLI; the PR body carries the same qualification.🤖 Addressed by Claude Code