diff --git a/docs/clean-room/CLEANROOM_NOTES.md b/docs/clean-room/CLEANROOM_NOTES.md index d99f407..944041c 100644 --- a/docs/clean-room/CLEANROOM_NOTES.md +++ b/docs/clean-room/CLEANROOM_NOTES.md @@ -2,10 +2,41 @@ ## Purpose -This document records the clean-room boundary for the IRTT client -reimplementation project. The goal is to ensure that the implementation -is developed from a behavioral/protocol specification only, without exposure -to the GPL-licensed upstream source code. +This document records the clean-room boundary for the IRTT reimplementation +project. The goal is to ensure that the implementation is developed from a +behavioral/protocol specification only, without exposure to the GPL-licensed +upstream source code. + +It covers three passes and a final sanitization: + +- the **client** pass (2026-04-28), producing + `../protocol/IRTT_CLIENT_PROTOCOL_SPEC.md`; +- the **server** pass (2026-08-11), producing + `../protocol/IRTT_SERVER_PROTOCOL_SPEC.md` and its behavioral vectors, and + applying corrections back to the client specification; +- the **midpoint timestamp follow-up** (2026-08-11); +- the **final sanitization pass** (2026-08-11), recorded at the end of this file. + +## How to Read This File + +This is a provenance record. It states *what class of material was excluded* and +*what discipline was applied*; it deliberately does **not** restate the excluded +material. A note that says "an upstream identifier was removed" and then quotes +the identifier has removed nothing. Earlier revisions of this file made exactly +that mistake, and the audit sections below have been rewritten to describe +categories rather than instances. + +Two things that are *not* contamination, recorded here so they are not +re-flagged: + +- **Parameter names.** ProtocolVersion, Duration, Interval, Length, + ReceivedStats, StampAt, Clock, DSCP and ServerFill appear throughout the + specifications. They come from upstream's **published** documentation of its + client's machine-readable output, not from source inspection. +- **Quoted diagnostic text.** Where a document quotes a message printed by the + upstream client or server, that is transcribed program output observed during + testing — the same thing any black-box tester sees — and is cited as evidence + of observed behavior. ## Clean-Room Boundary @@ -13,23 +44,21 @@ to the GPL-licensed upstream source code. upstream source code, documentation, tests, CLI behavior, and release notes to understand the IRTT protocol. -- **Clean side:** The implementation agent will receive only the - `IRTT_CLIENT_PROTOCOL_SPEC.md` document. The implementation agent MUST NOT +- **Clean side:** The implementation agent will receive only the documents + under `docs/protocol/` and this notes file. The implementation agent MUST NOT see the upstream source code. ## Source Materials Inspected -The following materials from the upstream IRTT repository (heistp/irtt, -GPLv2) were inspected to produce the protocol specification: +The following materials from the upstream IRTT project (heistp/irtt, GPLv2) were +inspected to produce the protocol specification: -1. **Documentation files:** - - `README.md` — project overview, FAQ, features, limitations - - `doc/irtt.md` — main man page - - `doc/irtt-client.md` — client man page with CLI options and JSON output format - - `doc/irtt-server.md` — server man page (referenced, not fully read) - - `CHANGES.md` — changelog +1. **Published end-user documentation:** the project overview and FAQ, the + general and per-command manual pages, and the changelog. This material is + public and may be relied on directly; it is cited in the specifications where + used. -2. **Source files (inspected for protocol behavior only):** +2. **Source (inspected for protocol behavior only):** - Protocol version and constants - Packet format, field layout, magic bytes, flags - Parameter serialization format (tag-value pairs, varint encoding) @@ -64,66 +93,60 @@ copied. ## Post-Drafting Audit -After the initial specification was drafted, a clean-room audit was -performed. The following issues were identified and corrected: +After the initial specification was drafted, a clean-room audit was performed. +The categories of problem found and corrected were: -1. **Upstream private type names leaked:** "ctoken" and "seqno" appeared - as parenthetical abbreviations in the terminology table. Removed. +1. **Upstream private abbreviations in the terminology table.** Removed and + replaced with descriptive names. -2. **Upstream implementation preference disclosed:** A note about IPv4 - preference in dual-stack resolution revealed an upstream implementation - choice. Removed. +2. **An upstream implementation preference disclosed** in the address-resolution + discussion. Removed; the spec now leaves the choice to the implementer. -3. **Go-specific function references:** References to `binary.PutUvarint`, - `binary.PutVarint`, and `encoding/binary` package named the upstream - language's standard library. Replaced with language-neutral descriptions - (LEB128, protobuf-style zigzag). +3. **Implementation-language standard-library references** in the parameter + encoding section. Replaced with language-neutral descriptions (LEB128, + protobuf-style zigzag). -4. **Internal scheduling algorithm described:** The send scheduling section - described a specific "snap-to-interval" drift-correction algorithm that - mirrored upstream control flow. Replaced with observable behavior - description (packets sent at approximately ideal times, drift - compensation is implementation-defined). +4. **An internal scheduling algorithm described** in the send-scheduling section, + mirroring upstream control flow. Replaced with observable behavior: packets + are sent at approximately their ideal times and drift compensation is + implementation-defined. -5. **Upstream timer algorithm named:** "The upstream uses exponential - averaging of timer errors" revealed an internal implementation choice. - Removed. +5. **An upstream timer-smoothing strategy named.** Removed. -6. **RTT edge case speculated:** The spec asserted that server processing - time exceeding raw RTT should cause subtraction to be skipped — a guard - the upstream does not actually implement. Moved to Open Questions. +6. **An RTT edge-case guard asserted** that upstream does not in fact implement. + Moved to Open Questions. -7. **Received window validity asserted normatively:** The behavior of - window=0 as an invalidity sentinel was stated as fact when it should be - verification-required. Softened with reference to Open Questions. +7. **Received-window validity asserted normatively** when it was + verification-required. Softened with a reference to Open Questions. -8. **Go concurrency term leaked:** "goroutine" appeared in the result model - section. Replaced with "operation." +8. **An implementation-language concurrency term** in the result-model section. + Replaced with a neutral word. -9. **Upstream test design referenced:** Section 18.2 described what "the - upstream test" was testing internally. Rewritten to describe the test - vector neutrally. +9. **Upstream test internals referenced** in the test-vector section. Rewritten + to describe the vector neutrally. -10. **Upstream constants asserted normatively:** The minimum restricted - interval (1 second) and maximum parameter buffer size (128 bytes) are - internal constants that were stated as protocol requirements. Moved to - Open Questions with verification suggestions. +10. **Internal constants asserted as protocol requirements** — a minimum + restricted interval and a maximum parameter buffer size. Moved to Open + Questions with verification suggestions. Both were subsequently settled by + measurement (Sections 19.9 and 19.10 of the client specification). -11. **Timestamp capture order stated as MUST:** Implementation-level timing - advice was stated as a protocol requirement. Changed to SHOULD with - measurement-accuracy rationale. +11. **Timestamp capture order stated as MUST** when it was implementation-level + timing advice. Downgraded, and later rewritten again in the final + sanitization pass to describe only the externally relevant relationship. -12. **Internal recording structure described:** Section 19.6 described a - "pre-send timestamp" used for send-call-time measurement, which is an - internal recording detail. Rewritten to focus on observable behavior. +12. **An internal recording structure described** in the send-timing discussion. + Rewritten to focus on observable behavior. -13. **Self-contradictory correction note:** Section 8.2 contained an inline - "Correction:" note that was confusing and revealed the drafting process. - Cleaned up. +13. **A self-contradictory inline correction note** that revealed the drafting + process. Cleaned up. -14. **Field ordering lacked verification note:** The critical field order - in Section 8.1.3 was derived from source and stated without any - verification caveat. Added verification note and new Open Question 19.11. +14. **Field ordering stated without a verification caveat** despite being + source-derived at the time. Verification note added, and the ordering was + subsequently confirmed by capture across six field combinations + (Section 19.11). + +No instance of the removed material is reproduced here. Restating a private +identifier in order to record that it was removed would defeat the removal. ## Clean-Room Compliance Checklist @@ -145,45 +168,445 @@ performed. The following issues were identified and corrected: ## Second-Pass Clean-Room Scrub -A second independent review was performed on 2026-04-28 by an agent that -had not seen the upstream source. The following issues were identified and -corrected: +A second review was performed on 2026-04-28 by an agent that had not seen the +upstream source. The categories of problem found and corrected were: -1. **Implementation architecture directive in scope:** Section 2 stated - "Library-first implementation target," which prescribes internal - architecture. Removed; the spec now says only that CLI design is not - prescribed. +1. **An implementation-architecture directive in the scope section**, which + prescribed internal structure. Removed; the specification now constrains only + observable behavior. -2. **Concurrency architecture leaked:** Section 6.3 described "a separate - receive path processes incoming reply packets concurrently," which - prescribes an internal threading/task model. Rewritten to state the - observable requirement (send and receive occur concurrently) without +2. **A concurrency architecture prescribed** in the active-test section, + implying an internal threading or task model. Rewritten to state the + observable requirement — sending and receiving proceed concurrently — without prescribing how. -3. **Source-derived design rationale in echo request:** Section 8.4 - explained that zeroed fields exist to "use the same field layout that - appears in echo replies," which reveals internal design reasoning. - Simplified to state only the observable purpose (reaching the - negotiated packet length). +3. **Source-derived design rationale** offered for the zeroed placeholder fields + in an echo request. Simplified to the observable purpose: reaching the + negotiated packet length. -4. **Source-exhaustive knowledge parenthetical:** Section 8.6 said - "(currently only ServerFill)" for string-typed values, revealing - knowledge of the complete parameter set from source inspection. - Removed the parenthetical. +4. **A parenthetical revealing source-exhaustive knowledge** of the complete + parameter set. Removed. -5. **Upstream CLI syntax leaked:** Section 10.10 used `NxD`/`NrD`/`D` - notation for wait variants, which mirrors upstream CLI option syntax. +5. **Upstream CLI option syntax mirrored** in the wait-variant notation. Replaced with plain-language descriptions. -6. **Algorithm name disclosed:** Section 12.9 named "Welford's online - algorithm" as a MAY recommendation. This is implementation guidance. - Replaced with "the method of computing running statistics is - implementation-defined." +6. **A named statistics algorithm recommended.** Replaced with a statement that + the method of computing running statistics is implementation-defined. -7. **Implementation-specific error category:** Section 14 listed - "Allocate results buffer failure," implying a pre-allocated buffer - strategy. Generalized to "Insufficient resources for test parameters." +7. **An implementation-specific error category** that implied a particular + allocation strategy. Generalized. -8. **Language-specific references in CLEANROOM_NOTES.md:** The Purpose - section named "the Rust implementation" and the checklist mentioned - "No Rust API design." Both replaced with language-neutral phrasing. +8. **Implementation-language references in this notes file itself.** Replaced + with language-neutral phrasing. + +As above, the removed instances are described by category and not reproduced. + +--- + +## Server Pass (2026-08-11) + +### Scope + +A second contaminated-side pass characterised the upstream **server** and +produced `../protocol/IRTT_SERVER_PROTOCOL_SPEC.md`, together with +`../protocol/test-vectors/SERVER_BEHAVIORAL_VECTORS.md` and four new captures. +Corrections and clarifications arising from it were applied to the client +specification. + +### Upstream Material Inspected + +Three upstream builds are involved in this work, and the sanitized output names +whichever one it means everywhere it matters: + +| Build | Role | +|-------|------| +| **0.9.1 release** | the behavioral baseline; every unqualified statement refers to it | +| **0.9.0 release** | comparison only, for the version difference in the midpoint follow-up | +| **development tree, six upstream commits past the 0.9.1 tag** | comparison only; post-release work, **not** part of 0.9.1, and never allowed to redefine the baseline | + +*Provenance correction (final sanitization pass):* an earlier revision described +that development tree as "eight commits past the 0.9.1 tag". The count was wrong +— it included two commits belonging to this clean-room project itself, which were +present in the same working tree but are not upstream work. The upstream count is +**six**, and the outgoing documents now say so consistently. + +Inspection covered, at a high level: + +1. **Published documentation:** the server manual page, the changelog, and the + project overview's security, FAQ and roadmap sections. +2. **Source (for behavior only):** request admission and validation, session + creation and lookup, parameter restriction, echo handling, reception + statistics, timestamp selection, payload filling, close handling, session + expiry, and server configuration defaults. +3. **Release history:** tags and commits between 0.9.1 and the checked-out + development head, to classify version differences. + +Source inspection was used only to decide **what to test**. Every normative +statement in the sanitized output rests on an observed experiment, a capture, or +published end-user documentation — not on the source. + +### Method, and What Each Part of It Establishes + +The distinction below is load-bearing and is preserved in the outgoing documents. + +- **Black-box observation — a raw UDP harness.** It drove a real upstream server + directly, so that requests no conforming client would emit could be tested + (out-of-order sequence numbers, truncated packets, invalid flag combinations, + foreign-endpoint packets, corrupted authentication, and so on). It shares no + code with any implementation. Its measurements are stated as observed wire + behavior. +- **Black-box observation — the real upstream client.** Driving the unmodified + upstream client and observing what it recorded, reported and exited with is an + observation of a genuinely independent second implementation. +- **Measurement, not constant-reading.** Timing boundaries (idle-expiry grace, + maximum-duration grace, interval caps) were established by measuring the + server's responses, not by reading values out of the source. +- **Contaminated-side consistency validation — a server written on this side.** + A server was built from the behavioral model and driven by the real upstream + client across eleven configurations. Because it was written on the contaminated + side it is **not** an independent implementation, and nothing in the outgoing + documents rests on it. What it establishes is that the specification as written + is sufficient to build something the upstream client accepts; the client's + acceptance is the black-box part. Earlier revisions called this an "independent + server"; that description was inaccurate and has been corrected throughout. +- All research scripts, logs, probe sources and notes were kept outside `docs/` + and are not part of the outgoing material. + +### Statement of Non-Copying (Server Pass) + +No upstream source code, comments, pseudocode, tests, or file/module names were +copied into any document under `docs/`. The sanitized output describes +externally observable behavior only. + +Where upstream behavior differs from what a naive reading of the wire format +would suggest — the received-window reset on reordering, the unenforced protocol +version, the midpoint dual-field emission — that difference is documented as an +**observation**, with the input that produces it and the output it yields. No +explanation of *why* the implementation behaves that way appears in the +sanitized output, because any such explanation would necessarily be +source-derived. + +*Amended 2026-08-11:* this remains true of the received-window and +protocol-version items. For the dual-field midpoint emission, one narrowly scoped +**source-assisted historical conclusion** was admitted and labelled — a judgement +that the measured 0.9.0-to-0.9.1 difference was unintended. See "Midpoint +Timestamp Follow-Up" below. + +Two upstream robustness defects are recorded in the server specification +(`../protocol/IRTT_SERVER_PROTOCOL_SPEC.md` Section 22) because both are +reachable from ordinary client traffic and a clean implementation has to decide +what to do instead. They are stated as observed outcomes plus **robustness +recommendations for the clean implementation** — explicitly not as +interoperability requirements, since no conforming client can observe which +choice a server made — with no description of the internal cause. + +### Server-Pass Sanitization Audit + +The following were checked for and are absent from all files under `docs/`: + +1. Upstream private function, type, field and variable names. +2. Upstream source file names, module names and package names. +3. Implementation-language identifiers, standard-library references and + concurrency vocabulary. +4. Copied or paraphrased upstream comments. +5. Source-shaped pseudocode and control-flow transcription. +6. Internal constants that are not externally observable. Timing values that + *are* observable (the idle-expiry grace, the maximum-duration grace, the + interval cap ratio) are stated with the measurement that establishes them and + are labelled as server policy, not protocol law. +7. Internal data-structure descriptions. Session lookup is described purely in + terms of which packets are accepted and which are dropped. +8. Token generation strategy. The specification states only the uniqueness + requirement and explicitly leaves generation to the implementer. + +The following items were considered and deliberately **excluded** from the +sanitized output as implementation-internal: + +- how session state is stored, indexed, or reclaimed; +- how packet buffers are managed or reused; +- the mechanism by which expired sessions are noticed; +- any description of why the received-window behavior takes the form it does. + +### Clean-Room Compliance Checklist (Server Pass) + +- [x] No upstream source code included. +- [x] No upstream comments quoted. +- [x] No upstream private function or type names included. +- [x] No upstream file/module layout described. +- [x] No implementation-language references remain. +- [x] No pseudocode derived from upstream implementation. +- [x] No internal data structures or algorithms described. +- [x] Observable behavior separated from server policy throughout. +- [x] Every normative statement backed by an experiment, capture, or public + documentation. +- [x] Version differences classified explicitly, with each build named (server + specification Section 21). +- [x] Unresolved questions recorded rather than inferred (server + specification Section 23). +- [x] Contaminated-side consistency validation performed against the real + upstream client, and labelled as such rather than as independent + verification. +- [x] Contaminated research notes, scripts and probe sources kept outside + `docs/`. + +--- + +## Midpoint Timestamp Follow-Up (2026-08-11) + +### Scope + +A focused contaminated-side investigation of one behavior: the timestamp layout +upstream 0.9.1 emits for StampAt = Midpoint. It produced changes to the server +specification (Sections 9.2, 11.3.1, 21.1, 21.2, 23.4, 24), the client +specification (Sections 8.5, 19.14), the verification report (Part III, +Finding S-18, plus a caveat on Section 19.2) and the behavioral vectors +(Section 5.2 and its new subsections). + +### A Different Provenance Mix From the Earlier Passes + +The earlier passes used source inspection **only to decide what to test**. This +pass went further: contaminated-side analysis was used to form expectations, +which were then tested. The sanitized output therefore contains one class of +statement the earlier passes did not, and it is labelled wherever it appears. +What crosses the boundary falls into four kinds: + +- **Black-box observed** — the wire order (wall field then monotonic field), the + affected StampAt/Clock combinations, the dependence of the reply length on the + negotiated length, the value an ordinary positional decoder reads in the + monotonic-only case, and the 0.9.0 / 0.9.1 / tested-post-0.9.1 comparison. All + measured with a raw UDP harness against three separate server builds, and + cross-checked at the application layer with the real upstream client. +- **Source-assisted historical conclusion** — one judgement only: that the + measured difference between 0.9.0 and 0.9.1 is an unintended regression rather + than a designed feature. It is labelled where it appears, it is not normative, + and the measured version difference is stated separately and stands on its own. +- **Interoperability requirement** — narrowly, what a clean client must tolerate + for this one verified case. +- **Robustness / implementation recommendation** — what a clean server should + emit. + +The specification wording keeps these four apart explicitly, so a clean-side +reader can tell which statements they may rely on as observed fact. + +Note in particular what is **not** in the list. An earlier revision also carried +a characterisation of how upstream conceptually represents a midpoint value. +That was an internal model, it was source-derived, and the useful wire behavior +is fully established without it, so it was removed from all implementation-facing +documentation in the final sanitization pass. + +### What Was Deliberately Not Transferred + +Detailed upstream implementation structure, internal representation, control +flow, change-level source evidence, and the source-derived reasoning supporting +the regression diagnosis were retained only on the contaminated side. + +That sentence is the whole of the disclosure, and deliberately so. An earlier +revision of this file enumerated the withheld findings in enough detail that +reading the list conveyed much of what the list claimed to be withholding. Naming +the categories is what a provenance record needs; restating the contents defeats +the exclusion it is recording. + +What crossed the boundary from that body of work is one sentence of conclusion — +labelled source-assisted, non-normative, and separable from every requirement in +the outgoing documents. + +### Method + +- A raw UDP harness performed the open negotiation, sent a single echo request, + and reported the exact reply datagram length and the bytes at each timestamp + offset. It shares no code with any implementation. +- Three server builds were exercised, each named wherever its results are used: + the **0.9.1 release**, the **0.9.0 release**, and a build of the upstream + development tree **six upstream commits past the 0.9.1 tag**. Builds and + harness live outside this tree. +- The real upstream client was used as a second, genuinely independent observer + for the application-layer effects. + +### Clean-Room Compliance Checklist (Midpoint Follow-Up) + +- [x] No upstream source code, comments or pseudocode included. +- [x] No upstream private function, type or field names included. +- [x] No upstream control flow described. +- [x] No upstream file or module layout described. +- [x] No internal conceptual model of upstream's timestamp representation + included. +- [x] The single source-assisted statement labelled as such, confined to a + historical judgement, and never presented as black-box verified. +- [x] Black-box observation, source-assisted conclusion, interoperability + requirement and robustness recommendation kept distinct. +- [x] Excluded contaminated-side material described by category only, without + restating its content. +- [x] Earlier over-general claim ("always 8 bytes longer") corrected rather than + left standing. + +--- + +## Final Sanitization Pass (2026-08-11) + +A last audit of the entire outgoing `docs/` tree before it crosses the airlock. +No new protocol research was performed; this pass only corrected, reclassified +and removed. + +### Provenance + +- Every statement now names the build it came from. The **0.9.1 release** is the + baseline; the **0.9.0 release** and the **development tree six upstream commits + past the 0.9.1 tag** appear only where explicitly labelled, and neither is + allowed to stand in for 0.9.1. +- The "eight commits past the 0.9.1 tag" figure was corrected to **six**. The + earlier count wrongly included two commits belonging to this clean-room project + itself. +- The server written on this side is now described everywhere as **contaminated- + side consistency validation**, never as an independent implementation. Only the + raw UDP harness and the real upstream client are treated as black-box + observers. + +### Statement classification + +The outgoing documents now separate five kinds of statement and never conflate +them: interoperability requirement; black-box observed upstream behavior; +upstream policy or default; robustness recommendation for the clean +implementation; and source-assisted historical conclusion. + +Specific reclassifications made in this pass: + +- Surviving a failed reply send is a **robustness recommendation**, not a + protocol MUST. No conforming client can observe which choice a server made. +- Upstream's absence of session and per-peer bounds is **observed policy**, and + the documents now say explicitly that it is not something a compatible server + should reproduce. The clean server is expected to be bounded; the shape of that + policy is left to the clean project. +- The fill-validation, DSCP-clamping and length-clamping items in the hazards + section were likewise moved from "requirement" to robustness recommendation. +- The server specification's conformance summary is split: interoperability + MUST / MUST NOT lists, and a separate list of robustness recommendations. + +### Architecture neutrality + +The outgoing documents prescribe no runtime or architecture for the clean server +— no blocking-versus-async choice, no runtime-free API, no core-type shape, no +socket ownership arrangement. Those are clean-project decisions and belong in the +clean repository after the airlock, not in protocol documentation. The +architecture-neutrality statement in the server specification's scope section was +extended to say so directly. + +### Wording removed or rewritten + +- **Source-derived timestamp placement.** Statements locating a timestamp + immediately before or after a particular system call were replaced with the + externally relevant property: the pair must bracket the server's handling of + the request, receive never later than send. Where an implementation takes its + readings is not observable and is no longer described. +- **Validation-order narrative.** The packet-admission section was rewritten from + a numbered processing sequence into a table of discard conditions plus the two + ordering relationships that were actually verified externally (Open-versus-Close + and Open-versus-echo-shaped-body interpretation precedence for an otherwise + admissible datagram, which overrides no independent rejection condition; and the + indistinguishability of an authentication failure from an unknown token). + Behavioral wording replaced control-flow wording throughout. +- **Internal midpoint model.** The characterisation of upstream's internal + representation of a midpoint timestamp was removed from all + implementation-facing documentation. What remains is the wire fact: a + dual-field midpoint representation, wall field followed by monotonic field. +- **DSCP.** The wire parameter is now stated plainly as the raw IP TOS / + Traffic-Class byte, range 0–255, with Expedited Forwarding at 0xb8 / 184. The + future user-facing notation for the clean project is marked out of scope rather + than baked into the protocol documents. + +### Corrected claims + +- **Midpoint reply length.** Every "negotiated length + 8" and "normal packet + length + 8" claim was removed. The documents now state the model uniformly: + `upstream_header` is `normal_header` plus one 8-byte timestamp field; + `compatible_reply_len = max(negotiated_length, normal_header)`; + `upstream_0_9_1_reply_len = max(negotiated_length, upstream_header)`; and the + observable excess is +8, then +7…+1, then 0 as the negotiated length rises. The + measured sweeps demonstrating this were preserved. +- **Overlong replies.** The blanket requirement that a client tolerate any reply + longer than the negotiated length was narrowed to the one verified case: + single-clock midpoint at **exactly** `upstream_0_9_1_reply_len`. That is one + additional accepted length per negotiation, not a `+1..+8` window — where + `upstream_0_9_1_reply_len > compatible_reply_len` the two values differ by + between 1 and 8 bytes, but a length strictly between them is rejected like any + other unexpected length. The documents say explicitly that this does not + generalise to arbitrary overlong echo replies and that strict validation of + other malformed or oversized packets is retained. +- **Normal minimum reply size.** Reply-length validation is stated against + `compatible_reply_len`, the normal compatible reply length, rather than against + the negotiated `Length` value alone; the two differ whenever the negotiated + Length is below the mandatory field block. +- **Monotonic-only midpoint.** The suggestion that a client may simply read the + second 8-byte region as the monotonic value was replaced with a conservative + rule. In the equal-length regime the conforming and upstream forms are + indistinguishable, so the second region may be ordinary payload; a correction + is permitted only where the dual-field form is otherwise identifiable, the + ambiguity is recorded rather than resolved by heuristic, and the packet must be + accepted either way. +- **0.9.0 → 0.9.1.** Any claim of no protocol-visible change between the two was + corrected. Published release-note content and measured wire behavior are now + stated separately, and they disagree on this point. +- **Protocol version.** Stale text using "protocol version mismatch" as an + example of a normal server rejection was removed. Upstream 0.9.1 accepts + version 0, 2, negative and absent, and returns 1; enforcement is client-side. +- **Received window.** The documents now state that `0x1` means no useful prior + received-history is represented, **not** that the previous 63 packets were + lost. Downstream/upstream loss classification is flagged as a separate question + scheduled for its own clean-side audit. +- **Close semantics.** No production standalone close-reply codec is specified. + Such a datagram was observed to be tolerated by upstream clients but has no + demonstrated purpose in protocol version 1, and the documents say so. +- **Parameter payload size.** The "believed to be 128 bytes" limit was replaced + with the measured result: no protocol limit on a received payload; 128 bytes is + a local allocation choice. + +### Contamination scan + +The whole outgoing `docs/` tree was scanned against a private-token list derived +on the contaminated side from the upstream tree. That list is **not** reproduced +here, and neither is any token from it. + +Scanned for and now absent: upstream private function, type, field and variable +identifiers; implementation-language identifiers and standard-library references; +upstream source filenames, module and package names; source-derived commit +identifiers; local filesystem paths; scratchpad paths; and temporary build paths. + +The residual hits found and fixed in this pass were all in the audit prose of +this file, which had been quoting the identifiers it claimed to have removed, and +one tooling-environment block in the verification report that carried a local +installation path and an implementation-language version string. + +Retained deliberately, and not contamination: the public upstream project +identity (`heistp/irtt`) where provenance requires it; the parameter names, which +come from upstream's published documentation; and quoted upstream program output, +which is observed black-box evidence. + +### Captures + +Every `.pcapng` intended to cross was metadata-reviewed. Four had already been +container-normalized during the server pass; the remaining ten still carried +capture-machine hardware and operating-system identification, the local capture +interface name and description, and the capture filter string. All ten were +regenerated to strip that metadata. + +No capture contained a hostname, a username, a filesystem path, a command string +or any reference to an upstream checkout. + +**Packet bytes were not altered.** For each regenerated file the full frame +hexdump and the per-frame epoch timestamps were compared before and after and +were identical. No capture had to be excluded from the outgoing set. + +### Outgoing tree hygiene + +The outgoing `docs/` tree contains only specifications, the verification report, +behavioral and packet vectors, and captures. No research scripts, probe sources, +upstream source archives, build outputs, temporary logs, dirty source notes or +private-identifier lists are present, and the contaminated-side material for the +midpoint work was left where it already lives, outside `docs/`. + +One incidental local file sits inside `docs/` and is **not** outgoing material: a +local tool-permissions file under a dotted directory. It is ignored by version +control, it is not tracked, and it may not be copied across the airlock. The +macOS Finder metadata file previously listed here is no longer present; the +exclusion still stands, since Finder can recreate one at any time. Re-verified +2026-08-11: the dotted directory is the only ignored entry under `docs/`, and no +Finder metadata file exists anywhere in the outgoing tree. diff --git a/docs/protocol/BLACKBOX_VERIFICATION_REPORT.md b/docs/protocol/BLACKBOX_VERIFICATION_REPORT.md index c7b09a0..bd31cae 100644 --- a/docs/protocol/BLACKBOX_VERIFICATION_REPORT.md +++ b/docs/protocol/BLACKBOX_VERIFICATION_REPORT.md @@ -10,23 +10,22 @@ ## Environment Check -``` -$ command -v irtt -/Users/lochnair/.local/bin/irtt +Tooling used for Part I. Installation paths are omitted deliberately; only the +versions matter for reproducing these results. +``` $ irtt version irtt version: 0.9.1 protocol version: 1 json format version: 1 -go version: go1.26.2 on darwin/arm64 - -$ command -v tshark -/opt/homebrew/bin/tshark $ tshark --version -TShark (Wireshark) 4.6.4 (Git commit f7c4a74874d9). +TShark (Wireshark) 4.6.4 ``` +Reference server and client: the upstream **0.9.1 release**, unmodified. +Capture platform: macOS Darwin 25.3.0 arm64, loopback interface. + --- ## Open Question Resolution @@ -126,6 +125,11 @@ Timestamp field count depends on StampAt and Clock: This is a **normative requirement** — implementations must compute the correct minimum to avoid malformed packets. +**Caveat added 2026-08-11:** the two single-clock midpoint rows describe the +*negotiated* layout, which is what a compatible implementation emits. Upstream +0.9.1 emits 2 fields (16 bytes) for midpoint with any clock; see Findings S-7 and +S-18. + --- ### 19.4 Varint Encoding — RESOLVED @@ -154,7 +158,8 @@ verified against known values. All decoded correctly using standard protobuf-style zigzag + LEB128. -**Verified independently with Python decoder.** +**Cross-checked with a separately written Python decoder**, implemented from the +wire-format description alone. **Conclusion:** The varint encoding is standard protobuf-compatible. This is a **normative requirement** for interoperability. @@ -166,8 +171,8 @@ is a **normative requirement** for interoperability. **Question:** Is the HMAC computed over the entire packet (including payload) with the HMAC field zeroed? -**Test setup:** Captured HMAC session, extracted packet bytes, independently -computed HMAC-MD5 with Python. +**Test setup:** Captured HMAC session, extracted packet bytes, and recomputed +HMAC-MD5 with a separately written Python script. **Commands:** ``` @@ -228,6 +233,11 @@ or a bug. can be used as an invalidity sentinel. This is a **compatibility recommendation** — implementations SHOULD treat window=0 as invalid. +**Superseded in part by Finding S-2 (Part II).** Upstream 0.9.1 was later +confirmed never to emit 0 at all when the window is negotiated. The value that +actually needs care is `0x1`, which is a *valid* window carrying no history and +must not be read as "nothing earlier was received". + --- ### 19.8 HMAC Mismatch Behavior — RESOLVED @@ -308,6 +318,11 @@ ServerFill string (32 bytes max), the parameter buffer stays well under Reclassified as **compatibility recommendation** — implementations should allocate at least 128 bytes for the parameter buffer. +**Superseded by Part II.** Upstream servers were later shown to accept an open +request carrying over 200 bytes of parameters. There is no protocol limit on a +*received* payload; 128 bytes is a local allocation choice only, and a receiver +should tolerate more rather than reject. + --- ### 19.11 Field Ordering Verification — RESOLVED @@ -344,22 +359,26 @@ allocate at least 128 bytes for the parameter buffer. --- -### 19.3 Server Close During Test — NOT DIRECTLY TESTED +### 19.3 Server Close During Test — NOT DIRECTLY TESTED (superseded) -**Limitation:** Triggering a server-initiated close mid-test requires -specific server conditions that are difficult to set up reliably in a +**Limitation at the time:** Triggering a server-initiated close mid-test requires +specific server conditions that were difficult to set up reliably in a black-box test. The spec's description (server MAY set Close flag on echo -reply) has not been verified externally. Reclassified as **open / -verification required** — the spec now recommends defensive handling -without asserting that this behavior occurs. +reply) had not been verified externally, and was reclassified as **open / +verification required**. + +**Resolved in Part II by Finding S-4.** The maximum-test-duration hard limit was +found to trigger it reliably, and the behavior was captured. This entry is +retained only as a record of the earlier state. --- ### 19.6 Send Timestamp Capture Timing — NOT TESTABLE -**Limitation:** The exact timing of client-side timestamp capture relative -to the send syscall is an implementation-internal detail that cannot be -verified through black-box testing. Remains an implementation choice. +**Limitation:** How closely a client's own timestamp capture sits to its send +and receive operations is internal to that client and cannot be verified through +black-box testing, in either direction. Remains an implementation choice, and +nothing normative depends on it. --- @@ -497,8 +516,8 @@ open request. ### Finding E: Test Vector Size Correction The test vectors in Section 18 are described as "target length 256 bytes -(shown truncated)." Independent HMAC verification confirms that both test -vectors (18.1 and 18.2) were computed over 92-byte packets (76-byte header +(shown truncated)." Recomputing the HMAC from the captured bytes confirms that +both vectors (18.1 and 18.2) were computed over 92-byte packets (76-byte header + 16-byte payload), not 256-byte packets. The "256 bytes" description is incorrect. @@ -560,6 +579,12 @@ irtt client -l 200 127.0.0.1 # request length 200 ## Captures Created +All capture files referenced in this report were metadata-reviewed and their +containers rewritten to strip capture comments, section and interface +descriptions, host/user/machine identification, capture filter strings and +filesystem paths. Packet bytes and timestamps were not altered — full frame +hexdumps and per-frame epoch timestamps were verified identical before and after. + | File | Contents | |------|----------| | `captures/full-session.pcapng` | Complete session: open, 3 echoes, close | @@ -608,3 +633,551 @@ See `test-vectors/` directory. defensive handling without asserting the behavior occurs 6. Received window late-packet behavior: removed unverified claim about late packets producing window=0; retained verified bit-0 sentinel + +--- +--- + +# Part II — Server-Side Verification + +**Date:** 2026-08-11 +**Baseline:** the upstream **0.9.1 release** (protocol version 1) +**OS:** macOS Darwin 25.5.0 arm64 (Apple Silicon) + +**Method, and what each part of it establishes:** + +- **Black-box observation.** A raw UDP harness drove a real upstream server + directly, so that requests no conforming client would emit could be tested. It + shares no code with any implementation. Everything it measured is stated here + as observed wire behavior. +- **Black-box observation at the application layer.** The real upstream 0.9.1 + *client* was driven against various servers, and its externally visible + reaction — what it recorded, what it reported, how it exited — was observed. + The client is an independent second implementation for this purpose. +- **Contaminated-side consistency validation.** A server was also written on the + contaminated side from the behavioral model and driven by the real upstream + client. Because it was produced on this side of the boundary it is **not** an + independent implementation, and no statement in this report rests on it. What + it provides is a consistency check: it demonstrates that the model as written + is sufficient to drive a real upstream client. The *client's* reaction to it is + black-box observation; the server itself is not. + +Findings feed `IRTT_SERVER_PROTOCOL_SPEC.md`. Input → output tables are in +`test-vectors/SERVER_BEHAVIORAL_VECTORS.md`. + +--- + +## Part II Environment + +``` +$ irtt version +irtt version: 0.9.1 +protocol version: 1 +json format version: 1 +``` + +**Provenance of the second binary.** A second server binary was built from the +upstream development tree as it stood **six upstream commits past the 0.9.1 +tag**, for the version-difference comparison in Finding S-13. Those six commits +are post-release work and are **not** part of 0.9.1; every result attributed to +that build is labelled as such and never folded into the 0.9.1 baseline. + +*(An earlier revision of this report said "8 commits past the 0.9.1 tag". That +count was wrong: it included two commits belonging to this clean-room project +itself, which were present in the same working tree but are not upstream work. +The upstream count is six.)* + +--- + +## Finding S-1: Session identity is token + source endpoint + +A post-open request is bound to a session by its token **and** by an exact match +on the source address, source port, and address family. + +| Case | Result | +|------|--------| +| Correct token, original endpoint | Served | +| Correct token, different source port | **Dropped, no reply; session intact** | +| Correct token, other address family's listener on the same port | Dropped | +| Unknown token | Dropped | +| Zero token | Dropped | +| Token of a closed session | Dropped | +| **Close** bearing a valid token from a foreign source port | **Dropped; session not closed** | + +Endpoint binding protects the close path as well as the echo path. A capture is +in `captures/server-session-identity.pcapng`. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-2: Received window resets to `0x1` on reordering and large gaps + +The single most consequential server behavior found. A request arriving with a +sequence number lower than the highest already seen, or 64 or more ahead of the +previous one, causes the reported window to become `0x1` — history is discarded, +not merely left unshifted, and it is never recovered. + +``` +seq 0 -> count 1 window 0x0000000000000001 +seq 1 -> count 2 window 0x0000000000000003 +seq 5 -> count 3 window 0x0000000000000031 +seq 3 -> count 4 window 0x0000000000000001 <- gap fill discards history +``` + +Duplicates leave the window unchanged and advance the count. 32-bit sequence +wraparound is handled as ordinary modular arithmetic. Window value 0 was never +observed. Full tables in `test-vectors/SERVER_BEHAVIORAL_VECTORS.md` Section 1; +capture in `captures/server-recv-window.pcapng`. + +The three-case rule stated in `IRTT_SERVER_PROTOCOL_SPEC.md` Section 10.2 was +cross-checked against a live server over 12 sessions of 40 requests each — 480 +replies — with sequence numbers drawn from a mix of in-order steps, duplicates, +forward gaps of 2 to 80 and backward steps of 1 to 5, and starting sequence +numbers of 0, 1, 7, 100 and 4294967294. **480 replies checked, 0 mismatches** in +either window or count. + +**Classification:** SERVER-ONLY NEW INFORMATION **and** CLIENT SPEC +CLARIFICATION (client spec Sections 12.8 and 19.15). + +## Finding S-3: Received count semantics + +The count advances by one for every request that reaches the reply stage, +including duplicates and out-of-order arrivals, and never for requests dropped +for bad magic, bad flags, bad authentication, unknown token, endpoint mismatch, +oversize, or rate limiting. It is per session and includes the request being +answered. Confirmed against a rate-limited server: 12 back-to-back requests +produced 5 replies with counts 1–5, and three later requests continued 6–8. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-4: Server-initiated close exists and is triggered by the duration limit + +Resolves client-spec open question 19.3. A server with a maximum test duration +answers the first echo request past its hard deadline with a complete echo reply +carrying flags `0x06` (Reply|Close), and removes the session at that moment. + +| Configured maximum duration | Close flag observed at | +|---|---| +| 1 s | 3.01 s after the first echo | +| 2 s | 4.06 s / 4.10 s after the first echo | + +The deadline is the maximum duration plus a 2-second grace, measured from the +session's first echo request — not from open. Capture: +`captures/server-close.pcapng`, frame 36. + +Driving an upstream 0.9.1 client against a server that sets the flag mid-test: +the client discards that reply's measurement, terminates with a "server closed +connection" condition, prints partial results, and exits successfully. + +No other condition — idle expiry, shutdown, rate limiting, oversize, auth +failure — was observed to set the flag. + +**Classification:** SERVER-ONLY NEW INFORMATION **and** CLIENT SPEC +CLARIFICATION (client spec Sections 13.7 and 19.3). + +## Finding S-5: Client-initiated close is never acknowledged, and a close reply is harmless + +Upstream 0.9.1 sends **nothing** in response to a close request. Repeated closes +and post-close echoes are dropped as unknown-token packets. Capture: +`captures/server-close-lifecycle.pcapng` ends with three unanswered client +packets. + +A contaminated-side server that *does* reply to close — magic + flags(Reply|Close) ++ [MAC] + token — was driven against a real 0.9.1 client and the run completed +normally with correct results, with nothing in the client's output changing. +Sending such a datagram is therefore harmless, but nothing was observed to make +use of it: it has no demonstrated purpose in protocol version 1, and the server +specification does not define it as required behavior. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-6: Protocol version is not enforced by the server + +Open requests with `ProtocolVersion` of 0, 2, −1, or the tag omitted entirely, +each produced a session-creating open reply (flags `0x03`, non-zero token) with +`ProtocolVersion = 1` returned. The client spec's claim that "the server will +also set the Close flag if there is a version mismatch" is not true of upstream +0.9.1. + +**Classification:** CLIENT SPEC CORRECTION (client spec Sections 6.2, 9.1, +19.13). + +## Finding S-7: Dual-field midpoint wire representation + +With StampAt = midpoint and a single negotiated clock, the reply carries both +midpoint fields — wall then monotonic — which is one 8-byte field more than the +negotiated parameters imply, and makes the datagram longer at small negotiated +lengths. Measured across every combination of received-stats and clock; confirmed +at the application layer by an upstream client reporting 36-byte packets while +receiving 44-byte datagrams and completing normally. + +A contaminated-side server emitting only the negotiated midpoint field was driven +by the 0.9.1 client for wall-only, monotonic-only and dual-clock configurations, +and the client completed every run cleanly. This is consistency validation, not +an independent observation; what it shows is that the *client* accepts the +conforming form. + +**Extended 2026-08-11 — see Finding S-18** for the negotiated-length dependence, +the version comparison, and the correction to the "always +8" characterisation. + +**Classification:** SERVER-ONLY NEW INFORMATION **and** CLIENT SPEC +CLARIFICATION (client spec Sections 8.5 and 19.14). + +## Finding S-8: Interval restriction resolves the client's safety floor + +A server restricts the negotiated interval to at most one quarter of its idle +timeout (measured: 4 s → 1 s, 8 s → 2 s, 20 s → 5 s, 40 s → 10 s, 60 s → 15 s). +This gives an ordinary, unmodified server that reduces a client's interval, which +resolves client-spec open question 19.9. + +| Server idle timeout | Returned interval | Client outcome | +|---|---|---| +| 2 s | 500 ms | Aborts — strict **and** loose | +| 3 s | 750 ms | Aborts — strict **and** loose | +| 4 s | 1 s (no reduction) | Proceeds | + +Reduction to exactly 1 s is an ordinary restriction: rejected in strict mode, +accepted in loose mode. The safety floor is **1 second**. + +**Classification:** CLIENT SPEC CLARIFICATION (client spec Sections 6.2, 19.9). + +## Finding S-9: Every rejection is a silent drop + +No error, reset, or NACK exists anywhere in the protocol. Verified for: short +datagrams, bad magic, reserved flag bits, the Reply flag on an inbound packet, +every authentication failure mode, malformed parameters, invalid parameter enum +values, unknown tokens, endpoint mismatch, oversize requests and rate limiting. +Full table in `IRTT_SERVER_PROTOCOL_SPEC.md` Section 17. + +Minimum accepted request sizes without authentication: open 4 bytes, close +12 bytes, echo 16 bytes; add 16 bytes with authentication. + +**Classification:** SERVER-ONLY NEW INFORMATION, CONSISTENT with the client +spec's HMAC-drop finding (19.8). + +## Finding S-10: Authentication must be applied to replies, including echo replies + +A contaminated-side server that authenticated its open reply correctly but +omitted the HMAC flag from echo replies caused the 0.9.1 client to discard every +echo reply and terminate abnormally with zero packets received. Fixing the flag +made all configurations pass. The load-bearing observation here is the **client's** +reaction, which is black-box; the server was merely the stimulus. + +All authentication failures — missing flag, missing field, wrong key, flipped +bit, zeroed MAC, truncated MAC, flag/MAC disagreement in either direction — +produce a silent drop with no session state change. A session survived eight +consecutive failing probes and served the next valid request normally. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-11: Idle expiry has a 5-second grace, and never-used sessions do not expire + +With a 2-second idle timeout, the boundary was measured between 6.9 s and 7.1 s, +i.e. timeout + 5 s. The first request past the deadline is still answered and +the session is released while handling it; only the next request is dropped. + +A session that has never carried an echo request does not expire at all: opened +and left idle for 9 seconds under a 2-second timeout, it was served normally. +Combined with the fact that every open — including a retransmitted one — creates +a new session, this means open retransmissions accumulate permanent state. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-12: No session or per-peer limits by default + +1000 sequential opens from a single source endpoint all succeeded with distinct +tokens, and the first, middle and last were all still usable afterwards. Two +opens from the same source port yield two independently usable sessions. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-13: Tested post-0.9.1 tree — CLI-only difference + +The development tree built for this study is **six upstream commits** past the +0.9.1 tag. Those commits are not part of the 0.9.1 release, and nothing in this +finding describes 0.9.1. The only functional change among them is an opt-in +server ECN option; the remainder are documentation, roadmap and dependency +updates. + +- The option's documentation states that it "disables UDP replies from server." + **Observed behavior contradicts this**: with the option enabled, sessions + completed normally over both IPv4 and IPv6 (4/4 packets, no loss). +- Enabling it forces the source-IP-pinning policy. +- ECN observation is a receive-side property of the IP header and does not + change the IRTT payload in either direction. + +A side-by-side run of both builds with an identical scripted request sequence +(open with full statistics and dual timestamps on both clocks; echoes 0, 1, 4, +3; close) produced identical structure: same open-reply layout, 60-byte replies, +same flags, counts 1–4, windows `0x1`, `0x3`, `0x19`, `0x1`, and no close reply +in either case. + +**Classification:** VERSION DIFFERENCE — CLI / server-policy. Not wire-visible. + +## Finding S-14: Reply length is capped by the interface MTU but the negotiated value is not + +On a 1500-byte-MTU interface, negotiated lengths of 1501, 2000, 3000 and 8000 +each produced 1500-byte replies while the open reply returned the requested +value unchanged. Clients treat a short reply as fatal, so such a session opens +and then fails on the first echo. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-15: Robustness hazards reachable from ordinary traffic + +Two conditions terminate the upstream server process. Both are recorded in +`IRTT_SERVER_PROTOCOL_SPEC.md` Section 22 so that a clean implementation does +not reproduce them. + +1. **Oversized reply.** If the negotiated length exceeds the host's maximum + outbound datagram size, the send fails with a non-transient error, which is + treated as fatal: the listener stops and the process exits. Observed on a + host with a 9216-byte maximum: a session negotiating 15000 bytes killed the + server on its first echo reply. The default configuration places no upper + bound on the negotiated length. +2. **Zero-length fill pattern.** A server-fill descriptor of the form + `pattern:` with an empty hex body terminates the server during open handling, + when the fill allow-list permits pattern fills. The default allow-list does + not, so a default-configured server is not exposed. An unparseable hex body + such as `pattern:zz` is handled correctly by falling back to the server + default. + +**Classification:** SERVER-ONLY NEW INFORMATION. + +## Finding S-16: Server fill is a continuous stream + +The default fill is the repeating pattern `69 72 74 74`, and its phase is not +reset per packet or per session — it advances continuously as bytes are consumed +and carries over between sessions on the same listener. + +``` +seq 0 payload: 72 74 74 69 72 74 74 +seq 1 payload: 69 72 74 74 69 72 74 +seq 2 payload: 74 69 72 74 74 69 72 +seq 3 payload: 74 74 69 72 74 74 69 +``` + +Under a no-fill policy the payload contains unspecified residual bytes; one +observed reply's trailing bytes were a fragment of a previous packet's parameter +payload. + +**Classification:** SERVER-ONLY NEW INFORMATION; also noted as a client-side +caution in client spec Section 8.5. + +## Finding S-17: Contaminated-side consistency validation of the model + +A server built from the behavioral model in `IRTT_SERVER_PROTOCOL_SPEC.md` — with +no upstream code — was driven by the real 0.9.1 client across: default +parameters; no statistics and no timestamps; midpoint with wall, monotonic and +both clocks; send-only with wall; receive-only with monotonic; a 200-byte packet +length; no-test mode; HMAC with a matching key; HMAC with a mismatched key; and +IPv6. Every configuration produced a clean, lossless run with correct results, +and the mismatched-key case correctly timed out. + +**This is contaminated-side consistency validation, not independent +verification.** The server was written on this side of the clean-room boundary, +so it cannot corroborate anything about upstream. What it does show is that the +specification as written is sufficient to build a server the real upstream client +accepts — the client's acceptance being the black-box part. No statement anywhere +in the outgoing documents rests on this exercise. + +**Classification:** Consistency validation, not a finding. + +--- + +## Part II Captures Created + +| File | Contents | +|------|----------| +| `captures/server-recv-window.pcapng` | Sequential, gap, duplicate, reorder and far-gap sequence numbers on one session | +| `captures/server-close-lifecycle.pcapng` | Client close, repeated close, post-close echo — none answered | +| `captures/server-close.pcapng` | Server-initiated close via the duration limit; frame 36 carries flags `0x06` | +| `captures/server-session-identity.pcapng` | Foreign-source-port echo and close, unknown token, original endpoint still served | + +## Part II Vectors Created + +`test-vectors/SERVER_BEHAVIORAL_VECTORS.md`. + +## Part II Client Spec Changes Applied + +| Section | Change | Class | +|---------|--------|-------| +| 2 | Pointer to the companion server specification | Editorial | +| 6.2, 9.1, 19.13 | Server does not enforce the protocol version | **CORRECTION** | +| 6.2, 19.9 | Interval safety floor pinned to 1 second, enforced in loose mode too | **CLARIFICATION** | +| 8.5, 19.14 | The one verified case in which a reply exceeds `compatible_reply_len`, scoped to the single extra length `upstream_0_9_1_reply_len`; `compatible_reply_len` rather than the negotiated Length is the normal minimum; payload content is not predictable | **CLARIFICATION** | +| 12.8, 19.15 | A window of `0x1` carries no history; window 0 is never emitted | **CLARIFICATION** | +| 13.7, 19.3 | Server-initiated close confirmed; observed client reaction | **CLARIFICATION** | +| 19.10 | No protocol limit on received parameter payload size | **CLARIFICATION** | + +## Part II Summary + +- Resolved client-spec open questions: **19.3**, **19.9**, **19.10**. +- New client-spec open questions resolved on discovery: **19.13**, **19.14**, + **19.15**. +- Still unresolved on the client side: **19.6** (not externally testable), + **19.12** (hard to trigger). +- Server-side unknowns are enumerated in `IRTT_SERVER_PROTOCOL_SPEC.md` + Section 23. + +--- + +# Part III — Midpoint Timestamp Follow-Up (2026-08-11) + +A focused follow-up on Finding S-7. Three server builds were exercised with a raw +UDP harness that performs the open negotiation, sends one echo request, and +reports the exact reply datagram length and the bytes at the timestamp offsets: + +| Build | Provenance | +|-------|-----------| +| **0.9.1** | the upstream release; the baseline for this study | +| **0.9.0** | the preceding upstream release, for comparison | +| **post-0.9.1 tree** | the upstream development tree six upstream commits past the 0.9.1 tag — post-release work, not part of any release | + +All on macOS arm64, loopback. + +**Provenance note.** The measurements below are black-box observations. Part III +additionally records one **source-assisted historical conclusion** — a judgement +about whether the measured 0.9.0/0.9.1 difference was intended — which is +labelled where it appears and is **not** presented as independently verified. +Nothing normative in the specifications depends on it, and the detailed +contaminated-side evidence behind it was retained on the contaminated side and +does not appear here. + +## Finding S-18: Dual-field midpoint emission is length-dependent in effect + +### The wire representation + +For StampAt = midpoint, upstream 0.9.1 emits two 8-byte timestamp fields — wall +first, monotonic second — for every negotiated Clock, occupying a fixed 16-byte +region. Full StampAt × Clock matrix at negotiated length 0, received-stats = +both, against 0.9.1: + +| StampAt | Clock = wall | Clock = monotonic | Clock = both | +|---------|--------------|-------------------|--------------| +| none | 28 | 28 | 28 | +| send | 36 | 36 | 44 | +| receive | 36 | 36 | 44 | +| both | 44 | 44 | 60 | +| **midpoint** | **44** (expected 36) | **44** (expected 36) | 44 | + +Every non-midpoint row matches the negotiated layout exactly. Only midpoint +deviates, and only when a single clock is negotiated. + +### The length dependence — correction to "always +8" + +Reply length is the larger of the negotiated length and the field block; the +extra field enlarges only the field block. Writing `normal_header` for the block +the negotiated parameters imply and `upstream_header` for that block plus one +8-byte timestamp field: + +``` +compatible_reply_len = max(negotiated_length, normal_header) +upstream_0_9_1_reply_len = max(negotiated_length, upstream_header) +``` + +Negotiated-length sweep, 0.9.1, midpoint + wall + received-stats = both +(`normal_header` 36, `upstream_header` 44): + +| Negotiated length | 0 | 16 | 32 | 36 | 37 | 40 | 43 | 44 | 45 | 48 | 64 | 128 | 1024 | 4096 | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| Reply datagram | 44 | 44 | 44 | 44 | 44 | 44 | 44 | 44 | 45 | 48 | 64 | 128 | 1024 | 4096 | +| Excess over a compatible server | +8 | +8 | +8 | +8 | +7 | +4 | +1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | + +Repeated for received-stats = none, count and window; the two minima shift to +24/32, 28/36 and 32/40 and the pattern is identical. The dual-clock control case +(midpoint + both clocks) showed zero excess at every length. + +At negotiated length 4096 the midpoint region still contains both fields, +followed by fill bytes — so the emission itself is unconditional; only its effect +on the datagram length depends on the negotiated length. + +**This corrects the earlier characterisation.** "Upstream replies are the +negotiated length plus 8 bytes", or "the normal packet length plus 8", is true +only while the negotiated length is at or below `normal_header`. Between the two +minima the excess falls through +7…+1, and once the negotiated length reaches +`upstream_header` the excess is **0** — the reply is exactly the negotiated +length and the extra field displaces payload. Any surviving "+8 always" wording +elsewhere is stale and wrong. + +**And it is a per-negotiation figure.** Each column of the sweep is a distinct +negotiation whose upstream reply has exactly one length, +`upstream_0_9_1_reply_len`. The +8 / +7…+1 / 0 values are what the difference +works out to across those different negotiations. No negotiation produced a range +of lengths, and no intermediate length between `compatible_reply_len` and +`upstream_0_9_1_reply_len` was ever observed. Wording that reads as "any reply up +to 8 bytes over is acceptable" is equally stale and wrong. + +### Reachable without requesting midpoint + +Against a 0.9.1 server restricted to a single timestamp, a request for StampAt = +both is answered with StampAt = midpoint (already recorded in the timestamp +allowance table). Measured: requested `3/wall` → negotiated `4/wall`, reply 44 +where 36 was implied; requested `3/monotonic` → same; requested `3/both` → +negotiated `4/both`, reply 44, no excess. + +### Value consequences for a positional decoder + +With Clock = monotonic the first 8 bytes of the midpoint region hold the wall +value, so a decoder that expects one midpoint field reads a +nanoseconds-since-epoch magnitude into its monotonic slot. Measured with the +upstream 0.9.1 client itself (`--tstamp=midpoint --clock=monotonic`): reported +server monotonic timestamps of 1786437872024631833 and 1786437872225745750 ns — +about 56.6 years, not process uptime — with the run completing normally and +reporting 72 bytes sent against 88 bytes received for two packets. + +**Limit of what this supports.** The misread is verified; a general correction is +not. Once the negotiated length reaches `upstream_header` the two forms are the +same size: + +``` +conforming single-clock: [mono][payload...] +upstream 0.9.1: [wall][mono][payload...] +``` + +and nothing in the datagram tells them apart. A client that unconditionally read +the second 8-byte region as monotonic would, against a conforming server, be +reading ordinary payload. The specifications therefore permit a correction only +where the dual-field form is otherwise identifiable, and record the equal-length +case as ambiguous rather than inventing a heuristic for it. + +### Version comparison — black-box measured + +Same harness, same parameters, negotiated length 0, received-stats = both: + +| Build | midpoint + wall | midpoint + monotonic | midpoint + both | +|-------|-----------------|----------------------|-----------------| +| 0.9.0 | 36 bytes, one field | 36 bytes, one field, process-relative value | 44 bytes, two fields | +| 0.9.1 | 44 bytes, two fields | 44 bytes, two fields | 44 bytes, two fields | +| tested post-0.9.1 tree | 44 bytes, two fields | 44 bytes, two fields | 44 bytes, two fields | + +0.9.0 emits exactly the negotiated fields; 0.9.1 emits the dual-field form; the +tested post-0.9.1 tree behaves like 0.9.1. The full matrix and the length sweep +were re-run against the post-0.9.1 build and matched 0.9.1 in every cell. + +This is a measurement. It says which bytes each build put on the wire, and +nothing about why. + +### Deliberate or defect — source-assisted historical conclusion + +**Labelled contaminated-side judgement. Not an independent observation, and not +load-bearing.** Contaminated-side investigation of upstream's release history +indicates that the 0.9.0-to-0.9.1 difference measured above is an **unintended +regression** rather than a designed protocol feature. Nothing in upstream's +published release notes or documentation refers to the behavior, and none of the +six post-0.9.1 upstream commits changes it. + +Three things about this paragraph: + +1. The measured 0.9.0-versus-0.9.1 difference is stated separately, above, and + stands entirely on its own. +2. That measurement is *consistent with* this judgement but does not establish + it. Intent is not observable from the wire. +3. The contaminated-side evidence supporting the judgement was retained on the + contaminated side and deliberately did not cross the boundary. Only the + conclusion appears here. + +No requirement in either specification depends on this paragraph. A reader who +disregards it loses nothing normative. + +**Classification:** SERVER SPEC CORRECTION (server spec Sections 9.2, 11.3.1, +21.1, 21.2, 23.4, 24) **and** CLIENT SPEC CORRECTION (client spec Sections 8.5, +19.14). diff --git a/docs/protocol/IRTT_CLIENT_PROTOCOL_SPEC.md b/docs/protocol/IRTT_CLIENT_PROTOCOL_SPEC.md index d53fb34..cae5bf0 100644 --- a/docs/protocol/IRTT_CLIENT_PROTOCOL_SPEC.md +++ b/docs/protocol/IRTT_CLIENT_PROTOCOL_SPEC.md @@ -1,8 +1,23 @@ # IRTT-Compatible Client Protocol Specification -**Version:** 1.1-verified -**Date:** 2026-04-28 -**Verified against:** irtt 0.9.1, protocol version 1, macOS Darwin 25.3.0 arm64 +- **Version:** 1.3-verified +- **Date:** 2026-08-11 +- **Verified against:** irtt 0.9.1 release, protocol version 1, macOS Darwin + 25.3.0 / 25.5.0 arm64 + +**Changes in 1.2:** corrections and clarifications arising from server-side +verification. See Sections 19.3, 19.9, 19.10, 19.13, 19.14 and 19.15, and the +companion `IRTT_SERVER_PROTOCOL_SPEC.md`. + +**Changes in 1.3:** the reply-length tolerance of Sections 8.5 and 19.14 is +narrowed from "accept any longer reply" to the one verified single-clock midpoint +case, and stated as a single additional accepted length — +`upstream_0_9_1_reply_len` exactly — rather than a `+1..+8` range; the normal +minimum and expected reply size is stated as `compatible_reply_len` rather than +the negotiated `Length` value alone; the monotonic-only decoding ambiguity is +recorded rather than papered over with a heuristic; the DSCP wire encoding is +separated from user-facing notation (Section 10.8); and the stale suggestion that +servers reject a protocol-version mismatch is removed (Section 6.2). --- @@ -25,11 +40,12 @@ as interoperability requirements, consistent with RFC 2119. ## 2. Scope - **Client-only implementation.** Server implementation is out of scope unless - server behavior is needed to understand client interoperability. + server behavior is needed to understand client interoperability. Server + behavior is specified separately in `IRTT_SERVER_PROTOCOL_SPEC.md`, which + shares this document's wire format. - **Compatible with existing IRTT servers** running protocol version 1. - This specification describes protocol and measurement behavior; CLI design is not prescribed. -- Server implementation is out of scope. --- @@ -187,8 +203,11 @@ A client session progresses through the following externally observable states: extracted from the reply payload. The client stores the connection token. Transitions to Active Test (or Completed if the no-test flag was set). - An open reply with the close flag set, **when the client did NOT set - Close in its request**: the server rejected the session (e.g., protocol - version mismatch). Transition to Failed. + Close in its request**: the server rejected the session. Transition to + Failed. Note that upstream 0.9.1 servers were never observed to produce + such a rejection — in particular they do **not** reject a mismatched + protocol version this way (see below and Section 19.13) — so a client MUST + handle the shape without depending on any server actually emitting it. - An open reply with the close flag set, **when the client DID set Close in its request** (no-test mode): the server acknowledged the open+close. The connection token is zero. Transition to Completed. @@ -203,12 +222,21 @@ A client session progresses through the following externally observable states: The client MUST reject (as an error) the following server parameter changes: - Duration increased beyond what was requested. - Length increased beyond what was requested. - - Interval reduced below a safety floor (see Section 19.9 for verification). + - Interval reduced below a safety floor of **1 second**. This rejection + applies in loose mode as well as strict mode. + [**Verified 2026-08-11** — see Section 19.9.] - **Protocol version check:** The client MUST verify that the server's protocol version matches exactly. Protocol version 1 is the current version. Mismatch MUST cause the session to fail. + The client cannot rely on the server to detect the mismatch. Upstream 0.9.1 + servers do **not** reject a mismatched protocol version: they rewrite it to 1, + open a normal session, and return `ProtocolVersion = 1` with the Close flag + clear. Version checking is therefore effectively client-side only. + [**Verified 2026-08-11** — see Section 19.13 and + `IRTT_SERVER_PROTOCOL_SPEC.md` Section 7.5.] + ### 6.3 Active Test - **Entry:** Open reply received and parameters accepted. @@ -276,7 +304,7 @@ parameters). | Echo Request (Test Packet) | Client → Server | Active test | Yes | Carry test payload with sequence number | | Echo Reply | Server → Client | Active test | Yes | Return test payload with timestamps and stats | | Close Request | Client → Server | Closing phase | Yes | Signal session end | -| Close Reply | Server → Client | Closing phase | No | Acknowledge close (server MAY send this) | +| Close Reply | Server → Client | Closing phase | No | Not part of normal operation. Upstream servers send nothing in response to a close; such a datagram is tolerated by clients but has no demonstrated purpose in protocol version 1 | --- @@ -445,7 +473,103 @@ set (see Section 19.3 — not yet verified by black-box testing). - Received Window (8 bytes) — if negotiated, contains 64-bit received bitmap - Timestamp fields — according to negotiated stamp_at and clock settings -**Payload:** Server-filled according to negotiation. +**Payload:** Server-filled according to negotiation. The client MUST NOT rely on +the payload's content: server fill is applied as a continuous stream, so a reply +payload does not necessarily begin at the start of the configured fill pattern. + +**Reply length:** The normal, expected size of an echo reply is +`compatible_reply_len` — the larger of the negotiated Length and the mandatory +field block, defined below. That, and not the negotiated `Length` value on its +own, is the minimum: the client MUST reject a reply *shorter* than +`compatible_reply_len`. (The two differ whenever the negotiated Length is smaller +than the field block, including the common case of a negotiated Length of 0.) +There is exactly one verified case in which a reply is *longer*, and the required +tolerance is scoped to it — see below. This is **not** a general rule that +overlong replies are acceptable; outside that one case, a client MUST keep +applying its ordinary strict validation to unexpectedly sized datagrams. + +**The one case: dual-field midpoint from upstream 0.9.1.** When midpoint +timestamps are negotiated, upstream 0.9.1 servers emit **two** midpoint timestamp +fields on the wire — the midpoint wall field followed by the midpoint monotonic +field, in that order — **even when the negotiated clock selects only one clock**. +A single-clock midpoint reply therefore carries one 8-byte field more than the +negotiated parameters imply. (This describes bytes on the wire only; it implies +nothing about how any server represents a midpoint internally.) + +The extra field enlarges the reply's *minimum* size; it does **not** add 8 bytes +to every reply. Writing `normal_header` for the field block the negotiated +parameters imply and `upstream_header` for that block plus one 8-byte timestamp +field: + +``` +normal_header = negotiated field block +upstream_header = normal_header + one 8-byte timestamp field + +compatible_reply_len = max(negotiated_length, normal_header) +upstream_0_9_1_reply_len = max(negotiated_length, upstream_header) +``` + +Three distinct quantities, not to be conflated: `negotiated_length` is the value +of the negotiated Length parameter; `normal_header` is the negotiated mandatory +field block; `compatible_reply_len` is the actual normal packet length implied by +both. + +So the excess is +8 while the negotiated length is at or below `normal_header`, +shrinks through +7…+1 between the two minima, and is **0** once the negotiated +length reaches `upstream_header` — there the extra field displaces 8 bytes of +payload instead of lengthening the datagram. Those figures are values the +difference takes **across different negotiations**. For any one set of negotiated +Params both quantities are fixed numbers, and the upstream reply has exactly one +length. + +**Required tolerance.** When the negotiated parameters are StampAt = Midpoint +with a single Clock, a conforming client MUST accept a reply whose length is +**exactly** `upstream_0_9_1_reply_len`. It MUST NOT require the extra field to be +present, and MUST NOT treat its presence as an error. + +Where `upstream_0_9_1_reply_len > compatible_reply_len`, that form is identifiable +by length alone and the difference between the two values is between 1 and 8 +bytes. This is one extra accepted length, **not** a `+1..+8` window: a client MUST +NOT accept an arbitrary length in that span, and MUST reject any length strictly +between `compatible_reply_len` and `upstream_0_9_1_reply_len` just as it rejects +any other unexpected length. + +**Accepted echo reply lengths, in full.** Compute both values once from the +negotiated Params. A conforming client then accepts exactly these lengths: + +| Received `packet_len` | Condition | Outcome | +|---|---|---| +| `== compatible_reply_len` | always | **accepted** — the normal exact form | +| `== upstream_0_9_1_reply_len` | single-clock Midpoint **and** `upstream_0_9_1_reply_len > compatible_reply_len` | **accepted** — the identifiable upstream midpoint compatibility form | +| any other value | — | **rejected** | + +When `upstream_0_9_1_reply_len == compatible_reply_len` the two rows collapse: the +packet is accepted at the normal exact length, the wire form is ambiguous, and no +generic monotonic correction is applied (see below). + +**Values read by a positional decoder.** The extra field is last, and field +offsets are derived from the negotiated parameters: + +- **Clock = wall:** the negotiated field comes first, so the value read is + correct. +- **Clock = monotonic:** the first 8-byte region holds the *wall*-domain value, so + an ordinary positional decoder reads a nanoseconds-since-epoch magnitude into + its monotonic slot instead of a process-relative one. + +**Do not correct for that blindly.** Once the negotiated length reaches +`upstream_header`, the upstream form and a conforming single-clock form produce +datagrams of the same size, and nothing distinguishes them — the second 8-byte +region is a timestamp under one form and ordinary payload under the other. A +client MAY take its monotonic value from the second region **only when the +dual-field form is otherwise identifiable** — that is, when +`upstream_0_9_1_reply_len > compatible_reply_len` for the negotiated Params and +the datagram length is exactly `upstream_0_9_1_reply_len`. In the equal-length +regime (`upstream_0_9_1_reply_len == compatible_reply_len`) the value SHOULD +be recorded as ambiguous rather than guessed. Either way the packet MUST still be +accepted, and a client that applies no correction MUST NOT fail on the resulting +implausible value. See the server specification, Section 11.3.1. +[**Verified 2026-08-11**, extended **2026-08-11** with a negotiated-length sweep +— see Section 19.14.] ### 8.5.1 Close Request Packet @@ -505,8 +629,10 @@ default to zero on the receiving end). Unknown parameter tags MUST be silently ignored by the receiver. -The maximum serialized parameter buffer size is believed to be 128 bytes -(see Section 19.10 for verification). +There is **no protocol limit** on the size of a received parameter payload; a +receiver MUST tolerate a payload larger than any it would itself produce. +Allocating at least 128 bytes for the parameter buffer is a safe local +implementation choice, not a protocol rule (see Section 19.10). **ReceivedStats enum values:** @@ -548,7 +674,14 @@ A conforming client MUST set `ProtocolVersion` to 1 in its open request. A conforming client MUST verify that the server's returned protocol version matches exactly. If there is a mismatch, the client MUST abort the session. -The server will also set the Close flag if there is a version mismatch. + +**Do not depend on the server to signal the mismatch.** Upstream 0.9.1 servers +accept any value in the `ProtocolVersion` parameter — including 0, 2, a negative +value, or the tag being absent entirely — rewrite it to 1, create a normal +session, and return an open reply with the Close flag clear and a non-zero +token. A server MAY reject an unknown version by replying with the Close flag +set and a zero token, but existing servers do not. +[**Verified 2026-08-11** — see Section 19.13.] ### 9.2 Unknown Fields and Extensions @@ -628,10 +761,18 @@ The following configuration parameters affect protocol behavior: - **Effect:** Sets the TOS/Traffic Class byte in the IP header for both client and server packets during the active test phase. - **Default:** 0 (best effort). -- **Valid values:** 0-255 (full TOS byte value). The DSCP field occupies - the upper 6 bits; the lower 2 bits (ECN) are typically zero. - Common values: 0 (best effort), 0xb8 (EF), 0xa0 (CS5), 0x20 (CS1). - [**Verified 2026-04-28** — see BLACKBOX_VERIFICATION_REPORT.md Finding B.] +- **Wire encoding (protocol fact):** the DSCP parameter carries the **raw IP TOS + / Traffic Class byte**, range **0–255** — not a 6-bit codepoint. The + differentiated-services codepoint occupies the upper 6 bits of that byte and + the lower 2 (ECN) are typically zero, so Expedited Forwarding is **0xb8 + (184)** on the wire. Other common values: 0 (best effort), 0xa0 (CS5), 0x20 + (CS1). [**Verified 2026-04-28** — see BLACKBOX_VERIFICATION_REPORT.md + Finding B.] +- **Out of scope — user-facing notation.** Whether an implementation's command + line and configuration accept a 0–63 codepoint (shifting it left by two to + produce the wire byte) or the raw 0–255 byte is a presentation decision for + that project, to be made and implemented separately. It is not a protocol + fact and this specification does not prescribe it. - **Protocol:** The server echoes packets with the negotiated DSCP value. The server MAY disallow DSCP and reset it to 0. - **Observed behavior:** DSCP/TOS is applied only to echo request and echo @@ -696,9 +837,14 @@ The following configuration parameters affect protocol behavior: - **Client receive timestamp:** SHOULD be captured as close as possible to the receive system call return, before any packet processing. - **Server timestamps:** Determined by the negotiated StampAt parameter: - - Receive: captured when the server receives the packet. - - Send: captured just before the server sends the reply. - - Midpoint: the average of receive and send times. + - Receive: the server's arrival instant for the request. + - Send: the server's departure instant for the reply. + - Midpoint: the average of the two. + + A client can rely on the pair bracketing the server's handling of that request, + and on receive never being later than send. It cannot observe, and MUST NOT + assume, anything about where a given server takes those readings internally or + how much of its own send and receive path falls outside them. ### 11.2 Clock Types @@ -857,22 +1003,46 @@ When the 64-bit received window is present in a reply: - A set bit means the server received that packet; a clear bit means it did not. -The window may not be valid for all replies. Bit 0 (LSB) is always set -for valid windows, since it represents the current packet which was -received. [**Verified 2026-04-28** — see Section 19.7.] Implementations -SHOULD treat a window value of 0 as potentially invalid and avoid using -it for loss classification. +Bit 0 (LSB) is always set in a window emitted by a server, since it represents +the current packet which was received. [**Verified 2026-04-28** — see +Section 19.7.] Implementations SHOULD treat a window value of 0 as potentially +invalid and avoid using it for loss classification. Note that upstream 0.9.1 +servers never emit 0 when the window is negotiated at all. +[**Verified 2026-08-11** — see Section 19.15.] + +**A window of `0x1` carries no historical information.** Upstream 0.9.1 servers +reset the window to `0x1` — not merely fail to update it — whenever a request +arrives with a sequence number lower than the highest already seen, or with a +gap of 64 or more from the previous request. `0x1` does **not** mean the previous +63 packets were lost; it means no useful prior received-history is represented in +the window after that transition. A single reordered packet discards everything +the window knew about the previous 63 sequence numbers, and later packets rebuild +it from scratch. A client that reads `0x1` literally will misclassify up to 63 +earlier packets as lost upstream after one reordering event. + +A conforming client SHOULD treat a window value of `0x1` on any reply other than +the first of the session as "no historical information available" rather than as +"the server received nothing earlier." +[**Verified 2026-08-11** — see Section 19.15 and +`test-vectors/SERVER_BEHAVIORAL_VECTORS.md` Section 1.] The client processes the received window to classify previous packets: - If a packet was not received by the client but the window says the server received it → lost downstream. - If a packet was not received by the client and the window says the server did not receive it → lost upstream. -- If the window cannot confirm (e.g., more than 64 packets ago, or window - was invalid) → generic loss. +- If the window cannot confirm (e.g., more than 64 packets ago, or the window + carries no history) → generic loss. The window has a maximum lookback of 63 packets (bits 1-63). +**Scope note.** The paragraphs above state what the window field *represents* on +the wire, which is what was verified. The downstream/upstream loss +classification built on top of it is a client-side interpretation question that +has not been audited to the same standard here, and it is scheduled for its own +review on the implementation side. Treat the classification rules as a starting +point rather than as verified protocol semantics. + ### 12.9 Statistics For RTT, send delay, receive delay, and IPDV, a conforming client SHOULD @@ -935,13 +1105,24 @@ no receive or send timestamp fields may also be present. ### 13.7 Close Flag in Echo Reply -If the server sets the Close flag in an echo reply, the client SHOULD -process the reply normally (record measurements) and then close the -connection. This is defensive handling for the possibility that the -server can forcibly end a session mid-test. +A server MAY forcibly end a session mid-test by setting the Close flag (0x04) on +an otherwise complete echo reply. This is the only in-band mechanism for a +server-initiated close; there is no standalone server close packet. -**Note:** This behavior has not been directly verified by black-box -testing (see Section 19.3). Implementations SHOULD handle it defensively. +A client MUST handle this case. On receiving an echo reply with the Close flag +set, the client MUST stop the test, MUST NOT send further test packets for that +session, and SHOULD report the partial results gathered so far as a successful +(if truncated) run rather than as an error. + +Whether the close-flagged reply's own measurement is recorded is an +implementation choice. Upstream 0.9.1 clients **discard** it: the run is +terminated on the "server closed" condition before the measurement is recorded. + +The session token is invalid from that moment; a close request the client sends +afterwards will be silently dropped by the server, which is normal. + +[**Verified 2026-08-11** — see Section 19.3 and +`IRTT_SERVER_PROTOCOL_SPEC.md` Section 15.2.] --- @@ -962,7 +1143,8 @@ testing (see Section 19.3). Implementations SHOULD handle it defensively. | Send error during test | SHOULD abort, report partial results. | | Receive error during test | SHOULD abort, report partial results. | | Individual malformed packet during test | SHOULD abort. | -| Short reply (shorter than negotiated length) | SHOULD abort. | +| Short reply (shorter than `compatible_reply_len`, the normal compatible reply length — not merely shorter than the negotiated Length) | SHOULD abort. | +| Reply at an unexpected length (any length other than `compatible_reply_len`, or the single-clock midpoint form at exactly `upstream_0_9_1_reply_len`) | SHOULD abort. See Section 8.5. | | Stamp-at mismatch in reply | SHOULD abort. | | Clock mismatch in reply | SHOULD abort. | | Unexpected sequence number | SHOULD abort. | @@ -1331,18 +1513,32 @@ combinations. See BLACKBOX_VERIFICATION_REPORT.md Section 19.2 for the complete table. The minimum is 16 bytes (no stats, no timestamps) and the maximum default is 76 bytes (both+both+both+HMAC). -### 19.3 Server Close During Test — OPEN / Verification Required +### 19.3 Server Close During Test — ✅ RESOLVED -**Status:** Not tested. Hard to trigger reliably with black-box testing. +**Status:** Verified 2026-08-11, irtt 0.9.1, macOS arm64. -**Question:** Can the server set the Close flag on an echo reply mid-test -(e.g., due to exceeding a duration limit)? If so, how should the client -respond? +**Result:** Yes. A server that enforces a maximum test duration answers the +first echo request arriving after its hard deadline (the configured maximum +duration plus a 2-second grace, measured from the session's first echo request) +with a complete, ordinary echo reply that additionally has the Close flag set, +and removes the session at the same moment. Measured: a 1-second maximum +produced a close-flagged reply 3.01 s after the first echo; a 2-second maximum +produced one at 4.06 s. -**Compatibility recommendation:** The client SHOULD check for the Close -flag in echo replies and, if set, process the reply normally (record -measurements) and then terminate the session. This is defensive handling -for a scenario that has not been directly observed in black-box testing. +A conforming client never reaches this deadline, because it stops sending at the +negotiated duration. The mechanism exists to bound sessions from clients that +ignore the negotiation. + +Observed upstream 0.9.1 client reaction: it discards the close-flagged reply's +measurement, terminates the run with a "server closed connection" condition, +prints the partial results, and exits with a success status. + +No other server condition was observed to set the Close flag on an echo reply: +idle expiry, shutdown, rate limiting, oversize requests and authentication +failure were each exercised and none produced one. + +See Section 13.7, `IRTT_SERVER_PROTOCOL_SPEC.md` Section 15.2, and +`captures/server-close.pcapng` (frame 36 carries flags `0x06`). ### 19.4 Varint Encoding Compatibility — ✅ RESOLVED @@ -1357,8 +1553,8 @@ verified. See BLACKBOX_VERIFICATION_REPORT.md Section 19.4. **Status:** Verified 2026-04-28, irtt 0.9.1, macOS arm64. **Result:** HMAC-MD5 is computed over the entire packet buffer (including -payload) with the 16-byte HMAC field zeroed. Independently verified with -Python for all four packet types (open request, echo request, echo reply, +payload) with the 16-byte HMAC field zeroed. Cross-checked with a separately +written Python script for all four packet types (open request, echo request, echo reply, close request). The spec's test vectors (18.1 and 18.2) also verify correctly at 92-byte packet size. @@ -1379,6 +1575,12 @@ window values: seq 0 → 0x01, seq 1 → 0x03, seq 2 → 0x07. A window value of 0 can be used as the invalid sentinel. See BLACKBOX_VERIFICATION_REPORT.md Section 19.7. +**Extended 2026-08-11.** Upstream 0.9.1 was subsequently confirmed never to emit +0 when the window is negotiated at all, and the far more common `0x1` was found +to be a *valid* window carrying no history — which is a distinct condition from +the 0 sentinel and must not be treated the same way. See Section 19.15 and +Section 12.8. + ### 19.8 Open Reply Without HMAC When Client Expects HMAC — ✅ RESOLVED **Status:** Verified 2026-04-28, irtt 0.9.1, macOS arm64. @@ -1388,24 +1590,43 @@ The client times out with `[OpenTimeout] no reply from server`. Same behavior for wrong key and no key. See BLACKBOX_VERIFICATION_REPORT.md Section 19.8. -### 19.9 Minimum Restricted Interval Safety Floor — PARTIALLY RESOLVED +### 19.9 Minimum Restricted Interval Safety Floor — ✅ RESOLVED -**Status:** Partially tested 2026-04-28. +**Status:** Verified 2026-08-11, irtt 0.9.1, macOS arm64. -**Result:** In strict mode, the client rejects ANY server parameter change -with a non-zero exit code. In loose mode, changes are accepted with a -warning. The exact safety floor for server-decreased intervals could not -be tested without a cooperative malicious server. See -BLACKBOX_VERIFICATION_REPORT.md Section 19.9. +**Result:** The safety floor is **1 second**, and it is enforced in loose mode +as well as strict mode. -### 19.10 Maximum Parameter Buffer Size — PARTIALLY RESOLVED +The test uses an ordinary, unmodified server: a server restricts the negotiated +interval to at most one quarter of its idle timeout, so an idle timeout below +4 seconds forces a reduction below 1 second for a client requesting 1 second. -**Status:** Partially tested 2026-04-28. +| Server idle timeout | Returned interval | Client outcome | +|---------------------|-------------------|----------------| +| 2 s | 500 ms | Aborts: reduction below 1 s — strict **and** loose | +| 3 s | 750 ms | Aborts: reduction below 1 s — strict **and** loose | +| 4 s | 1 s (no reduction for a 1 s request) | Proceeds | -**Result:** Largest observed parameter payload was 52 bytes (all params -including 24-character ServerFill). Upper bound estimate is ~82 bytes. The -128-byte limit appears safe. See BLACKBOX_VERIFICATION_REPORT.md -Section 19.10. +A reduction to exactly 1 second (requesting 2 s against a 4 s idle timeout) is +treated as an ordinary server restriction: rejected in strict mode, accepted in +loose mode. + +See `test-vectors/SERVER_BEHAVIORAL_VECTORS.md` Section 4.4. + +### 19.10 Maximum Parameter Buffer Size — ✅ RESOLVED (as a local choice) + +**Status:** Verified 2026-08-11, irtt 0.9.1, macOS arm64. + +**Result:** There is no protocol limit on the size of a received parameter +payload. An open request carrying over 200 bytes of parameters (80 repetitions +of an unknown tag) was accepted and answered normally by an upstream server. + +The largest payload a version-1 peer can legitimately *produce* remains small: +the observed maximum from a real client was 52 bytes, and the theoretical +maximum with all tags at their widest encodings plus a 32-byte ServerFill string +is under 90 bytes. Allocating at least 128 bytes for the parameter buffer is a +safe **implementation choice**, not a protocol requirement, and a receiver +SHOULD tolerate larger payloads rather than rejecting them. ### 19.11 Field Ordering Verification — ✅ RESOLVED @@ -1429,6 +1650,97 @@ negative value? This edge case requires server processing time to exceed the raw client-measured RTT, which is extremely unlikely under normal conditions. +### 19.13 Server-Side Protocol Version Enforcement — ✅ RESOLVED + +**Status:** Verified 2026-08-11, irtt 0.9.1, macOS arm64. + +**Result:** Upstream servers do not enforce the protocol version. Open requests +carrying `ProtocolVersion` values of 0, 2 and −1, and requests omitting the tag +entirely, each produced a session-creating open reply with flags `0x03` +(Open|Reply), a non-zero token, and `ProtocolVersion = 1` in the returned +parameters. + +**Consequence for the client:** the mismatch check in Sections 6.2 and 9.1 is +the only line of defence and MUST be implemented. A client that assumes the +server will reject it will silently run a test against an incompatible peer. + +### 19.14 Reply Length Longer Than Negotiated — ✅ RESOLVED + +**Status:** Verified 2026-08-11, irtt 0.9.1, macOS arm64. Extended 2026-08-11 +with a negotiated-length sweep and a version comparison. + +**Result:** Yes, in one specific, verified case. When midpoint timestamps are +negotiated with a single clock (wall only or monotonic only), upstream 0.9.1 +servers emit both midpoint fields on the wire — wall field then monotonic field — +which is one 8-byte field more than the negotiated parameters imply. Measured +across all combinations of received-stats and clock with midpoint timestamps; +confirmed at the application layer by an upstream client reporting 36-byte +packets while receiving 44-byte datagrams, and completing normally. + +**The excess is bounded by the ordinary length rule, not constant.** Reply length +is the larger of the negotiated length and the field block; the extra field +enlarges only the field block. Measured with received-stats = both, midpoint, +wall (`normal_header` 36, `upstream_header` 44): negotiated lengths 0, 16, 32, +36 → 44-byte replies; 37 → 44 (+7); 40 → 44 (+4); 43 → 44 (+1); 44 → 44 (0); +45, 64, 1024, 4096 → exactly the negotiated length (0). At 4096 the midpoint +region still contains both fields — the extra field displaces payload rather than +extending the datagram. Any claim that upstream replies are simply "the +negotiated length plus 8" is wrong outside the first regime. Each cell of that +sweep is a different negotiation with a single reply length; the +7 / +4 / +1 +figures are differences between negotiations, never alternative lengths available +within one. + +**Scope of the requirement.** For single-clock midpoint the client MUST accept a +reply at **exactly** `upstream_0_9_1_reply_len`. Where that value exceeds +`compatible_reply_len` the form is identifiable by length alone and the two +values differ by between 1 and 8 bytes — but that is one additional accepted +length, not a `+1..+8` window, and a length strictly between the two MUST be +rejected like any other unexpected length. That is the whole of the tolerance: it +does **not** generalise to accepting arbitrary overlong echo replies, and the +client MUST continue to reject replies *shorter* than `compatible_reply_len` — +the normal compatible reply length, not merely the negotiated `Length` value — +and to validate every other unexpectedly sized datagram strictly. It MUST NOT +require the extra field to be present. + +**The monotonic-only ambiguity.** Once the negotiated length reaches +`upstream_header`, the upstream and conforming forms are the same size and are not +distinguishable from the datagram alone. A client MUST NOT unconditionally read +the second 8-byte region as the monotonic value whenever midpoint + monotonic is +negotiated — against a conforming server that region is ordinary payload. Apply a +correction only when the dual-field form is otherwise identifiable, meaning +`upstream_0_9_1_reply_len > compatible_reply_len` and the datagram is exactly +that length; otherwise record the value as ambiguous, and accept the packet +either way. See the server specification, Section 11.3.1. + +A server that emits only the negotiated fields is also fully interoperable: a +server built to do so was driven by the upstream 0.9.1 client for wall-only, +monotonic-only and dual-clock midpoint configurations and every run completed +cleanly. Upstream 0.9.0 behaves that way on the wire; the dual-field form appears +in 0.9.1 and is unchanged in the tested post-0.9.1 tree. + +### 19.15 Received Window After Reordering or a Large Gap — ✅ RESOLVED + +**Status:** Verified 2026-08-11, irtt 0.9.1, macOS arm64. + +**Result:** The window is **reset to `0x1`**, not merely left unshifted, in both +of these cases: + +- a request whose sequence number is lower than the highest already seen + (reordering, or a late gap fill); +- a request whose sequence number is 64 or more ahead of the previous one. + +The count still increments. A gap of exactly 63 preserves the older packet in +bit 63; a gap of 64 discards everything. + +The window value 0 was never observed; bit 0 is always set. 32-bit sequence +wraparound is handled as ordinary modular arithmetic and does not disturb the +window. + +Full input → output tables are in +`test-vectors/SERVER_BEHAVIORAL_VECTORS.md` Section 1, with a capture in +`captures/server-recv-window.pcapng`. See Section 12.8 for the classification +consequence. + --- ## 20. Non-Goals diff --git a/docs/protocol/IRTT_SERVER_PROTOCOL_SPEC.md b/docs/protocol/IRTT_SERVER_PROTOCOL_SPEC.md new file mode 100644 index 0000000..be73762 --- /dev/null +++ b/docs/protocol/IRTT_SERVER_PROTOCOL_SPEC.md @@ -0,0 +1,1929 @@ +# IRTT-Compatible Server Protocol Specification + +**Version:** 1.1-verified +**Date:** 2026-08-11 +**Compatibility baseline:** IRTT 0.9.1, protocol version 1 +**Verification platform:** macOS Darwin 25.5.0 arm64 + +--- + +## 1. Status + +This document describes the externally observable behavior required for an +independent **server** implementation that interoperates with existing IRTT +(Isochronous Round-Trip Tester) clients speaking protocol version 1. + +It is the server-side counterpart to `IRTT_CLIENT_PROTOCOL_SPEC.md`. The two +documents share a wire format; this one adds only what a server must do that a +client does not. Where a rule is already stated in the client specification +(magic bytes, flag bits, field ordering, parameter encoding, HMAC computation), +this document references it rather than restating it, and states only the +server-specific reading of it. + +This document does **not** describe upstream source structure, internal +algorithms, module layout, or implementation organization. It is intended for a +clean-room implementation that has never seen the upstream source code. + +The key words "MUST", "MUST NOT", "REQUIRED", "SHOULD", "SHOULD NOT", +"RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as interoperability +requirements, consistent with RFC 2119. + +Five distinct levels of statement are used throughout, and they are not +interchangeable. Nothing in one level implies anything at another. + +| Marker | Meaning | +|--------|---------| +| **Protocol requirement** | Required for interoperability with existing clients. Stated with MUST / MUST NOT / SHOULD. Every one rests on observed behavior. | +| **Upstream 0.9.1 behavior** | What the reference server was observed to do on the wire. A compatible implementation is **not** obliged to reproduce it unless a separate protocol requirement says so. | +| **Server policy** | A configurable or arbitrary choice of the reference server, including its defaults. A compatible implementation may choose differently, and in several cases should. | +| **Robustness recommendation** | Advice for the clean implementation's own resilience or safety. It is **not** an interoperability requirement and no existing client can observe whether it was followed. | +| **Source-assisted historical conclusion** | A judgement about upstream's release history reached with contaminated-side access to upstream material. Never load-bearing: no requirement in this document depends on one. Used once, in Section 11.3.1. | + +Where the reference server's own behavior would be a poor choice for a new +implementation — an unbounded resource policy, a fatal reaction to a single +failed send — this document records the observation and the recommendation +separately, and does not turn either into a protocol rule. + +--- + +## 2. Scope and Compatibility Baseline + +- **Server-only.** Client behavior appears only where a server's obligations + depend on it. +- **Protocol version 1**, as implemented by IRTT 0.9.0 and 0.9.1. +- The baseline for every unqualified statement is the **upstream 0.9.1 + release**. +- Two other upstream builds were exercised, and each is named explicitly + wherever its results are used: + - the **0.9.0 release**, for the version comparison in Sections 11.3.1 and + 21.1; + - a build of the **upstream development tree as it stood six commits past the + 0.9.1 tag**, covered separately in Section 21.2. Those six commits are + *not* part of the 0.9.1 release, and no result measured against them + redefines the 0.9.1 baseline. +- Development-era (0.1.x) protocol variants are out of scope and are not + interoperable. +- CLI design, logging, output formats, configuration file syntax, and process + supervision are out of scope. + +Implementations are not required to use any specific internal architecture, +data structure, naming convention, concurrency model, threading or task model, +socket ownership arrangement, or source organization. This document deliberately +prescribes none of those: they are decisions for the implementing project, not +protocol facts. Only externally observable behavior is normative. + +--- + +## 3. Clean-Room Provenance + +This specification was produced on the contaminated side of the clean-room +boundary described in `../clean-room/CLEANROOM_NOTES.md`. It records only +externally observable behavior: + +- packet bytes on the wire, in both directions; +- accept / drop / reply decisions; +- session state transitions inferred from subsequent packet exchanges; +- negotiated parameter values returned to the client; +- timing boundaries measured by observation; +- documented command-line policy knobs and their observable effects. + +Every normative statement in this document is supported by a black-box +experiment against a running reference server, by a packet capture in +`captures/`, or by published end-user documentation. Statements that could only +be derived from internal structure have been excluded. + +**One labelled exception.** Section 11.3.1 carries a single sentence of +*historical* judgement — that a particular measured difference between 0.9.0 and +0.9.1 is an unintended regression rather than a designed feature — which was +reached with contaminated-side access to upstream material. It is labelled where +it appears, it is not normative, and no requirement anywhere in this document +depends on it. The measured 0.9.0-versus-0.9.1 wire difference is stated +separately and stands on its own. The contaminated-side evidence behind the +judgement was retained on the contaminated side and deliberately not carried +into this document; see `../clean-room/CLEANROOM_NOTES.md`. + +**Naming.** The parameter names used throughout (ProtocolVersion, Duration, +Interval, Length, ReceivedStats, StampAt, Clock, DSCP, ServerFill) are taken +from upstream's **published** documentation of its client's machine-readable +output, not from source inspection. No other upstream identifier appears in this +document. + +--- + +## 4. Roles and Directionality + +A server: + +1. listens on one or more UDP endpoints; +2. accepts session-open requests and assigns session tokens; +3. echoes test packets, annotating them with reception statistics and + timestamps as negotiated; +4. releases session state on client request, on policy limits, or on idle + expiry. + +Every packet a server emits **MUST** have the Reply flag (0x02) set. Every +packet a server accepts **MUST** have the Reply flag clear. + +**Protocol requirement.** A reply without the Reply flag is not merely ignored +by existing clients — upstream 0.9.1 clients abort the run and terminate +abnormally when a reply arrives with the flag clear. This was confirmed by +driving an upstream client against a deliberately non-conforming server. + +--- + +## 5. Transport and Packet Envelope + +### 5.1 Transport + +UDP only. There is no TCP fallback, no connection setup at the transport layer, +and no fragmentation logic in the protocol itself. + +The conventional server port is **2112** (server policy: the reference server's +default bind is all addresses on port 2112). + +### 5.2 Address Families + +A server MUST support IPv4. IPv6 support is RECOMMENDED and is provided by the +reference server. + +**Upstream 0.9.1 behavior.** Binding to an unspecified address creates two +independent listeners, one per address family, on the same port. Tokens are +scoped to the listener that issued them: a token issued by the IPv4 listener is +not recognized by the IPv6 listener on the same port, and a request bearing it +is dropped as an unknown token. See `captures/server-session-identity.pcapng` and +the vectors file for the equivalent same-family case. + +**Protocol requirement.** Session tokens need only be unique within the scope in +which they are looked up. A compatible implementation MAY use one global table +or one per listener; clients never observe the difference except through the +drop behavior above. + +### 5.3 Reply Addressing + +**Protocol requirement.** A reply MUST be sent to the exact source address and +port of the request, and MUST be sent **from** the address and port to which the +request was addressed. Clients use connected UDP sockets and silently discard +datagrams whose source does not match the address they sent to. + +**Server policy.** On a multi-homed host with a listener bound to an unspecified +address, choosing the reply source address by routing table rather than by the +request's destination address will break clients on the non-preferred address. +The reference server offers an opt-in policy to pin the reply source address to +the request's destination address, and its documentation recommends binding to +explicit addresses instead. On single-homed and loopback configurations both +modes were observed to produce correct source addresses. + +### 5.4 Packet Admission + +This section states which datagrams a server answers and which it discards. It +describes outcomes, not an internal processing pipeline: any implementation that +produces these accept/discard decisions is conformant, whatever order it +evaluates them in. + +**Discard conditions.** Each of the following was independently observed to +cause the datagram to be **silently discarded**, with no reply of any kind and +no change to the state of any live session: + +| Condition | Notes | +|-----------|-------| +| Datagram too short to carry magic + flags | With an HMAC key configured, the authentication field is part of the floor, so the floor rises by 16 bytes | +| First three bytes are not the protocol magic | | +| Any flag bit above the four defined bits is set (flags > 0x0F) | | +| The Reply flag is set | A server never answers a datagram carrying it | +| Authentication mismatch of any kind | Section 14; includes the flag/key mismatch in either direction | +| A non-open datagram too short to carry the token | | +| A non-open datagram whose token matches no live session | Section 8 | +| A non-open datagram whose token matches a session bound to a different source endpoint | Section 8; the session itself is unaffected | +| An echo datagram too short to carry the sequence number | | +| An echo datagram exceeding a configured maximum packet length | Section 9.4, server policy | +| An echo datagram arriving with no rate allowance left | Section 9.3, server policy | +| An open datagram whose parameter payload is malformed or out of range | Section 7.2 | + +**Protocol requirement.** Every rejection is a **silent discard**. The protocol +defines no error reply, no reset, and no NACK. A client distinguishes "rejected" +from "lost" only by timing out. A compatible server MUST NOT answer any of the +conditions above. + +**Observable precedence.** Only two ordering relationships are externally +visible, and both were verified directly. Both describe how an *admissible* +datagram is interpreted; neither weakens any discard condition in the table +above. + +1. **For an otherwise admissible datagram with Open set, Open interpretation + takes precedence over Close interpretation and over an echo-shaped body.** A + datagram with both Open and Close set is a *no-test* open (Section 16), not a + close; and an echo-shaped datagram that also carries the Open flag is answered + as an open, with the bytes that would have been the token and sequence number + parsed as Open parameter data rather than as those fields. This precedence is + about interpretation only. It does **not** override the independent rejection + conditions above: a datagram carrying the Open flag is still silently + discarded if the Reply flag is also set, if any undefined flag bit is set, if + authentication fails, or if its Open parameter payload is malformed or out of + range. A compatible server SHOULD behave the same way, so that a client's + retransmitted open is never mistaken for an echo. +2. **An authentication failure is indistinguishable from an unknown token.** A + datagram carrying a *valid* token with a bad MAC produces exactly the same + silence as one carrying a token that was never issued (Section 14.2). A + compatible server MUST preserve this: no reply behavior may reveal whether a + token exists, for a datagram that failed authentication. + +**Request bytes beyond the sequence number carry no meaning to a server.** The +midpoint/receive-send exclusivity and clock-consistency rules that govern +timestamp fields apply to *replies*; they have no effect on inbound traffic. An +echo request carrying arbitrary bytes in the region a reply would use for +timestamps is accepted and answered normally. + +### 5.5 Minimum and Maximum Datagram Sizes + +Observed minimum accepted request sizes (no authentication): + +| Request kind | Minimum accepted datagram | +|--------------|---------------------------| +| Open (with empty parameter payload) | 4 bytes | +| Close | 12 bytes | +| Echo | 16 bytes | + +With authentication enabled, add 16 bytes to each figure. + +**Upstream 0.9.1 behavior.** Trailing bytes beyond the fields a request kind +requires are ignored: + +- an echo request longer than the negotiated length is accepted and answered + with a reply of the *negotiated* length, not the request length; +- a close request with trailing bytes is accepted and closes the session. + +The only length ceiling applied to requests is the configured maximum packet +length (Section 19). + +--- + +## 6. Server Lifecycle + +### 6.1 Startup + +**Server policy.** The reference server resolves its bind specifications, +creates one listener per resolved address/family, and serves until interrupted. +A failure on any single listener causes the whole server to stop. + +### 6.2 Shutdown + +**Upstream 0.9.1 behavior.** Termination is abrupt from the client's +perspective. No close, reset, or notification packet of any kind is emitted for +live sessions when the server shuts down; subsequent client packets simply go +unanswered. Verified by signalling a running server mid-session and observing +that nothing arrived at the client before the socket went silent. + +**Protocol requirement.** None. Clients MUST already tolerate an unresponsive +server. A compatible implementation MAY emit close replies to live sessions on +shutdown; upstream clients ignore unsolicited close replies harmlessly +(Section 15.1). + +### 6.3 Reply Send Failures + +**Upstream 0.9.1 behavior.** If sending a reply fails with a non-transient +socket error, the listener terminates and the whole server process exits. This +is reachable in ordinary use: a client may negotiate a packet length larger than +the host's maximum outbound datagram size, and the first echo reply then ends the +server. See Section 22. + +**Protocol requirement:** none. A reply that is never sent is +indistinguishable, to the client, from a reply that was lost in the network. No +client can observe how the server reacted internally. + +**Robustness recommendation (clean implementation).** Because the outcome above +is reachable from ordinary client traffic and takes every other session down +with it, a clean server should treat a per-packet send failure as a per-packet +event: drop that reply, keep the listener and the process running, and continue +serving other sessions. This is a resilience choice for the implementing +project, not something compatibility requires. + +--- + +## 7. OPEN Negotiation + +### 7.1 Open Request + +**Direction:** client → server. + +**Flags:** Open (0x01) set, Reply (0x02) clear, HMAC (0x08) set iff the client +is authenticating, Close (0x04) set iff the client requests *no-test* +(Section 16). + +**Layout:** magic (3) + flags (1) + [authentication (16)] + parameter payload. + +**Protocol requirement.** There is **no** token field in an open request. The +parameter payload begins immediately after the fixed header (and the +authentication field, when present). + +The parameter payload uses the tag/value encoding defined in the client +specification, Section 8.6. A server MUST decode it with the same rules. + +### 7.2 Parameter Acceptance and Rejection + +A server parses the whole parameter payload. Observed acceptance rules: + +| Condition | Observed result | +|-----------|-----------------| +| Empty parameter payload | Accepted; all parameters default to zero, then restriction applies | +| Unknown tag with a well-formed value | Silently ignored; the rest of the payload is still parsed | +| Truncated tag or truncated value varint | **Dropped, no reply** | +| Varint overflow | **Dropped, no reply** | +| Duration present and ≤ 0 | **Dropped, no reply** | +| Interval present and ≤ 0 | **Dropped, no reply** | +| Duration or Interval absent (implicitly 0) | Accepted | +| ReceivedStats outside 0–3 | **Dropped, no reply** | +| StampAt outside 0–4 | **Dropped, no reply** | +| Clock outside 1–3 | **Dropped, no reply** | +| Length any value, including negative | Accepted (see Section 7.3) | +| DSCP any value, including negative or > 255 | Accepted (see Section 7.3) | +| ServerFill string longer than 32 bytes | **Dropped, no reply** | +| ServerFill declared length exceeding the remaining payload | **Dropped, no reply** | +| Parameter payload far larger than any real request (200+ bytes of unknown tags tested) | Accepted | + +**Protocol requirement.** Unknown tags MUST be silently ignored. A malformed +parameter payload MUST NOT produce a reply — a rejection reply would let an +unauthenticated peer distinguish a live server from a firewall drop, which is +contrary to the design of the authenticated mode. + +**Clarification.** Upstream servers impose no 128-byte ceiling on *incoming* +parameter payloads. A payload of over 200 bytes composed of unknown tags was +accepted and answered normally. Any such limit in an implementation is a local +buffer-sizing choice, not a protocol rule. + +### 7.3 Server Restriction of Parameters + +A server MAY reduce or replace requested parameters. The restricted values are +what it returns in the open reply and what it enforces thereafter. + +Observed restriction rules of upstream 0.9.1 (each is **server policy**, gated +on the corresponding configuration knob): + +| Parameter | Restriction applied | +|-----------|--------------------| +| ProtocolVersion | Always rewritten to 1 (see Section 7.5) | +| Duration | Clamped down to the configured maximum test duration, when one is set | +| Interval | Raised to the configured minimum interval, when one is set | +| Interval | Then clamped down to (idle timeout ÷ 4), when an idle timeout is set | +| Length | Clamped down to the configured maximum packet length, when one is set | +| StampAt | Reduced per the configured timestamp allowance (Section 11.4) | +| DSCP | Forced to 0 when DSCP is disallowed or unsupported on the socket | +| ServerFill | Replaced with the server's own default fill descriptor when the requested fill does not match the allow-list | + +Measured confirmations of the interval cap: idle timeout 4 s → maximum interval +1 s; 8 s → 2 s; 20 s → 5 s; 40 s → 10 s; 60 s → 15 s. + +**Interoperability note.** The interval cap can *reduce* an interval below what +the client asked for. Upstream clients enforce a hard floor of **1 second** on a +server-reduced interval and abort the session — in both strict and loose mode — +when the returned interval is below it. A reduction to exactly 1 s is accepted +in loose mode. A server whose idle timeout is below 4 s will therefore be unable +to serve clients requesting a 1 s interval. Measured: idle timeout 2 s reduces +1 s → 500 ms and the client aborts; idle timeout 3 s reduces to 750 ms and the +client aborts; idle timeout 4 s produces no reduction at all for a 1 s request. + +**Server-policy hazard.** If the configured minimum interval is larger than +(idle timeout ÷ 4), the negotiated interval is the smaller of the two while the +enforced minimum remains the larger. A fully conforming client then sends at the +negotiated interval and is rate-limited (Section 9.3). Observed with a 5 s +minimum interval and an 8 s idle timeout: negotiated interval 2 s, but only the +first request in each 5 s window was answered. + +### 7.4 Open Reply + +**Direction:** server → client. + +**Flags:** Open (0x01) set, Reply (0x02) set, HMAC (0x08) set iff the server is +authenticating, Close (0x04) set iff the request had it set (no-test) — see +Section 7.5 for the one case where upstream does *not* set it. + +**Layout:** magic (3) + flags (1) + [authentication (16)] + token (8) + +restricted parameter payload. + +**Protocol requirement.** + +- The token field MUST be present in an open reply. +- For a session-creating open, the token MUST be non-zero. Clients treat a zero + token without the Close flag as an error. +- The parameter payload MUST contain the **restricted** values — the values the + server will actually enforce. Clients compare them field-by-field against + what they requested and, by default, abort on any difference. +- Parameters whose restricted value is zero are omitted from the payload + entirely; the client reconstructs them as zero. +- The open reply is **not** padded to the negotiated length. Its size is + header + token + parameter payload. + +**Upstream 0.9.1 behavior.** The open reply preserves whatever flags the request +carried, adding Reply. There is no separate "reject" reply shape. + +### 7.5 Protocol Version — Important Deviation + +**Upstream 0.9.1 behavior.** The server does **not** reject a mismatched +protocol version. Any value in the ProtocolVersion parameter — 0, 2, −1, or the +tag omitted entirely — is silently rewritten to 1, a normal session is created, +and the open reply carries ProtocolVersion = 1 with the Close flag **clear**. +Verified by black-box test: requests with ProtocolVersion 0, 2 and −1 each +produced a session-creating open reply. + +Consequence: version mismatch is detected **client-side only**, by the client +comparing the returned version against its own. A client speaking a future +version would send its own version, receive 1, and abort locally. + +**Protocol requirement.** A compatible server MUST return ProtocolVersion = 1 in +the open reply so that version-1 clients accept the session. A compatible server +MAY additionally reject unknown versions by replying with the Close flag set and +a zero token; existing clients treat such a reply as a rejection and abort. Both +behaviors are interoperable with version-1 clients; only the second is +interoperable with a hypothetical future client that relies on server-side +detection. + +**Do not assume** that an upstream server will tell you your version is wrong. + +### 7.6 Repeated and Retransmitted Opens + +**Upstream 0.9.1 behavior.** Every open request creates a new, independent +session with a fresh token. There is no deduplication of retransmitted opens and +no per-endpoint session limit: + +- two opens from the *same* source port yield two distinct tokens, and **both** + remain usable simultaneously; +- an open request sent twice (as a client retransmission would) yields two + distinct tokens; the client keeps only the token from the reply it processes, + and the other session is orphaned; +- 1000 sequential opens from a single source endpoint all succeeded, all tokens + were distinct, and the first, middle and last were all still usable + afterwards. + +**Resource consequence (upstream policy).** An orphaned session created by a +retransmitted open never carries an echo request, and — under upstream's idle +policy — a session that has never carried an echo request **never expires** +(Section 18.2). On a lossy path, open retransmissions therefore accumulate +permanent state. + +**Protocol requirement:** none. Clients cannot observe orphaned sessions, and +nothing about the behavior above is required for interoperability. + +**Robustness recommendation (clean implementation).** Bound the lifetime of +sessions that have never carried an echo request, so that open retransmissions +cannot accumulate. See Section 18.3. + +--- + +## 8. Session Identity and Tokens + +### 8.1 What Identifies a Session + +**Protocol requirement.** A post-open request is bound to a session by the +**token**, and the server MUST additionally verify that it arrived from the +same source endpoint the session was opened from. + +Observed identity components, all of which must match: + +| Component | Must match | +|-----------|-----------| +| Token (64-bit, little-endian) | Yes | +| Source IP address | Yes | +| Source UDP port | Yes | +| IPv6 zone/scope, when present | Yes | +| Address family / listener | Yes — tokens are not shared between the IPv4 and IPv6 listeners of the same port | + +Observed outcomes: + +| Case | Result | +|------|--------| +| Correct token, original endpoint | Served normally | +| Correct token, **different source port**, same IP | **Dropped, no reply.** Session left intact and still usable from the original port | +| Correct token, different address family | **Dropped, no reply** (different listener, unknown token there) | +| Unknown token | **Dropped, no reply** | +| Zero token | **Dropped, no reply** (treated as any other unknown token) | +| Token of a session already closed | **Dropped, no reply** | +| **Close** request bearing a valid token from a foreign source port | **Dropped**; the session is **not** closed | + +The last row matters: endpoint binding protects the close path as well as the +echo path. A capture of these cases is in +`captures/server-session-identity.pcapng`. + +**Practical consequence.** Any NAT rebinding, source-port change, or address +change during a session terminates it from the client's point of view — all +subsequent packets go unanswered until the idle timeout reclaims the state. +There is no rebinding or migration mechanism in protocol version 1. + +### 8.2 Token Semantics + +| Property | Value | +|----------|-------| +| Width | 64 bits | +| Encoding | Little-endian in the token field | +| Assigned by | Server, during open | +| Zero token | Reserved in practice: clients reject a zero token in a session-creating open reply, and a zero token is used in no-test replies (Section 16) | +| Uniqueness | MUST be unique among the server's live sessions in the scope where lookups occur | +| Lifetime | From open until close, policy limit, or idle expiry | +| Reuse | A token that has been released MAY be issued again later; no client-visible barrier exists | + +**Protocol requirement.** The server MUST echo the session token back in every +reply it sends for that session. + +**Upstream 0.9.1 client behavior (informative).** Upstream clients do **not** +validate the token in echo replies — a reply carrying a wrong token was +accepted and recorded. Do not rely on this: other implementations may validate, +and matching the token is required by this specification. + +**Token generation is deliberately unspecified.** A compatible implementation +MAY choose any generation strategy that satisfies the uniqueness requirement. +Because the token is the only thing standing between an off-path attacker and a +usable session on a non-authenticated server, generating tokens from a +cryptographically strong source is RECOMMENDED. + +--- + +## 9. ECHO Processing + +### 9.1 Echo Request + +**Direction:** client → server. + +**Flags:** Open clear, Reply clear, Close clear, HMAC set iff authenticating. + +**Layout:** magic (3) + flags (1) + [authentication (16)] + token (8) + +sequence number (4) + client padding to the negotiated length. + +**Protocol requirement.** A server MUST read only the token and the 32-bit +little-endian sequence number from a request. Everything after the sequence +number is opaque to the server: clients place zeroed placeholders there so the +request reaches the negotiated length, and a server MUST NOT interpret them. + +**Upstream 0.9.1 behavior.** The datagram length of a request is not compared +against the negotiated length. A 16-byte request on a session that negotiated +64 bytes is accepted and answered with a 64-byte reply. A request *longer* than +the negotiated length is likewise accepted, and the reply is still the +negotiated length. + +### 9.2 Echo Reply + +**Direction:** server → client. + +**Flags:** Reply (0x02) set; HMAC (0x08) set iff authenticating; Close (0x04) +set only in the server-initiated close case (Section 15.2). Open MUST be clear — +upstream clients abort if an echo reply has the Open flag set. + +**Layout (fields in the order defined in the client specification, +Section 8.1.3):** + +``` +magic (3) | flags (1) | [auth (16)] | token (8) | sequence (4) + | [received count (4)] -- iff negotiated stats include "count" + | [received window (8)] -- iff negotiated stats include "window" + | [timestamp fields] -- per negotiated StampAt and Clock + | payload padding to the negotiated length +``` + +**Protocol requirement.** + +- The sequence number MUST be copied from the request unchanged. +- The token MUST be copied from the request unchanged. +- The reply length MUST be at least the length of the fields it contains, and + SHOULD be exactly the negotiated length when that is larger. +- When the negotiated length is smaller than the field block (including the + common case of a negotiated length of 0), the reply is the field block's + length. Measured: a session negotiating length 0 or 20 with full statistics + and dual timestamps on both clocks produced 60-byte replies. + +Writing `normal_header` for the block of fields the negotiated parameters imply: + +``` +compatible_reply_len = max(negotiated_length, normal_header) +``` + +For StampAt = Midpoint with a single negotiated clock, upstream 0.9.1 emits one +8-byte timestamp field more than the negotiated parameters imply, which changes +the reply length in exactly the cases where the field block wins. See +Section 11.3.1. + +**Upstream 0.9.1 behavior — reply length is capped by the local interface MTU.** +The negotiated length is echoed back to the client unchanged, but the actual +reply datagram is truncated to the detected MTU of the listener's interface. +Measured on a 1500-byte-MTU interface: negotiated 1501, 2000, 3000 and 8000 all +produced 1500-byte replies, while the open reply still reported the requested +value. Upstream clients treat a reply shorter than the negotiated length as a +fatal error, so this configuration fails on the first echo. See Section 22. + +### 9.3 Rate Limiting + +**Server policy.** The reference server enforces the negotiated minimum send +interval with a token-bucket allowance: + +- a burst allowance of *N* requests is available immediately (default N = 5); +- allowance is replenished at one request per configured minimum interval and + is capped at *N*; +- a request arriving with no allowance is **dropped without a reply**. + +Measured with a 100 ms minimum interval and a burst of 5: a blast of 12 requests +produced exactly 5 replies; after a 600 ms pause, the next 3 spaced requests +were all answered. With a burst of 1, a blast of 4 produced 1 reply. + +**Protocol requirement.** A rate-limited request MUST NOT be answered and MUST +NOT affect the reception statistics (Section 10). Measured: the received count +advanced 1, 2, 3, 4, 5 across the accepted burst and continued 6, 7, 8 for the +later requests — dropped requests were not counted. + +### 9.4 Maximum Request Length + +**Server policy.** When a maximum packet length is configured, a request whose +**datagram length** exceeds it is dropped without a reply. Measured with a +maximum of 64: 16-, 32- and 64-byte requests were answered with 64-byte replies; +65- and 100-byte requests were dropped. + +The maximum applies to echo requests only. An open request larger than the +maximum was still accepted and answered. + +If the configured maximum is below the minimum echo field block, **every** echo +request is dropped while opens still succeed. Measured with a maximum of 10: the +session opened, negotiated a length of 10, and every 16-byte echo request was +dropped. + +--- + +## 10. Sequence Handling and Received Statistics + +This section is the most behaviorally distinctive part of the server and is +specified strictly as input → observed output. The reference implementation's +observable semantics differ from a naive reading of "a 64-packet reception +bitmap", and clients depend on the observed behavior. + +### 10.1 Received Count + +| Property | Value | +|----------|-------| +| Field width | 32 bits, little-endian | +| Present when | Negotiated ReceivedStats includes "count" (values 1 or 3) | +| Initial value | 0 at session creation; the first accepted request is answered with 1 | +| Scope | Per session. A new session starts at 0 again | +| Counts | Every echo request that reaches the reply stage | +| Does **not** count | Requests dropped for bad magic, bad flags, bad authentication, unknown token, endpoint mismatch, oversize, or rate limiting | +| Duplicates | **Counted.** A repeated sequence number increments the count | +| Reordered / late requests | **Counted** | +| Close and open requests | Not counted | + +**Protocol requirement.** The count reported in a reply MUST include the request +being answered. + +**Wraparound.** The field is 32 bits. Behavior at 2^32 requests in one session +was not observed and is listed in Section 23. + +### 10.2 Received Window + +| Property | Value | +|----------|-------| +| Field width | 64 bits, little-endian | +| Present when | Negotiated ReceivedStats includes "window" (values 2 or 3) | +| Bit 0 (LSB) | The request being answered | +| Bit *k* | The request with sequence number (current − *k*), for 1 ≤ *k* ≤ 63 | +| Set bit | That sequence number was received by the server in this session | +| Initial value | 0 at session creation | + +**Observed upstream 0.9.1 semantics.** The table below is a summary of the +measured input → output vectors in +`test-vectors/SERVER_BEHAVIORAL_VECTORS.md` Section 1, expressed as a relation +between two values a client can see on the wire: the window in the previous +reply and the window in the next one. It is a description of observed output, +not a prescription of how to compute it — any method producing these values is +conformant. + +Let `S` be the sequence number of the incoming request, `L` the sequence number +carried by the previous accepted request of the session, `W` the window value +carried by the previous reply (`W = 0` before the session's first request), and + +``` +Δ = (S − L) mod 2^32 -- unsigned 32-bit difference +``` + +Three cases account for every observed vector: + +| Condition | Resulting window | +|-----------|------------------| +| Δ = 0 | `W \| 1` | +| 1 ≤ Δ ≤ 63 | `(W << Δ) \| 1` | +| Δ ≥ 64 | `0x1` | + +Read through the cases a client actually encounters: + +| Situation | Δ | Resulting window | +|-----------|---|------------------| +| First request of a session | any | `0x1` (because `W` is 0) | +| In order, no gap | 1 | `(W << 1) \| 1` | +| Duplicate of the most recent sequence number | 0 | `W`, unchanged | +| Gap ahead, within range | 2 … 63 | `(W << Δ) \| 1` | +| Gap ahead of 64 or more | ≥ 64 | **`0x1`** | +| **Out of order / late arrival** | large, because the unsigned difference wraps | **`0x1`** | + +**The last two rows are the critical ones.** Because the difference is unsigned, +a sequence number *lower* than the previous one produces a very large Δ and +therefore falls into the third case. A late or reordered request does **not** +set a historical bit; it resets the window to `0x1`. The information that +earlier packets were received is lost, and it is not recovered by later packets +— the window rebuilds from scratch. + +**Protocol requirement.** Bit 0 is always set in a window a server emits. A +compatible server MUST set bit 0. Consequently a window value of 0 is never +produced by upstream 0.9.1 when the window field is negotiated at all; clients +MAY continue to treat 0 defensively as "no information". + +**Client-facing consequence, and why this must be stated.** A window of `0x1` is +a *valid* window that carries no information about earlier packets. It does +**not** mean the previous 63 sequence numbers were lost; it means no useful +prior received-history is represented in the window after the transition that +produced it. A client that reads `0x1` literally will classify up to 63 earlier +packets as lost upstream after a single reordering event or after any gap of 64 +or more. Implementations of either side SHOULD treat `0x1` on a non-first packet +as "no historical information available" rather than "nothing earlier was +received". + +This is a statement about what the window *represents*. How a client turns +window bits into an upstream/downstream loss classification is a separate +question, out of scope here, and is scheduled for its own audit on the clean +side; nothing in this section should be read as settling it. + +**Sequence wraparound.** The comparison is modular over 32 bits, so a session +that crosses the u32 boundary continues normally. Measured with sequence numbers +4294967293, 4294967294, 4294967295, 0, 1: windows advanced +`0x1, 0x3, 0x7, 0xf, 0x1f` with counts 1…5, exactly as for a non-wrapping run. + +**Sequence numbers need not start at 0.** A session whose first request carries +sequence 100 behaves identically to one starting at 0: window `0x1`, then `0x3`, +then `0x7`. + +### 10.3 Behavioral Vectors + +See `test-vectors/SERVER_BEHAVIORAL_VECTORS.md` for the full input → output +tables, and `captures/server-recv-window.pcapng` for a capture of the +gap / duplicate / reorder / far-gap sequence. + +--- + +## 11. Timestamp Semantics + +### 11.1 What Each Field Means + +| Negotiated StampAt | Fields emitted | Instant represented | +|--------------------|----------------|---------------------| +| None (0) | none | — | +| Send (1) | send wall and/or send monotonic | The server's departure instant for this reply | +| Receive (2) | receive wall and/or receive monotonic | The server's arrival instant for this request | +| Both (3) | receive and send fields | Both of the above | +| Midpoint (4) | midpoint fields | The arithmetic mean of the receive and send instants | + +The externally relevant property is the *relationship* between the two instants, +not where in a server's internals they are taken. The pair must bracket the +server's own handling of the request as tightly as the implementation can manage, +so that the difference between them is a usable estimate of server processing +time and the client's round-trip correction is not systematically wrong. How +closely a given implementation can bracket that interval is not externally +observable; see Section 23.7. + +**Protocol requirement.** The receive instant MUST NOT be later than the send +instant for the same reply. Measured server-side processing durations on +loopback were on the order of a few microseconds. + +### 11.2 Clock Encodings + +| Clock | Field | Encoding | +|-------|-------|----------| +| Wall | wall fields | Signed 64-bit little-endian, **nanoseconds since the Unix epoch** | +| Monotonic | monotonic fields | Signed 64-bit little-endian, **nanoseconds since an arbitrary server-chosen origin** | + +**Observed monotonic origin.** The origin is fixed for the life of the server +process and is shared by every session and every listener of that process. Two +sessions opened seconds apart on different address families reported monotonic +values differing by exactly the elapsed wall time between them, and values +tracked the server process's uptime. A client therefore MUST NOT assume the +origin is per-session, and MUST NOT compare monotonic values against its own +clock — only differences between two server monotonic values from the same +server are meaningful. + +**Protocol requirement.** A compatible server MAY choose any monotonic origin, +provided it is stable for as long as any session it reports into may live. + +### 11.3 Absent Fields + +A timestamp field that is not negotiated is simply not present in the reply; it +is not zero-filled. Field presence is positional — receivers derive offsets from +the negotiated parameters, not from the packet. + +### 11.3.1 Upstream 0.9.1 — Dual-Field Midpoint Wire Representation + +**Black-box observed.** For StampAt = Midpoint, upstream 0.9.1 emits the midpoint +as **two** 8-byte timestamp fields — the midpoint **wall** field followed by the +midpoint **monotonic** field, in that order — **even when the negotiated Clock +selects only one clock**. The midpoint occupies that fixed two-field, 16-byte +region whenever it is present at all, so a single-clock midpoint reply carries +one 8-byte field more than the negotiated parameters imply. + +This is a statement about bytes on the wire. It says nothing about how upstream +represents a midpoint internally, and nothing in this section depends on any +such model. + +**Which negotiations are affected.** + +| StampAt | Clock | Midpoint region on the wire | +|---------|-------|-----------------------------| +| Midpoint (4) | Wall (1) | wall + monotonic — **one field more than negotiated** | +| Midpoint (4) | Monotonic (2) | wall + monotonic — **one field more than negotiated** | +| Midpoint (4) | Both (3) | wall + monotonic — matches the negotiated layout | + +Receive-only, send-only and both-timestamp modes are **not** affected: each of +those emits exactly the fields the negotiated Clock implies, for all three Clock +values. Measured for all 15 StampAt × Clock combinations; only the two +single-clock midpoint rows deviate. + +A client can reach this configuration without asking for midpoint timestamps: a +server restricted to a single timestamp answers a request for both receive and +send with Midpoint (Section 11.4), so a Wall-only client that requested StampAt = +Both also receives the dual-field form. + +**Effect on reply length.** The extra field enlarges the *minimum* reply length; +it does **not** add 8 bytes to every reply. Reply length follows the ordinary +rule of Section 9.2 — the larger of the negotiated length and the field block — +applied to a field block one timestamp field larger. Writing `normal_header` for +the block of fields the negotiated parameters imply: + +``` +normal_header = negotiated field block +upstream_header = normal_header + one 8-byte timestamp field + +compatible_reply_len = max(negotiated_length, normal_header) +upstream_0_9_1_reply_len = max(negotiated_length, upstream_header) +``` + +The observable excess is therefore `upstream_0_9_1_reply_len − +compatible_reply_len`, which takes one of three forms depending on the +negotiated length: + +| Negotiated length | Upstream 0.9.1 reply | Excess over a compatible server | +|-------------------|----------------------|---------------------------------| +| ≤ `normal_header` | `upstream_header` | **+8** | +| between `normal_header` and `upstream_header` | `upstream_header` | **+7 … +1**, shrinking as the negotiated length rises | +| ≥ `upstream_header` | exactly the negotiated length | **0** — the extra field displaces 8 bytes of payload | + +Read the excess column across rows, not within one session: for any *single* set +of negotiated Params the upstream reply has exactly one length, +`upstream_0_9_1_reply_len`. The +8, +7…+1 and 0 values are what that one length +works out to under different negotiations. No negotiation admits a range of +lengths. + +Measured with ReceivedStats = Both, StampAt = Midpoint, Clock = Wall +(`normal_header` 36, `upstream_header` 44): + +| Negotiated length | 0 | 16 | 32 | 36 | 37 | 40 | 43 | 44 | 45 | 64 | 1024 | 4096 | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| Reply datagram | 44 | 44 | 44 | 44 | 44 | 44 | 44 | 44 | 45 | 64 | 1024 | 4096 | +| Excess | +8 | +8 | +8 | +8 | +7 | +4 | +1 | 0 | 0 | 0 | 0 | 0 | + +The same pattern was measured for ReceivedStats = None, Count and Window, with +the two minima shifting accordingly (24/32, 28/36 and 32/40 respectively). At a +negotiated length of 4096 the midpoint region still contains both fields, +followed by fill — so the dual-field emission is unconditional, and only its +effect on the *total datagram length* depends on the negotiated length. + +**Correction to earlier material.** Any statement that an upstream reply is "the +negotiated length plus 8 bytes", or "the normal packet length plus 8", is wrong +as a general claim. It holds only in the first regime above. For sufficiently +large negotiated packets the datagram length is exactly the negotiated length, +and the excess field is visible only as displaced payload. + +**Effect on the values a positional decoder reads.** Field offsets are derived +from the negotiated parameters, and the extra field is the last field before the +payload. The two forms compare as follows: + +``` +Clock = Wall + conforming single-clock form: [midpoint_wall][payload...] + upstream 0.9.1 form: [midpoint_wall][midpoint_mono][payload...] + +Clock = Monotonic + conforming single-clock form: [midpoint_mono][payload...] + upstream 0.9.1 form: [midpoint_wall][midpoint_mono][payload...] +``` + +- **Clock = Wall:** the negotiated field comes first in both forms, so an + ordinary positional decoder reads the correct value. +- **Clock = Monotonic:** an ordinary positional decoder reads the first 8-byte + region — which upstream fills with the **wall-domain** value — as the + negotiated monotonic timestamp. Black-box verified with the upstream client: a + 0.9.1 client requesting monotonic-only midpoint reported server monotonic + timestamps of approximately 1.79 × 10¹⁸ ns — a nanoseconds-since-epoch + magnitude, not a process-relative one — and completed the test without error. + +**The two forms are not generally distinguishable.** Once the negotiated length +reaches `upstream_header`, the equality + +``` +upstream_0_9_1_reply_len == compatible_reply_len +``` + +holds, and the conforming and upstream forms produce datagrams of **identical +length**: + +``` +conforming: [mono][payload...] +upstream: [wall][mono][payload...] +``` + +In that regime nothing in the datagram identifies which form was used. The eight +bytes following the first midpoint region are a monotonic timestamp under the +upstream form and ordinary payload under a conforming one. This ambiguity is +recorded here as a limitation, not resolved: no heuristic for telling the two +apart in the equal-length regime has been established, and this document does not +invent one. + +**Version behavior.** Black-box measured: upstream 0.9.0 emits only the +negotiated midpoint field for a single clock (a monotonic-only midpoint reply +carries a process-relative value at the negotiated length). The dual-field form +appears in 0.9.1 and is **unchanged** in the tested post-0.9.1 development tree +(Section 21). + +**Source-assisted historical conclusion (non-normative).** Contaminated-side +investigation of upstream's release history indicates that the difference between +0.9.0 and 0.9.1 recorded above is an **unintended regression** rather than a +designed protocol feature. This is a judgement reached with access to upstream +material on the contaminated side; it is **not** an independent observation, and +the measured 0.9.0-versus-0.9.1 difference stated above does not by itself +establish intent. Nothing in upstream's published release notes or documentation +refers to the behavior. No requirement below depends on this paragraph, and a +reader who discards it entirely loses nothing normative. + +**Protocol requirement.** + +- A compatible server SHOULD emit exactly the fields implied by the negotiated + StampAt and Clock, including for midpoint. Emitting the conforming form is known + to interoperate: a contaminated-side server built to emit only the negotiated + midpoint field was driven by the upstream 0.9.1 client for Wall-only, + Monotonic-only and dual-clock midpoint configurations, and the client — an + independent implementation — completed every run cleanly. +- A compatible **client** MUST tolerate the verified upstream single-clock + midpoint dual-field form. Specifically, when the negotiated parameters are + StampAt = Midpoint with a single Clock, it MUST accept a reply whose length is + **exactly** `upstream_0_9_1_reply_len`. It MUST NOT require the extra field to + be present, and it MUST NOT treat its presence as an error. +- That tolerance is a **single additional accepted length, not a range**. For any + one set of negotiated Params, `compatible_reply_len` and + `upstream_0_9_1_reply_len` are two fixed numbers, and those are the only two + lengths an echo reply may have. When + `upstream_0_9_1_reply_len > compatible_reply_len` the upstream form is + identifiable by length alone, and the difference between the two values is + between 1 and 8 bytes. The +8, +7…+1 and 0 figures in the excess table earlier + in this section are differences **across different negotiations**; they are not + a range of acceptable lengths within one negotiation. A client MUST NOT apply a + rule of the form "accept anything up to 8 bytes over", and MUST reject a length + strictly between `compatible_reply_len` and `upstream_0_9_1_reply_len` exactly + as it rejects any other unexpected length. +- The tolerance is otherwise **narrow and specific**. It is not a general rule + that replies longer than expected are valid, and it does not license accepting + arbitrary overlong echo replies. A compatible client MUST continue to reject + replies *shorter* than `compatible_reply_len` — the normal compatible reply + length, which is the larger of the negotiated Length and the mandatory field + block, not the negotiated Length on its own — and MUST continue to apply its + ordinary strict validation to every other malformed or unexpectedly sized + datagram. +- A client that negotiates monotonic-only midpoint MAY read its monotonic value + from the second 8-byte region of the midpoint area **only when the dual-field + form is otherwise identifiable** — that is, when + `upstream_0_9_1_reply_len > compatible_reply_len` for the negotiated Params and + the datagram length is exactly `upstream_0_9_1_reply_len`, which happens only + while the negotiated length is below `upstream_header`. A client MUST NOT apply + that correction unconditionally + whenever midpoint and monotonic are negotiated: in the equal-length regime it + would interpret ordinary payload bytes from a conforming server as a timestamp. +- In the equal-length regime a client SHOULD record the value as ambiguous rather + than guess. Whatever it decides, it MUST still accept the packet: a reply is + not to be rejected merely because upstream might have produced the ambiguous + form, and a client that applies no correction MUST NOT fail on the resulting + implausible value. +- All of the above constrains a *client*. None of it is a licence, or a + requirement, for a clean implementation's **server** to emit the extra field. + +**Accepted echo reply lengths, in full.** For a given set of negotiated Params, +compute `compatible_reply_len` and `upstream_0_9_1_reply_len` once. A conforming +client then accepts exactly these lengths and no others: + +| Received `packet_len` | Condition | Outcome | +|---|---|---| +| `== compatible_reply_len` | always | **accepted** — the normal exact form | +| `== upstream_0_9_1_reply_len` | single-clock Midpoint **and** `upstream_0_9_1_reply_len > compatible_reply_len` | **accepted** — the identifiable upstream midpoint compatibility form | +| any other value | — | **rejected** | + +When `upstream_0_9_1_reply_len == compatible_reply_len` the two rows collapse: +the packet is accepted at the normal exact length, the wire form is ambiguous +between `[mono][payload…]` and `[wall][mono][payload…]`, and no generic monotonic +correction is applied. This is behavioral wording; it prescribes no particular +decoder structure. + +### 11.4 Timestamp Allowance (Server Policy) + +The reference server can restrict which timestamp modes it will provide. The +observed mapping from requested StampAt to negotiated StampAt: + +| Requested | Allowance = dual | Allowance = single | Allowance = none | +|-----------|------------------|--------------------|------------------| +| None (0) | 0 | 0 | 0 | +| Send (1) | 1 | 1 | 0 | +| Receive (2) | 2 | 2 | 0 | +| Both (3) | 3 | **4 (midpoint)** | 0 | +| Midpoint (4) | 4 | 4 | 0 | + +Note the substitution: under a single-timestamp allowance, a request for both +receive and send is answered with **midpoint**, not with one of the two. Clients +comparing negotiated against requested parameters will see this as a server +restriction. + +--- + +## 12. Server Timing Relationships + +A server's timestamps exist so a client can separate server processing time from +network time. The relationships a compatible server MUST make derivable: + +``` +server_processing = server_send − server_receive (both from the same clock) +adjusted_rtt = raw_client_rtt − server_processing +send_delay = server_best_receive_wall − client_send_wall +receive_delay = client_receive_wall − server_best_send_wall +``` + +where *best receive* is the receive timestamp when present, otherwise the send +timestamp, and symmetrically for *best send*. For midpoint timestamps both +resolve to the midpoint value, which makes `server_processing` identically zero +— measured: an upstream client using midpoint timestamps reports a server +processing time of 0 s in all four statistics. + +**Protocol requirement.** + +- When both receive and send timestamps are negotiated, they MUST be taken from + the same clock domain, and MUST bracket the server's own handling of that + request. +- One-way delays are only meaningful with externally synchronized wall clocks; + a server has no obligation beyond reporting its own wall clock honestly. +- A server MUST NOT rewrite or smooth timestamps between packets. + +Client-side statistics (IPDV, loss classification, aggregate reporting) are out +of scope here; see the client specification, Section 12. + +--- + +## 13. Server Fill and Payload + +### 13.1 What the Payload Is + +Everything after the last present field in a reply, up to the reply's length, is +payload. Its content carries no protocol meaning; it exists to reach the +negotiated packet length. + +### 13.2 Fill Negotiation + +The ServerFill parameter (tag 9) is a length-prefixed UTF-8 string, maximum 32 +bytes. Observed descriptor forms: + +| Descriptor | Meaning | +|------------|---------| +| absent / empty | Client expresses no preference; the server uses its own configured fill | +| `none` | Do not fill; see Section 13.4 | +| `rand` | Fill with random bytes | +| `pattern:HH…` | Fill with the repeating byte pattern given as hex | + +**Server policy.** The reference server matches the requested descriptor against +an allow-list of glob patterns (default: `rand` only). A descriptor that does +not match is replaced with the server's **default fill descriptor**, which is +returned to the client in the open reply. + +Measured on a default-configuration server: + +| Requested | Negotiated value returned | Actual reply payload | +|-----------|---------------------------|----------------------| +| `rand` | `rand` | random bytes | +| `none` | `pattern:69727474` | repeating `69 72 74 74` | +| `pattern:aabb` | `pattern:69727474` | repeating `69 72 74 74` | +| `bogus` | `pattern:69727474` | repeating `69 72 74 74` | +| 32-byte string | `pattern:69727474` | repeating `69 72 74 74` | +| 33-byte string | — | **open dropped, no reply** | + +**Note.** The descriptor returned on refusal is the server's *default* fill +descriptor and does not necessarily describe the bytes the server will actually +send — a server configured with a different fill still returns the default +descriptor while filling with its configured fill. Clients MUST NOT rely on the +returned descriptor to predict payload bytes. + +With a permissive allow-list, requested descriptors are honored and returned +verbatim: `pattern:aabb` produced `aa bb aa bb …`, `pattern:00` produced zeros, +`pattern:ff00` produced `ff 00 ff 00 …`. An unparseable hex body +(`pattern:zz`) falls back to the server default and returns +`pattern:69727474`. + +### 13.3 Default Fill Bytes and Pattern Phase + +**Upstream 0.9.1 behavior.** The default fill is the repeating four-byte pattern +`69 72 74 74`. The pattern's **phase is not reset per packet or per session**; +it advances continuously as bytes are consumed. Observed across four consecutive +replies with a 7-byte payload on one session: + +``` +seq 0 payload: 72 74 74 69 72 74 74 +seq 1 payload: 69 72 74 74 69 72 74 +seq 2 payload: 74 69 72 74 74 69 72 +seq 3 payload: 74 74 69 72 74 74 69 +``` + +and across separate sessions on the same listener, the phase carried over. + +**Protocol requirement.** A client MUST NOT assume a reply payload begins at the +start of the fill pattern, and MUST NOT use payload bytes for any protocol +purpose. A compatible server MAY reset the phase per packet. + +### 13.4 No Fill + +**Upstream 0.9.1 behavior.** When the effective fill is `none`, no bytes are +written into the payload region. The bytes that appear there are unspecified +residual buffer content. In practice: + +- when the request was at least as long as the reply, the client's own request + payload is returned; +- when the request was shorter, the trailing bytes are residue from earlier + traffic on that listener — observed to include fragments of a previous + packet's parameter payload. + +The reference server's own documentation describes `none` as insecure on public +servers, and its default allow-list excludes it. + +**Protocol requirement.** A compatible server that supports a no-fill mode MUST +NOT emit residual bytes from other clients' traffic. Zero-filling or echoing +only the bytes actually supplied by the requester are both acceptable. + +### 13.5 Interaction with Length and Authentication + +- The payload region is what remains after all present fields. Enabling + authentication consumes 16 bytes of the packet, reducing the payload for the + same negotiated length. +- If the negotiated length is smaller than the field block, there is no payload. +- The fill is applied after all fields are written, so it never overwrites + protocol data. + +--- + +## 14. Authentication (HMAC) + +### 14.1 Mechanism + +When a server is configured with a key, an authentication field of 16 bytes +immediately follows the flags byte in **every** packet in both directions, and +the HMAC flag (0x08) is set. + +The MAC is HMAC-MD5 over the **entire datagram** with the 16-byte +authentication field zeroed, exactly as specified in the client specification, +Section 13.4. A server computes and verifies it identically. + +**Protocol requirement.** A server that is authenticating MUST set the HMAC flag +**and** write a valid MAC in every reply it sends, including echo replies. This +was verified the hard way: a server that authenticated open replies correctly +but omitted the HMAC flag from echo replies caused every echo reply to be +discarded by the upstream client, which then terminated abnormally with no +packets received. + +### 14.2 Observed Behavior on Authentication Failure + +Every case below results in a **silent drop**: no reply of any kind, and no +change to any session's state. + +| Condition (server has a key) | Result | +|------------------------------|--------| +| Request has no HMAC flag and no field | Dropped | +| Request has the HMAC flag, MAC computed with a different key | Dropped | +| Request has the HMAC flag, one bit flipped in the MAC | Dropped | +| Request has the HMAC flag, MAC field all zeros | Dropped | +| Request too short to contain the MAC field (< 20 bytes) | Dropped | +| MAC field truncated (field shortened, rest shifted) | Dropped | +| MAC computed correctly but the HMAC flag cleared | Dropped | +| Valid open, valid echo, valid close | Served normally | + +| Condition (server has **no** key) | Result | +|-----------------------------------|--------| +| Request has the HMAC flag set | Dropped | + +**Protocol requirement.** Authentication failure MUST NOT produce a reply. A +server MUST reject a packet whose HMAC flag disagrees with its own +configuration, in **both** directions of mismatch. MAC comparison SHOULD be +constant-time. + +Applies to all three request kinds: open, echo, and close. An unauthenticated +close request cannot tear down an authenticated session. + +**Indistinguishability.** An authentication failure is externally +indistinguishable from every other drop: a packet bearing a *valid* token with a +bad MAC produces exactly the same silence as one bearing an unknown token. + +**Protocol requirement.** A compatible server MUST NOT allow the result of a +session lookup to leak through its reply behavior for a packet that failed +authentication. Otherwise an unauthenticated peer could probe for live tokens. + +### 14.3 Session Effect + +An authentication failure leaves the session completely untouched. Measured: a +sequence of eight malformed or wrongly-keyed packets against a live session was +followed by a correctly authenticated echo that was served normally, with the +received count advancing only for the accepted packets. + +--- + +## 15. CLOSE Lifecycle + +### 15.1 Client-Initiated Close + +**Request.** Direction client → server. + +**Flags:** Close (0x04) set, Reply clear, Open clear, HMAC set iff +authenticating. + +**Layout:** magic (3) + flags (1) + [authentication (16)] + token (8). Total +12 bytes unauthenticated, 28 bytes authenticated. There is no sequence number +and no payload. + +**Observed server behavior:** + +| Event | Result | +|-------|--------| +| Valid close from the session's endpoint | Session removed. **No reply of any kind is sent.** | +| Trailing bytes after the token | Accepted; session removed | +| Repeated close with the same token | Dropped as an unknown token; no reply | +| Echo after close | Dropped as an unknown token; no reply | +| Close from a foreign source port | Dropped; session **not** removed | +| Close bearing an unknown or zero token | Dropped; no reply | + +**Protocol requirement.** A server MUST accept the close request and release the +session. It MUST NOT be expected to reply, and a compatible server that sends +nothing is fully conformant. + +- **Upstream 0.9.1 sends no reply at all** to a normal client close. Verified in + `captures/server-close-lifecycle.pcapng`, where the close request is the last + packet in the exchange. +- Once the session is gone, a repeated close or a post-close echo is handled the + same way as any unknown or stale token: silently discarded, no reply. +- A standalone Reply|Close datagram is *tolerated* by upstream clients but has no + demonstrated purpose in protocol version 1. As a one-off experiment, an + upstream client was driven against a server that answered close with + magic + flags(Reply|Close) + [auth] + token; the run completed normally with + correct results and nothing in the client's output changed, showing no sign the + reply had been used. That establishes only that such a datagram is harmless. + +**No production standalone close-reply codec is specified here.** Because no +version-1 client has been shown to do anything with such a datagram, this +document does not define one as required — or even as recommended — behavior. An +implementation that wants to emit one is not prohibited from doing so, but it +gains nothing observable and should not treat the shape above as a normative +encoding. + +**The exact point at which the session becomes unavailable** is the moment the +close request is processed. The client has no acknowledgement and therefore no +way to distinguish "close delivered" from "close lost"; a lost close leaves the +session to be reclaimed by idle expiry (Section 18). + +### 15.2 Server-Initiated Close + +**When it happens.** A server signals close by setting the Close flag (0x04) on +an otherwise ordinary **echo reply**. There is no standalone server-initiated +close packet in protocol version 1. + +**Upstream 0.9.1 behavior.** The single observed trigger is the maximum test +duration hard limit: + +- when a maximum test duration is configured, the server clamps the negotiated + duration to it during negotiation; +- it additionally enforces a hard deadline of **maximum duration + 2 seconds**, + measured from the **first echo request** of the session; +- the first echo request arriving after that deadline is answered with a normal, + complete echo reply — correct sequence number, token, statistics and + timestamps — with the Close flag additionally set; +- the session is removed at the same time, so every later request is dropped as + an unknown token. + +Measured: maximum duration 1 s → close flag on the reply at 3.01 s after the +first echo; maximum duration 2 s → close flag at 4.06 s and again at 4.1 s in a +separate run. Captured in `captures/server-close.pcapng` (frame 36 carries +flags `0x06`). + +A conforming client never reaches this deadline, because it stops sending at the +negotiated duration. The mechanism exists to bound sessions from clients that +ignore the negotiated duration. + +**Observed client reaction.** An upstream 0.9.1 client that receives an echo +reply with the Close flag set: + +1. **does not record** that reply's measurement; +2. terminates the run immediately, reporting a "server closed connection" + condition; +3. prints the partial results gathered up to that point and exits with a success + status; +4. sends its own close request for the session. + +**Protocol requirement.** + +- A server that terminates a session mid-test MUST do so by setting the Close + flag on an echo reply; there is no other in-band mechanism. +- The close-flagged reply MUST otherwise be a well-formed echo reply for the + request that triggered it — clients parse it before noticing the flag. +- A server MUST NOT expect an acknowledgement. The client's subsequent close + request will be dropped as an unknown token, which is normal and harmless. +- The close-flagged reply MUST be authenticated when the session is + authenticated. +- The session token MUST be released at the same time; it MUST NOT remain usable + after a close-flagged reply. + +--- + +## 16. No-Test Mode + +A client may open and immediately close a session without running a test, in +order to validate parameters against a server. + +**Request:** an open request with **both** Open (0x01) and Close (0x04) set. + +**Reply:** Open | Reply | Close (0x07, or 0x0F with authentication), a **zero +token**, and the fully restricted parameter payload. + +**Observed properties:** + +| Property | Result | +|----------|--------| +| Session state created | **None.** No token is issued and nothing is retained | +| Token in the reply | All zeros | +| Parameters in the reply | The same restricted values a real open would return | +| Subsequent echo with the zero token | Dropped, no reply | +| Subsequent close with the zero token | Dropped, no reply | +| With authentication | Reply is authenticated normally; flags `0x0F` | +| Cleanup required | None | + +**Protocol requirement.** A server MUST NOT create a usable session for a +no-test open, and MUST return a zero token. The Close flag in the reply here +means "acknowledged and finished", not "rejected"; the distinction is made by +the client from whether *it* set Close in the request. See the client +specification, Sections 6.2 and 8.3. + +--- + +## 17. Malformed, Unexpected and Adversarial Packets + +Compact reference table. "Session effect: none" means the referenced session (if +any) is left fully usable. + +| Request condition | Session state | Auth state | Server response | Session effect | +|---|---|---|---|---| +| Datagram shorter than the fixed header | any | any | drop | none | +| Bad magic | any | any | drop | none | +| Any reserved flag bit set (flags > 0x0F) | any | any | drop | none | +| Reply flag set | any | any | drop | none | +| HMAC flag set, server has no key | any | n/a | drop | none | +| No HMAC flag, server has a key | any | fail | drop | none | +| Wrong / corrupted / zeroed / truncated MAC | any | fail | drop | none | +| Open, truncated parameter varint | n/a | ok | drop | none | +| Open, duration ≤ 0 or interval ≤ 0 | n/a | ok | drop | none | +| Open, out-of-range stats / stamp-at / clock enum | n/a | ok | drop | none | +| Open, server-fill string > 32 bytes or short buffer | n/a | ok | drop | none | +| Open, unknown parameter tags | n/a | ok | **open reply**, tags ignored | new session | +| Open, empty parameter payload | n/a | ok | **open reply** | new session | +| Open, protocol version ≠ 1 | n/a | ok | **open reply with version 1** | new session | +| Open with Close flag (no-test) | n/a | ok | **reply, zero token** | none created | +| Non-open, too short for the token | any | ok | drop | none | +| Non-open, unknown token | — | ok | drop | none | +| Non-open, zero token | — | ok | drop | none | +| Echo, correct token, foreign source port | live | ok | drop | none | +| Close, correct token, foreign source port | live | ok | drop | **none — stays open** | +| Echo, too short for the sequence number | live | ok | drop | none | +| Echo exceeding the configured maximum length | live | ok | drop | none | +| Echo arriving faster than the rate allowance | live | ok | drop | none | +| Echo with trailing bytes past the negotiated length | live | ok | **reply at negotiated length** | normal | +| Echo shorter than the negotiated length | live | ok | **reply at negotiated length** | normal | +| Echo with a duplicate sequence number | live | ok | **reply** | count +1, window unchanged | +| Echo with a lower sequence number than the last | live | ok | **reply** | count +1, window reset to `0x1` | +| Echo after the maximum-duration deadline | live | ok | **reply with Close flag** | session removed | +| Echo after idle expiry deadline | expired | ok | **reply** (served once) | session removed | +| Any request on a closed / expired session | gone | ok | drop | none | +| Close with trailing bytes | live | ok | **accepted**; no reply, as for any close | session removed | +| Echo-shaped datagram with the Open flag set, otherwise admissible | — | ok | treated as an **open**; the token bytes are parsed as parameters | new session if those bytes parse | +| Open flag set together with the Reply flag, or with an undefined flag bit | any | any | drop | none | + +The last row restates the observable precedence of Section 5.4 in its narrow +form: for an otherwise admissible datagram, Open interpretation wins over Close +interpretation and over an echo-shaped body, so an echo-shaped datagram with the +Open flag set produces an open reply rather than an echo reply. That precedence +does not override the drop rows above — a datagram carrying the Open flag is +still dropped when the Reply flag is set, when an undefined flag bit is set, on +an authentication failure, or when its Open parameter payload is malformed or out +of range. A compatible server SHOULD behave the same way, so that a client's +retransmitted open is never mistaken for an echo. + +--- + +## 18. Session Lifetime and Resource Policy + +### 18.1 Lifetime Events + +| Event | Effect | +|-------|--------| +| Open (non-no-test) | Session created, token issued | +| Open with Close (no-test) | No session created | +| Client close from the bound endpoint | Session removed immediately, no reply | +| Maximum test duration exceeded | Close-flagged echo reply, session removed (Section 15.2) | +| Idle expiry | Session removed (Section 18.2) | +| Server shutdown | All sessions gone, no notification | +| Client disappears | Nothing until idle expiry | + +### 18.2 Idle Expiry — Observed Semantics + +**Server policy.** The reference server has a configurable idle timeout, +default **1 minute**, with `0` meaning "never expire". + +**Observed behavior**, measured with a 2-second idle timeout: + +| Gap since the previous accepted echo | First request after the gap | The request after that | +|---|---|---| +| 5.0 s | answered | answered | +| 6.5 s | answered | answered | +| 6.9 s | answered | answered | +| 7.1 s | answered | **dropped** | +| 7.5 s | answered | **dropped** | +| 9.0 s | answered | **dropped** | + +Two things follow, and both are externally observable: + +1. **The effective expiry deadline is the configured idle timeout plus a + 5-second grace period.** Measured boundary between 6.9 s and 7.1 s for a + 2-second timeout. +2. **The first request that arrives after the deadline is still answered**, and + the session is torn down as part of handling it. Only the *next* request is + dropped. A compatible implementation MAY drop the first post-deadline request + instead; clients cannot depend on either behavior, because from their side + the deadline coincides with an unresponsive server. + +**Upstream 0.9.1 behavior — a session that has never carried an echo request +never expires.** The idle clock starts at the first echo request, not at open. +Measured with a 2-second idle timeout: a session opened and left idle for 9 +seconds with no echo request was still served afterwards. See Section 7.6 for +why this matters (open retransmissions). + +**Protocol requirement.** Expiry itself is entirely server policy, and a +compatible server MAY choose any deadline or none. The one interoperability +constraint is negative: a server MUST NOT signal expiry to the client — the +client observes it only as silence. + +**Robustness recommendation (clean implementation).** Apply an expiry deadline to +sessions that have never carried an echo request as well, rather than leaving +them resident indefinitely as upstream does. + +Requests dropped for rate limiting still refresh the idle clock, so a client +sending faster than the allowance keeps its session alive without receiving +replies. + +### 18.3 Resource Limits + +**Observed upstream 0.9.1 policy.** The table below records what the reference +server was seen to do with its default configuration. Every row is an +observation about upstream's *policy*, not a protocol rule, and none of it is +behavior a compatible server is expected to reproduce. + +| Limit | Observed | +|-------|----------| +| Maximum concurrent sessions | **None observed.** 1000 sessions from one source endpoint all succeeded and all remained usable | +| Per-peer session limit | **None** | +| Per-peer or global request rate limit | **None** beyond the per-session minimum-interval allowance | +| Maximum request length | Only if configured; unlimited by default | +| Maximum test duration | Only if configured; unlimited by default | +| Server-fill descriptor length | 32 bytes (hard) | + +**Protocol requirement.** None. Nothing here is observable to a conforming +client, and a compatible server MAY impose session caps, per-peer caps and +admission limits freely. Because the protocol has no rejection reply, the only +way to refuse is to drop — a client hitting a cap sees an open timeout, which it +already has to tolerate. + +**Explicitly not an interoperability requirement.** Upstream's absence of session +and per-peer bounds is an observed policy choice. It is **not** something a +compatible irtt-rs server should reproduce, and reproducing it would not improve +interoperability by any measure. + +**Robustness recommendation (clean implementation).** A session is created by a +single unauthenticated datagram, opens are never deduplicated, and upstream never +expires a session that has not carried an echo request. The clean irtt-rs server +is therefore expected to operate under a **bounded** resource policy: bounded +total session state, and an expiry deadline that also covers never-used sessions. +The specific limits, the eviction strategy and the shape of that policy are a +clean-project design decision and are deliberately not prescribed here. + +--- + +## 19. Protocol-Visible Limits and Defaults + +Values marked *(protocol)* are fixed by the wire format. Values marked *(policy)* +are the reference server's defaults and are freely changeable by a compatible +implementation. + +| Item | Value | Kind | +|------|-------|------| +| Magic | `0x14 0xA7 0x5B` | protocol | +| Flag bits defined | 0x01 Open, 0x02 Reply, 0x04 Close, 0x08 HMAC | protocol | +| Reserved flag bits | 0x10–0x80, must be zero | protocol | +| Token width | 64 bits, little-endian | protocol | +| Sequence width | 32 bits, little-endian, wraps modularly | protocol | +| Received count width | 32 bits, little-endian | protocol | +| Received window width | 64 bits, little-endian, bit 0 = current | protocol | +| Timestamp width | 64 bits signed, little-endian, nanoseconds | protocol | +| Authentication field | 16 bytes, HMAC-MD5, packet-wide with the field zeroed | protocol | +| Server-fill descriptor maximum | 32 bytes | protocol | +| Protocol version | 1 | protocol | +| Default port | 2112 | policy | +| Default minimum send interval | 10 ms | policy | +| Default burst allowance | 5 requests | policy | +| Default idle timeout | 1 minute | policy | +| Idle-expiry grace beyond the timeout | 5 seconds (measured) | policy | +| Maximum negotiable interval | idle timeout ÷ 4 (measured) | policy | +| Maximum-duration grace | 2 seconds (measured) | policy | +| Default maximum test duration | unlimited | policy | +| Default maximum packet length | unlimited | policy | +| Default fill pattern | repeating `69 72 74 74` | policy | +| Default fill allow-list | random fills only | policy | +| Default timestamp allowance | dual | policy | +| DSCP allowed by default | yes | policy | + +**Client-side floor worth knowing.** Upstream clients reject any server-reduced +interval below **1 second**, in strict and loose mode alike. A server policy that +would produce a smaller interval makes the server unusable by those clients. + +--- + +## 20. DSCP + +**Protocol requirement — what the wire parameter is.** The DSCP parameter +(tag 8) carries the **raw IP TOS / Traffic Class byte**, with a useful range of +**0–255**. It is not a 6-bit codepoint. A server applies the negotiated value +directly as the TOS / Traffic Class byte of its echo replies, and a compatible +implementation MUST encode and interpret it the same way. The common Expedited +Forwarding marking is therefore **0xb8 (184)** on the wire, not 46. Verified by +capture; see `BLACKBOX_VERIFICATION_REPORT.md` Finding B and +`captures/dscp-session.pcapng`. + +> **Out of scope: user-facing DSCP notation.** Whether an implementation's +> command line or configuration accepts a 6-bit codepoint (0–63) and shifts it +> left by two to produce the wire byte, or accepts the raw byte directly, is a +> presentation decision for that project. The clean irtt-rs project will decide +> and implement its own convention separately. Nothing about that choice is +> upstream wire behavior, and it does not belong in this specification. + +**Server policy.** When DSCP is permitted and supported on the socket, the server +applies the negotiated value to its echo replies. When DSCP is disallowed, the +negotiated value is forced to 0 and returned as absent from the parameter +payload. + +**Observed range handling.** The negotiated value is **not** range-checked +during negotiation. Values 0–255 are applied normally. A value above 255 is +accepted and echoed back in the open reply, but then every echo reply for that +session fails to send and is dropped — the client sees a session that opens and +then answers nothing. + +**Robustness recommendation (clean implementation).** Reject or clamp DSCP +values outside 0–255 during negotiation rather than accepting a value that cannot +be applied, so that the value returned in the open reply is one the server will +honour. + +DSCP is applied to echo replies only; open and close packets are unmarked (see +the client specification, Section 10.8). + +--- + +## 21. Version Differences + +### 21.1 0.9.0 → 0.9.1 + +Two independent things are recorded here and must not be conflated. + +**What upstream's published release notes say.** They cover platform time +precision, server logging transport, a probe integration, build changes and bug +fixes. None of it is presented as a protocol change, and the protocol version +remained 1. Taken alone, the release notes would suggest no wire-visible +difference. + +**What was measured on the wire.** One wire-visible difference exists that the +release notes do not mention: the single-clock midpoint layout. A 0.9.0 server +emits only the negotiated midpoint field; a 0.9.1 server emits both midpoint +fields. Measured side by side with identical scripted sessions: + +| Server | StampAt = 4, Clock = Wall | StampAt = 4, Clock = Monotonic | StampAt = 4, Clock = Both | +|--------|---------------------------|--------------------------------|---------------------------| +| 0.9.0 | 36 bytes, one field | 36 bytes, one field, process-relative value | 44 bytes, two fields | +| 0.9.1 | **44 bytes, two fields** | **44 bytes, two fields** | 44 bytes, two fields | + +(Received stats = both, negotiated length 0.) See Section 11.3.1. + +**Correction to earlier material.** Any earlier statement that there were no +protocol-visible changes between 0.9.0 and 0.9.1 is wrong. That claim followed +the release notes rather than measurement; the measured difference above stands. + +**Classification: one wire-visible difference in 0.9.1, alongside changes that +are not wire-visible.** Section 11.3.1 additionally records a source-assisted, +non-normative judgement that the wire-visible difference is unintended. That +judgement is separate from, and not required by, the measurement above. + +### 21.2 0.9.1 → the tested post-0.9.1 development tree + +**Provenance.** The development tree exercised for this study is the 0.9.1 +release plus **six** subsequent upstream commits on the development branch: an +ECN change and its merge, a roadmap note, a documentation correction, and a +dependency update. Only one of them is a functional change. These six commits are +**not** part of the 0.9.1 release; everything in this subsection was measured +against a build of that development tree, and nothing here describes 0.9.1. + +**Change: an opt-in server ECN option.** + +- Adds a server command-line option that, per its documentation, ships ECN bits + for the client to log, and states that it is IPv6-only, that it forces the + source-IP-pinning option, and that it "disables UDP replies from server". +- **Observed behavior contradicts the documentation on the last point.** With + the option enabled, sessions completed normally over both IPv4 and IPv6: + packets sent/received 4/4, no loss, ordinary results. Replies are **not** + disabled. +- Enabling it forces the source-IP-pinning behavior described in Section 5.3. +- ECN observation is a receive-side property read from the IP header; it does + not change the IRTT payload in either direction. + +**Timestamp layout is unchanged.** The full StampAt × Clock matrix and the +midpoint length sweep of Section 11.3.1 were re-measured against a build of this +development tree and produced results identical to 0.9.1, byte-position for +byte-position. The tested post-0.9.1 tree therefore behaves like 0.9.1 in this +respect, and none of the six commits changes the dual-field midpoint emission. + +**Wire comparison.** A side-by-side run of the 0.9.1 release build and the +post-0.9.1 build, using an identical scripted request sequence (open with full +statistics and dual timestamps on both clocks; echo sequence 0, 1, 4, 3; close), +produced byte-identical structure: same open-reply layout, same reply lengths +(60 bytes), same flags, same counts (1, 2, 3, 4) and the same windows +(`0x1`, `0x3`, `0x19`, `0x1`), and no close reply in either case. + +**Classification: CLI / server-policy difference. Not wire-visible.** + +Nothing in this document changes for the tested post-0.9.1 tree. + +### 21.3 0.1.x + +Incompatible predecessor protocol. Out of scope. + +--- + +## 22. Implementation Hazards Observed Upstream + +These are behaviors a clean implementation should deliberately **not** +reproduce. They are listed because each is reachable from ordinary client +traffic, so a clean server has to decide what to do instead. + +Everything in this section is an **observation plus a robustness +recommendation**. None of it is an interoperability requirement: no conforming +client can tell which choice a server made, because in every case the visible +effect is either silence or an ordinary reply. + +### 22.1 Oversized Reply Terminates the Server + +If the negotiated packet length exceeds the host's maximum outbound UDP +datagram size, the first echo reply fails to send with a non-transient error, +and upstream 0.9.1 treats that as fatal: the listener stops and the server +process exits. Observed on a host whose maximum outbound datagram size was +9216 bytes: a session negotiating 15000 bytes killed the server on its first +echo reply, and all subsequent clients received nothing. + +The default configuration places no upper bound on the negotiated length, so +this is reachable without any malformed packet. + +**Robustness recommendation (restated from Section 6.3).** Treat a per-packet +send failure as a per-packet event rather than a fatal one, and refuse to +negotiate a length the server cannot transmit. + +### 22.2 Zero-Length Fill Pattern Terminates the Server + +A server-fill descriptor of the form `pattern:` with an empty hex body +terminates upstream 0.9.1 during open handling, when the fill allow-list permits +pattern fills. The default allow-list does not, so a default-configured server +is not exposed; a server configured with a permissive allow-list is. + +**Robustness recommendation.** Validate a fill descriptor before using it and +reject a zero-length pattern. Falling back to the server default — the behavior +already observed for an unparseable hex body such as `pattern:zz` — is the +natural response, and is what upstream already does for that neighbouring case. + +### 22.3 Negotiated Length Not Bounded by Path or Interface + +The negotiated length reported to the client is not clamped to the interface +MTU, but the reply is (Section 9.2). Clients treat a short reply as fatal, so +this produces a session that opens successfully and then fails immediately. + +**Robustness recommendation.** Clamp the negotiated length to what the server can +actually emit, so that the value returned in the open reply is honest. (The +resulting reply length is then covered by the ordinary rule of Section 9.2, which +*is* an interoperability requirement — a reply shorter than the negotiated length +is fatal to upstream clients.) + +### 22.4 Residual Payload Disclosure Under No-Fill + +See Section 13.4. A no-fill mode that emits uninitialized buffer content can +leak bytes from other clients' traffic. The requirement not to emit another +peer's data is stated in Section 13.4; it is a security property of the +implementation rather than a wire-compatibility rule. + +--- + +## 23. Remaining Unknowns + +Each entry records the question, why compatibility might depend on it, what was +inspected, what was attempted, and what evidence would settle it. + +### 23.1 Received-count wraparound + +- **Question:** What does the 32-bit received count do at 2^32 requests in a + single session — wrap, saturate, or something else? +- **Why it matters:** A client computing upstream loss from the count would see + a large negative or nonsensical delta at the wrap point. Only reachable in + extremely long, high-rate sessions. +- **Evidence inspected:** Field width and encoding confirmed from captures; + counter observed to advance by exactly one per accepted request. +- **Attempted:** Not attempted. Reaching 2^32 requests is not feasible in a + targeted experiment. +- **Would resolve it:** A server-side test harness able to preload a session + counter, or a very long-running instrumented session. + +### 23.2 Behavior under genuine resource exhaustion + +- **Question:** What does a server do when session state or buffers can no + longer be allocated? +- **Why it matters:** Determines whether a client sees an open timeout, a + partial session, or a dead server. +- **Evidence inspected:** 1000 concurrent sessions from one endpoint were + created with no refusals and no degradation; no admission control was + observed. +- **Attempted:** Scaling to exhaustion was deliberately not attempted — it is a + resource-starvation test, not a protocol test. +- **Would resolve it:** A memory-capped container running the reference server + while a controlled harness opens sessions until allocation fails. + +### 23.3 IPv6 zone / link-local session identity + +- **Question:** Is the IPv6 zone index part of session identity in practice, and + how are scoped addresses compared? +- **Why it matters:** A server bound to a link-local address on multiple + interfaces could confuse two peers with the same address in different zones. +- **Evidence inspected:** IPv6 sessions over loopback work identically to IPv4; + the IPv4 and IPv6 listeners of one port were confirmed to have separate token + scopes. +- **Attempted:** Link-local multi-interface testing was not possible on the + single-host test platform. +- **Would resolve it:** Two interfaces with the same link-local address in + different zones, with clients on each. + +### 23.4 Midpoint dual-field emission and non-upstream clients + +- **Question:** Do client implementations other than upstream 0.9.1 tolerate the + extra midpoint field, and do any require it? +- **Why it matters:** Section 11.3.1 recommends emitting only the negotiated + fields; a client that computes an exact expected reply length and rejects + longer replies would break against upstream servers, not against a compatible + one. +- **Evidence inspected:** All 60 combinations of ReceivedStats × StampAt × Clock + measured against the reference server; a negotiated-length sweep across the two + minima for four received-stats settings; upstream client verified to accept + both forms. The 0.9.0 build and the tested post-0.9.1 build were measured for + comparison. +- **Note:** the question is narrower than it was. Because the length difference + disappears once the negotiated length reaches `upstream_header`, a third-party + client is only exposed to the length discrepancy at small negotiated lengths; + at larger ones it is exposed only to the field *contents* in the monotonic-only + case — which is also the regime in which the two forms cannot be told apart + (Section 11.3.1). +- **Attempted:** Only upstream 0.9.1 was available as a second implementation. +- **Would resolve it:** Testing against a third-party client, or a survey of + other implementations. + +### 23.5 Reply-length capping on other platforms + +- **Question:** Is the reply-length cap always the listener interface's MTU, or + does it vary with how the platform reports interface MTUs? +- **Why it matters:** It determines the largest length a server can honestly + negotiate. +- **Evidence inspected:** Measured on one host: replies capped at exactly the + bind interface's MTU (1500 on an Ethernet interface); loopback with a + 16384-byte MTU was masked by a smaller maximum outbound datagram size. +- **Attempted:** Two interfaces with different MTUs on one host. +- **Would resolve it:** The same measurement on Linux and Windows hosts with + jumbo-frame and tunnel interfaces. + +### 23.6 Whether any trigger other than the duration limit sets Close on a reply + +- **Question:** Are there conditions besides the maximum-duration hard limit + that cause upstream to set the Close flag on an echo reply? +- **Why it matters:** A client must handle the flag whenever it appears; a + server implementer needs to know which conditions warrant it. +- **Evidence inspected:** The duration limit was reproduced and captured. Idle + expiry, shutdown, rate limiting, oversize requests and authentication failure + were each exercised and none of them produced a close-flagged reply. +- **Attempted:** All the above, plus repeated close and foreign-endpoint close. +- **Would resolve it:** Exhaustive exercise of every server policy knob against + a rogue client that keeps sending regardless. + +### 23.7 How tightly server timestamps bracket the request + +- **Question:** How much of a server's own send and receive path falls outside + the interval its two timestamps describe? +- **Why it matters:** It bounds the accuracy of the client's server-processing + correction. +- **Evidence inspected:** Receive-to-send deltas of a few microseconds on + loopback; ordering (receive ≤ send) always held. +- **Attempted:** Cannot be separated from scheduling noise externally. +- **Would resolve it:** Hardware timestamping on both endpoints. Nothing about + where an implementation takes its timestamps is externally observable, so this + is likely to remain permanently open, and no requirement in this document + depends on it. Section 11.1 states the relationship a compatible server should + aim to make derivable. + +--- + +## 24. Conformance Summary + +The two lists below are **interoperability** requirements only: each is +observable by a conforming client, and each rests on a black-box experiment. +Robustness recommendations for the clean implementation are collected separately +in Section 24.3 and are deliberately not mixed in. + +### 24.1 A conforming server MUST: + +1. Speak UDP, validate magic and flag bits, and reject any packet with the Reply + flag set. +2. Set the Reply flag on every packet it emits, and never the Open flag on an + echo reply. +3. Silently drop every rejected packet — no error replies, ever. +4. Accept open requests without a token field, decode the parameter payload per + the shared encoding rules, and return the **restricted** parameters together + with a non-zero token. +5. Return ProtocolVersion = 1 in the open reply. +6. Return a zero token and the Close flag for a no-test open, and create no + session for it. +7. Bind each session to its token **and** its source endpoint, and drop packets + that fail either check without disturbing the session. +8. Echo the token and sequence number unchanged in every echo reply. +9. Emit the negotiated fields in the specified order, and pad to the negotiated + length. For StampAt = Midpoint this means the fields the negotiated Clock + implies and no others — a conforming server does not reproduce the upstream + dual-field emission of Section 11.3.1. +10. Report a received count that includes the current request and excludes every + dropped one. +11. Report a received window with bit 0 set, bit *k* meaning "sequence + (current − *k*) was received". +12. Emit receive and send timestamps from a consistent clock domain, such that + the pair brackets the server's own handling of that request and the receive + instant is never later than the send instant. +13. Release the session on a valid close request. Sending nothing in response is + correct and is what upstream does; no close-reply datagram is required, and + none is specified here (Section 15.1). +14. Signal a server-initiated close only by setting the Close flag on an + otherwise complete echo reply, and release the token at that point. +15. When authenticating: set the HMAC flag and a valid MAC on **every** reply, + and drop silently on any authentication failure, with no reply behavior that + could distinguish a known token from an unknown one. + +### 24.2 A conforming server MUST NOT: + +1. Reply to any malformed, unauthenticated, unknown-token or foreign-endpoint + packet. +2. Interpret any request bytes beyond the token and sequence number. +3. Emit another peer's data, or uninitialized memory, as payload. +4. Return a negotiated packet length in the open reply that it will not + actually emit. + +### 24.3 Robustness recommendations for the clean implementation + +Not interoperability requirements. No conforming client can observe whether these +were followed; they are here because each addresses a behavior observed upstream +that a new implementation should not copy. + +1. Treat a per-packet send failure as a per-packet event, not a fatal one + (Sections 6.3, 22.1). +2. Operate under a bounded session-resource policy, and expire sessions that have + never carried an echo request (Sections 7.6, 18.2, 18.3). The specific limits + and eviction policy are a clean-project decision. +3. Validate a fill descriptor before use, and reject a zero-length pattern + (Section 22.2). +4. Reject or clamp DSCP values the socket cannot carry, during negotiation + (Section 20). +5. Clamp the negotiated length to what the server can actually transmit + (Sections 22.1, 22.3). + +--- + +## 25. Related Documents + +- `IRTT_CLIENT_PROTOCOL_SPEC.md` — wire format, parameter encoding, HMAC + computation, client-side measurement semantics. +- `BLACKBOX_VERIFICATION_REPORT.md` — verification evidence, including the + server-side section. +- `test-vectors/SERVER_BEHAVIORAL_VECTORS.md` — input → output vectors for this + document. +- `test-vectors/README.md` — packet-level test vectors. +- `captures/` — packet captures referenced throughout. +- `../clean-room/CLEANROOM_NOTES.md` — clean-room boundary and audit record. diff --git a/docs/protocol/captures/basic-session.pcapng b/docs/protocol/captures/basic-session.pcapng index ead1d08..3d40218 100644 Binary files a/docs/protocol/captures/basic-session.pcapng and b/docs/protocol/captures/basic-session.pcapng differ diff --git a/docs/protocol/captures/dscp-46.pcapng b/docs/protocol/captures/dscp-46.pcapng index 0094955..37dd222 100644 Binary files a/docs/protocol/captures/dscp-46.pcapng and b/docs/protocol/captures/dscp-46.pcapng differ diff --git a/docs/protocol/captures/dscp-session.pcapng b/docs/protocol/captures/dscp-session.pcapng index d593111..ddc96c7 100644 Binary files a/docs/protocol/captures/dscp-session.pcapng and b/docs/protocol/captures/dscp-session.pcapng differ diff --git a/docs/protocol/captures/dscp-values.pcapng b/docs/protocol/captures/dscp-values.pcapng index b3bea20..5ac1dfb 100644 Binary files a/docs/protocol/captures/dscp-values.pcapng and b/docs/protocol/captures/dscp-values.pcapng differ diff --git a/docs/protocol/captures/full-session.pcapng b/docs/protocol/captures/full-session.pcapng index d6aa5d4..c8fd1f6 100644 Binary files a/docs/protocol/captures/full-session.pcapng and b/docs/protocol/captures/full-session.pcapng differ diff --git a/docs/protocol/captures/hmac-session.pcapng b/docs/protocol/captures/hmac-session.pcapng index 54e0bd1..06ee6e8 100644 Binary files a/docs/protocol/captures/hmac-session.pcapng and b/docs/protocol/captures/hmac-session.pcapng differ diff --git a/docs/protocol/captures/large-packet.pcapng b/docs/protocol/captures/large-packet.pcapng index 02626b6..50d906c 100644 Binary files a/docs/protocol/captures/large-packet.pcapng and b/docs/protocol/captures/large-packet.pcapng differ diff --git a/docs/protocol/captures/long-sfill.pcapng b/docs/protocol/captures/long-sfill.pcapng index 9afbefe..c65947e 100644 Binary files a/docs/protocol/captures/long-sfill.pcapng and b/docs/protocol/captures/long-sfill.pcapng differ diff --git a/docs/protocol/captures/no-test.pcapng b/docs/protocol/captures/no-test.pcapng index 53e9999..02b49cc 100644 Binary files a/docs/protocol/captures/no-test.pcapng and b/docs/protocol/captures/no-test.pcapng differ diff --git a/docs/protocol/captures/server-close-lifecycle.pcapng b/docs/protocol/captures/server-close-lifecycle.pcapng new file mode 100644 index 0000000..e3655b0 Binary files /dev/null and b/docs/protocol/captures/server-close-lifecycle.pcapng differ diff --git a/docs/protocol/captures/server-close.pcapng b/docs/protocol/captures/server-close.pcapng new file mode 100644 index 0000000..d25c4e4 Binary files /dev/null and b/docs/protocol/captures/server-close.pcapng differ diff --git a/docs/protocol/captures/server-recv-window.pcapng b/docs/protocol/captures/server-recv-window.pcapng new file mode 100644 index 0000000..b4fec96 Binary files /dev/null and b/docs/protocol/captures/server-recv-window.pcapng differ diff --git a/docs/protocol/captures/server-session-identity.pcapng b/docs/protocol/captures/server-session-identity.pcapng new file mode 100644 index 0000000..f83be39 Binary files /dev/null and b/docs/protocol/captures/server-session-identity.pcapng differ diff --git a/docs/protocol/captures/tstamp-none.pcapng b/docs/protocol/captures/tstamp-none.pcapng index 186ecff..ca15501 100644 Binary files a/docs/protocol/captures/tstamp-none.pcapng and b/docs/protocol/captures/tstamp-none.pcapng differ diff --git a/docs/protocol/test-vectors/README.md b/docs/protocol/test-vectors/README.md index d9f2cbc..db25337 100644 --- a/docs/protocol/test-vectors/README.md +++ b/docs/protocol/test-vectors/README.md @@ -160,8 +160,8 @@ b9 07 05 06 06 06 07 06 - Bytes 4-19: HMAC `ff 90 16 a7 aa 53 78 16 9e c3 a2 d5 54 dc 30 36` - Bytes 20-39: Parameter payload -**HMAC verified independently with Python (HMAC-MD5 over entire packet -with HMAC field zeroed).** +**HMAC cross-checked with a separately written Python script (HMAC-MD5 over the +entire packet with the HMAC field zeroed).** --- diff --git a/docs/protocol/test-vectors/SERVER_BEHAVIORAL_VECTORS.md b/docs/protocol/test-vectors/SERVER_BEHAVIORAL_VECTORS.md new file mode 100644 index 0000000..3316028 --- /dev/null +++ b/docs/protocol/test-vectors/SERVER_BEHAVIORAL_VECTORS.md @@ -0,0 +1,801 @@ +# IRTT Server Behavioral Vectors + +**Source:** Black-box observation of an upstream IRTT **0.9.1 release** server on +macOS Darwin 25.5.0 arm64, driven by a raw UDP harness. +**Date:** 2026-08-11 +**Companion to:** `../IRTT_SERVER_PROTOCOL_SPEC.md` + +These vectors are stated as **input → observed output**. They describe what a +server was seen to emit for a given request sequence. They do not describe how +any implementation produces those bytes. + +Unless a row says otherwise, the server under test is the unmodified upstream +0.9.1 release. Two other builds appear only in Section 5.2.3, each named +explicitly: the 0.9.0 release, and a build of the upstream development tree six +upstream commits past the 0.9.1 tag. Results from those builds never stand in for +0.9.1. + +Addresses in the accompanying captures are loopback only. Session tokens are +server-assigned random values and differ per run; where a token appears below it +is the value from that specific run and carries no meaning. + +--- + +## 1. Received Window and Received Count + +All vectors in this section were produced on a single session negotiated with +ReceivedStats = both (3), StampAt = none (0). Each row is one echo request and +the reply it produced. `count` is the 32-bit received count field; `window` is +the 64-bit received window field. + +### 1.1 Sequential, no loss + +| # | request seq | reply count | reply window | +|---|-------------|-------------|--------------| +| 1 | 0 | 1 | `0x0000000000000001` | +| 2 | 1 | 2 | `0x0000000000000003` | +| 3 | 2 | 3 | `0x0000000000000007` | +| 4 | 3 | 4 | `0x000000000000000f` | +| 5 | 4 | 5 | `0x000000000000001f` | +| 6 | 5 | 6 | `0x000000000000003f` | + +### 1.2 One gap + +| # | request seq | reply count | reply window | note | +|---|-------------|-------------|--------------|------| +| 1 | 0 | 1 | `0x0000000000000001` | | +| 2 | 1 | 2 | `0x0000000000000003` | | +| 3 | 3 | 3 | `0x000000000000000d` | bit 1 clear — sequence 2 missing | +| 4 | 4 | 4 | `0x000000000000001b` | bit 2 clear — sequence 2 missing | + +### 1.3 Multiple gaps + +| # | request seq | reply count | reply window | +|---|-------------|-------------|--------------| +| 1 | 0 | 1 | `0x0000000000000001` | +| 2 | 3 | 2 | `0x0000000000000009` | +| 3 | 7 | 3 | `0x0000000000000091` | +| 4 | 8 | 4 | `0x0000000000000123` | + +### 1.4 Later gap fill — window collapses + +| # | request seq | reply count | reply window | note | +|---|-------------|-------------|--------------|------| +| 1 | 0 | 1 | `0x0000000000000001` | | +| 2 | 1 | 2 | `0x0000000000000003` | | +| 3 | 5 | 3 | `0x0000000000000031` | gap over 2, 3, 4 | +| 4 | 3 | 4 | **`0x0000000000000001`** | filling the gap **discards** history | + +A late arrival does not set the corresponding historical bit. The window is +reset to "only the current packet is known". + +### 1.5 Duplicate sequence number + +| # | request seq | reply count | reply window | note | +|---|-------------|-------------|--------------|------| +| 1 | 0 | 1 | `0x0000000000000001` | | +| 2 | 1 | 2 | `0x0000000000000003` | | +| 3 | 1 | 3 | `0x0000000000000003` | count advances, window unchanged | +| 4 | 2 | 4 | `0x0000000000000007` | | + +### 1.6 Reordering + +| # | request seq | reply count | reply window | note | +|---|-------------|-------------|--------------|------| +| 1 | 0 | 1 | `0x0000000000000001` | | +| 2 | 1 | 2 | `0x0000000000000003` | | +| 3 | 2 | 3 | `0x0000000000000007` | | +| 4 | 1 | 4 | **`0x0000000000000001`** | out-of-order arrival resets the window | +| 5 | 3 | 5 | `0x0000000000000005` | rebuilds from the reset state | + +Note row 5: the window says sequence 1 was received (bit 2) and sequence 3 was +received (bit 0), but bit 1 is clear even though sequence 2 *was* received +earlier. History before the reset is unrecoverable. + +### 1.7 Strictly descending sequence numbers + +| # | request seq | reply count | reply window | +|---|-------------|-------------|--------------| +| 1 | 5 | 1 | `0x0000000000000001` | +| 2 | 4 | 2 | `0x0000000000000001` | +| 3 | 3 | 3 | `0x0000000000000001` | +| 4 | 2 | 4 | `0x0000000000000001` | +| 5 | 1 | 5 | `0x0000000000000001` | +| 6 | 0 | 6 | `0x0000000000000001` | + +Every reply is answered; only the count carries information. + +### 1.8 Gap at and beyond the representable window + +| first seq | second seq | gap | second reply window | +|-----------|------------|-----|---------------------| +| 0 | 63 | 63 | `0x8000000000000001` — highest bit set | +| 0 | 64 | 64 | `0x0000000000000001` | +| 0 | 65 | 65 | `0x0000000000000001` | +| 0 | 1000 | 1000 | `0x0000000000000001` | + +The window represents the current packet plus 63 predecessors. A gap of exactly +63 still records the older packet in the top bit; a gap of 64 or more discards +all history. + +### 1.9 Non-zero starting sequence number + +| # | request seq | reply count | reply window | +|---|-------------|-------------|--------------| +| 1 | 100 | 1 | `0x0000000000000001` | +| 2 | 101 | 2 | `0x0000000000000003` | +| 3 | 102 | 3 | `0x0000000000000007` | + +The first request of a session always yields window `0x1`, whatever its +sequence number. + +### 1.10 32-bit sequence wraparound + +| # | request seq | reply count | reply window | +|---|-------------|-------------|--------------| +| 1 | 4294967293 | 1 | `0x0000000000000001` | +| 2 | 4294967294 | 2 | `0x0000000000000003` | +| 3 | 4294967295 | 3 | `0x0000000000000007` | +| 4 | 0 | 4 | `0x000000000000000f` | +| 5 | 1 | 5 | `0x000000000000001f` | + +Wraparound is handled as ordinary modular arithmetic: crossing 2^32 − 1 → 0 is +treated as a step of one, not as a gap of 2^32 − 1 and not as reordering. + +Starting a session at 4294967295 and following with 0 gives `0x1` then `0x3`. + +### 1.11 Randomized cross-check + +The window and count rules in `../IRTT_SERVER_PROTOCOL_SPEC.md` Section 10 were +checked against a live server over 12 sessions of 40 requests each — 480 replies +in total — with sequence numbers drawn from a mix of in-order steps, duplicates, +forward gaps of 2 to 80, and backward steps of 1 to 5, and with starting +sequence numbers of 0, 1, 7, 100 and 4294967294. + +**Result: 480 replies checked, 0 mismatches** in either the window value or the +count value. + +### 1.12 Count excludes dropped requests + +Server configured with a minimum interval of 100 ms and a burst allowance of 5. +Twelve requests sent back to back, then three more after a 600 ms pause. + +| requests sent | replies received | counts in replies | +|---------------|------------------|-------------------| +| 12 back to back | 5 | 1, 2, 3, 4, 5 | +| 3 after a pause | 3 | 6, 7, 8 | + +Rate-limited requests produce no reply and do not advance the count. + +--- + +## 2. Session Identity + +Capture: `../captures/server-session-identity.pcapng`. + +Session opened from source port A; token `88a76d2fd0cd5f32`. + +| # | source | packet | observed | +|---|--------|--------|----------| +| 1 | A | open request | open reply, token assigned | +| 2 | A | echo seq 0 | reply, count 1, window `0x1` | +| 3 | **B** | echo seq 1, correct token | **no reply** | +| 4 | **B** | close, correct token | **no reply**, session **not** closed | +| 5 | A | echo seq 2 | reply, count 2, window `0x5` | +| 6 | **C** | echo seq 0, token `1111111111111111` | **no reply** | +| 7 | A | close | **no reply**, session removed | + +Row 5 is the important one: the count advanced from 1 to 2, not to 3 — the +foreign-endpoint request in row 3 was not counted. The window `0x5` reflects +sequence 2 (bit 0) and sequence 0 (bit 2), with bit 1 clear because sequence 1 +never reached the session. + +Row 4 confirms that a close from a foreign endpoint does not tear down the +session. + +Additional cases from the same class of experiment: + +| Case | Result | +|------|--------| +| Two opens from the same source port | Two distinct tokens, both simultaneously usable | +| Same token replayed to the other address family's listener on the same port | No reply | +| Echo with an all-zero token | No reply | +| Echo with the token of an already-closed session | No reply | +| Repeated close with the same token | No reply | +| 1000 opens from one source endpoint | 1000 distinct tokens, all usable | + +--- + +## 3. Close Lifecycle + +### 3.1 Client-initiated close + +Capture: `../captures/server-close-lifecycle.pcapng`. Session token +`eef1e3fb1ab2eb66`, negotiated ReceivedStats = both, StampAt = none. + +| frame | direction | bytes | +|-------|-----------|-------| +| 1 | C → S | `14a75b01 01020280e0ba84bf030380a8d6b9070506` | +| 2 | S → C | `14a75b03 eef1e3fb1ab2eb66 01020280e0ba84bf030380a8d6b9070506` | +| 3 | C → S | `14a75b00 eef1e3fb1ab2eb66 00000000 00000000 0000000000000000` | +| 4 | S → C | `14a75b02 eef1e3fb1ab2eb66 00000000 01000000 0100000000000000` | +| 5 | C → S | `14a75b00 eef1e3fb1ab2eb66 01000000 00000000 0000000000000000` | +| 6 | S → C | `14a75b02 eef1e3fb1ab2eb66 01000000 02000000 0300000000000000` | +| 7 | C → S | `14a75b00 eef1e3fb1ab2eb66 02000000 00000000 0000000000000000` | +| 8 | S → C | `14a75b02 eef1e3fb1ab2eb66 02000000 03000000 0700000000000000` | +| 9 | C → S | `14a75b04 eef1e3fb1ab2eb66` — close request | +| 10 | C → S | `14a75b04 eef1e3fb1ab2eb66` — repeated close | +| 11 | C → S | `14a75b00 eef1e3fb1ab2eb66 09000000 …` — echo after close | + +Frames 9, 10 and 11 receive **no response**. Frame 9 removes the session; 10 and +11 are dropped as unknown-token packets. The capture ends with three unanswered +client packets, which is the complete and correct close exchange for +protocol version 1. + +Close request layout: magic (3) + flags `0x04` (1) + token (8) = 12 bytes. With +authentication: magic (3) + flags `0x0c` (1) + MAC (16) + token (8) = 28 bytes. + +### 3.2 Server-initiated close + +Capture: `../captures/server-close.pcapng`. Server configured with a maximum +test duration of 2 s; client requested 600 s and was restricted to 2 s. Requests +sent every ~250 ms. + +| frame | direction | bytes (abridged) | note | +|-------|-----------|------------------|------| +| 2 | S → C | `14a75b03 f0592d9fb129558c 010202 80d0acf30e 03…` | duration restricted to 2 s | +| 34 | S → C | `14a75b02 … 0f000000 10000000 ffff000000000000` | ordinary reply, seq 15 | +| 35 | C → S | `14a75b00 … 10000000 …` | echo seq 16 | +| **36** | S → C | `14a75b06 f0592d9fb129558c 10000000 11000000 ffff010000000000` | **flags `0x06` = Reply\|Close** | +| 37 | C → S | `14a75b00 … 63000000 …` | echo seq 99 — **no reply** | + +Frame 36 is a complete, ordinary echo reply — correct sequence number (16), +correct token, count 17, window `0x1ffff` — with the Close flag additionally +set. The session is gone from that moment; frame 37 is unanswered. + +Timing measurements of the deadline, from the **first** echo request of the +session: + +| configured maximum duration | close flag observed at | +|-----------------------------|------------------------| +| 1 s | 3.01 s | +| 2 s | 4.06 s | +| 2 s (second run) | 4.10 s | + +The deadline is the configured maximum duration plus a 2-second grace period. + +### 3.3 Observed upstream client reaction to a close-flagged reply + +Driving an upstream 0.9.1 client against a server that sets the Close flag from +sequence 2 onward: + +``` +seq=0 recorded +seq=1 recorded +seq=2 reply carries Close -> run terminates +``` + +The client reports a "server closed connection" condition, prints the statistics +for sequences 0 and 1 only — the close-flagged reply is **not** recorded — and +exits with a success status. + +--- + +## 4. Open Negotiation + +### 4.1 Protocol version is not rejected + +Server default configuration. Each row is an open request carrying only the +listed ProtocolVersion value. + +| requested ProtocolVersion | server response | reply parameter payload | +|---------------------------|-----------------|-------------------------| +| 1 | open reply, session created | `0102` (version 1) | +| 0 | open reply, session created | `0102` (version 1) | +| 2 | open reply, session created | `0102` (version 1) | +| −1 | open reply, session created | `0102` (version 1) | +| tag absent entirely | open reply, session created | `0102` (version 1) | + +In every case the reply flags were `0x03` (Open\|Reply) with the Close flag +**clear** and a non-zero token. Reply length 14 bytes: magic (3) + flags (1) + +token (8) + `0102`. + +### 4.2 Parameter payload acceptance + +| open request parameter payload | response | +|-------------------------------|----------| +| empty | open reply, 14 bytes | +| tag 2 only, no value bytes | no reply | +| tag 2 with an incomplete multi-byte varint | no reply | +| version 1 + duration 0 | no reply | +| version 1 + duration −1 | no reply | +| version 1 + interval −5 | no reply | +| version 1 + ReceivedStats 4 | no reply | +| version 1 + ReceivedStats −1 | no reply | +| version 1 + StampAt 5 | no reply | +| version 1 + StampAt 4 + Clock 3 | open reply | +| version 1 + Clock 0 | no reply | +| version 1 + Clock 4 | no reply | +| version 1 + unknown tag 42 | open reply, tag ignored | +| version 1 + unknown tag 200 | open reply, tag ignored | +| version 1 + 80 repetitions of unknown tag 42 (~200 bytes) | open reply, all ignored | +| version 1 + ServerFill declared 40 bytes | no reply | +| version 1 + ServerFill declaring 10 bytes with only 2 present | no reply | +| 50 bytes of arbitrary data | no reply | + +### 4.3 No-test open + +| direction | bytes | +|-----------|-------| +| C → S | `14a75b05` + parameter payload — flags Open\|Close | +| S → C | `14a75b07 0000000000000000` + restricted parameters — flags Open\|Reply\|Close, **zero token** | + +With authentication the reply flags are `0x0f` and the reply carries a valid MAC. +The zero token is not usable: subsequent echo and close requests bearing it get +no reply. + +### 4.4 Interval restriction against the idle timeout + +Client requests an interval of one hour; server has no configured minimum +interval. + +| server idle timeout | negotiated interval | +|---------------------|---------------------| +| 4 s | 1 s | +| 8 s | 2 s | +| 20 s | 5 s | +| 40 s | 10 s | +| 60 s | 15 s | + +Client-side floor, measured with an upstream 0.9.1 client requesting a 1 s +interval: + +| server idle timeout | server-returned interval | client outcome | +|---------------------|--------------------------|----------------| +| 2 s | 500 ms | aborts — reduction below 1 s, in strict **and** loose mode | +| 3 s | 750 ms | aborts — reduction below 1 s, in strict **and** loose mode | +| 4 s | 1 s (no reduction) | proceeds | + +Requesting a 2 s interval against a 4 s idle timeout produces a reduction to +exactly 1 s: rejected in strict mode as an ordinary restriction, accepted in +loose mode. + +### 4.5 Timestamp allowance + +Requested StampAt → negotiated StampAt, by server timestamp allowance: + +| requested | dual | single | none | +|-----------|------|--------|------| +| 0 (none) | 0 | 0 | 0 | +| 1 (send) | 1 | 1 | 0 | +| 2 (receive) | 2 | 2 | 0 | +| 3 (both) | 3 | **4** | 0 | +| 4 (midpoint) | 4 | 4 | 0 | + +--- + +## 5. Reply Length + +### 5.1 Length as a function of negotiated fields + +Reply lengths, unauthenticated, measured across every combination. `S` is +ReceivedStats, `A` is StampAt, `C` is Clock. + +| S | A | C | reply length | +|---|---|---|--------------| +| 0 | 0 | any | 16 | +| 1 | 0 | any | 20 | +| 2 | 0 | any | 24 | +| 3 | 0 | any | 28 | +| 3 | 1 or 2 | 1 or 2 | 36 | +| 3 | 1 or 2 | 3 | 44 | +| 3 | 3 | 1 or 2 | 44 | +| 3 | 3 | 3 | 60 | +| 3 | 4 | 1 or 2 | **44** — see below | +| 3 | 4 | 3 | 44 | + +Add 16 bytes to every figure when authentication is in use. + +### 5.2 Dual-field midpoint wire representation + +For StampAt = 4 (midpoint), upstream 0.9.1 emits **two** 8-byte timestamp fields +— the midpoint wall field followed by the midpoint monotonic field, in that order +— for every negotiated Clock, occupying a fixed 16-byte region. With a single +negotiated clock this is one field more than the negotiated parameters imply. +These rows record bytes observed on the wire and nothing else. All vectors below +at negotiated length 0. + +| S | A | C | expected from negotiation | observed | +|---|---|---|---------------------------|----------| +| 0 | 4 | 1 | 24 | **32** | +| 0 | 4 | 2 | 24 | **32** | +| 0 | 4 | 3 | 32 | 32 | +| 1 | 4 | 1 | 28 | **36** | +| 1 | 4 | 2 | 28 | **36** | +| 2 | 4 | 1 | 32 | **40** | +| 2 | 4 | 2 | 32 | **40** | +| 3 | 4 | 1 | 36 | **44** | +| 3 | 4 | 2 | 36 | **44** | +| 3 | 4 | 3 | 44 | 44 | + +Raw reply for S=0, A=4, C=1 (wall only), 32 bytes: + +``` +14 a7 5b 02 00 00 00 00 +a4 c1 74 90 47 83 ca 18 <- midpoint wall, 1786384545595310500 +da 45 a2 3b 00 00 00 00 <- midpoint monotonic, 1000490458 +``` + +The second field was not negotiated. An upstream 0.9.1 client run in this +configuration reports "packet length: 36 bytes" while receiving 44-byte +datagrams, and completes normally. + +No other StampAt value deviates. Measured at negotiated length 0, S=3: `A=1` and +`A=2` give 36 bytes for C=1 or C=2 and 44 for C=3; `A=3` gives 44 for C=1 or C=2 +and 60 for C=3 — all exactly as the negotiated layout implies. + +### 5.2.1 The excess is bounded by the negotiated length + +The extra field enlarges the field block, not every reply. Reply length remains +the larger of the negotiated length and the field block (Section 5.3): + +``` +normal_header = negotiated field block +upstream_header = normal_header + one 8-byte timestamp field + +compatible_reply_len = max(negotiated_length, normal_header) +upstream_0_9_1_reply_len = max(negotiated_length, upstream_header) +``` + +Sweep for S=3, A=4, C=1 — `normal_header` 36, `upstream_header` 44: + +| negotiated length | 0 | 16 | 32 | 36 | 37 | 40 | 43 | 44 | 45 | 48 | 64 | 128 | 1024 | 4096 | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| reply datagram | 44 | 44 | 44 | 44 | 44 | 44 | 44 | 44 | 45 | 48 | 64 | 128 | 1024 | 4096 | +| excess | +8 | +8 | +8 | +8 | +7 | +4 | +1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | + +Same pattern for the other received-stats settings, with the two minima at 24/32 +(S=0), 28/36 (S=1) and 32/40 (S=2). The C=3 control case shows zero excess at +every negotiated length. + +At negotiated length 4096 the midpoint region still holds both fields, followed +by fill — the extra field displaces payload rather than extending the datagram. +The excess is therefore **+8**, then **+7…+1**, then **0**, depending on the +negotiated length. There is no negotiated length at which the reply is +universally "negotiated length + 8". + +Read that row across columns, not within a column. Each column is a separate +negotiation whose upstream reply has exactly one length, +`upstream_0_9_1_reply_len`. The +8 / +7…+1 / 0 figures are the values the +difference takes across different negotiations; no single negotiation produces a +range of lengths, and nothing here says an intermediate length between +`compatible_reply_len` and `upstream_0_9_1_reply_len` was ever emitted or should +be accepted. + +**Consequence for decoding.** In the zero-excess regime +(`upstream_0_9_1_reply_len == compatible_reply_len`) the upstream form +`[wall][mono][payload…]` and a conforming single-clock form `[mono][payload…]` +produce datagrams of identical length, and the vectors above provide no way to +tell them apart. That ambiguity is recorded, not resolved. + +### 5.2.2 Reached without requesting midpoint + +Server restricted to a single timestamp, S=3, negotiated length 0: + +| requested A/C | negotiated A/C | expected from negotiation | observed | +|---------------|----------------|---------------------------|----------| +| 3 / 1 | 4 / 1 | 36 | **44** | +| 3 / 2 | 4 / 2 | 36 | **44** | +| 3 / 3 | 4 / 3 | 44 | 44 | +| 4 / 1 | 4 / 1 | 36 | **44** | + +### 5.2.3 Version comparison + +Negotiated length 0, S=3: + +| build | A=4, C=1 | A=4, C=2 | A=4, C=3 | +|-------|----------|----------|----------| +| 0.9.0 release | 36, one field | 36, one field (process-relative value) | 44, two fields | +| 0.9.1 release | **44, two fields** | **44, two fields** | 44, two fields | +| tested post-0.9.1 tree (six upstream commits past the 0.9.1 tag) | **44, two fields** | **44, two fields** | 44, two fields | + +The 0.9.0 release emits exactly the negotiated fields; 0.9.1 emits the dual-field +form; the tested post-0.9.1 tree behaves like 0.9.1. The six post-0.9.1 commits +are post-release work and are not part of the 0.9.1 release. + +A contaminated-side server emitting **only** the negotiated midpoint field was +driven by an upstream 0.9.1 client for wall-only, monotonic-only and dual-clock +midpoint configurations, and the client completed every run cleanly. This is +contaminated-side consistency validation; the black-box part of it is the +client's acceptance of the conforming form. + +### 5.3 Negotiated length versus actual reply length + +Server bound to an interface with a 1500-byte MTU, ReceivedStats = none, +StampAt = none (16-byte field block). + +| requested length | negotiated length returned | reply datagram | +|------------------|----------------------------|----------------| +| 0 | 0 | 16 | +| 10 | 10 | 16 | +| 16 | 16 | 16 | +| 17 | 17 | 17 | +| 1000 | 1000 | 1000 | +| 1400 | 1400 | 1400 | +| 1500 | 1500 | 1500 | +| 1501 | 1501 | **1500** | +| 2000 | 2000 | **1500** | +| 3000 | 3000 | **1500** | +| 8000 | 8000 | **1500** | + +The negotiated value returned to the client is not clamped; the datagram is. + +A session negotiating a length of 0 or 20 with full statistics and dual +timestamps on both clocks produced 60-byte replies — the field block wins when +it is larger than the negotiated length. + +### 5.4 Request length versus reply length + +| negotiated length | request datagram | reply datagram | +|-------------------|------------------|----------------| +| 16 | 216 (200 trailing bytes) | 16 | +| 64 | 16 | 64 | +| 64 | 32 | 64 | +| 64 | 64 | 64 | + +The reply length depends only on the negotiation, never on the request. + +### 5.5 Maximum request length policy + +Server configured with a maximum packet length of 64; client requested 200 and +was restricted to 64. + +| request datagram | result | +|------------------|--------| +| 16 | reply, 64 bytes | +| 32 | reply, 64 bytes | +| 64 | reply, 64 bytes | +| 65 | **no reply** | +| 100 | **no reply** | +| open request larger than 64 | **accepted**, open reply sent | + +Server configured with a maximum packet length of 10 (below the minimum field +block): the session opens and negotiates a length of 10, and every 16-byte echo +request is dropped. + +--- + +## 6. Server Fill + +### 6.1 Default fill bytes and pattern phase + +Default-configuration server, ReceivedStats = none, StampAt = none, so the +payload begins at offset 16. + +| negotiated length | reply payload | +|-------------------|---------------| +| 17 | `69` | +| 20 | `72 74 74 69` | +| 24 | `72 74 74 69 72 74 74 69` | +| 48 | `72 74 74 69` repeated 8 times | + +Four consecutive replies on one session with a 7-byte payload: + +``` +seq 0: 72 74 74 69 72 74 74 +seq 1: 69 72 74 74 69 72 74 +seq 2: 74 69 72 74 74 69 72 +seq 3: 74 74 69 72 74 74 69 +``` + +The four-byte pattern `69 72 74 74` is emitted as a continuous stream; its phase +is not reset per packet, and it carries over between sessions on the same +listener. + +### 6.2 Fill descriptor negotiation + +Default allow-list (random fills only), negotiated length 32 so the payload is +16 bytes: + +| requested descriptor | negotiated descriptor returned | payload | +|----------------------|-------------------------------|---------| +| `rand` | `rand` | random, differs per packet | +| `none` | `pattern:69727474` | `72747469…` | +| `pattern:aabb` | `pattern:69727474` | `72747469…` | +| `pattern:69727474` | `pattern:69727474` | `72747469…` | +| `bogus` | `pattern:69727474` | `72747469…` | +| 32-character string | `pattern:69727474` | `72747469…` | +| 33-character string | — | **open dropped** | + +Permissive allow-list: + +| requested descriptor | negotiated descriptor returned | payload | +|----------------------|-------------------------------|---------| +| `rand` | `rand` | random | +| `pattern:aabb` | `pattern:aabb` | `aabbaabbaabbaabb…` | +| `pattern:00` | `pattern:00` | `0000000000000000…` | +| `pattern:ff00` | `pattern:ff00` | `ff00ff00ff00ff00…` | +| `pattern:zz` | `pattern:69727474` | `69727474…` (invalid hex, falls back) | +| `none` | `none` | **unspecified residual bytes** | +| `pattern:` | — | **server terminated** | + +Empty allow-list: every requested descriptor, including `rand`, is answered with +`pattern:69727474`. + +### 6.3 The refused descriptor does not describe the payload + +Two servers with different configured fills, default allow-list (`rand` only), +negotiated length 32 so the payload is 16 bytes: + +| Server fill | Requested descriptor | Negotiated descriptor returned | Payload | +|-------------|----------------------|-------------------------------|---------| +| random | none | absent | `25ae387a787d641d82eb514f2bfca15f` (random) | +| random | `pattern:aabb` | `pattern:69727474` | `29ce866b6d192c74376e7161dd71975d` (random) | +| random | `rand` | `rand` | random | +| `pattern:abcd` | none | absent | `abcdabcdabcdabcdabcdabcdabcdabcd` | +| `pattern:abcd` | `pattern:aabb` | `pattern:69727474` | `abcdabcdabcdabcdabcdabcdabcdabcd` | +| `pattern:abcd` | `rand` | `rand` | random | + +On refusal, the descriptor returned to the client is a **fixed default string**, +not a description of the bytes the server actually sends. A client must not +predict payload content from it. + +### 6.4 No-fill payload content + +Server configured with no fill and a permissive allow-list, negotiated length 32 +(16-byte payload): + +| request payload | reply payload | +|-----------------|---------------| +| `000102…0f` (16 bytes) | `000102…0f` — the request's own bytes | +| none (16-byte request) | `000102…0f` — **residue from the previous packet** | + +On a `none` fill negotiated through the descriptor, one observed 16-byte payload +was `c0cbacf6220380897a044009 046e6f6e`, whose trailing bytes are a fragment of +the preceding open reply's parameter payload. Payload content under no-fill is +unspecified and may contain data from unrelated traffic. + +--- + +## 7. Authentication + +Server configured with key `correctkey`. + +| request | result | +|---------|--------| +| Open, flags `0x09`, valid MAC | Open reply, flags `0x0b`, valid MAC, 43 bytes | +| Echo, flags `0x08`, valid MAC | Echo reply, flags `0x0a`, valid MAC, 44 bytes | +| Close, flags `0x0c`, valid MAC | Session removed, **no reply** | +| No-test open, flags `0x0d`, valid MAC | Reply flags `0x0f`, zero token, valid MAC | +| Open with no HMAC flag and no MAC field | no reply | +| Echo with no HMAC flag and no MAC field | no reply | +| Close with no HMAC flag and no MAC field | no reply, **session survives** | +| Open with MAC computed under a different key | no reply | +| Echo with MAC computed under a different key | no reply | +| Echo with one bit flipped in the MAC | no reply | +| Echo with the MAC field all zeros | no reply | +| Echo with the MAC field truncated to 8 bytes | no reply | +| 19-byte datagram (too short for the MAC field) | no reply | +| 20-byte datagram, flags `0x08`, valid MAC, nothing else | no reply | +| Valid MAC but the HMAC flag cleared in flags | no reply | +| MAC recomputed with the flag cleared, flag cleared | no reply | + +Against a server with **no** key, any request with the HMAC flag set gets no +reply. + +**Session integrity across failures.** After the eight failing cases above were +sent to a live session, the next correctly authenticated echo was served +normally and the received count had advanced only for the accepted packets. + +**Reply authentication is mandatory.** A contaminated-side server that +authenticated its open reply correctly but omitted the HMAC flag from echo +replies caused an upstream 0.9.1 client to discard every echo reply and terminate +abnormally with zero packets received. The observation that matters is the +*client's* reaction, which is black-box; the server was only the stimulus. + +--- + +## 8. Idle Expiry + +Server configured with a 2-second idle timeout. Each trial: open, one echo to +start the idle clock, wait, then two echoes back to back. + +| gap after the last accepted echo | first request after the gap | the request after that | +|---|---|---| +| 5.0 s | answered | answered | +| 6.5 s | answered | answered | +| 6.9 s | answered | answered | +| 7.1 s | answered | **dropped** | +| 7.5 s | answered | **dropped** | +| 9.0 s | answered | **dropped** | + +Deadline = idle timeout + 5 s. The first request past the deadline is still +answered, and the session is released while handling it. + +**A session that has never carried an echo request does not expire.** With the +same 2-second timeout, a session left completely idle for 9 seconds after open +was served normally on its first echo request. + +With the idle timeout disabled, a session idle for 9 seconds was served normally +and remained usable. + +--- + +## 9. Malformed Packet Admission + +Session established; probes sent on the same socket. Prefix used: +`14a75b00` + token + `00000000`. + +| probe | result | +|-------|--------| +| 0–3 byte datagram | no reply | +| 4–11 byte datagram (flags `0x00`) | no reply | +| 12–15 byte datagram (flags `0x00`) | no reply | +| 16-byte datagram (flags `0x00`) | **reply, 28 bytes** | +| magic `15a75b` | no reply | +| magic `000000` | no reply | +| flags `0x10`, `0x20`, `0x40`, `0x80`, `0xf0`, `0xff`, `0x1f` | no reply | +| flags `0x02`, `0x03`, `0x06`, `0x07`, `0x0a` (Reply set) | no reply | +| flags `0x09` (Open\|HMAC) to a non-authenticating server | no reply | +| flags `0x01` (Open) | open reply, new session | +| flags `0x05` (Open\|Close) | open reply, zero token | +| echo-shaped datagram with flags `0x01` | treated as an open; the token bytes are parsed as parameters | +| close with 20 trailing bytes | no reply, session removed | + +A 12-byte datagram with flags `0x04` is a valid close request; a 12-byte +datagram with flags `0x00` is too short to be an echo request and is dropped. + +The Open flag decides *interpretation*, not admission. For an otherwise +admissible datagram it wins over Close (`0x05` is the no-test open form) and over +an echo-shaped body (the token/sequence bytes are read as Open parameter data). +It does not rescue a datagram from an independent rejection: `0x03` (Open|Reply) +is dropped for the Reply flag, `0x09` (Open|HMAC) to a non-authenticating server +is dropped for the authentication mismatch, `0x1f` is dropped for the undefined +flag bits, and an Open whose parameter payload is malformed or out of range is +dropped as well (Section 4.2). + +--- + +## 10. DSCP Negotiation and Application + +The values below are **raw IP TOS / Traffic Class byte** values, which is what +the wire parameter carries — not 6-bit codepoints. Expedited Forwarding is +therefore 184 (0xb8) in this table, not 46. + +| requested DSCP | negotiated value returned | echo replies | +|----------------|---------------------------|--------------| +| 0 | absent (0) | delivered | +| 46 | 46 | delivered | +| 184 | 184 | delivered | +| 255 | 255 | delivered | +| 256 | 256 | **all dropped** | +| 300 | 300 | **all dropped** | +| −5 | −5 | delivered | + +With DSCP disallowed by server policy, every requested value is returned as +absent (0) and replies are delivered normally. + +A value the server cannot apply to its socket is still accepted during +negotiation; the session then opens successfully and answers nothing. + +--- + +## 11. Captures + +| File | Contents | +|------|----------| +| `../captures/server-recv-window.pcapng` | One session exercising sequential, gap, duplicate, reorder and far-gap sequence numbers | +| `../captures/server-close-lifecycle.pcapng` | Client-initiated close, repeated close, and an echo after close — none answered | +| `../captures/server-close.pcapng` | Server-initiated close via the maximum-duration limit; frame 36 carries flags `0x06` | +| `../captures/server-session-identity.pcapng` | Foreign-source-port echo and close, unknown token, and the original endpoint continuing to be served | + +All captures are loopback traffic with the server on a fixed port and the client +on an ephemeral port. + +**Capture container metadata.** Every `.pcapng` under `../captures/` was reviewed +and its container rewritten so that it carries no capture comment, no +section/interface description, no host, user or capture-machine identification, +no capture filter string, and no filesystem path — only the packet records and +the rewriting tool's own version string. Packet bytes and timestamps were **not** +altered: for each file, the full frame hexdump and the per-frame epoch timestamps +were verified byte-identical before and after the rewrite.