Real signal delivery on Linux (observed via x86_64-musl workspace
testing) can interrupt blocking and nonblocking recv/recvmsg calls
with EINTR. Several production and test receive paths treated that as
a fatal error rather than a transient interruption:
- Client::open's blocking receive loop and Client::recv_once/
recv_available's blocking receive returned ClientError::Socket on
Interrupted instead of retrying within their existing timeout
budget.
- AsyncClient::poll_open and poll_recv's nonblocking recvmsg path
(Linux ancillary) failed the same way instead of retrying through
the existing readiness/work-budget machinery.
- The server's ancillary receive loop could terminate the listener on
an interrupted nonblocking recvmsg.
- The client's MSG_ERRQUEUE TX-timestamp drain turned an interrupted
read of optional metadata into a fatal ClientError::Socket.
- Fake-server test helpers panicked or misclassified EINTR as a
timeout instead of retrying it.
Interrupted is now retried transparently everywhere a receive can
observe it, while preserving each path's existing timing contract:
- Blocking Open reuses its already-absolute per-attempt deadline, so
interruptions resume within the same attempt without retransmitting
or extending the deadline.
- recv_once/recv_available compute one absolute deadline per logical
receive (when a timeout is configured) so repeated interruptions
cannot extend it past the configured timeout, and restore the
socket's configured read timeout afterward so a later call in the
same recv_available budget is unaffected.
- AsyncClient's poll_open already bounds its retry loop with
OPEN_POLL_WORK_BUDGET; poll_recv gains an equivalent
RECV_POLL_WORK_BUDGET, since unlike WouldBlock, Interrupted does not
clear Tokio's readiness state, so an unbounded retry loop was
otherwise possible under sustained signal delivery.
- The server's receive loop retries Interrupted the same way it
already retries WouldBlock, yielding to the runtime after a bounded
number of consecutive interruptions for the same reason.
- The error-queue drain treats an interrupted read as no record
obtained this drain (deferring to the next natural drain point)
rather than a socket error, matching its existing EWOULDBLOCK
handling.
Deterministic regression tests use a small test-only injection hook
(mirroring existing send-injection test hooks) rather than real signal
timing, covering: Open completing despite interruptions with no
retransmit, Open's deadline not extending under repeated
interruptions, recv_once classifying a reply normally despite
interruptions, recv_once's timeout not extending under repeated
interruptions, and the AsyncClient equivalents for open and recv.
No protocol, session, or timeout-classification behavior changes; no
new dependencies; send-side EINTR handling is unchanged and out of
scope.
Moves the client userspace probe sent_at measurement to immediately
after a successful UDP send, improving raw RTT and the userspace
upstream-OWD fallback by excluding the socket send-call interval and
the small amount of local work preceding it.
All fallible probe-commit preflight (timeout deadline checked_add,
capacity/collision checks) still runs before the send, from a private
pre-send anchor (SessionMachine::finalize_probe_commit). The public
sent_at is now supplied afterward to an infallible
SessionMachine::commit_probe_sent, so DurationOverflow still guarantees
nothing was transmitted and a WouldBlock try_send still commits
nothing.
The operational probe timeout deadline stays anchored to the pre-send
timestamp on purpose, not to the new post-send sent_at, preserving
existing timeout/loss timing. Linux kernel TX_SOFTWARE plausibility
keeps its own pre-send wall-clock lower bound
(PendingProbe::tx_not_before_wall) instead of the post-send sent_at,
since a legitimate TX_SOFTWARE timestamp can be generated while the
send syscall is still in flight and would otherwise be wrongly
rejected.
Also flags (without touching production code) that irtt-stats'
ReplySample::from_reply_parts feeds the public sent_at directly into
IpdvSample.client_send_mono/client_send_wall_ns, so the upstream/send
IPDV component naturally shifts along with the sent_at semantics
change; per scope this is reported rather than compensated for.
Upstream one-way delay now prefers the correlated Linux TX_SOFTWARE wall
timestamp retained by PendingProbe when it is locally plausible (sent_at
<= kernel_tx <= received_at, all on the client's own wall clock), falling
back to the userspace sent_at.wall sample otherwise. RTT, downstream OWD,
IPDV, and timeout/scheduling semantics are unchanged.
On Linux, best-effort upgrade the socket's SO_TIMESTAMPING flags to
TX_SOFTWARE | OPT_ID | OPT_TSONLY after a successful Open and before the
first Echo probe. Promote the #103 error-queue foundation to production:
MSG_ERRQUEUE is drained opportunistically with a fixed per-call budget at
several natural choke points (after a probe send, before processing a
reply, before evaluating timeouts), and each automatic OPT_ID timestamp is
correlated directly with wire_seq — safe because Linux rolls back its
speculative sk_tskey increment on every send failure, wire_seq and the
kernel counter start together at every Open, and a socket can never be
reopened for a second session. The observed kernel TX wall timestamp is
retained on PendingProbe (surviving pending -> timed-out retention) as
dormant metadata only; RTT, OWD, and IPDV are unchanged and continue to use
the userspace send timestamp exclusively. Capability failure is fully
non-fatal and RX timestamping is unaffected either way. Non-Linux and
non-ancillary builds are unchanged.
The MSG_ERRQUEUE control buffer was a plain [u8; N], which only
guarantees byte alignment. recvmsg's cmsgs() walks cmsghdr references
out of that storage and requires cmsghdr alignment, so this mirrors
the parent module's #[repr(align(8))] ControlBuffer instead of relying
on incidental stack alignment.
Migrate the Linux client's receive kernel timing from SO_TIMESTAMPNS to
SO_TIMESTAMPING with RX_SOFTWARE + SOFTWARE, reported via
SCM_TIMESTAMPING instead of SCM_TIMESTAMPNS. Linux documents fallback
quirks when SO_TIMESTAMP[NS] and SO_TIMESTAMPING are both enabled on a
socket, so this establishes one unambiguous software-timestamp facility
before client TX timestamp work builds on the same API.
ReceiveMeta.kernel_rx_timestamp semantics, PacketMeta propagation, and
all PR #100 downstream OWD/RTT/IPDV behavior are unchanged. The receive
control buffer is now sized deliberately from cmsg_space::<T>() instead
of an unexplained fixed constant.
Also add safe, test-only Linux primitives that a follow-up change needs
for client TX timestamping: SO_TIMESTAMPING TX flag construction,
nonblocking MSG_ERRQUEUE parsing, and extended-error classification
that distinguishes a genuine send-timestamp completion from a malformed
timestamp record and from a real socket/network error. Production
sockets do not enable SOF_TIMESTAMPING_TX_SOFTWARE and nothing drains
the error queue in production; these primitives only run under
#[cfg(test)].
The zip loops stopped at the shorter list, and the sequence helpers filter
out everything that is not a probe or reply, so a driver emitting an extra
trailing event of another kind would still have passed. Event-list lengths
are now asserted before the element-wise comparison.
The counter tests instantiated the private TargetCounters and mirrored its
match arms, duplicating outcome-level coverage that already asserts
replies_received and late through ManagedTargetOutcome, and coupling a
private refactor to persistent test churn.
The new counter documentation also defined late replies in timeout-only
terms. A reply that arrives behind an already-received sequence is
classified late by process_echo_reply even when its own probe has not timed
out, so both counters are now described in terms of order and timeout.
The removed test only checked that every failure kind appeared somewhere in
the same hand-maintained case table the mapping test already walks. It could
not prove exhaustiveness or the absence of a fallback, which the compiler
enforces, and it pinned the incidental fact that each category currently has
an input.
The existing conformance suite compared the two driver styles on open
negotiation with HMAC and DSCP, no-test completion, and peer close. It said
nothing about failed-open rollback, multi-probe scheduling, or the errors
each driver promises for out-of-state operations.
Three tests close those gaps against externally observable behavior only:
a first open that times out against a deliberately mute peer, followed by a
retry that opens, probes, and closes with a fresh sequence; three
caller-paced probes against a real irtt-server, matched sequence for
sequence in both drivers; and equivalent errors for probing before open,
opening twice, and probing or closing after close.
No production code changed, and no divergence between the drivers was
found.
Target-local accounting was spread across the managed driver: the event
counter increments were inline in publish_client_events, the authoritative
packets_sent was copied from the client at fifteen separate call sites, and
ManagedTargetOutcome was constructed twice from the same six fields.
TargetCounters::observe now owns the per-event accounting,
TargetRuntime::sync_packets_sent/record_packets_sent own the authoritative
sent count, and TargetRuntime::outcome owns outcome construction, replacing
the free target_outcome function and the inline copy in finish_target. No
counter is touched outside TargetRuntime any more.
The replies_received semantics are unchanged and now documented on both the
private counter and the public outcome field: on-time unique echo replies
only, with late unique replies counted by late. observe matches ClientEvent
exhaustively, so a new event variant must be classified deliberately.
classify_client_error now matches every ClientError variant explicitly, so a
new variant fails to compile here until its real classification is chosen.
NoTokioRuntime classifies as InvalidState and DatagramLengthMismatch as
Socket; ManagedTargetFailureKind::Other survives only as a deliberate
category with no default path into it.
ManagedTargetFailurePhase and ManagedTargetFailureKind gain controlled
Display impls, and the CLI client/TUI failure diagnostics use them instead
of leaking Debug enum spelling into user-facing text.
`Params::decode` seeded `Params::default()` and overwrote only the tags it
saw, so an omitted tag and one explicitly encoded as zero were
indistinguishable afterwards. The protocol gives those the same value but not
the same meaning, and a server cannot apply its open admission rules without
telling them apart: an absent Duration or Interval is accepted as the wire
default zero, while one explicitly encoded as zero is invalid.
Add `Params::decode_with_presence`, returning the decoded values alongside a
`ParamPresence` recording which known tags appeared. `Params::decode` now
delegates to it, so there is still exactly one parser and the two cannot
diverge. Unknown tags stay ignored and untracked, and repeated known tags keep
last-value-wins.
Add `Clock::Unspecified = 0` as the default, which is the wire default for an
absent Clock tag and what an empty parameter payload decodes to. An explicit
Clock tag encoding zero remains an invalid enum value, so `Unspecified` is
only ever reached by omission; `encode` omits the tag for it rather than
emitting an explicit zero. It selects no clock, so `has_wall`/`has_mono` are
false and no timestamp field is laid out.
`ClientConfig::clock` is public and therefore now constructible as
`Unspecified`, which is a wire default rather than a client mode. Reject it
during config validation. The client default stays `Clock::Both`, so normal
client-generated params and wire vectors are unchanged.
Replace the three per-kind request encoders and decoders with one
directional pair, ahead of introducing irtt-server.
Sender side: encode_request(RequestToEncode, hmac_key) constructs OPEN,
CLOSE and ECHO requests from borrowed values. Params and the ECHO payload
are borrowed rather than owned, removing a Params clone and a payload Vec
from the client's hot paths. Wire output is byte-identical; all captured
vectors still pass unchanged.
Receiver side: decode_request(packet) structurally classifies an inbound
request with no HMAC key, no negotiated Params, no session state and no
allocation. A server must extract a token before it knows which key
applies, so authentication is a separate step. Classification follows the
verified server spec: reserved bits and FLAG_REPLY are rejected, OPEN
takes precedence over CLOSE and over an echo-shaped body, and only the
structural minimums (4/12/16, plus 16 with FLAG_HMAC) are enforced.
Trailing bytes are tolerated, no negotiated length ceiling is applied, and
a zero token is structurally valid. OPEN parameters stay encoded bytes so
an authenticating receiver can verify before decoding untrusted data, and
everything after an ECHO sequence number is exposed as an opaque tail.
Add verify_packet_hmac(key, packet), which locates the standard field
itself and rejects a packet lacking FLAG_HMAC. hmac_present reports packet
layout, never authentication. The offset-based verify_hmac stays public as
the low-level primitive for hand-built and captured wire vectors, matching
the existing compute_hmac pair.
Refactor the private envelope into structural header decoding, codec flag
rules, HMAC presence policy and verification as separate steps. The reply
codecs compose the same steps in the same order, so decode_open_reply and
decode_echo_reply keep their exact error ordering and the client's
fatal-versus-ignore classification is unchanged. Reply decoding stays
state-specific: the client already knows which reply to expect, and the
two need different semantic context.
Migrate the production client and every fake peer to the new API; the
fake peers now admit requests the way a server does, instead of parsing
packets by hand.
An audit of the server-reported receive window found no defect in the
statistics pipeline: the bitmap is decoded raw, stored raw, never
interpreted, and never feeds the directional loss estimates, which use
the cumulative server receive count.
The public rustdoc was silent on all of that, leaving consumers of
PacketCounts free to read a window of 0x1 as a 63-packet loss burst.
Document the field's contract, name the count as the only server input
to LossStats, and cover the window-only case with a regression test.
Eliminate unsafe code blocks and manual pointer arithmetic in Linux-specific receive paths by utilizing nix for socket options and control message parsing. This improves safety and maintainability while preserving the existing kernel RX timestamp and traffic class functionality.
Nils Andreas Svee