Add proptest-based generative tests for irtt-stats's stateful rolling
windows and IPDV pairing.
The rolling_count and rolling_time bounds turn out to be two
independent VecDeques (rolling.rs), not a single window bounded by
both, so the main property test proves that independence: it drives a
StatsCollector configured with both bounds simultaneously through
generated operation streams, checking after every operation that each
window matches a from-scratch reference model applying only its own
bound (and that the cumulative snapshot matches a full-history
replay). Since both the production window and the reference replay
the same events through the same deterministic aggregation code, the
comparison is exact equality, not a tolerance.
Also add two smaller property tests for IpdvTracker's bounded FIFO
state and at-most-once adjacent-pair emission under arbitrary
(including non-monotonic and duplicate) sequence streams, complementing
the existing strictly-increasing hand test.
No discrepancies between production and the reference model were
found.
Add proptest-based property tests for irtt-proto's varint and Params
codecs, covering the full u64/i64 domains for varint/zigzag round trips
and generated valid Params for semantic round trips, presence
omission rules, repeated-tag last-wins behavior, and unknown-tag
tolerance.
cargo-deny's licenses check requires a license expression even on an
unpublished crate; add the workspace's Apache-2.0 license so the check
passes rather than treating irtt-integration-tests as unlicensed.
irtt-client and irtt-server mutually dev-depended on each other for
cross-crate integration tests. That cycle blocks `cargo publish` for
either crate whenever the other doesn't already exist on the registry
at the required version, which is exactly the situation for a crate's
first-ever release (irtt-server has never been published before).
Introduce a new `irtt-integration-tests` crate (publish = false, no
library target) to own tests whose actual subject is the interaction
between two or more irtt-* production crates:
crate-local behavior -> crate's own tests/
cross-crate/system behavior -> irtt-integration-tests
Move every test that imports both irtt_client and irtt_server there:
the client-side support harness (hmac, large_packets, protocol_options,
fill_negotiation) and the server-side real-socket tests that drive a
real AsyncClient against a real Server (async_client, negotiation_policy,
fill_wire_bytes, traffic_class, wildcard_source). server_set.rs stays in
irtt-server/tests since it only uses irtt_proto as test machinery to
drive irtt-server directly, which is a normal one-directional test
dependency, not a peer relationship.
irtt-server's dev-dependency on irtt-client is dropped entirely now
that nothing under crates/irtt-server/tests needs it, which fully
breaks the cycle from that side.
irtt-client keeps a one-directional dev-dependency on irtt-server: a
handful of white-box unit tests in src/async_client/tests.rs assert on
private AsyncClient internals (socket, remote, test_hooks) while also
needing a real in-tree server for blocking/async equivalence checks, so
they can't move to an external crate without losing that access. Their
InTreeServer helper is duplicated locally rather than shared, matching
the existing precedent of irtt-cli's own tests/support/in_tree_server.rs.
Each workspace crate now owns its own version instead of the previous
lockstep scheme; irtt-proto, irtt-client, irtt-server, irtt-stats, and
irtt-cli all become 0.5.0 as the first independently-versioned release.
Workspace dependency requirements are bumped to match.
Add a root CHANGELOG.md with a real per-crate 0.5.0 section drawn from
archaeology of every merge since v0.4.0, not just path-filtered commits.
Move cargo-dist to a crate-qualified `irtt-cli/vX.Y.Z` tag namespace so
only irtt-cli (the one dist-able application) triggers a GitHub Release;
library crate tags (irtt-proto/irtt-client/irtt-server/irtt-stats) exist
purely to pin the source for a crates.io release and no longer risk
tripping the release workflow. Regenerate the renamed
irtt-cli-release.yml via `dist generate` and update the few other files
that referenced the old release.yml path by name. Also fix a stale
`features = ["full"]` in dist-workspace.toml left over from a feature-flag
refactor that dropped the "full" feature entirely, which was silently
breaking every dist build.
Rescope semver.yml's tag trigger to the same five crate-qualified tags
so semver checks keep running under the new convention.
- Error-queue drain: an interrupted MSG_ERRQUEUE read no longer ends
the bounded drain early as if the queue were empty (which could
strand a still-queued TX timestamp past the point its probe is
looked up and removed). It now reports ErrorQueueRecord::Ignored,
consuming one attempt of the drain's existing per-call work budget
and retrying the same slot instead.
- Fake-server recv_from test helpers (client_multi_target.rs and
tests/support/mod.rs) now keep one absolute deadline across
interruptions, re-applying only the remaining time on each retry
and restoring the configured timeout afterward, mirroring the
production blocking-receive fix these tests exist to guard rather
than restarting the full timeout on every interruption.
Real signal delivery on Linux (observed via x86_64-musl workspace
testing) can interrupt blocking and nonblocking recv/recvmsg calls
with EINTR. Several production and test receive paths treated that as
a fatal error rather than a transient interruption:
- Client::open's blocking receive loop and Client::recv_once/
recv_available's blocking receive returned ClientError::Socket on
Interrupted instead of retrying within their existing timeout
budget.
- AsyncClient::poll_open and poll_recv's nonblocking recvmsg path
(Linux ancillary) failed the same way instead of retrying through
the existing readiness/work-budget machinery.
- The server's ancillary receive loop could terminate the listener on
an interrupted nonblocking recvmsg.
- The client's MSG_ERRQUEUE TX-timestamp drain turned an interrupted
read of optional metadata into a fatal ClientError::Socket.
- Fake-server test helpers panicked or misclassified EINTR as a
timeout instead of retrying it.
Interrupted is now retried transparently everywhere a receive can
observe it, while preserving each path's existing timing contract:
- Blocking Open reuses its already-absolute per-attempt deadline, so
interruptions resume within the same attempt without retransmitting
or extending the deadline.
- recv_once/recv_available compute one absolute deadline per logical
receive (when a timeout is configured) so repeated interruptions
cannot extend it past the configured timeout, and restore the
socket's configured read timeout afterward so a later call in the
same recv_available budget is unaffected.
- AsyncClient's poll_open already bounds its retry loop with
OPEN_POLL_WORK_BUDGET; poll_recv gains an equivalent
RECV_POLL_WORK_BUDGET, since unlike WouldBlock, Interrupted does not
clear Tokio's readiness state, so an unbounded retry loop was
otherwise possible under sustained signal delivery.
- The server's receive loop retries Interrupted the same way it
already retries WouldBlock, yielding to the runtime after a bounded
number of consecutive interruptions for the same reason.
- The error-queue drain treats an interrupted read as no record
obtained this drain (deferring to the next natural drain point)
rather than a socket error, matching its existing EWOULDBLOCK
handling.
Deterministic regression tests use a small test-only injection hook
(mirroring existing send-injection test hooks) rather than real signal
timing, covering: Open completing despite interruptions with no
retransmit, Open's deadline not extending under repeated
interruptions, recv_once classifying a reply normally despite
interruptions, recv_once's timeout not extending under repeated
interruptions, and the AsyncClient equivalents for open and recv.
No protocol, session, or timeout-classification behavior changes; no
new dependencies; send-side EINTR handling is unchanged and out of
scope.
timeout_at is pre-send anchored (F-04), so it can precede the same
probe's post-send sent_at when probe_timeout is small relative to the
send-call duration. Rolling-window eviction assumes non-decreasing
event timestamps, so clamp the internal StatsEvent::Loss windowing
value without changing the public EchoLoss.timeout_at field. Also
correct an AGENTS.md claim that sent_at feeds the downstream one-way
delay, which it does not.
The post-send sent_at change does shift the upstream IPDV component's
endpoint, since ReplySample copies sent_at into client_send_mono/wall
for send_ipdv_ns. Document the shift instead of claiming IPDV is
unaffected.
Moves the client userspace probe sent_at measurement to immediately
after a successful UDP send, improving raw RTT and the userspace
upstream-OWD fallback by excluding the socket send-call interval and
the small amount of local work preceding it.
All fallible probe-commit preflight (timeout deadline checked_add,
capacity/collision checks) still runs before the send, from a private
pre-send anchor (SessionMachine::finalize_probe_commit). The public
sent_at is now supplied afterward to an infallible
SessionMachine::commit_probe_sent, so DurationOverflow still guarantees
nothing was transmitted and a WouldBlock try_send still commits
nothing.
The operational probe timeout deadline stays anchored to the pre-send
timestamp on purpose, not to the new post-send sent_at, preserving
existing timeout/loss timing. Linux kernel TX_SOFTWARE plausibility
keeps its own pre-send wall-clock lower bound
(PendingProbe::tx_not_before_wall) instead of the post-send sent_at,
since a legitimate TX_SOFTWARE timestamp can be generated while the
send syscall is still in flight and would otherwise be wrongly
rejected.
Also flags (without touching production code) that irtt-stats'
ReplySample::from_reply_parts feeds the public sent_at directly into
IpdvSample.client_send_mono/client_send_wall_ns, so the upstream/send
IPDV component naturally shifts along with the sent_at semantics
change; per scope this is reported rather than compensated for.
Upstream one-way delay now prefers the correlated Linux TX_SOFTWARE wall
timestamp retained by PendingProbe when it is locally plausible (sent_at
<= kernel_tx <= received_at, all on the client's own wall clock), falling
back to the userspace sent_at.wall sample otherwise. RTT, downstream OWD,
IPDV, and timeout/scheduling semantics are unchanged.
On Linux, best-effort upgrade the socket's SO_TIMESTAMPING flags to
TX_SOFTWARE | OPT_ID | OPT_TSONLY after a successful Open and before the
first Echo probe. Promote the #103 error-queue foundation to production:
MSG_ERRQUEUE is drained opportunistically with a fixed per-call budget at
several natural choke points (after a probe send, before processing a
reply, before evaluating timeouts), and each automatic OPT_ID timestamp is
correlated directly with wire_seq — safe because Linux rolls back its
speculative sk_tskey increment on every send failure, wire_seq and the
kernel counter start together at every Open, and a socket can never be
reopened for a second session. The observed kernel TX wall timestamp is
retained on PendingProbe (surviving pending -> timed-out retention) as
dormant metadata only; RTT, OWD, and IPDV are unchanged and continue to use
the userspace send timestamp exclusively. Capability failure is fully
non-fatal and RX timestamping is unaffected either way. Non-Linux and
non-ancillary builds are unchanged.
The MSG_ERRQUEUE control buffer was a plain [u8; N], which only
guarantees byte alignment. recvmsg's cmsgs() walks cmsghdr references
out of that storage and requires cmsghdr alignment, so this mirrors
the parent module's #[repr(align(8))] ControlBuffer instead of relying
on incidental stack alignment.
Migrate the Linux client's receive kernel timing from SO_TIMESTAMPNS to
SO_TIMESTAMPING with RX_SOFTWARE + SOFTWARE, reported via
SCM_TIMESTAMPING instead of SCM_TIMESTAMPNS. Linux documents fallback
quirks when SO_TIMESTAMP[NS] and SO_TIMESTAMPING are both enabled on a
socket, so this establishes one unambiguous software-timestamp facility
before client TX timestamp work builds on the same API.
ReceiveMeta.kernel_rx_timestamp semantics, PacketMeta propagation, and
all PR #100 downstream OWD/RTT/IPDV behavior are unchanged. The receive
control buffer is now sized deliberately from cmsg_space::<T>() instead
of an unexplained fixed constant.
Also add safe, test-only Linux primitives that a follow-up change needs
for client TX timestamping: SO_TIMESTAMPING TX flag construction,
nonblocking MSG_ERRQUEUE parsing, and extended-error classification
that distinguishes a genuine send-timestamp completion from a malformed
timestamp record and from a real socket/network error. Production
sockets do not enable SOF_TIMESTAMPING_TX_SOFTWARE and nothing drains
the error queue in production; these primitives only run under
#[cfg(test)].
An echo reply's receive wall timestamp now prefers the kernel arrival
reading the Linux listener captured, where that reading is plausible for
the datagram it describes. That removes socket-queue and runtime wakeup
latency from the upstream one-way delay a client computes, and it is the
only thing the observation is allowed to change.
The runtime reaches the core through a crate-private entry point taking
the optional observation as an explicit per-datagram argument; the public
ServerCore::handle_datagram is unchanged in signature and in behavior, and
both run one implementation. There is no "next receive timestamp" side
channel and no clock wrapper.
ClockSample stays one paired userspace instant: the preferred wall value
is carried beside it on the classified echo, never inside it. Lifecycle,
rate and maximum-duration policy remain on received_at.mono_ns, midpoints
remain the mean of the paired userspace samples, and no monotonic instant
is estimated from a wall reading. A reading that does not convert, that is
later than the core's own sample, or that lags it by more than one second
falls back to the userspace wall; it is never a reason to drop a datagram.
Rolling snapshots are recomputed with `SampleMode::RunningOnly` regardless
of the configured sample mode, so the new `SampleMode::Exact` wording — that
every timing metric a `Snapshot` exposes reports a median — was not true of
`rolling_count()` and `rolling_time()`.
Scope the promise to the cumulative snapshot and say plainly on both rolling
accessors that they use running statistics only and report no median, since
a rolling window is a bounded recent view rather than the run's retained
history. A test pins both halves so the wording cannot quietly go stale.
Rolling retention behavior itself is unchanged.
`SampleMode::Exact` retained exact samples for eight of the eleven public
timing metrics. Send call, timer error, and server processing always built
their `TimeMetric` with retention disabled, so a finite run reported no
median for them even though they are the same public `TimeStats` type and
appear in the same summary table with a Median column.
Derive one `retain_exact` decision from the sample mode in `CoreStats::new`
and apply it to every timing metric, so exact mode means the same thing for
all of them. Running-only mode is unchanged and still retains none of them.
The retention estimator counted eight exact sample streams per measurable
reply. Rename it to the eleven exact samples an ordinary successful probe
can retain and document the wider optional-measurement upper bound, which
raises the finite estimate on this target from 668 to 764 bytes per probe.
A test counts the metrics a fully measured probe actually retains and
compares it against that constant, so retention and its estimate cannot
drift apart unnoticed.
No new median-capability type was needed: `TimeStats::median_ns` already
distinguishes a retained median from an absent one, and an optional
measurement the server never supplied still produces no samples at all.
Both client applets duplicated the step from parsed arguments to a runnable
managed setup: target specs, managed target preparation, a ClientConfig
built from an arbitrary primary target, pacing conversion, and an identical
ManagedClientConfig skeleton with its own copy of the event-capacity
constant. The primary-target step was the worst of it, since the managed
driver replaces server_addr per target, so the address the applets went out
of their way to find was immediately discarded.
shared::client::prepare now performs that step once: it validates the target
set first, then returns the validated targets, one target-free ClientConfig
template, and the pacing, with managed_config()/managed_targets() for the
driver. CommonClientArgs::to_client_config no longer takes a server address.
Applet-specific semantics are unchanged: each command keeps its own duration
default and help text, the client still accepts no target for
--list-columns, and the TUI still rejects an empty target list at parse
time.
The frame echoes most panel content elsewhere - the session token appears in
recent events, a replying target names a chart series, and the sample
panel's last-warning row satisfies a loose warning search - so whole-frame
assertions kept claiming coverage they did not have.
Panels are now addressed structurally: has_panel matches a title in a top
border, panel_rows slices a panel's interior to its own columns, and
assertions run against the owning panel. That fixes the session token,
recent-events warning line, graph-view panel exclusions and graph-view
target rows, and converts the remaining whole-frame checks in the module so
the class of defect does not come back.
Each tightened assertion was confirmed to fail against a deliberate
regression of the behavior it protects.
Four assertions did not protect what they described. Target labels were
searched across the whole frame, where the first target also appears in the
session header and a replying target appears in recent events and the graph
legend; labels are now matched against the target table's own columns, with
the status column and the replying target's last sample. The sample panel's
effective RTT is now asserted as a whole row, since the fixture's adjusted
and effective values coincide and a bare "2.5ms" still matched with the
effective field dropped. The minimum-height case now rejects dashboard
content like the minimum-width case does, and the compact layout checks for
the timing panel title rather than one incidental run of spaces from the
large timing formatter.
Each tightened assertion was confirmed to fail against a deliberate
regression of the behavior it protects.
The zip loops stopped at the shorter list, and the sequence helpers filter
out everything that is not a probe or reply, so a driver emitting an extra
trailing event of another kind would still have passed. Event-list lengths
are now asserted before the element-wise comparison.
The counter tests instantiated the private TargetCounters and mirrored its
match arms, duplicating outcome-level coverage that already asserts
replies_received and late through ManagedTargetOutcome, and coupling a
private refactor to persistent test churn.
The new counter documentation also defined late replies in timeout-only
terms. A reply that arrives behind an already-received sequence is
classified late by process_echo_reply even when its own probe has not timed
out, so both counters are now described in terms of order and timeout.
The whole-frame substring assertions passed as long as the warning appeared
anywhere, so a sample panel regressing to "last warning: -" would still have
been reported as covered. The assertion now locates the last-warning row and
requires it to carry the warning itself.
The removed test only checked that every failure kind appeared somewhere in
the same hand-maintained case table the mapping test already walks. It could
not prove exhaustiveness or the absence of a fallback, which the compiler
enforces, and it pinned the incidental fact that each category currently has
an input.
The existing conformance suite compared the two driver styles on open
negotiation with HMAC and DSCP, no-test completion, and peer close. It said
nothing about failed-open rollback, multi-probe scheduling, or the errors
each driver promises for out-of-state operations.
Three tests close those gaps against externally observable behavior only:
a first open that times out against a deliberately mute peer, followed by a
retry that opens, probes, and closes with a fresh sequence; three
caller-paced probes against a real irtt-server, matched sequence for
sequence in both drivers; and equivalent errors for probing before open,
opening twice, and probing or closing after close.
No production code changed, and no divergence between the drivers was
found.
Target-local accounting was spread across the managed driver: the event
counter increments were inline in publish_client_events, the authoritative
packets_sent was copied from the client at fifteen separate call sites, and
ManagedTargetOutcome was constructed twice from the same six fields.
TargetCounters::observe now owns the per-event accounting,
TargetRuntime::sync_packets_sent/record_packets_sent own the authoritative
sent count, and TargetRuntime::outcome owns outcome construction, replacing
the free target_outcome function and the inline copy in finish_target. No
counter is touched outside TargetRuntime any more.
The replies_received semantics are unchanged and now documented on both the
private counter and the public outcome field: on-time unique echo replies
only, with late unique replies counted by late. observe matches ClientEvent
exhaustively, so a new event variant must be classified deliberately.
The TUI had pure state and graph-viewport tests but nothing that rendered a
frame, so layout selection and panel presence were unprotected.
Eleven tests drive draw_dashboard through Ratatui's TestBackend across the
durable layouts: below the minimum width and height, the smallest usable
terminal, compact, large, multi-target, and the graph view in both target
modes, including active sample data, warning and error presentation, and
status-line state. Assertions are on visible text and layout facts rather
than full-screen snapshots.
Test-only change; production rendering is untouched.
The TUI suppressed late-reply timing by rebuilding the event with sent_at,
rtt, server_timing, one_way and received_stats erased. That leaked stats
policy into event construction and, as a side effect, reclassified a matched
late reply as an untracked late reply.
StatsConfig now carries an explicit LateReplyMode. Measure keeps today's
behavior for every existing caller. CountOnly keeps the reply a late unique
reply and keeps packet, byte and late counters updating, while suppressing
timing, one-way, server-processing and IPDV contributions and reporting
contributed_sample == false. Cumulative and rolling snapshots share the
policy. Genuinely untracked late replies stay untracked in both modes.
The TUI selects CountOnly and passes the original event straight to the
collector.
Stream output, the run summary, and the TUI each carried their own copy of
the same scalar vocabulary, and they had drifted: the TUI spelled
microseconds "us" while the others used "µs", percentages guarded
non-finite values in one place but not another, and the absent-value
placeholder was spelled inline everywhere.
cmd::format now owns nanosecond-scalar, percentage, and count formatting
with one documented unit-selection and rounding policy, µs everywhere, and
signed values preserved. The frontends keep their own metric sets and
layouts; the TUI keeps its coarse wall-clock span policy, now named
format_span and documented as deliberately distinct, and uses the shared
scalar vocabulary below one second.
classify_client_error now matches every ClientError variant explicitly, so a
new variant fails to compile here until its real classification is chosen.
NoTokioRuntime classifies as InvalidState and DatagramLengthMismatch as
Socket; ManagedTargetFailureKind::Other survives only as a deliberate
category with no default path into it.
ManagedTargetFailurePhase and ManagedTargetFailureKind gain controlled
Display impls, and the CLI client/TUI failure diagnostics use them instead
of leaking Debug enum spelling into user-facing text.
Remove the optional "stats" feature by merging it into the "client" and "tui" features to reduce conditional compilation complexity. Replace the "client-runtime" internal feature with direct dependency sets.
`--bind` becomes repeatable, so one process can serve several addresses:
an explicit IPv4 and IPv6 pair on one port, or two interface addresses
that must stay distinct.
The applet now builds a `ServerSet` for every invocation, including a
single bind. There is deliberately no bind-count branch and no direct
path from the CLI to `Server`: one orchestration path means startup
errors, shutdown and listener-failure handling are identical whatever was
configured, and the single-listener case — the common one — keeps that
path exercised in normal use rather than only in tests.
The configuration stays one `ServerConfig`; addresses never enter it, and
`ServerSet` clones it into each listener. Bounds in it are therefore per
listener, which `--max-sessions` help now says. Listener lines are
printed only once the whole set is up, so a set whose second bind fails
no longer announces the first as listening.
The shutdown flag, its polling future and the current-thread runtime are
unchanged: exactly one shutdown future is created and the set fans it out
internally.
`Server` answers on one address. A host that must serve several — an
explicit IPv4 and IPv6 pair, or two interface addresses — needs several
of them, and something has to own their construction, their shutdown and
each other's failures.
`ServerSet` is that owner, and deliberately not a protocol object: it
holds no session state, no tokens and no clock, and nothing it does is
visible on the wire. Each listener stays a complete `Server` with its own
socket, core, session table and tokens, so a token issued by one is
unknown at the others and every bound in the cloned `ServerConfig` —
`max_sessions` included — applies per listener rather than per process.
Construction binds every listener or none, preserving requested order and
recording the addresses actually bound. `run` moves each listener into
its own task, fans one shutdown future out over a watch channel, and
joins them all: a listener that fails, or that returns before shutdown
was asked for, shuts its siblings down and fails the set, because a
service configured for two families must not continue silently as one.
A genuine IPv6 listener sharing an explicit port with an IPv4 sibling is
created `IPV6_V6ONLY` before it is bound, since a dual-stack socket would
otherwise claim the IPv4 port too. That is the only pre-bind step, it
uses safe `socket2` APIs, and an IPv4-mapped address is read as the IPv4
listener it is — never as the IPv6 half of a pair. Standalone
`Server::bind` semantics are unchanged.
Nils Andreas Svee