docs(client): clarify TX timestamp correlation

This commit is contained in:
Nils Andreas Svee committed 2026-09-15 12:31:26 +07:00
1 parent ed805e854f
commit 2882beb2c9
4 files changed
+43 -53

No files matched your search

+19 -22
View File
@@ -26,30 +26,27 @@ configuration (`try_enable_tx_timestamping` restores it explicitly) and
timing falls back to userspace `sent_at` as it always has. TX capability timing falls back to userspace `sent_at` as it always has. TX capability
failure is never a public error and is never logged from the library. failure is never a public error and is never logged from the library.
### Correlation invariant: kernel ID == wire sequence ### Best-effort kernel-ID correlation
The client uses the kernel's automatic `OPT_ID` directly as the probe's wire The client uses the kernel's automatic `OPT_ID` as the probe's wire-sequence
sequence number — no separate correlation map. This is safe because: correlation ID — no separate correlation map. On the normal successful-send
path, both counters start at zero after Open and advance together. This is
not a permanent identity: a kernel ID can theoretically be consumed while a
send is being prepared and then survive a subsequent send failure, leaving
kernel IDs one or more steps ahead of `next_wire_seq`. Do not restore an
assumption that failed or nonblocking sends cannot cause that desynchrony.
- **A failed/nonblocking send never consumes an ID.** Linux increments the The effect is confined to the optional timing enhancement. A kernel timestamp
per-socket ID counter (`sk->sk_tskey`) speculatively while building a whose ID cannot be matched to a pending or retained timed-out probe is
datagram and rolls the increment back (`atomic_dec`) on every error path discarded, as is any implausible timestamp; `sent_at` remains the ordinary
out of `__ip_append_data`/`__ip6_append_data` userspace fallback. This is safe because:
(`net/ipv4/ip_output.c`, `net/ipv6/ip6_output.c`), guarded by a local
`hold_tskey` flag. A `WouldBlock` `try_send` therefore can never leave a - **Normal successful sends track together.** `wire_seq` starts at 0 on every
gap in, or otherwise advance, the ID the kernel will assign to the next successful Open (`SessionMachine::commit_open` builds a fresh
*successful* submission. (`include/net/sock.h`'s `_sock_tx_timestamp` `ActiveSession` with `next_wire_seq: 0`) and advances by exactly 1 only via
performs the same increment for the non-corked path.) Client-side, the `commit_probe_sent`, after a confirmed successful socket send. TX
existing send transaction only calls `commit_probe_sent` — which is the timestamping is enabled after that same successful Open and before any probe
only thing that advances `next_wire_seq` — after a successful socket send; is sent, so the normal zero points line up.
a `WouldBlock` result never reaches it.
- **The two counters start together and advance together.** `wire_seq`
starts at 0 on every successful Open (`SessionMachine::commit_open` builds
a fresh `ActiveSession` with `next_wire_seq: 0`) and advances by exactly 1
only via `commit_probe_sent`, which only runs after a confirmed successful
send — the same condition under which the kernel's counter advances by 1.
TX timestamping is enabled only after that same successful Open and before
any probe is sent, so both counters' zero points line up exactly.
- **One socket, one session, no reopen.** `SessionMachine`'s state machine - **One socket, one session, no reopen.** `SessionMachine`'s state machine
only reaches `Open` from `Connected`, and `commit_open`/`commit_local_close` only reaches `Open` from `Connected`, and `commit_open`/`commit_local_close`
never route back to `Connected`. A `Client`/`AsyncClient` cannot reopen a never route back to `Connected`. A `Client`/`AsyncClient` cannot reopen a
+9 -8
View File
@@ -12,14 +12,15 @@
//! After a successful Open, the client adapter best-effort upgrades the //! After a successful Open, the client adapter best-effort upgrades the
//! socket from [`RX_TIMESTAMPING_FLAGS`] to [`error_queue::TX_TIMESTAMPING_FLAGS`] //! socket from [`RX_TIMESTAMPING_FLAGS`] to [`error_queue::TX_TIMESTAMPING_FLAGS`]
//! via [`try_enable_tx_timestamping`]. When that succeeds, the automatic //! via [`try_enable_tx_timestamping`]. When that succeeds, the automatic
//! `SOF_TIMESTAMPING_OPT_ID` counter the kernel assigns to each successfully //! `SOF_TIMESTAMPING_OPT_ID` counter normally tracks the probe wire sequence
//! submitted datagram is used directly as the probe's wire sequence number //! after successful sends, so it is used as a best-effort correlation ID.
//! (see the client crate's `AGENTS.md` for the correlation invariant this //! A kernel ID can theoretically be consumed by a send that later fails,
//! relies on). [`drain_tx_timestamps`] performs a small bounded, nonblocking //! desynchronizing that correlation. [`drain_tx_timestamps`] discards
//! read of `MSG_ERRQUEUE` so the adapter can opportunistically collect those //! unmatched or implausible records; the userspace `sent_at` timestamp remains
//! timestamps without ever waiting for one. The observed kernel TX wall time //! the fallback. It performs a small bounded, nonblocking read of
//! is retained as metadata only; it is not consumed by any measurement in //! `MSG_ERRQUEUE` so the adapter can opportunistically collect timestamps
//! this change. //! without ever waiting for one. The observed kernel TX wall time is retained
//! as metadata only; it is not consumed by any measurement in this change.
use std::{ use std::{
io::{self, IoSliceMut}, io::{self, IoSliceMut},
@@ -624,22 +624,14 @@ mod tests {
)); ));
} }
// NOTE: a live test proving that a failed nonblocking `sendmsg` does not // NOTE: There is no live test for whether a failed nonblocking `sendmsg`
// consume an automatic OPT_ID is not included here. Forcing a reliable, // consumes an automatic OPT_ID. Forcing a reliable, non-flaky `WouldBlock`
// non-flaky `WouldBlock` on a connected loopback UDP socket (e.g. via // on a connected loopback UDP socket (for example through socket-buffer
// socket-buffer saturation) has no deterministic OS-level mechanism // saturation) has no deterministic OS-level mechanism in this environment,
// available in this environment, and the repository's testing policy // and the repository's testing policy forbids sleep/saturation-based
// explicitly forbids faking coverage with sleep/saturation-based // flakiness. The correlation is intentionally best effort: a kernel ID can
// flakiness. That safety is instead established from Linux kernel // theoretically be consumed by a send that later fails. The deterministic
// source: `_sock_tx_timestamp` (include/net/sock.h) and // adapter test covers only the local half: an injected `WouldBlock` does
// `__ip_append_data`/`__ip6_append_data` (net/ipv4/ip_output.c, // not advance the local wire-sequence counter. Any later unmatched kernel
// net/ipv6/ip6_output.c) increment `sk->sk_tskey` speculatively before // timestamp is discarded and the userspace `sent_at` remains the fallback.
// a datagram is fully built and explicitly `atomic_dec` it back out on
// every error path (tracked via a local `hold_tskey` flag), so a failed
// send can never leave a gap in — or otherwise advance — the ID space
// the kernel hands out to successfully submitted datagrams. See the
// client crate's `AGENTS.md` for the full citation. The client-side
// half of this invariant (a `WouldBlock` `try_send` never advances the
// local wire-sequence counter used as the correlation ID) is covered by
// a deterministic test in the adapter module that injects `WouldBlock`.
} }
+5 -5
View File
@@ -427,11 +427,11 @@ impl SessionMachine {
/// automatic `SOF_TIMESTAMPING_OPT_ID` the kernel assigned when the /// automatic `SOF_TIMESTAMPING_OPT_ID` the kernel assigned when the
/// datagram was submitted. /// datagram was submitted.
/// ///
/// `wire_seq` doubles as that correlation ID: both counters start at /// `wire_seq` normally matches that correlation ID after successful
/// zero for a session's first timestamped send and advance by exactly /// sends. This is best effort rather than an invariant: a kernel ID can
/// one only on a confirmed successful submission, so they stay /// theoretically be consumed by a send that later fails, leaving the
/// identical for the life of one open session (see the client crate's /// counters desynchronized. An unmatched or implausible ID is discarded;
/// `AGENTS.md` for the full invariant). /// the probe's userspace `sent_at` remains the fallback.
/// ///
/// Updates a still-pending or already-timed-out probe in place. Never /// Updates a still-pending or already-timed-out probe in place. Never
/// resurrects a completed, evicted, or unknown probe, and never touches /// resurrects a completed, evicted, or unknown probe, and never touches