docs(client): clarify TX timestamp correlation
This commit is contained in:
4 files changed
+43
-53
No files matched your search
@@ -26,30 +26,27 @@ configuration (`try_enable_tx_timestamping` restores it explicitly) and
|
|||||||
timing falls back to userspace `sent_at` as it always has. TX capability
|
timing falls back to userspace `sent_at` as it always has. TX capability
|
||||||
failure is never a public error and is never logged from the library.
|
failure is never a public error and is never logged from the library.
|
||||||
|
|
||||||
### Correlation invariant: kernel ID == wire sequence
|
### Best-effort kernel-ID correlation
|
||||||
|
|
||||||
The client uses the kernel's automatic `OPT_ID` directly as the probe's wire
|
The client uses the kernel's automatic `OPT_ID` as the probe's wire-sequence
|
||||||
sequence number — no separate correlation map. This is safe because:
|
correlation ID — no separate correlation map. On the normal successful-send
|
||||||
|
path, both counters start at zero after Open and advance together. This is
|
||||||
|
not a permanent identity: a kernel ID can theoretically be consumed while a
|
||||||
|
send is being prepared and then survive a subsequent send failure, leaving
|
||||||
|
kernel IDs one or more steps ahead of `next_wire_seq`. Do not restore an
|
||||||
|
assumption that failed or nonblocking sends cannot cause that desynchrony.
|
||||||
|
|
||||||
- **A failed/nonblocking send never consumes an ID.** Linux increments the
|
The effect is confined to the optional timing enhancement. A kernel timestamp
|
||||||
per-socket ID counter (`sk->sk_tskey`) speculatively while building a
|
whose ID cannot be matched to a pending or retained timed-out probe is
|
||||||
datagram and rolls the increment back (`atomic_dec`) on every error path
|
discarded, as is any implausible timestamp; `sent_at` remains the ordinary
|
||||||
out of `__ip_append_data`/`__ip6_append_data`
|
userspace fallback. This is safe because:
|
||||||
(`net/ipv4/ip_output.c`, `net/ipv6/ip6_output.c`), guarded by a local
|
|
||||||
`hold_tskey` flag. A `WouldBlock` `try_send` therefore can never leave a
|
- **Normal successful sends track together.** `wire_seq` starts at 0 on every
|
||||||
gap in, or otherwise advance, the ID the kernel will assign to the next
|
successful Open (`SessionMachine::commit_open` builds a fresh
|
||||||
*successful* submission. (`include/net/sock.h`'s `_sock_tx_timestamp`
|
`ActiveSession` with `next_wire_seq: 0`) and advances by exactly 1 only via
|
||||||
performs the same increment for the non-corked path.) Client-side, the
|
`commit_probe_sent`, after a confirmed successful socket send. TX
|
||||||
existing send transaction only calls `commit_probe_sent` — which is the
|
timestamping is enabled after that same successful Open and before any probe
|
||||||
only thing that advances `next_wire_seq` — after a successful socket send;
|
is sent, so the normal zero points line up.
|
||||||
a `WouldBlock` result never reaches it.
|
|
||||||
- **The two counters start together and advance together.** `wire_seq`
|
|
||||||
starts at 0 on every successful Open (`SessionMachine::commit_open` builds
|
|
||||||
a fresh `ActiveSession` with `next_wire_seq: 0`) and advances by exactly 1
|
|
||||||
only via `commit_probe_sent`, which only runs after a confirmed successful
|
|
||||||
send — the same condition under which the kernel's counter advances by 1.
|
|
||||||
TX timestamping is enabled only after that same successful Open and before
|
|
||||||
any probe is sent, so both counters' zero points line up exactly.
|
|
||||||
- **One socket, one session, no reopen.** `SessionMachine`'s state machine
|
- **One socket, one session, no reopen.** `SessionMachine`'s state machine
|
||||||
only reaches `Open` from `Connected`, and `commit_open`/`commit_local_close`
|
only reaches `Open` from `Connected`, and `commit_open`/`commit_local_close`
|
||||||
never route back to `Connected`. A `Client`/`AsyncClient` cannot reopen a
|
never route back to `Connected`. A `Client`/`AsyncClient` cannot reopen a
|
||||||
|
|||||||
@@ -12,14 +12,15 @@
|
|||||||
//! After a successful Open, the client adapter best-effort upgrades the
|
//! After a successful Open, the client adapter best-effort upgrades the
|
||||||
//! socket from [`RX_TIMESTAMPING_FLAGS`] to [`error_queue::TX_TIMESTAMPING_FLAGS`]
|
//! socket from [`RX_TIMESTAMPING_FLAGS`] to [`error_queue::TX_TIMESTAMPING_FLAGS`]
|
||||||
//! via [`try_enable_tx_timestamping`]. When that succeeds, the automatic
|
//! via [`try_enable_tx_timestamping`]. When that succeeds, the automatic
|
||||||
//! `SOF_TIMESTAMPING_OPT_ID` counter the kernel assigns to each successfully
|
//! `SOF_TIMESTAMPING_OPT_ID` counter normally tracks the probe wire sequence
|
||||||
//! submitted datagram is used directly as the probe's wire sequence number
|
//! after successful sends, so it is used as a best-effort correlation ID.
|
||||||
//! (see the client crate's `AGENTS.md` for the correlation invariant this
|
//! A kernel ID can theoretically be consumed by a send that later fails,
|
||||||
//! relies on). [`drain_tx_timestamps`] performs a small bounded, nonblocking
|
//! desynchronizing that correlation. [`drain_tx_timestamps`] discards
|
||||||
//! read of `MSG_ERRQUEUE` so the adapter can opportunistically collect those
|
//! unmatched or implausible records; the userspace `sent_at` timestamp remains
|
||||||
//! timestamps without ever waiting for one. The observed kernel TX wall time
|
//! the fallback. It performs a small bounded, nonblocking read of
|
||||||
//! is retained as metadata only; it is not consumed by any measurement in
|
//! `MSG_ERRQUEUE` so the adapter can opportunistically collect timestamps
|
||||||
//! this change.
|
//! without ever waiting for one. The observed kernel TX wall time is retained
|
||||||
|
//! as metadata only; it is not consumed by any measurement in this change.
|
||||||
|
|
||||||
use std::{
|
use std::{
|
||||||
io::{self, IoSliceMut},
|
io::{self, IoSliceMut},
|
||||||
|
|||||||
@@ -624,22 +624,14 @@ mod tests {
|
|||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
|
||||||
// NOTE: a live test proving that a failed nonblocking `sendmsg` does not
|
// NOTE: There is no live test for whether a failed nonblocking `sendmsg`
|
||||||
// consume an automatic OPT_ID is not included here. Forcing a reliable,
|
// consumes an automatic OPT_ID. Forcing a reliable, non-flaky `WouldBlock`
|
||||||
// non-flaky `WouldBlock` on a connected loopback UDP socket (e.g. via
|
// on a connected loopback UDP socket (for example through socket-buffer
|
||||||
// socket-buffer saturation) has no deterministic OS-level mechanism
|
// saturation) has no deterministic OS-level mechanism in this environment,
|
||||||
// available in this environment, and the repository's testing policy
|
// and the repository's testing policy forbids sleep/saturation-based
|
||||||
// explicitly forbids faking coverage with sleep/saturation-based
|
// flakiness. The correlation is intentionally best effort: a kernel ID can
|
||||||
// flakiness. That safety is instead established from Linux kernel
|
// theoretically be consumed by a send that later fails. The deterministic
|
||||||
// source: `_sock_tx_timestamp` (include/net/sock.h) and
|
// adapter test covers only the local half: an injected `WouldBlock` does
|
||||||
// `__ip_append_data`/`__ip6_append_data` (net/ipv4/ip_output.c,
|
// not advance the local wire-sequence counter. Any later unmatched kernel
|
||||||
// net/ipv6/ip6_output.c) increment `sk->sk_tskey` speculatively before
|
// timestamp is discarded and the userspace `sent_at` remains the fallback.
|
||||||
// a datagram is fully built and explicitly `atomic_dec` it back out on
|
|
||||||
// every error path (tracked via a local `hold_tskey` flag), so a failed
|
|
||||||
// send can never leave a gap in — or otherwise advance — the ID space
|
|
||||||
// the kernel hands out to successfully submitted datagrams. See the
|
|
||||||
// client crate's `AGENTS.md` for the full citation. The client-side
|
|
||||||
// half of this invariant (a `WouldBlock` `try_send` never advances the
|
|
||||||
// local wire-sequence counter used as the correlation ID) is covered by
|
|
||||||
// a deterministic test in the adapter module that injects `WouldBlock`.
|
|
||||||
}
|
}
|
||||||
@@ -427,11 +427,11 @@ impl SessionMachine {
|
|||||||
/// automatic `SOF_TIMESTAMPING_OPT_ID` the kernel assigned when the
|
/// automatic `SOF_TIMESTAMPING_OPT_ID` the kernel assigned when the
|
||||||
/// datagram was submitted.
|
/// datagram was submitted.
|
||||||
///
|
///
|
||||||
/// `wire_seq` doubles as that correlation ID: both counters start at
|
/// `wire_seq` normally matches that correlation ID after successful
|
||||||
/// zero for a session's first timestamped send and advance by exactly
|
/// sends. This is best effort rather than an invariant: a kernel ID can
|
||||||
/// one only on a confirmed successful submission, so they stay
|
/// theoretically be consumed by a send that later fails, leaving the
|
||||||
/// identical for the life of one open session (see the client crate's
|
/// counters desynchronized. An unmatched or implausible ID is discarded;
|
||||||
/// `AGENTS.md` for the full invariant).
|
/// the probe's userspace `sent_at` remains the fallback.
|
||||||
///
|
///
|
||||||
/// Updates a still-pending or already-timed-out probe in place. Never
|
/// Updates a still-pending or already-timed-out probe in place. Never
|
||||||
/// resurrects a completed, evicted, or unknown probe, and never touches
|
/// resurrects a completed, evicted, or unknown probe, and never touches
|
||||||
|
|||||||
Reference in new issue
Block a user