diff --git a/AGENTS.md b/AGENTS.md index 69fda30..c2c54de 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,7 +57,7 @@ Tokio must remain optional for `irtt-client`; default builds must remain runtime ### Server architecture (`irtt-server`) -`irtt-server` is a first-class reusable server crate. It contains the deterministic OPEN/ECHO/session core — packet admission, authentication policy, open negotiation, the bounded session table, open reply construction, echo processing with its per-session receive state and timestamps, per-session rate limiting, session lifetime with idle expiry, the maximum-duration server-initiated close, and client-initiated close — plus a reusable Tokio UDP `Server` that runs one sequential `ServerCore` per listener with caller-controlled shutdown and scheduled idle maintenance. Logical expiry remains exact on authenticated, structurally valid requests, while the runtime also reclaims expired sessions once per second when no traffic arrives. Each reply carries the raw traffic class it must be sent with, which the runtime applies to the listener before every send. The full server fill policy and wildcard-bind packet-info handling remain unimplemented; see `crates/irtt-server/AGENTS.md`. The rest of this section describes the implemented boundary and the parts not yet written. +`irtt-server` is a first-class reusable server crate. It contains the deterministic OPEN/ECHO/session core — packet admission, authentication policy, open negotiation, the bounded session table, open reply construction, echo processing with its per-session receive state and timestamps, per-session rate limiting, session lifetime with idle expiry, the maximum-duration server-initiated close, and client-initiated close — plus a reusable Tokio UDP `Server` that runs one sequential `ServerCore` per listener with caller-controlled shutdown and scheduled idle maintenance. Logical expiry remains exact on authenticated, structurally valid requests, while the runtime also reclaims expired sessions once per second when no traffic arrives. Each reply carries the raw traffic class it must be sent with, which the runtime applies to the listener before every send. A wildcard listener additionally recovers each request's local destination from packet metadata and sends that request's reply from it, on Linux, macOS and FreeBSD; a wildcard bind on a target without that path is refused at construction rather than served from a routing-table source address. Explicit-address listeners are unaffected everywhere. The full server fill policy remains unimplemented; see `crates/irtt-server/AGENTS.md`. The rest of this section describes the implemented boundary and the parts not yet written. - It shares wire semantics with the client through `irtt-proto` and follows the same clean-room boundary. - It must **not** depend on `irtt-client` merely to reuse protocol behavior. diff --git a/README.md b/README.md index 36ab907..70b8ee5 100644 --- a/README.md +++ b/README.md @@ -180,11 +180,14 @@ The same applet is reachable through the dispatcher: irtt-rs server --bind 192.0.2.10:2112 ``` -An explicit interface address is preferred. A wildcard bind such as -`--bind 0.0.0.0:2112` is accepted, but reply source-address selection is then -left to the kernel: per-packet destination-address handling for multi-homed -hosts is not implemented. One process serves exactly one address, so run a -second process to serve a second address or family. +An explicit interface address works on every supported system. A wildcard bind +such as `--bind 0.0.0.0:2112` reads each request's local destination from packet +metadata and sends that request's reply from the same address, so a client on a +multi-homed host is answered from the endpoint it contacted. That is implemented +on Linux, macOS and FreeBSD; elsewhere a wildcard bind is refused rather than +served from a routing-table source address a client would discard. One process +serves exactly one address, so run a second process to serve a second address or +family. The bound endpoint is printed on startup, which also resolves a port of `0`. `Ctrl-C` stops the server gracefully. @@ -304,7 +307,7 @@ The client, event stream, machine-readable output, multi-target execution, local The server library and the server applet are implemented: open negotiation, session state, echo processing, per-session rate limiting, idle expiry, the -maximum-duration close, HMAC authentication, and the negotiated per-session -reply traffic class, which is applied on sockets that support it. The full -server fill policy, per-packet source-address selection for wildcard binds on -multi-homed hosts, and multiple listeners in one process are not implemented. +maximum-duration close, HMAC authentication, the negotiated per-session reply +traffic class, which is applied on sockets that support it, and wildcard +reply-source selection on Linux, macOS and FreeBSD. The full server fill policy +and multiple listeners in one process are not implemented. diff --git a/crates/irtt-cli/src/cmd/server/args.rs b/crates/irtt-cli/src/cmd/server/args.rs index 4963101..ec21582 100644 --- a/crates/irtt-cli/src/cmd/server/args.rs +++ b/crates/irtt-cli/src/cmd/server/args.rs @@ -17,7 +17,7 @@ pub struct ServerArgs { #[arg( long, value_name = "ADDR", - long_help = "Local UDP address to bind, as ADDR:PORT, for example 127.0.0.1:2112 or [::1]:2112. Host names are not resolved.\n\nAn explicit interface address is preferred. A wildcard bind such as 0.0.0.0:2112 is allowed, but reply source-address selection is then left to the kernel: per-packet destination-address handling for multi-homed hosts is not implemented." + long_help = "Local UDP address to bind, as ADDR:PORT, for example 127.0.0.1:2112 or [::1]:2112. Host names are not resolved.\n\nA wildcard bind such as 0.0.0.0:2112 answers each request from the address it was sent to, using per-packet destination metadata. That is supported on Linux, macOS and FreeBSD; on other systems a wildcard bind is refused and an explicit interface address is required." )] pub bind: SocketAddr, diff --git a/crates/irtt-server/AGENTS.md b/crates/irtt-server/AGENTS.md index c78aea1..6cd962d 100644 --- a/crates/irtt-server/AGENTS.md +++ b/crates/irtt-server/AGENTS.md @@ -53,14 +53,56 @@ material does not establish a semantic, state the ambiguity instead of guessing. the listener. Core state is not rolled back after a transport send failure. - Runtime maintenance calls the core's expiry hook. It must not inspect session internals or grow a second expiry rule. -- Explicit-address binds are the source-address-safe supported path. A wildcard - bind on a multi-homed host remains subject to kernel source-address selection - until packet-info handling is implemented. - Do not add a blocking server, an alternate-runtime variant, a transport trait, a runtime abstraction layer, or a mirrored client-style API family. There is no product requirement for any of them, and `ServerCore` being public is not a promise of runtime independence. +## Reply source address + +A reply must leave from the address its request was sent to, or a client reading +from a connected socket discards it. An explicit-address listener satisfies that +by construction and keeps the plain `recv_from`/`send_to` path — do not route it +through ancillary machinery merely because the helper exists. + +A wildcard listener does not, so it recovers each request's local destination +and sends that reply from it. The supported targets are Linux, macOS and +FreeBSD; a wildcard bind anywhere else fails at construction. Correct refusal is +the point: a listener that starts and then answers from a routing-table source +address fails invisibly, as loss. + +- **Destination metadata is runtime transport state.** It never enters + `ServerCore`, `Session` or `OutboundDatagram`. The core knows peers and + sessions; which of the host's addresses a request arrived on is not session + policy, and adding it there would make the core platform-shaped. +- **A wildcard listener must never process a datagram whose local destination it + could not recover.** Missing destination metadata, `MSG_CTRUNC` and `MSG_TRUNC` + all drop the datagram *before* the core, so no receive, rate or lifetime state + moves for a request that cannot be answered correctly. There is no fall back + to kernel source selection: the listener promised correct sources when it + accepted the bind. +- Wildcard-ness is decided once, from the bound address, and never inferred from + a peer. +- Source selection is **per-packet send metadata**; the traffic class stays the + existing socket-wide setting applied before every send. Two transport + mechanisms, deliberately: the marking policy is already implemented and tested + across a wider target matrix, and sequential send ownership is what keeps it + race-free. Do not migrate it into control messages merely because `sendmsg` is + now in reach. +- `recvmsg` and `sendmsg` integrate through Tokio readiness — `readable()` / + `writable()` with `try_io` — following the client's ancillary receive path. + No blocking call, no `spawn_blocking`, no dedicated receive thread, and still + one socket and one task. +- The crate forbids unsafe code and that is not negotiable for this feature. Use + `nix`'s safe `recvmsg`/`sendmsg`, `ControlMessage`, `ControlMessageOwned` and + `sockopt` wrappers; no raw syscall, no `CMSG_*` walking, no hand-written ABI + layer. A target that cannot be served this way is refused, not excepted. +- The receive and send structures are not interchangeable, and the BSD IPv4 + options are not Linux's under other names. Build each direction's control + message explicitly rather than echoing a received one back. +- A send failure is per-packet loss, exactly as an ordinary `send_to` failure + is. It never terminates the listener and never rolls back core state. + ## Rejection is silence Malformed, unauthenticated, unsupported and policy-refused input is discarded diff --git a/crates/irtt-server/src/lib.rs b/crates/irtt-server/src/lib.rs index eace3c9..7b46657 100644 --- a/crates/irtt-server/src/lib.rs +++ b/crates/irtt-server/src/lib.rs @@ -39,14 +39,29 @@ //! not carried an echo request are explicitly not compatibility targets: an //! `irtt-rs` session ages from the moment it is opened. //! +//! # Reply source address +//! +//! A reply leaves from the address its request was sent to. An +//! explicit-address listener gets that from the bind and uses ordinary +//! receive/send. A wildcard listener (`0.0.0.0` or `[::]`) instead reads each +//! request's local destination from packet metadata and sends that request's +//! reply from it, so a client on the host's second address is answered from the +//! endpoint it contacted rather than from whichever one the routing table would +//! have chosen. +//! +//! That path is implemented for Linux, macOS and FreeBSD. On other targets a +//! wildcard bind is refused by [`Server::from_socket`] — and so by +//! [`Server::bind`] — rather than served incorrectly; explicit-address +//! listeners work everywhere they did before. +//! //! # Current scope //! //! Open handling and negotiation, session creation, normal echo processing, //! per-session rate limiting, idle expiry, the maximum-duration -//! server-initiated close, the negotiated per-session reply traffic class and -//! Tokio UDP orchestration. The full server fill policy and wildcard-bind -//! destination-address metadata remain separate slices. Echo payloads are -//! zero-filled. +//! server-initiated close, the negotiated per-session reply traffic class, +//! wildcard reply-source selection and Tokio UDP orchestration. The full server +//! fill policy and running several listeners in one process remain separate +//! slices. Echo payloads are zero-filled. #![forbid(unsafe_code)] mod clock; diff --git a/crates/irtt-server/tests/wildcard_source.rs b/crates/irtt-server/tests/wildcard_source.rs new file mode 100644 index 0000000..2789dda --- /dev/null +++ b/crates/irtt-server/tests/wildcard_source.rs @@ -0,0 +1,214 @@ +//! A wildcard listener answers from the address the request was sent to. +//! +//! Every test here drives a real [`AsyncClient`], which reads from a *connected* +//! UDP socket. That connection is the assertion: the kernel discards any +//! datagram whose source is not the endpoint the client sent to, so a reply +//! that left from another of the host's addresses is indistinguishable from no +//! reply at all. An open, an echo and a close completing therefore prove the +//! source address, without the test having to inspect it. +//! +//! Compiled on the targets that implement wildcard source selection. Elsewhere +//! a wildcard bind is refused outright, which the runtime's own tests cover. + +#![cfg(any(target_os = "linux", target_os = "macos", target_os = "freebsd"))] + +use std::{ + io, + net::{Ipv4Addr, SocketAddr, UdpSocket}, + time::Duration, +}; + +use irtt_client::{AsyncClient, ClientConfig, ClientEvent, OpenOutcome, PacketMeta, SocketConfig}; +use irtt_proto::{Clock, ReceivedStats, StampAt}; +use irtt_server::{Server, ServerConfig, ServerRuntimeError}; +use tokio::{sync::oneshot, task::JoinHandle, time::timeout}; + +/// EF, as the client's configuration takes it, and the raw TOS byte it occupies +/// the upper six bits of. +const EF_DSCP: u8 = 46; +const EF_TRAFFIC_CLASS: u8 = 0xb8; + +#[tokio::test(flavor = "current_thread")] +async fn a_wildcard_ipv4_listener_replies_from_the_requested_address() { + timeout(Duration::from_secs(5), async { + let server = Server::bind("0.0.0.0:0".parse().unwrap(), unthrottled()) + .await + .expect("a wildcard IPv4 listener must be supported on this target"); + let port = server.local_addr().unwrap().port(); + // A second loopback address makes this a genuine multi-homed case: the + // listener is bound to none of the host's addresses in particular, and + // the client contacts one the routing table would not have picked on + // its own. Hosts carrying only 127.0.0.1 — macOS, unless an alias is + // configured — still exercise the whole ancillary path, one address + // wide. + let target = SocketAddr::from((secondary_ipv4_loopback(), port)); + let running = spawn(server); + + let mut client = connected_client(target).await; + assert_started(&mut client, target).await; + + let meta = echo_once(&mut client).await; + // Source selection is per-packet ancillary data and the traffic class + // is a socket-wide setting applied before the send. Both act on the + // same reply, so this is where they have to agree. + if cfg!(target_os = "linux") { + assert_eq!( + meta.traffic_class, + Some(EF_TRAFFIC_CLASS), + "a source-selected reply still carries its session's marking" + ); + assert_eq!(meta.dscp, Some(EF_DSCP)); + assert_eq!(meta.ecn, Some(0)); + } + + assert!(matches!( + client.close().await.unwrap().as_slice(), + [ClientEvent::SessionClosed { .. }] + )); + running.stop().await; + }) + .await + .expect("the wildcard IPv4 source-selection test exceeded its bounded runtime"); +} + +#[tokio::test(flavor = "current_thread")] +async fn a_wildcard_ipv6_listener_replies_from_the_requested_address() { + timeout(Duration::from_secs(5), async { + if !ipv6_loopback_available() { + eprintln!("skipping wildcard IPv6 test: IPv6 loopback unavailable"); + return; + } + let server = match Server::bind("[::]:0".parse().unwrap(), unthrottled()).await { + Ok(server) => server, + Err(ServerRuntimeError::Bind { source, .. }) if is_family_unavailable(&source) => { + eprintln!("skipping wildcard IPv6 test: IPv6 unavailable"); + return; + } + Err(error) => panic!("unexpected wildcard IPv6 bind failure: {error}"), + }; + let port = server.local_addr().unwrap().port(); + let target: SocketAddr = format!("[::1]:{port}").parse().unwrap(); + let running = spawn(server); + + // Loopback is one address, so this does not prove multi-homed IPv6. It + // does exercise the parts that differ from IPv4: the received packet + // info, the interface index carried with it, and the source-selected + // IPv6 send. + let mut client = connected_client(target).await; + assert_started(&mut client, target).await; + echo_once(&mut client).await; + assert!(matches!( + client.close().await.unwrap().as_slice(), + [ClientEvent::SessionClosed { .. }] + )); + + running.stop().await; + }) + .await + .expect("the wildcard IPv6 source-selection test exceeded its bounded runtime"); +} + +/// A local IPv4 address that is not the primary loopback one, where the host +/// has one. +/// +/// Bindability is the test: an address this process can bind is one the host +/// accepts datagrams for. `127.0.0.0/8` supplies a second local destination on +/// hosts that carry the whole range, without touching interface configuration. +fn secondary_ipv4_loopback() -> Ipv4Addr { + let candidate = Ipv4Addr::new(127, 0, 0, 2); + if UdpSocket::bind((candidate, 0)).is_ok() { + return candidate; + } + eprintln!("{candidate} is not a local address on this host; using 127.0.0.1 instead"); + Ipv4Addr::LOCALHOST +} + +fn ipv6_loopback_available() -> bool { + match UdpSocket::bind("[::1]:0") { + Ok(_) => true, + Err(error) if is_family_unavailable(&error) => false, + Err(error) => panic!("unexpected IPv6 loopback bind failure: {error}"), + } +} + +fn is_family_unavailable(error: &io::Error) -> bool { + matches!( + error.kind(), + io::ErrorKind::AddrNotAvailable | io::ErrorKind::Unsupported + ) +} + +fn unthrottled() -> ServerConfig { + ServerConfig::default().with_min_send_interval(Duration::ZERO) +} + +async fn connected_client(server_addr: SocketAddr) -> AsyncClient { + let config = ClientConfig { + server_addr: server_addr.to_string(), + duration: Some(Duration::from_secs(30)), + interval: Duration::from_millis(10), + length: 64, + dscp: EF_DSCP, + received_stats: ReceivedStats::Both, + stamp_at: StampAt::Both, + clock: Clock::Both, + open_timeouts: vec![Duration::from_millis(500)], + socket_config: SocketConfig { + recv_timeout: Some(Duration::from_millis(500)), + ..SocketConfig::default() + }, + probe_timeout: Duration::from_millis(500), + ..ClientConfig::default() + }; + AsyncClient::connect(config).await.unwrap() +} + +async fn assert_started(client: &mut AsyncClient, server_addr: SocketAddr) { + match client.open().await.unwrap() { + OpenOutcome::Started { remote, token, .. } => { + assert_eq!(remote, server_addr); + assert_ne!(token, 0); + } + OpenOutcome::NoTestCompleted { .. } => panic!("a normal client unexpectedly ran no-test"), + } +} + +/// Sends one echo and returns its reply's receive metadata. +async fn echo_once(client: &mut AsyncClient) -> PacketMeta { + client.send_probe().await.unwrap(); + loop { + for event in client.recv().await.unwrap() { + if let ClientEvent::EchoReply { + seq, packet_meta, .. + } = event + { + assert_eq!(seq, 0); + return packet_meta; + } + } + } +} + +struct Running { + shutdown: oneshot::Sender<()>, + task: JoinHandle>, +} + +impl Running { + async fn stop(self) { + self.shutdown.send(()).unwrap(); + self.task.await.unwrap().unwrap(); + } +} + +fn spawn(mut server: Server) -> Running { + let (shutdown, shutdown_rx) = oneshot::channel(); + let task = tokio::spawn(async move { + server + .run(async { + let _ = shutdown_rx.await; + }) + .await + }); + Running { shutdown, task } +}