feat(cli): route server through server set

`--bind` becomes repeatable, so one process can serve several addresses:
an explicit IPv4 and IPv6 pair on one port, or two interface addresses
that must stay distinct.

The applet now builds a `ServerSet` for every invocation, including a
single bind. There is deliberately no bind-count branch and no direct
path from the CLI to `Server`: one orchestration path means startup
errors, shutdown and listener-failure handling are identical whatever was
configured, and the single-listener case — the common one — keeps that
path exercised in normal use rather than only in tests.

The configuration stays one `ServerConfig`; addresses never enter it, and
`ServerSet` clones it into each listener. Bounds in it are therefore per
listener, which `--max-sessions` help now says. Listener lines are
printed only once the whole set is up, so a set whose second bind fails
no longer announces the first as listening.

The shutdown flag, its polling future and the current-thread runtime are
unchanged: exactly one shutdown future is created and the set fans it out
internally.
This commit is contained in:
Nils Andreas Svee committed 2026-08-14 16:34:02 +07:00
1 parent 09f0d6e02e
commit 2171d02203
4 files changed
+126 -31

No files matched your search

+3 -2
View File
@@ -57,7 +57,7 @@ Tokio must remain optional for `irtt-client`; default builds must remain runtime
### Server architecture (`irtt-server`) ### Server architecture (`irtt-server`)
`irtt-server` is a first-class reusable server crate. It contains the deterministic OPEN/ECHO/session core — packet admission, authentication policy, open negotiation, the bounded session table, open reply construction, echo processing with its per-session receive state and timestamps, per-session rate limiting, session lifetime with idle expiry, the maximum-duration server-initiated close, and client-initiated close — plus a reusable Tokio UDP `Server` that runs one sequential `ServerCore` per listener with caller-controlled shutdown and scheduled idle maintenance. Logical expiry remains exact on authenticated, structurally valid requests, while the runtime also reclaims expired sessions once per second when no traffic arrives. Each reply carries the raw traffic class it must be sent with, which the runtime applies to the listener before every send. A wildcard listener additionally recovers each request's local destination from packet metadata and sends that request's reply from it, on Linux, macOS and FreeBSD; a wildcard bind on a target without that path is refused at construction rather than served from a routing-table source address. Explicit-address listeners are unaffected everywhere. Echo replies fill their payload from the session's negotiated server fill; the policy and its deliberate divergences are in `crates/irtt-server/AGENTS.md`. Two optional negotiation policies restrict what a session may ask the server to provide — a timestamp allowance and DSCP permission — both off by default and both settled during open negotiation rather than in the runtime. Serving multiple listeners in one process is not implemented; that, and ordinary process-level polish and observability decisions, is what remains of the server architecture. `irtt-server` is a first-class reusable server crate. It contains the deterministic OPEN/ECHO/session core — packet admission, authentication policy, open negotiation, the bounded session table, open reply construction, echo processing with its per-session receive state and timestamps, per-session rate limiting, session lifetime with idle expiry, the maximum-duration server-initiated close, and client-initiated close — plus a reusable Tokio UDP `Server` that runs one sequential `ServerCore` per listener with caller-controlled shutdown and scheduled idle maintenance. Logical expiry remains exact on authenticated, structurally valid requests, while the runtime also reclaims expired sessions once per second when no traffic arrives. Each reply carries the raw traffic class it must be sent with, which the runtime applies to the listener before every send. A wildcard listener additionally recovers each request's local destination from packet metadata and sends that request's reply from it, on Linux, macOS and FreeBSD; a wildcard bind on a target without that path is refused at construction rather than served from a routing-table source address. Explicit-address listeners are unaffected everywhere. Echo replies fill their payload from the session's negotiated server fill; the policy and its deliberate divergences are in `crates/irtt-server/AGENTS.md`. Two optional negotiation policies restrict what a session may ask the server to provide — a timestamp allowance and DSCP permission — both off by default and both settled during open negotiation rather than in the runtime. Above that one-listener primitive sits `ServerSet`, the service-level owner of one or more independent `Server`s: it binds them all or none, runs each in its own Tokio task, fans one external shutdown out to them, joins them all, and fails the group if any listener fails or stops early. Ordinary process-level polish and observability decisions are what remains of the server architecture.
- It shares wire semantics with the client through `irtt-proto` and follows the same clean-room boundary. - It shares wire semantics with the client through `irtt-proto` and follows the same clean-room boundary.
- It must **not** depend on `irtt-client` merely to reuse protocol behavior. - It must **not** depend on `irtt-client` merely to reuse protocol behavior.
@@ -65,7 +65,8 @@ Tokio must remain optional for `irtt-client`; default builds must remain runtime
- There is currently no requirement for a blocking server API, alternate async runtimes, a runtime abstraction layer, or a runtime-free server feature. Do not build blocking/runtime variants merely for symmetry with the client, and do not invent a mirrored client-style API family (`BlockingServer`, `AsyncServer`, `ManagedServer`, `BlockingManagedServer`, or similar) ahead of agreement. - There is currently no requirement for a blocking server API, alternate async runtimes, a runtime abstraction layer, or a runtime-free server feature. Do not build blocking/runtime variants merely for symmetry with the client, and do not invent a mirrored client-style API family (`BlockingServer`, `AsyncServer`, `ManagedServer`, `BlockingManagedServer`, or similar) ahead of agreement.
- Protocol/session logic should be separated from socket/runtime orchestration where that materially improves deterministic testing, ownership clarity, state-machine reasoning, or maintainability. A `ServerCore`-like internal boundary is therefore desirable, but its exact name/API is not prescribed, it need not be public, it is not a promise of runtime independence, and simple code should not be tortured merely to make the core artificially pure. - Protocol/session logic should be separated from socket/runtime orchestration where that materially improves deterministic testing, ownership clarity, state-machine reasoning, or maintainability. A `ServerCore`-like internal boundary is therefore desirable, but its exact name/API is not prescribed, it need not be public, it is not a promise of runtime independence, and simple code should not be tortured merely to make the core artificially pure.
- Runtime/server orchestration belongs around that core and may freely use Tokio for UDP sockets, `recv_from`/`send_to`, IPv4/IPv6 listeners, timers, session expiry, max-duration deadlines, shutdown, bounded control/config channels where useful, and `select!`-style orchestration. - Runtime/server orchestration belongs around that core and may freely use Tokio for UDP sockets, `recv_from`/`send_to`, IPv4/IPv6 listeners, timers, session expiry, max-duration deadlines, shutdown, bounded control/config channels where useful, and `select!`-style orchestration.
- The CLI server applet exists behind the `irtt-cli` `server` feature, which is part of that crate's default build. It is thin orchestration/configuration over reusable `irtt-server` — one current-thread Tokio runtime, one explicitly bound listener, one `Server` — and must not grow a second server state machine. Multi-listener orchestration, hostname resolution, daemonization, and configuration files are deliberately absent. - Ownership is layered and the layers do not blur: `Server` is the one-listener primitive (one socket, one `ServerCore`, one sequential loop), and `ServerSet` supervises one or more of them without owning protocol or session state itself. A common `ServerConfig` is cloned into each listener, so its bounds — `max_sessions` included — are per listener and not per process. Listeners share nothing else: no shared `ServerCore`, no global session table or token namespace, no shared clock, and one sequential sender per socket, which is what keeps the socket-wide reply traffic class race-free.
- The CLI server applet exists behind the `irtt-cli` `server` feature, which is part of that crate's default build. It is thin orchestration/configuration over reusable `irtt-server` — one current-thread Tokio runtime, one repeatable `--bind`, one `ServerSet` — and must not grow a second server state machine. **It always uses `ServerSet`, including for a single bind**: there is no direct CLI-to-`Server` path and no bind-count branch, so startup, shutdown and listener-failure semantics are identical whatever was configured, and the singleton case exercises the same orchestration in normal use. Hostname resolution, interface expansion, daemonization, and configuration files are deliberately absent.
- Resource policy: the server should use bounded session/resource policy. Upstream's observed lack of useful default session/per-peer bounds, and its never expiring a session that has not carried an echo request (see `IRTT_SERVER_PROTOCOL_SPEC.md`), are not interoperability requirements and must not be reproduced merely for compatibility. - Resource policy: the server should use bounded session/resource policy. Upstream's observed lack of useful default session/per-peer bounds, and its never expiring a session that has not carried an echo request (see `IRTT_SERVER_PROTOCOL_SPEC.md`), are not interoperability requirements and must not be reproduced merely for compatibility.
- Testing: prefer `irtt-server` for normal compliant fake-peer behavior where practical, as far as its implemented behavior reaches. Keep narrow adversarial/raw peers for intentionally malformed, ambiguous, or non-compliant wire behavior. - Testing: prefer `irtt-server` for normal compliant fake-peer behavior where practical, as far as its implemented behavior reaches. Keep narrow adversarial/raw peers for intentionally malformed, ambiguous, or non-compliant wire behavior.
+27 -10
View File
@@ -168,7 +168,7 @@ Quit with `q` or `Ctrl-C`.
## Server ## Server
The server applet serves one UDP listener and requires an explicit bind address: The server applet requires at least one explicit bind address:
```sh ```sh
irtt-server --bind 127.0.0.1:2112 irtt-server --bind 127.0.0.1:2112
@@ -180,17 +180,34 @@ The same applet is reachable through the dispatcher:
irtt-rs server --bind 192.0.2.10:2112 irtt-rs server --bind 192.0.2.10:2112
``` ```
Repeat `--bind` to serve several addresses from one process, in the order given:
```sh
irtt-server \
--bind 0.0.0.0:2112 \
--bind [::]:2112
```
Every invocation runs through the same multi-listener path, so one address is an
ordinary set of one rather than a separate mode. The policy options below are
applied to every listener, but the listeners are otherwise independent: each has
its own sessions and tokens, so a session belongs to the address it was opened
on and `--max-sessions` bounds each listener rather than the process. Binding is
all or nothing — if any address cannot be bound, none are served — and a port of
`0` selects an unused port per listener. If a listener fails while running, the
others are shut down with it rather than leaving a service configured for two
families answering on one.
An explicit interface address works on every supported system. A wildcard bind An explicit interface address works on every supported system. A wildcard bind
such as `--bind 0.0.0.0:2112` reads each request's local destination from packet such as `--bind 0.0.0.0:2112` reads each request's local destination from packet
metadata and sends that request's reply from the same address, so a client on a metadata and sends that request's reply from the same address, so a client on a
multi-homed host is answered from the endpoint it contacted. That is implemented multi-homed host is answered from the endpoint it contacted. That is implemented
on Linux, macOS and FreeBSD; elsewhere a wildcard bind is refused rather than on Linux, macOS and FreeBSD; elsewhere a wildcard bind is refused rather than
served from a routing-table source address a client would discard. One process served from a routing-table source address a client would discard. Wildcard
serves exactly one address, so run a second process to serve a second address or IPv4 and IPv6 listeners may share one port, as above.
family.
The bound endpoint is printed on startup, which also resolves a port of `0`. Each bound endpoint is printed on startup once every listener is up, which also
`Ctrl-C` stops the server gracefully. resolves a port of `0`. `Ctrl-C` stops the server gracefully.
Session policy is set with options that map directly onto the server library's Session policy is set with options that map directly onto the server library's
configuration; anything left unset keeps the library default: configuration; anything left unset keeps the library default:
@@ -326,10 +343,10 @@ The client, event stream, machine-readable output, multi-target execution, local
The server library and the server applet are implemented: open negotiation, The server library and the server applet are implemented: open negotiation,
session state, echo processing, per-session rate limiting, idle expiry, the session state, echo processing, per-session rate limiting, idle expiry, the
maximum-duration close, HMAC authentication, the negotiated per-session reply maximum-duration close, HMAC authentication, the negotiated per-session reply
traffic class, which is applied on sockets that support it, server fill, and traffic class, which is applied on sockets that support it, server fill,
wildcard reply-source selection on Linux, macOS and FreeBSD. Multiple listeners wildcard reply-source selection on Linux, macOS and FreeBSD, the optional
in one process are not implemented, and optional timestamp and DSCP restriction timestamp and DSCP restriction controls, and serving several listeners from one
controls remain separate concerns. process with independent sessions per listener.
Server replies fill their payload with the negotiated ServerFill mode: `none`, Server replies fill their payload with the negotiated ServerFill mode: `none`,
which is zero-filled, `rand`, or `pattern:` with a repeating hexadecimal which is zero-filled, `rand`, or `pattern:` with a repeating hexadecimal
+61 -9
View File
@@ -10,16 +10,17 @@ use irtt_server::{ServerConfig, TimestampAllowance};
#[command( #[command(
name = "irtt-server", name = "irtt-server",
about = "Minimal IRTT-compatible UDP server", about = "Minimal IRTT-compatible UDP server",
after_help = "Policy options left unset keep the irtt-server library defaults.\nOne process serves exactly one listener; run a second process for a second address." after_help = "Policy options left unset keep the irtt-server library defaults.\nRepeat --bind to serve multiple addresses in one process. All listeners use the same policy but keep independent session namespaces."
)] )]
pub struct ServerArgs { pub struct ServerArgs {
/// Local UDP address to bind, as ADDR:PORT. /// Local UDP address to bind, as ADDR:PORT. Repeatable.
#[arg( #[arg(
long, long,
required = true,
value_name = "ADDR", value_name = "ADDR",
long_help = "Local UDP address to bind, as ADDR:PORT, for example 127.0.0.1:2112 or [::1]:2112. Host names are not resolved.\n\nA wildcard bind such as 0.0.0.0:2112 answers each request from the address it was sent to, using per-packet destination metadata. That is supported on Linux, macOS and FreeBSD; on other systems a wildcard bind is refused and an explicit interface address is required." long_help = "Local UDP address to bind, as ADDR:PORT, for example 127.0.0.1:2112 or [::1]:2112. Host names are not resolved.\n\nRepeat the option to serve several addresses from one process, in the order given. Every listener applies the same policy options, but each keeps its own sessions and tokens, so a session belongs to the address it was opened on. Binding is all or nothing: if any address cannot be bound, none are served.\n\nA wildcard bind such as 0.0.0.0:2112 answers each request from the address it was sent to, using per-packet destination metadata. That is supported on Linux, macOS and FreeBSD; on other systems a wildcard bind is refused and an explicit interface address is required.\n\nA port of 0 selects an unused port per listener, so two such binds get two different ports."
)] )]
pub bind: SocketAddr, pub bind: Vec<SocketAddr>,
/// HMAC key; every request must then carry a valid MAC. /// HMAC key; every request must then carry a valid MAC.
#[arg( #[arg(
@@ -29,11 +30,11 @@ pub struct ServerArgs {
)] )]
pub hmac: Option<String>, pub hmac: Option<String>,
/// Maximum number of simultaneously live sessions. /// Maximum simultaneously live sessions per listener.
#[arg( #[arg(
long, long,
value_name = "COUNT", value_name = "COUNT",
long_help = "Maximum number of simultaneously live sessions. Once the table is full, a session-creating open is dropped silently; nothing is evicted. Zero refuses every session-creating open." long_help = "Maximum number of simultaneously live sessions per listener. Once a listener's table is full, a session-creating open to it is dropped silently; nothing is evicted. Zero refuses every session-creating open.\n\nThis is a per-listener bound, not a process-wide one: with two --bind options, each listener admits up to this many sessions."
)] )]
pub max_sessions: Option<usize>, pub max_sessions: Option<usize>,
@@ -226,16 +227,67 @@ mod tests {
#[test] #[test]
fn bind_is_required_and_parses_both_families() { fn bind_is_required_and_parses_both_families() {
assert!(parse(&[]).is_err()); assert!(
parse(&[]).is_err(),
"a server with no address serves nothing"
);
let ipv4 = parse(&["--bind", "127.0.0.1:2112"]).unwrap(); let ipv4 = parse(&["--bind", "127.0.0.1:2112"]).unwrap();
assert_eq!(ipv4.bind, SocketAddr::from((Ipv4Addr::LOCALHOST, 2112))); assert_eq!(ipv4.bind, [SocketAddr::from((Ipv4Addr::LOCALHOST, 2112))]);
let ipv6 = parse(&["--bind", "[::1]:2112"]).unwrap(); let ipv6 = parse(&["--bind", "[::1]:2112"]).unwrap();
assert_eq!(ipv6.bind, SocketAddr::from((Ipv6Addr::LOCALHOST, 2112))); assert_eq!(ipv6.bind, [SocketAddr::from((Ipv6Addr::LOCALHOST, 2112))]);
assert!(parse(&["--bind", "127.0.0.1"]).is_err()); assert!(parse(&["--bind", "127.0.0.1"]).is_err());
assert!(parse(&["--bind", "localhost:2112"]).is_err()); assert!(parse(&["--bind", "localhost:2112"]).is_err());
// Still one address per option: comma-separated syntax is not a thing
// here, and accepting it silently would be worse than rejecting it.
assert!(parse(&["--bind", "127.0.0.1:2112,[::1]:2112"]).is_err());
}
#[test]
fn bind_is_repeatable_and_keeps_the_requested_order() {
let args = parse(&[
"--bind",
"[::1]:2112",
"--bind",
"127.0.0.1:2112",
"--bind",
"127.0.0.1:2113",
])
.unwrap();
assert_eq!(
args.bind,
[
SocketAddr::from((Ipv6Addr::LOCALHOST, 2112)),
SocketAddr::from((Ipv4Addr::LOCALHOST, 2112)),
SocketAddr::from((Ipv4Addr::LOCALHOST, 2113)),
],
"listeners are served in the order they were requested"
);
}
#[test]
fn the_configuration_is_one_policy_however_many_listeners_there_are() {
// Addresses never enter the server configuration: every listener is
// built from a clone of this one value.
let one = parse(&["--bind", "127.0.0.1:2112", "--max-sessions", "4"])
.unwrap()
.server_config();
let several = parse(&[
"--bind",
"127.0.0.1:2112",
"--bind",
"[::1]:2112",
"--max-sessions",
"4",
])
.unwrap()
.server_config();
assert_eq!(one, several);
assert_eq!(several.max_sessions(), 4, "and it is a per-listener bound");
} }
#[test] #[test]
+35 -10
View File
@@ -3,21 +3,24 @@ use std::{
time::Duration, time::Duration,
}; };
use irtt_server::Server; use irtt_server::ServerSet;
use super::args::ServerArgs; use super::args::ServerArgs;
/// How often the shutdown future samples the process-wide signal flag. /// How often the shutdown future samples the process-wide signal flag.
/// ///
/// The flag a signal handler sets has no async wake mechanism, so the future /// The flag a signal handler sets has no async wake mechanism, so the future
/// polls it on a timer rather than spinning. `Server::run` owns everything after /// polls it on a timer rather than spinning. `ServerSet::run` owns everything
/// that future becomes ready. /// after that future becomes ready, including fanning it out to the listeners.
const SHUTDOWN_POLL_INTERVAL: Duration = Duration::from_millis(50); const SHUTDOWN_POLL_INTERVAL: Duration = Duration::from_millis(50);
/// Runs one listener until the process is asked to shut down. /// Runs the configured listeners until the process is asked to shut down.
/// ///
/// The applet owns exactly one current-thread Tokio runtime, one bound socket /// The applet owns exactly one current-thread Tokio runtime and one
/// and one reusable [`Server`]; the server's own loop is the event loop. /// [`ServerSet`], whatever number of addresses was requested. A single `--bind`
/// is an ordinary set of one and takes the same path as a set of several: one
/// construction, one shutdown future, one supervision rule. Listener
/// concurrency is async I/O concurrency and needs no worker threads.
pub fn run_server( pub fn run_server(
args: ServerArgs, args: ServerArgs,
shutdown_requested: &AtomicBool, shutdown_requested: &AtomicBool,
@@ -32,9 +35,14 @@ async fn serve(
args: ServerArgs, args: ServerArgs,
shutdown_requested: &AtomicBool, shutdown_requested: &AtomicBool,
) -> Result<(), Box<dyn std::error::Error>> { ) -> Result<(), Box<dyn std::error::Error>> {
let mut server = Server::bind(args.bind, args.server_config()).await?; let config = args.server_config();
eprintln!("irtt-server: listening on {}", server.local_addr()?); let set = ServerSet::bind(args.bind, config).await?;
server.run(shutdown_signal(shutdown_requested)).await?; // Only once every listener is up. Announcing one address and then failing
// to bind the next would report a service that never started.
for addr in set.local_addrs() {
eprintln!("irtt-server: listening on {addr}");
}
set.run(shutdown_signal(shutdown_requested)).await?;
Ok(()) Ok(())
} }
@@ -50,11 +58,28 @@ mod tests {
use super::*; use super::*;
/// The ordinary invocation, and the one that keeps the single-listener case
/// on the same orchestration path as every other.
#[test] #[test]
fn an_already_requested_shutdown_binds_and_returns() { fn an_already_requested_shutdown_binds_one_listener_and_returns() {
let args = ServerArgs::try_parse_from(["irtt-server", "--bind", "127.0.0.1:0"]).unwrap(); let args = ServerArgs::try_parse_from(["irtt-server", "--bind", "127.0.0.1:0"]).unwrap();
let shutdown_requested = AtomicBool::new(true); let shutdown_requested = AtomicBool::new(true);
run_server(args, &shutdown_requested).unwrap(); run_server(args, &shutdown_requested).unwrap();
} }
#[test]
fn an_already_requested_shutdown_binds_every_listener_and_returns() {
let args = ServerArgs::try_parse_from([
"irtt-server",
"--bind",
"127.0.0.1:0",
"--bind",
"127.0.0.1:0",
])
.unwrap();
let shutdown_requested = AtomicBool::new(true);
run_server(args, &shutdown_requested).unwrap();
}
} }