Skip to content
icmonoPublic

About

mtr from both ends at once — bidirectional path measurement over iroh

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

bimtr

mtr from both ends at once. Run one agent on each host; they connect to each other (through NAT, via iroh), trace both forward paths, and probe every hop of each path from both sides. The result shows how the routers on the B→A path behave toward A, which neither mtr A→B nor mtr B→A can show. See SPEC.md for the design.

bimtr  bucharest-1 ⇄ fra-2  direct 192.0.2.2  icmp
┌ fra-2 → bucharest-1 ─────────────────────────────────────────────────────────────────────────────┐
│                       │ ──── from fra-2 · TTL-exceeded ──── │ ───── from bucharest-1 · echo ─────│
│  #    Host            │   Loss  Snt  Last   Avg  Best  Wrst │   Loss  Snt  Last   Avg  Best  Wrst│
│  1 ⇄↺ 10.0.0.13       │   0.0%   10   3.0   3.0   3.0   3.0 │   0.0%   10   5.0   5.0   5.0   5.0│
│  2    10.0.0.14       │   2.0%   10   6.0   6.0   6.0   6.0 │   2.1%   10   5.0   5.0   5.0   5.0│
│  3    10.0.0.15       │   2.0%   10   9.0   9.0   9.0   9.0 │ no echo reply (10 sent)            │

Both column groups describe the same routers on the same path. The left group is the path's sender tracing it (TTL-exceeded); the right group is the receiver pinging each of those routers directly (echo).

Only ICMP is used: each side traces (TTL-exceeded) toward the other's public address and pings (echo) the routers on the other side's path. There are no separate end-to-end probes; the last row of a trace (the target answering echo) is the end-to-end loss and RTT.

NAT

Traces don't use the iroh connection, so they work whether it is direct or relayed. Each side traces to the remote address of the direct path iroh selected or, when relayed, to the peer's public IPv4 address:

  • One side behind NAT: the trace toward it ends at its NAT. That's the internet path into its network; only the last step, from the NAT to the host, can't be seen.
  • Both behind NAT: each traces to the other's NAT.
  • No public IPv4 address known: that direction isn't traced; the pane says so.

Each side checks whether the address the peer traces to is one of its own interface addresses and reports the answer. A target that isn't is marked [NAT] (in the pane, the report title and the JSON: local_target_is_nat / remote_target_is_nat), and the header lists who is behind NAT. Many NATs don't answer echo, so the trace often stops a hop or two before the target, just as mtr does. The [NAT] row marker then has no row to sit on; the header and the report title still carry the verdict.

Overlays and private networks

The trace follows the path iroh picked. If both hosts also share a LAN or an overlay (Nebula, WireGuard, Tailscale), iroh usually prefers it, and the trace shows that network: often a single hop. That is the path the hosts actually use. To measure the internet path instead, start both sides with --bind <public-ip>:<port> so only the public address is advertised.

Status

Milestone State
M0 spike (engine ADR, iroh connect) done — ADR 0001
M1 control channel, traces, PathUpdate, text report done
M2 reverse probing, StatsUpdate, correlation, TUI done — netns lab hop-level test passes
M3 E2E probes, clock offset, E2E header dropped: replaced by the trace target's own loss/RTT, plus NAT marking (see "NAT")
M4 UDP/TCP, IPv6, rDNS/ASN, path-change log partial: rDNS/ASN, JSON schema and path-change log done; field runs documented
M5 packaging partial: static musl build works; no CI release job, systemd unit or macOS

The MVP is Linux, IPv4 and ICMP.

Build

cargo build --release                                            # glibc
cargo build --release --target x86_64-unknown-linux-musl -p bimtr  # static, runs on any distro

Privileges

Raw ICMP sockets are preferred:

sudo setcap cap_net_raw+ep "$(command -v bimtr)"

Without CAP_NET_RAW, bimtr falls back to unprivileged ICMP datagram sockets, which need your group in net.ipv4.ping_group_range. Traceroute then reads TTL errors from the socket error queue (IP_RECVERR). The mode in use is logged and sent to the peer in Hello.caps. If neither works, startup fails with an explanation.

Use

# host B (e.g. a server): prints a ticket
bimtr listen --headless

# host A: interactive
bimtr connect <ticket>

# or non-interactive
bimtr report <ticket> --cycles 30 [--json]
  • bimtr id prints this host's node ID; pass it to listen --allow <id>. Without --allow, a listener pins the first peer that completes the handshake and refuses all others until restarted. A ticket only says where to connect; --allow is the real access control.
  • Probe options (--interval, --max-ttl, --protocol) are proposed by the connecting side. The listener clamps them to its limits (interval ≥ 100 ms and large enough that a round fits the 200 probes/s cap, e.g. ≥ 450 ms at --max-ttl 30; TTL ≤ 64), and the connector refuses an answer more aggressive than it asked for.
  • --relay <url> uses a self-hosted iroh-relay; --relay none disables relaying (direct addresses only).
  • --bind <ip:port> binds and advertises only one address (multi-homed hosts).
  • --json-stream <file> appends one NDJSON snapshot per second (all modes).
  • Logs go to stderr when headless and to ~/.cache/bimtr/bimtr.log (mode 0600) with the TUI, never to stdout.
  • The TUI shows a rolling 60 s window ("last 60 s" in each pane title); the text and JSON reports' tables are cumulative since start.
  • p freezes the display only; measurement and the peer's view continue.
  • If the peer's stats stop arriving (the control stream stalls when the path breaks), the header shows "peer stats Ns old": its columns are no longer current.

TUI keys: q quit · p pause · r reset · tab focus · ↑↓ scroll · h path-change log · d names on/off · n IP next to name · a ASN column. Columns adapt to width: ≥110 adds ASN, ≥121 adds StDev, <100 drops Best/Wrst.

Names and AS numbers

Each agent looks up the hops on its own path and sends the results with its PathUpdate, so a private hop is named by its own network's resolver:

  • reverse DNS through the system resolver (/etc/hosts, local DNS), like mtr;
  • origin AS from Team Cymru's DNS service (origin.asn.cymru.com), like mtr -z, for globally routable addresses only. AS??? means Cymru has no record (e.g. unannounced infrastructure addresses).

Each IP is looked up once per session; temporary DNS failures are retried after 1, 5 and 30 minutes. --no-dns / --no-asn turn off the local lookups (no hop IPs leave the host in DNS queries); names the peer looked up are still shown.

Trust and privacy

  • A malicious peer can't make us probe our own loopback, link-local, multicast or broadcast addresses (reverse targets are filtered), can't raise our probe rate (both sides clamp the config; the engine caps everything at 200 probes/s and 64 reverse targets), and can't reach the terminal with escape sequences (every peer string is sanitized on receipt; list sizes are bounded). It can ask us to ping up to 64 unicast IPv4 addresses, private ranges included (providers route those internally and they do answer), and learn the results. Use --allow on listeners.
  • Third parties: by default the n0 public relays and discovery service learn node IDs and IPs (--relay <url> or --relay none avoids that), and every globally routable hop IP goes to Team Cymru in a DNS query (--no-asn).

Reading the panes

Each direction X→Y has two column groups, measuring the same routers from both ends with the only probe each end can use (an end that isn't on the X→Y path can't trace it, so it pings):

  • from X: TTL-exceeded RTT/loss from X's own trace, as in mtr.
  • from Y: echo RTT/loss from Y probing that hop directly. The reply comes back along the tail of the X→Y path, so this column measures the X→Y path in the direction toward Y.

⇄ marks a hop that is on both paths; ↺ means the probe's outbound leg is known too. no reply means the hop never answered echo, which is distinct from loss. Loss is highlighted only when it persists along the path: to later hops in the "from X" column, or to earlier hops in the "from Y" column (their replies cross the same lossy link). Isolated loss, usually ICMP rate limiting, is dimmed. The last answering hop's loss in the "from X" column is the end-to-end figure and is always shown; the first hop's in the "from Y" column is checked against it.

Tests

cargo test                                   # unit + render + schema tests
sudo BIMTR_LAB=1 cargo test -p bimtr --test lab -- --test-threads=1   # netns lab, see docs/lab.md
tests/field/run.sh ./bimtr 30 hostA:hostB # real hosts, see docs/field-results/

Open issues

  1. SPEC §12's lab assertion is imprecise. With loss on R4→A, the "from A" column shows loss at R3 and R4, not "starting at R4", because R3's replies to A also cross R4. The lab asserts the physically correct behaviour; see docs/lab.md.
  2. --cycles N is wall-clock. Counts land slightly above N because of the settle time for the final round's timeouts.
  3. NAT + relay netns variant is not automated. NAT traversal was verified on real hosts instead (eu, behind NAT, hole-punched a direct path both as connector and as listener).
  4. ECMP: the primary responder per TTL is the most frequent one, so on load-balanced hops it can flip, logging path changes and resetting that hop's reverse stats. Secondary responders are pinged, but their echo stats aren't shown.
  5. ↺ adds nothing over ⇄ today: a hop on the other path is exactly a hop whose outbound leg is known. It would carry information only with router-level alias resolution.
  6. Reset (r) is local: the peer-supplied columns keep the peer's cumulative counters.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option. Unless you explicitly state otherwise, any contribution you intentionally submit for inclusion in this work, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

mtr from both ends at once — bidirectional path measurement over iroh

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages