Encrypt VLAN/VXLAN tunnel traffic with per-tunnel AES-256 keys#117
Draft
antoncxx wants to merge 6 commits into
Draft
Encrypt VLAN/VXLAN tunnel traffic with per-tunnel AES-256 keys#117antoncxx wants to merge 6 commits into
antoncxx wants to merge 6 commits into
Conversation
Every VLAN/VXLAN link the server builds now gets its own random AES-256 key, sent to both endpoints alongside the existing VlanSetup/VxlanSetup control messages. VLAN traffic is encrypted/decrypted in nullnet-client's userspace forwarder (AES-256-GCM); VXLAN traffic is protected by a per-tunnel kernel IPsec (XFRM/ESP) SA, since that path is handled entirely by the kernel's native vxlan interface. Each VXLAN tunnel also gets its own UDP dstport (replacing the shared 4789 default) so XFRM policies can tell concurrent tunnels between the same host pair apart.
Previously, tunnels whose two endpoints happened to be colocated on the same physical host got no encryption at all: VXLAN's same-host branch used a plain veth pair with no IPsec, and VLAN traffic between two same-host access ports was switched directly by OVS without ever transiting the encrypting userspace forwarder. Neither is reachable by a network sniffer, but both are readable by anything else with sufficient privilege on that same host (a differently-privileged container, a host-level process) — which matters for a project whose whole premise is not trusting other things on the network by default. VXLAN: wrap the same-host veth pair in MACsec (802.1AE, GCM-AES-256), keyed with the same per-tunnel key already delivered for the cross-host IPsec path. Cascades away on teardown when the underlying veth is deleted; also deleted explicitly for clarity. VLAN: replace the single default-normal OVS flow with two more specific ones — traffic arriving from the trunk (already decrypted) is delivered by normal switching; traffic arriving from any access port always exits via the trunk, never directly to another access port. This forces every packet through the TAP and therefore through the existing encrypt/decrypt path in forward/send.rs and forward/receive.rs, with no changes needed there.
iproute2's macsec option parser is positional: `port` (part of this device's own SCI) must appear before `cipher`, not after. The previous order failed with "macsec: unknown command \"port\"?" every time, silently, because the command's stderr was suppressed to tolerate an unrelated race (the veth-pair creation a few lines earlier). None of the four macsec setup commands actually race against the sibling script invocation — each side only touches its own uniquely-named interface — so their stderr is no longer suppressed either, making a real failure here loud instead of silent next time. Found by live-testing on a real same-host deployment: `ip macsec show` was empty despite the veth pair (and its MTU bump) being created correctly, which pointed straight at the interface-creation line right after it.
Diagnostic aid for tracking down intermittent "connection reset" on proxied SSH-over-VLAN: every silent-drop point (missing key, decrypt/auth failure, malformed datagram) and every firewall verdict on both send and receive now logs to stderr with [DEBUG] prefix. In particular, receive.rs now logs before crafting a REJECT reply — that's the one code path that sends a real TCP RST back to the peer, so if a REJECT verdict is firing intermittently on legitimate traffic, this will show it directly instead of us guessing from symptoms alone. Intended to be reverted once the root cause is found — not a permanent logging addition.
The same-host defense-in-depth fix replaced OVS's single default flow
(priority=0,actions=normal) with a generic "everything from an access
port goes to the trunk" rule using a raw `output:<TAP>` action. Unlike
`actions=normal`, `output` does not re-add the 802.1Q tag that access
ports only carry internally (as an implicit port association, not part
of the packet's own bytes) — so every redirected frame arrived at
nullnet-client's TAP already stripped of its VLAN tag, and got dropped
by forward/send.rs as malformed ("Packet missing VLAN tag"), silently,
for every single packet on every VLAN tunnel that hit this path.
Replaced the one generic redirect rule with one precise rule per access
port (installed alongside the port itself in configure_access_port, torn
down alongside it in remove_vlan), each matching only its own in_port and
explicitly pushing the correct 802.1Q tag (push_vlan + mod_vlan_vid)
before sending to the trunk. The original priority=0 default-normal rule
is restored as a safety fallback for the narrow window before a fresh
access port's own rule lands.
Found via live testing: intermittent SSH-over-proxy connection resets
that traced back to a flood of "Packet missing VLAN tag" errors in
nullnet-client's own logs.
Same diagnostic purpose as the earlier debug commit — need to see what this frame actually was (IPv6 NDP/multicast noise vs. something that should have been carried) rather than just knowing the match fell through to the catch-all arm.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every VLAN/VXLAN link the server builds now gets its own random AES-256 key, sent to both endpoints alongside the existing VlanSetup/VxlanSetup control messages. VLAN traffic is encrypted/decrypted in nullnet-client's userspace forwarder (AES-256-GCM); VXLAN traffic is protected by a per-tunnel kernel IPsec (XFRM/ESP) SA, since that path is handled entirely by the kernel's native vxlan interface. Each VXLAN tunnel also gets its own UDP dstport (replacing the shared 4789 default) so XFRM policies can tell concurrent tunnels between the same host pair apart.