What eBPF is
eBPF (extended Berkeley Packet Filter) is a facility in the Linux kernel that lets you load small, sandboxed programs into kernel space and attach them to specific hooks: a tracepoint, a socket, a tc qdisc, a cgroup, a syscall entry, etc. When a packet traverses the hook, or the syscall fires, your program runs inside the kernel with access to that event’s context (skb, registers, process state, whatever the hook exposes). It reads data, updates counters in shared maps, optionally mutates or drops the event, and returns. The crucial property: the kernel verifies your program is safe to run before it loads. You can’t crash the kernel, leak memory, or loop forever. Either the verifier accepts the program as provably safe and it runs with near-native performance, or it rejects the program at load time and nothing happens. That’s what makes eBPF usable in a production datapath. You get kernel-level observability without kernel-level blast radius. It started life as a packet-filtering assembler fortcpdump in 1993 (McCanne & Jacobson’s original BSD Packet Filter) and was extended in 2014 by Alexei Starovoitov into a general-purpose in-kernel VM. Today it’s the substrate underneath Cilium, Pixie, Katran, Cloudflare’s DDoS mitigation, Falco, Tetragon, bpftrace, most modern Linux tracing tools, and an increasing slice of the kernel’s own networking stack. Origin story on LWN.
How the safety story actually works
Three things keep eBPF code from taking down the kernel:- The verifier walks every possible execution path before load. It proves: every memory read is in-bounds against a known-size region, every loop terminates (originally: no loops at all; now: bounded loops with a fixed upper count), every helper call gets the right argument types, pointer arithmetic stays within the pointer’s allowed range, and the program’s total instruction count is finite. A program that fails any of these checks doesn’t load; the kernel returns
EINVALtobpf(BPF_PROG_LOAD). Kernel verifier docs. - Memory access goes through a narrow helper API, not raw pointers. You can’t just dereference arbitrary kernel addresses. You use
bpf_probe_read_kernel(),bpf_skb_load_bytes(), map lookup helpers, etc. Each of these is a well-known function with verifier-enforced bounds. - JIT + runtime isolation. The verifier-accepted program is JIT-compiled into native instructions (on x86_64, aarch64). At runtime the program runs to completion under a bounded-instruction cap; there is no preemption and no kernel-mode recursion into arbitrary code.
svc/heimdall/internal/network/bpf/network.bpf.c cannot crash the kernel and cannot corrupt customer packets. If it has a bug, the worst case is the verifier rejects the next build (we catch it at loadBpfObjects time) or the counters undercount. There is no scenario where a malformed eBPF program takes down the node.
Why eBPF here
Per-pod network byte accounting on a Kubernetes node is a place where eBPF is genuinely the best tool. The alternatives and why they fail:
Our program does the bare minimum: skip the Ethernet header, read the IP header, classify destination IP as RFC1918/public,
__sync_fetch_and_add into one of four counter slots in a shared map keyed by the pod’s netns cookie (bpf_get_netns_cookie, a stable per-netns 64-bit identifier). It returns TC_ACT_UNSPEC (non-terminating; see heimdall.mdx for why this matters with Cilium in the chain), never modifies the packet, and runs alongside Cilium’s own programs without interfering. Userspace (the Go side) periodically reads the map and writes the raw counter values to ClickHouse. Same pattern as CPU/memory; the only difference is that the counters live in a BPF map instead of a sysfs file.
What eBPF gives us that no other tool does
- Per-packet execution at native speed. Our program runs for every packet crossing the pod’s veth, adding roughly 100ns of overhead. At 100k packets/second that’s 10ms of CPU per second. Unmeasurable in practice.
- Namespace-aware. The hook runs in the right context to see the pod’s real packets, before or after Cilium’s datapath depending on where we attach and how we chain.
- Shared userspace/kernel state via maps. A BPF map is a kernel data structure accessible from both the BPF program and userspace through file descriptors. We atomically increment counters in the program; Go reads them with
map.Lookup(). No IPC, no syscalls per packet. - Safe under the customer load path. Because of the verifier, our code is structurally incapable of breaking customer pod networking. See the Safety invariants section of heimdall.mdx.
Further reading
- ebpf.io - What is eBPF?. The canonical overview. Start here.
- Brendan Gregg’s eBPF landing page. Entry point to his tools, book, and many worked performance-analysis examples.
- “Learning eBPF” by Liz Rice. The friendliest book-length intro, from O’Reilly.
- Cilium’s BPF and XDP reference guide. Dense but the best single architecture doc outside the kernel tree. Covers hook types, helpers, maps, verifier idioms.
- Linux kernel verifier docs. The safety story, in source form.
- Meta’s Katran L4 load balancer. The canonical “eBPF at hyperscale” reference. Terabits per second of traffic through BPF programs.
- Cloudflare blog, eBPF tag. A steady stream of production eBPF writeups from the L7 edge of the internet.
Where eBPF lives in this repo
svc/heimdall/internal/network/bpf/network.bpf.c. The only eBPF C program we ship. Around 180 lines, compiled bybpf2gointo an embedded.oplus Go bindings. Runmise run generate-bpfafter editing.svc/heimdall/internal/network/bpf/network_helpers.h. Hand-rolled subset of kernel + libbpf headers (around 180 lines). Replaces a 4.5 MB vmlinux.h that we’d otherwise need to vendor.svc/heimdall/internal/network/network_linux.go. The Go loader + async attach worker pool.svc/heimdall/internal/network/sandbox_linux.go. Resolves each pod’s CNI netns through containerd.svc/heimdall/internal/network/veth_linux.go. Enters the pod netns viasetns, finds the pod-side eth0, reads the netns cookie viaSO_NETNS_COOKIE, and attaches both TCX programs in-netns.