FAQ
What is the minimum Linux kernel version to run Tetragon?
Tetragon needs Linux kernel version 4.19 or greater.
We currently run tests on stable long-term support kernels 4.19, 5.4, 5.10, 5.15 and bpf-next, see this test workflow for up to date information. Not all Tetragon features work with older kernel versions. BPF evolves rapidly and we recommend you use the most recent stable kernel possible to get the most out of Tetragon’s features.
Note that Tetragon also needs BTF support which might take some work on older kernels.
See the recommended Linux kernel configuration options
# CORE BPF
CONFIG_BPF
CONFIG_BPF_JIT
CONFIG_BPF_JIT_DEFAULT_ON
CONFIG_BPF_EVENTS
CONFIG_BPF_SYSCALL
CONFIG_HAVE_BPF_JIT
CONFIG_HAVE_EBPF_JIT
CONFIG_FTRACE_SYSCALLS
# BTF
CONFIG_DEBUG_INFO_BTF
CONFIG_DEBUG_INFO_BTF_MODULES
# Enforcement
CONFIG_BPF_KPROBE_OVERRIDE
# CGROUP and Process tracking
CONFIG_CGROUPS=y Control Group support
CONFIG_MEMCG=y Memory Control group
CONFIG_BLK_CGROUP=y Generic block IO controller
CONFIG_CGROUP_SCHED=y
CONFIG_CGROUP_PIDS=y Process Control group
CONFIG_CGROUP_FREEZER=y Freeze and unfreeze tasks controller
CONFIG_CPUSETS=y Manage CPUSETs
CONFIG_PROC_PID_CPUSET=y
CONFIG_CGROUP_DEVICE=Y Devices Control group
CONFIG_CGROUP_CPUACCT=y CPU accouting controller
CONFIG_CGROUP_PERF=y
CONFIG_CGROUP_BPF=y Attach eBPF programs to a cgroup
CGROUP_FAVOR_DYNMODS=y (optional) >= 6.0
Reduces the latencies of dynamic cgroup modifications at the
cost of making hot path operations such as forks and exits
more expensive.
Platforms with frequent cgroup migrations could enable this
option as a potential alleviation for pod and containers
association issues.
Tetragon failed to start complaining about a missing BTF file
You might have encountered the following issues:
level=info msg="BTF discovery: default kernel btf file does not exist" btf-file=/sys/kernel/btf/vmlinux
level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/metadata/vmlinux-5.15.49-linuxkit
level=info msg="BTF discovery: candidate btf file does not exist" btf-file=/var/lib/tetragon/btf
[...]
level=fatal msg="Failed to start tetragon" error="tetragon, aborting kernel autodiscovery failed: Kernel version \"5.15.49-linuxkit\" BTF search failed kernel is not included in supported list. Please check Tetragon requirements documentation, then use --btf option to specify BTF path and/or '--kernel' to specify kernel version"
Tetragon needs BTF (BPF Type Format) support to load its BPF programs using CO-RE (Compile Once - Run Everywhere). In brief, CO-RE is useful to load BPF programs that have been compiled on a different kernel version than the target kernel. Indeed, kernel structures change between versions and BPF programs need to access fields in them. So CO-RE uses the BTF file of the kernel in which you are loading the BPF program to know the differences between the struct and patch the fields offset in the accessed structures. CO-RE allows portability of the BPF programs but requires a kernel with BTF enabled.
Most of the common Linux distributions
now ship with BTF enabled and do not require any extra work, this is kernel
option CONFIG_DEBUG_INFO_BTF=y
. To check if BTF is enabled on your Linux
system and see the BTF data file of your kernel, the standard location is
/sys/kernel/btf/vmlinux
. By default, Tetragon will look for this file (this
is the first line in the log output above).
If your kernel does not support BTF you can:
- Retrieve the BTF file for your kernel version from an external source.
- Build the BTF file from your kernel debug symbols. You will need pahole to add BTF metadata to the debugging symbols and LLVM minimize the medata size.
- Rebuild your kernel with
CONFIG_DEBUG_INFO_BTF
toy
.
Tetragon will also look into /var/lib/tetragon/btf
for the vmlinux
file
(this is the third line in the log output above). Or you can use the --btf
flag to directly indicate Tetragon where to locate the file.
If you encounter this issue while using Docker Desktop on macOS, please refer to can I run Tetragon on Mac computers.
Can I install and use Tetragon in standalone mode (outside of k8s)?
Yes! Check the Container deployment or Package deployment guides for alternative install methods.
Otherwise you can build Tetragon from source by running make
to generate standalone
binaries.
Make sure to take a look at the Development Setup
guide for the build requirements. Then use sudo ./tetragon --bpf-lib bpf/objs
to run Tetragon.
CI is complaining about Go module vendoring, what do I do?
You can run make vendor
then add and commit your changes.
CI is complaining about a missing “signed-off-by” line. What do I do?
You need to add a signed-off-by line to your commit messages. The easiest way
to do this is with git fetch origin/main && git rebase --signoff origin/main
.
Then push your changes.
Can I run Tetragon on Mac computers?
Yes! You can run Tetragon locally by running a Linux virtual machine on your Mac.
On macOS running on amd64 (also known as Intel Mac) and arm64 (also know as Apple Silicon Mac), open source and commercial solutions exists to run virtual machines, here is a list of popular open source projects that you can use:
- Lima: Linux virtual machines: website lima-vm.io.
- UTM: Virtual machines for iOS and macOS: website mac.getutm.app.
- VirtualBox: website virtualbox.org (arm64 in developer preview).
You can use these solutions to run a recent Linux distribution that ships with BTF debug information support.
Please note that you cannot use Docker Desktop on macOS, because the Linux virtual machine provided by Docker lacks support for the BTF debug information. The BTF debug information file is needed for CO-RE in order to load sensors of Tetragon. While it is technically possible to manually generate the BTF metadata, you will need the kernel debug symbols, and Docker stopped pushing the kernel images containing those debug symbols.
Issues are open on the GitHub repository for Docker Desktop on macOS and linuxkit, the project used to build the kernel used by Docker Desktop. If you would like Docker Desktop to support BTF, feel free to support those issues.
Go compiler is complaining about missing symbols in pkg/bpf
You might end up with the output of the Go compiler looking similar to this:
# github.com/cilium/tetragon/pkg/bpf
pkg/bpf/map_linux.go:174:9: undefined: bpfAttrObjOp
pkg/bpf/map_linux.go:266:19: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:310:19: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:318:16: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:415:9: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:422:9: undefined: UpdateElementFromPointers
pkg/bpf/map_linux.go:455:9: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:466:9: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:496:9: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:515:9: undefined: bpfAttrMapOpElem
pkg/bpf/map_linux.go:515:9: too many errors
It is likely because this package uses CGO and you might be missing a C compiler or the Go toolchain might fail detecting one. Also note from the CGO documentation that CGO is disabled by default when cross-compiling:
The cgo tool is enabled by default for native builds on systems where it is expected to work. It is disabled by default when cross-compiling as well as when the CC environment variable is unset and the default C compiler (typically gcc or clang) cannot be found on the system PATH. You can override the default by setting the CGO_ENABLED environment variable when running the go tool: set it to 1 to enable the use of cgo, and to 0 to disable it. […]
If you encounter the issue under Linux:
- Check your Go environment information with
go env CC
. The output should be similar togcc
. Verify thatgcc
is installed withgcc --version
. If it’s not the case, installgcc
. - If you prefer to use clang, you can manually set the
CC
env variable toclang
, for example building tetragon withCC=clang make tetragon
. To persistently change the value of the CC go environment variable you can usego env -w "CC=clang"
.