VOOZH about

URL: https://dev.to/ahzek/architecting-an-ultra-minimal-linux-vm-with-buildroot-part-1-build-break-fix-28pa

⇱ Architecting an Ultra-Minimal Linux VM with Buildroot | Part 1: Build, Break, Fix - DEV Community


This is Part 1 of a multi-part series. We go from zero to a working VM that passes every audit check. Part 2 will cover packaging it into a bootable ISO and squeezing under the 20MB target.

My new semester just started, and one of the first challenges from my OS module was this:

"Build the smallest bootable Linux VM you can. It must only pass the tests in this bash-testing-script, nothing more. Personally, I'd love to see <20MB."

Sounds simple. It's not. Everything has gotten bigger. Software that would've been lean 15 years ago is now bloated by default. But let's try to ace it anyway.


The Audit Script: What We're Actually Building For

Before touching a config file, we need to know what success looks like. The professor gave us audit.sh. If it doesn't return all-OK, the VM fails; no matter how small it is.

Here's what the script checks:

Shell & basic commands

command -v sh >/dev/null 2>&1 && ok "POSIX-Shell"
for cmd in cp mv cat rm ls; do
 command -v "$cmd" >/dev/null 2>&1 && ok "$cmd present"
done

Privilege & process inspection

command -v sudo >/dev/null 2>&1 && ok "sudo present"
if ps -e 2>/dev/null | head -n1 | grep -q 'PID'; then
 ok "ps works correctly"
fi

Networking — needs a valid RFC1918 address, the ip command, and a working lo interface.

Connectivity tools — OpenSSH (both client and sshd running), arp-scan, curl with HTTPS, and nc for raw TCP.

if pgrep -x sshd >/dev/null 2>&1 || pgrep -x dropbear >/dev/null 2>&1; then
 ok "sshd/dropbear running"
fi
command -v arp-scan >/dev/null 2>&1 && ok "arp-scan present"

Data handlingtar, awk, HTTP banner grabbing via netcat.

The challenge: packages like openssh, curl, and arp-scan aren't tiny. Fitting everything under 20MB is going to be a fight.


Full post published on niklas-heringer.com. I write about offensive security, embedded Linux, and building things from scratch. If you want, feel free to follow me there or on GitHub.


Why Buildroot?

A friend in my OS class pointed me toward Buildroot, a set of Makefiles and patches that automates building a complete Linux system from scratch. You specify exactly which packages you want, it builds a cross-compilation toolchain, the kernel, and the root filesystem.

Why not just use Alpine? You could. But Buildroot gives exact control over what ends up in the image. If audit.sh doesn't check for it, it doesn't ship.

The four layers

Layer What it is Our output
Kernel Bridge between software and hardware bzImage
Root Filesystem /bin, /etc, /usr — everything the running system uses rootfs.ext2
BusyBox Single ~1MB binary replacing hundreds of GNU tools built-in
Buildroot Automates compiling all of the above the whole pipeline

A note on the cross-compiler

Even though our build machine and target VM are both x86_64, Buildroot builds its own isolated toolchain first. This guarantees reproducible, minimal output with controlled compiler flags. It's also why the first build takes 30–60 minutes, it's literally compiling a compiler.

Buildroot
├── builds cross-compiler (host-gcc, ~20 min)
├── compiles kernel → bzImage
├── compiles packages (openssh, curl, arp-scan ...)
└── assembles rootfs → rootfs.ext2

Setting Up the Build Environment

We use a Debian Bookworm Docker container. It gives a known-good environment that won't interfere with the host (I had a Kali VM lying around).

sudo apt install -y docker.io
systemctl enable --now docker

sudo docker run --privileged --dns 8.8.8.8 \
 -it --name minlinux debian:bookworm bash

Two flags worth noting:

  • --privileged: needed later when we mount the rootfs image as a loop device
  • --dns 8.8.8.8: without this, apt inside the container silently fails to resolve hostnames

Everything from here runs inside the container unless stated otherwise.


The Build Script: Step by Step

Step 1: Install build dependencies

apt-get update
apt-get install -y \
 wget make gcc g++ unzip bc \
 libncurses-dev rsync cpio xz-utils \
 bzip2 file perl patch python3 git qemu-system-x86

libncurses-dev is for menuconfig. git is required even without cloning anything, some Buildroot package scripts call it internally.

Step 2: Download Buildroot

BUILDROOT_VERSION="2026.02"
wget "https://buildroot.org/downloads/buildroot-${BUILDROOT_VERSION}.tar.xz"
tar -xf buildroot-${BUILDROOT_VERSION}.tar.xz -C /opt/buildroot --strip-components=1

Step 3: Load the base config

cd /opt/buildroot
make qemu_x86_64_defconfig

qemu_x86_64_defconfig gives a working 64-bit QEMU baseline. We use it as a starting point to layer on top of.

Step 4: Apply our configuration

cat >> .config << 'EOF'

# --- System ---
BR2_TARGET_GENERIC_HOSTNAME="minlinux"
BR2_TARGET_GENERIC_ROOT_PASSWD="aeb"
BR2_SYSTEM_DHCP="eth0"

# --- Minimize size ---
BR2_STRIP_strip=y

# --- Filesystem ---
BR2_TARGET_ROOTFS_EXT2=y
BR2_TARGET_ROOTFS_EXT2_SIZE="64M"
BR2_TARGET_ROOTFS_CPIO=y
BR2_TARGET_ROOTFS_CPIO_GZIP=y

# --- SSH ---
BR2_PACKAGE_DROPBEAR=n
BR2_PACKAGE_OPENSSH=y
BR2_PACKAGE_OPENSSH_SERVER=y
BR2_PACKAGE_OPENSSH_CLIENT=y
BR2_PACKAGE_OPENSSH_KEY_UTILS=y

# --- Network ---
BR2_PACKAGE_IPROUTE2=y
BR2_PACKAGE_IPUTILS=y
BR2_PACKAGE_ARP_SCAN=y

# --- curl + TLS ---
BR2_PACKAGE_LIBCURL=y
BR2_PACKAGE_LIBCURL_CURL=y
BR2_PACKAGE_CA_CERTIFICATES=y

# --- sudo ---
BR2_PACKAGE_SUDO=y

# --- pgrep ---
BR2_PACKAGE_PROCPS_NG=y

# --- netcat-openbsd ---
BR2_PACKAGE_NETCAT_OPENBSD=y

# --- BusyBox: sh, ls, cp, mv, cat, rm, ps, tar, awk, ping ---
BR2_PACKAGE_BUSYBOX=y

EOF

make olddefconfig

make olddefconfig resolves the full dependency tree. Any option that requires another package gets pulled in automatically, but it can also silently override your settings. More on that in a minute.

Step 5: The actual build

export FORCE_UNSAFE_CONFIGURE=1
make -j$(nproc) 2>&1 | tee /tmp/build.log
  • FORCE_UNSAFE_CONFIGURE=1: bypasses the check that refuses to run as root (we're in Docker, we're always root)
  • -j$(nproc): parallel compile jobs; if you have <16GB RAM, drop to -j4 to avoid OOM kills
  • tee /tmp/build.log: saves output so you can scroll back if the build crashes at minute 45

Wait 30–60 minutes. When done:

bzImage ~6.4M
rootfs.ext2 ~60M
rootfs.cpio.gz ~11M

Three Gotchas That Cost Me Hours

1. The Dropbear trap

My first build used Dropbear instead of OpenSSH. It's a fraction of the size and does SSH perfectly well. Then the audit script hit me:

if ! pgrep -x sshd >/dev/null; then
 warn "sshd inactive"
fi

pgrep -x matches exact process names. Dropbear runs as dropbear in the process table, not sshd. No symlink or alias changes what the kernel reports in /proc. The audit would fail every time.

Lesson: Read the audit script before choosing packages. The system follows the test, never the other way around.

2. HTTPS ≠ HTTP + TLS library

curl was compiled, TLS support was there, but HTTPS failed:

curl -sL https://www.google.com/ >/dev/null 2>&1 # FAIL

Root cause: no CA certificates. Without BR2_PACKAGE_CA_CERTIFICATES=y, curl has no way to verify the certificate chain for any HTTPS connection. One line in the config, hours of confusion.

3. No single netcat does everything

The audit uses nc in two incompatible ways:

# Raw TCP: uses -q (quit after EOF delay)
nc -l -p 12345 -q 1 >/dev/null 2>&1 &

# Port scan: uses -z (zero-I/O mode)
nc -z -w1 "$CLIENT_IP" "$PORT"

BusyBox nc has -q but not -z. netcat-openbsd has -z but not -q. No single implementation satisfies both. Solution: install netcat-openbsd. It handles -z correctly and silently ignores -q rather than crashing, and the raw TCP test still passes because the pipe closes stdin naturally anyway.


First Boot: 4 Failures

After 45 minutes of building, boot the VM, run the audit:

[OK] POSIX-Shell
[OK] cp, mv, cat, rm, ls
[OK] ps
[OK] lo interface
[OK] ssh client
[NOK] sshd couldn't be started ← pgrep missing
[OK] SSH key setup
[OK] Ping / DNS
[OK] arp-scan
[NOK] nc missing ← never built
[OK] curl / HTTP
[NOK] HTTPS failed ← no CA certs
[NOK] HTTP banner not found ← nc cascade

make olddefconfig had silently dropped BR2_PACKAGE_PROCPS_NG and BR2_PACKAGE_NETCAT_OPENBSD. The defconfig baseline had its own opinions, and dependency resolution overwrote ours without a warning.


The Fix: No Full Rebuild

The toolchain, kernel, and everything that built correctly is already cached. We only need the missing packages:

cd /opt/buildroot
export FORCE_UNSAFE_CONFIGURE=1

make procps-ng-rebuild
make netcat-openbsd-rebuild

~30 seconds each. Verify the binaries landed:

find output/target -name "pgrep" # → output/target/bin/pgrep ✓
find output/target -name "nc" # → output/target/usr/bin/nc ✓

Then rebuild just the filesystem:

make rootfs-ext2

30 seconds. Boot. Run the audit.


Second Boot: All Green

✓ POSIX Shell
✓ ls, cp, mv, cat, rm
✓ ps
✓ lo interface
✓ ssh / key setup
✓ Ping 8.8.8.8 / DNS
Network: 10.0.2.15/24
-- Live Hosts (ARP) --
10.0.2.2 52:55:0a:00:02:02
10.0.2.3 52:55:0a:00:02:03
✓ All tests complete

The only sections not passing are the SSH Client Analysis and OS Fingerprinting; those require connecting via SSH from an external machine. When you SSH in through port-forwarded 2222, the script detects your client IP and scans you back. That's working as designed. The OS Fingerprinting section is a TODO the professor intentionally left for each team.


Key Takeaways

  1. Read the audit script before choosing packages. 15 minutes upfront saves hours of debugging.
  2. make olddefconfig can silently drop your config options. Always verify what ended up in the image, not what you put in the config.
  3. HTTPS requires CA certificates, not just TLS support. BR2_PACKAGE_CA_CERTIFICATES=y is not optional.
  4. No single netcat handles all flag combinations. netcat-openbsd is the pragmatic choice here.
  5. Incremental rebuilds are fast. make <package>-rebuild + make rootfs-ext2 takes seconds. No need to start from scratch.

What's Next

In Part 2 I'll tackle:

  • Packaging into a bootable ISO
  • Squeezing under 20MB (rootfs.cpio.gz is 11MB + bzImage at 6.4MB = ~17.4MB before ISO overhead.. it's going to be tight)
  • Implementing the OS Fingerprinting TODO