Containerizing Web Applications: Docker, systemd-nspawn, and FreeBSD Jails – Isolation, Performance, and Security
This article compares three leading containerization technologies – Docker (OCI), systemd-nspawn, and FreeBSD Jails – from the perspective of kernel architecture, storage mechanisms (overlay2 vs ZFS), networking (NAT, macvlan, VNET/epair), and security (seccomp, AppArmor, Capsicum). Particular attention is given to performance problems under database workloads, including the io_uring block in Docker containers and native ZFS integration with FreeBSD Jails.
Introduction
Web application deployment has undergone a fundamental transformation over the past decade – from full hardware virtualization (hypervisors) towards OS-level virtualization. This shift brought a dramatic reduction in CPU and memory overhead, making it possible to run hundreds of isolated environments on a single physical server.
The evolution has not been uniform, however. In the Linux ecosystem, Docker – built on the Open Container Initiative (OCI) specification – became the dominant standard, revolutionizing software distribution through the concept of ephemeral, application-centric containers. An alternative within the same ecosystem is systemd-nspawn, a tool integrated with the systemd init manager that treats containers not as single processes but as full OS-level virtual machines with their own init system.
In the BSD world, FreeBSD Jails provide a native, deeply kernel-integrated approach to isolation. The technology dates to 1999, making it considerably more mature than its Linux counterparts. Combined with the VNET virtual network stack and ZFS, FreeBSD Jails create an environment of exceptional coherence and security. ZFS configuration details for database workloads are covered in the article Advanced ZFS Optimization for PostgreSQL and MySQL.
This analysis compares these three technologies from the perspective of kernel architecture, storage mechanisms, networking, and security – with a particular focus on database workloads and web applications.
Isolation architecture: namespaces and cgroups v2 on Linux versus FreeBSD Jails
The Linux container – an illusion assembled from kernel primitives
In Linux there is no single coherent kernel object called a "container." A Linux container is an abstraction that emerges from simultaneously applying two independent kernel subsystems: namespaces and control groups (cgroups). Docker and systemd-nspawn act as orchestrators that configure and wire these subsystems around a target process.
Namespaces change the global visibility of resources for a given process tree:
- Mount (mnt) – isolates the filesystem topology. New mount points are visible only within the namespace.
- PID – processes inside the container see an independent process tree starting from a virtual PID 1.
- Network (net) – virtualizes the entire network stack: interfaces, IP addresses, routing tables, and firewall rules (
iptables/nftables). - User – maps UID/GID identifiers inside the container to unprivileged identifiers on the host. A process may hold root (UID 0) privileges inside the container while remaining an ordinary user on the host – this is the rootless containers architecture.
- IPC – isolates shared memory (System V IPC).
- UTS – isolates the hostname and domain name.
Control groups (cgroups v2) enforce hard limits on resource consumption: CPU time (CPU quotas), RAM (host-level OOM prevention), I/O throughput (throttling), and limits on child processes and file descriptors. Cgroups v2 introduced a Unified Hierarchy that resolved the inconsistency problems present in the older v1 implementation.
Docker versus systemd-nspawn – two approaches on the same kernel
The differences between these tools come down to how they use the same primitives:
Docker assembles namespaces around a single application process (for example an Nginx server or a MariaDB daemon). Docker containers are by design ephemeral and stateless.
systemd-nspawn loads a full init system (systemd) as PID 1 inside the container. It restricts access to critical kernel interfaces by mounting /sys, /proc/sys, and /sys/fs/selinux read-only. It prevents rebooting the host or loading kernel modules from within the container. It integrates with the machinectl facility, enabling containers to be managed as systemd services.
FreeBSD Jails – a monolithic kernel object
FreeBSD implements isolation in a fundamentally different way. The Jails mechanism is not a composition of independent subsystems – the jail_set(2) syscall creates an indivisible kernel object. The kernel always knows whether a given process is inside a jail because the process data structure directly references the jail structure.
A jail extends the classic chroot environment with absolute separation of user spaces, shared memory (System V IPC), the process subsystem, and the network layer. The kernel categorically refuses operations on global objects from within a jail, creating a significantly narrower attack surface than Linux, where namespace abstractions have been bypassed multiple times due to errors in permission context validation.
Resource management is handled through the RCTL (Resource Limits) mechanism, which operates hierarchically across users, processes, and jail objects. Modern deployments are built on the Jails v2 model (hierarchical jails) and the fully virtualized VNET (VIMAGE) network stack.
| Parameter | Docker (Linux OCI) | systemd-nspawn (Linux) | FreeBSD Jails (VNET) |
|---|---|---|---|
| Isolation mechanism | Composition of namespaces and cgroups v2 | Composition of namespaces and cgroups v2 | Monolithic kernel object |
| Root process | Application process (PID 1) | systemd init system (PID 1) | /etc/rc init environment |
| Resource limits | cgroups via the Docker daemon | Native systemd service integration (cgroups v2) | Hierarchical RCTL mechanism |
| Persistence model | Ephemeral, stateless with volumes | Stateful – OS-level virtual machine | Stateful – full system instance |
| Network isolation | Virtual bridges and port mapping | Macvlan or veth pair | Full stack virtualization (VNET) |
Storage: ZFS as the infrastructure foundation
The storage layer is a critical factor in the performance of containerized databases. Understanding I/O behavior beneath the virtualization layer prevents write amplification and throughput degradation.
Docker and storage drivers
Docker containers are built on a layered topology (image layers). Each layer represents a Dockerfile instruction and is read-only. Modifications are written to an ephemeral writable layer via a Copy-on-Write mechanism.
The default driver is overlay2, which operates at the file level (file-level CoW). If a process inside a container needs to modify a single byte in a gigabyte database file from a lower layer, overlay2 must copy the entire file to the write layer. For databases handling continuous UPDATE/INSERT operations, this generates catastrophic performance degradation.
The solution is to use Docker volumes or bind mounts for data directories (for example /var/lib/mysql). These mechanisms bypass the driver's CoW layer entirely, allowing the container to write data directly to the host filesystem at near-native speed.
An alternative is the dedicated ZFS storage driver, which operates at the block level (block-level CoW). Modifying a single byte copies only the changed block (for example 128 KB), which is nearly instantaneous. The driver provides data integrity via checksums and transparent compression (LZ4). The downside is a risk of slower ZFS administrative operations when a large number of snapshots accumulate during image pulls.
systemd-nspawn and direct ZFS access
Due to its stateful nature, systemd-nspawn integrates cleanly with ZFS at the dataset level. Each container can operate on its own ZFS dataset with individual parameters (recordsize, compression, quota). Dataset snapshots allow instant backups and rollbacks without the overhead of any intermediate layer.
FreeBSD Jails and native ZFS integration
In the FreeBSD ecosystem the ZFS–Jails integration goes even deeper. The zfs jail command delegates entire datasets into a jail. The jail can manage its own snapshots and ZFS parameters without access to the global pool tree.
The configuration requires the appropriate security flags: enforce_statfs=1, allow.mount, and allow.mount.zfs in the jail definition, plus a devfs ruleset (devfs_ruleset) that exposes the /dev/zfs device without granting access to the host's physical disks.
Container networking architecture
Linux: virtual bridges, NAT, and macvlan
Docker's default network configuration relies on a virtual bridge (docker0) and NAT. Each container receives a veth interface connected to the bridge, and outbound traffic passes through iptables/nftables rules on the host.
Under heavy load – tens of gigabits per second – the overhead of NAT operations (rewriting MAC addresses, IP checksums, and TCP sequence numbers) degrades throughput significantly.
The solution is the macvlan driver. The virtual interface gets an individual MAC address and is connected directly to the host physical NIC. The container reaches the physical switch without NAT, achieving throughput close to the physical link. Macvlan is available in both Docker and systemd-nspawn (via the MACVLAN= parameter in the profile file).
FreeBSD: VNET (VIMAGE) and epair pairs
FreeBSD's VNET (VIMAGE) system instantiates a complete copy of the kernel network stack in memory for each jail: isolated ARP tables, unique lo0 interfaces, independent transport protocol instances, IPsec, and individual packet filter instances.
Communication is based on epair interface pairs connected through a virtual bridge. The "a" end (for example epair0a) stays on the host as a bridge member, and the "b" end (for example epair0b) is handed to the jail, where it receives an individual IP address.
A notable issue in nested virtualization environments (for example FreeBSD inside a KVM hypervisor) is pathological behavior of Hardware Checksum Offloading. Virtual VNET bridges combined with NAT corrupt TCP headers, causing a severe throughput drop. The fix is to disable checksum offloading: set hw.vtnet.X.csum_disable=1 and hw.vtnet.lro_disable=1 in /boot/loader.conf, and disable bridge pfil passthrough with net.link.bridge.pfil_member=0 and net.link.bridge.pfil_bridge=0.
| Parameter | Linux: bridge + NAT (Docker) | Linux: macvlan | FreeBSD: VNET + Jails |
|---|---|---|---|
| Mechanism | veth interface and virtual docker0 bridge | Virtual MAC interface on the primary NIC | Virtualized stack (VIMAGE) with epair pairs and bridge |
| CPU overhead | High under heavy traffic (NAT) | Low – no NAT | Requires disabling checksum offloading |
| Firewall isolation | Per-container iptables/nftables instance | Independent container firewall | Full pf instance inside each jail |
Security: seccomp and AppArmor versus Capsicum
Linux: layered filter barriers
Linux systems defend against container breakout attacks through a layered methodology:
Seccomp-bpf (Secure Computing with Berkeley Packet Filter) filters syscalls using JSON profiles containing an allowlist and blocklist. Default profiles block calls such as CLONE_NEWUSER, clock_settime, key_ctl, and cryptographic device operations.
AppArmor (Mandatory Access Control) enforces path-based access restrictions regardless of the user's system permissions. AppArmor profiles are assigned to containers automatically.
The combination of seccomp and AppArmor creates a multi-level barrier: seccomp blocks dangerous syscalls; AppArmor restricts filesystem path access.
FreeBSD: Capsicum – capability-based security
FreeBSD takes a fundamentally different approach – the Capsicum model, built on capabilities rather than filter lists.
An application calls cap_enter(), after which it is confined to capability mode. From that point on, the program loses access to the global filesystem – it cannot call a simple open("/etc/passwd") by path. All references to global resources (processes, memory, sockets) are blocked by the kernel.
The only way to access resources is through file descriptors allocated before entering capability mode. The program operates exclusively on descriptors passed to it from outside, using restricted syscall variants (for example openat(2) instead of open(2)).
This approach – known as oblivious sandboxing – makes a program far more resilient to Remote Code Execution (RCE) attacks than syscall filtering with seccomp, because even exploiting a kernel vulnerability does not grant the attacker access to global system resources.
Security restrictions and performance: the io_uring problem in Docker
The io_uring interface, introduced in Linux kernel 5.1, delivers a step change in I/O performance through ring queues (Submission Queue and Completion Queue) mapped in shared memory between userspace and the kernel. This eliminates the overhead of continuous context switching, enabling full utilization of NVMe disk throughput.
The problem is that Docker's default seccomp profile blocks all io_uring syscalls. This is a deliberate security decision: the asynchronous nature of io_uring and its shared memory ring structure create a large attack surface – the subsystem has produced multiple security vulnerabilities, which led Google (gVisor) and Android to block it entirely.
The practical effect is a silent performance regression for databases inside Docker containers – database engines fall back to slow synchronous interfaces without any warning or error message. This issue was also discussed in the context of MariaDB in the article ZFS Optimization for Databases.
Options available to administrators:
- Disable seccomp for the database container with the
--security-opt seccomp=unconfinedflag – requires consciously accepting an elevated risk posture. - Load a custom JSON profile that selectively permits
io_uringsyscalls. - Increase the locked memory limit (
memlock) in Docker Compose or the systemd unit (LimitMEMLOCK), becauseio_uringrequires pinning memory pages.
On FreeBSD the problem does not exist – jails do not use seccomp, and the native Capsicum architecture does not block I/O operations at the syscall level.
Conclusion and recommendations
Each of the three technologies – Docker, systemd-nspawn, and FreeBSD Jails – implements isolation in a fundamentally different way and excels in different scenarios.
Docker is the optimal choice for environments requiring rapid deployment, horizontal scaling, and integration with the Kubernetes ecosystem. The ephemeral nature of containers and the rich image registry infrastructure make it the standard for microservice architectures. However, it requires awareness of its limitations: overlay2 overhead on databases, io_uring blocking by seccomp, and the need to use volumes for persistent data.
systemd-nspawn fits where a stateful, multi-service machine with a full init system is needed – for example development environments, staging, or isolation of legacy applications. It integrates well with ZFS and machinectl, but does not have an image ecosystem comparable to Docker Hub.
FreeBSD Jails with VNET and ZFS offer the deepest kernel integration, the narrowest attack surface (monolithic kernel object plus Capsicum), and native ZFS dataset delegation without any intermediate layer. They are the ideal solution for demanding database workloads and environments where I/O stability and security are the top priorities. They do require deeper configuration expertise (VNET, epair, devfs_ruleset, RCTL) and have a smaller community than the Docker ecosystem.
Regardless of which technology you choose, understanding the host kernel's behavior is essential – memlock limits, seccomp restrictions, Copy-on-Write characteristics in the filesystem, and hardware network offloading mechanisms. These details, not the choice of containerization tool itself, ultimately determine the performance and security of production infrastructure.
At WebOptimo we specialize in Linux and FreeBSD server administration, web application containerization, and database performance optimization. If you are planning a container deployment in a production environment or need an audit of your existing infrastructure, get in touch. You can also explore our server administration, Linux administration, and FreeBSD administration services. See also our other articles: ZFS Optimization for Databases, LEMP Stack Architecture, and PostgreSQL High Availability.
FAQ – Containerization
Docker runs a single application process in an ephemeral OCI-compliant container. systemd-nspawn loads a full init system (systemd) as PID 1, creating a stateful OS-level virtual machine. Both use the same Linux kernel primitives (namespaces and cgroups v2), but differ in philosophy: Docker targets microservices, systemd-nspawn targets complete system environments.
FreeBSD Jails are a monolithic kernel object, not a composition of independent subsystems. The kernel checks on every syscall whether a process is inside a jail and categorically refuses operations on global resources. FreeBSD also uses the Capsicum capability model, which after cap_enter() severs the program from the global filesystem, whereas Linux relies on seccomp filters that have been bypassed multiple times.
The default overlay2 storage driver operates at the file level (file-level CoW). Modifying a single byte in a gigabyte database file requires copying the entire file. The default seccomp profile also blocks io_uring, silently forcing databases to fall back to slower synchronous interfaces. The fix is to use Docker volumes (bind mounts) for data directories and optionally relax the seccomp profile.
Macvlan is a network driver that attaches a container directly to the host physical NIC with its own MAC address, bypassing NAT. Throughput is close to wire speed. Use it for high-traffic workloads where the NAT overhead on the docker0 bridge becomes a bottleneck.
Yes, with caveats. The ZFS storage driver operates at the block level (block-level CoW), eliminating the whole-file copy problem on modification. It provides data integrity checksums and transparent LZ4 compression. The downside is a risk of slowdowns in ZFS administrative operations when many snapshots accumulate during image pulls.
VNET instantiates a complete copy of the kernel network stack for each jail: isolated ARP tables, interfaces, transport protocol instances, IPsec, and packet filter instances. Docker creates only isolated network namespaces with virtual bridges. VNET provides deeper isolation but requires configuring epair pairs and attention to the checksum offloading problem in nested virtualization environments.
Docker's default seccomp profile blocks io_uring syscalls because its asynchronous nature and shared memory ring structure create a large attack surface. The subsystem has produced multiple security vulnerabilities, leading Google (gVisor) and Android to block it entirely. Databases silently fall back to slower synchronous interfaces. The issue does not affect FreeBSD Jails.
