Incus: Containers and VMs Side by Side

I spent a long weekend a few years back running two separate stacks on the same box: LXD for my system containers, and a hand-rolled libvirt setup for the two Windows VMs I kept around for testing. Two command sets, two networking models, two sets of storage pools to keep in sync. It worked, but every time I wanted to move a workload from a container to a VM — because it needed a real kernel, or GPU passthrough, or just more isolation — I was rewriting the whole deployment instead of flipping a flag.

Incus fixes exactly that annoyance. It’s a fork of LXD, created in 2023 after Canonical moved LXD’s copyright assignment terms in a direction the community didn’t love, and it’s now hosted under the Linux Containers project alongside LXC itself. Functionally it’s almost a drop-in replacement for LXD as it stood at the fork point, plus its own steady stream of improvements since. The pitch that matters for a homelab: one daemon, one CLI, one API, and incus launch can hand you either a system container or a full VM depending on one flag. Same profiles, same storage pools, same network bridges, same command syntax for exec, snapshot, and migrate.

Why unify containers and VMs at all

Containers and VMs solve different problems and pretending otherwise gets you in trouble. A system container shares the host kernel — cheap, fast to start, but anything that needs its own kernel (a different distro’s kernel modules, nested virtualisation, GPU passthrough for certain workloads, or software that pokes at low-level kernel interfaces) is out of scope. A VM gives you a full virtual machine with its own kernel and hardware boundary, at the cost of more overhead and slower boot.

In practice a homelab needs both. My Pi-hole equivalent, my reverse proxy, my Git server — all fine as containers, because they’re just userspace daemons. But my pfSense-adjacent firewall VM and my one Windows box for testing print drivers genuinely need a kernel boundary. Before Incus (and LXD, which pioneered this model), that meant learning libvirt/QEMU as a second toolchain: different XML-based config, different networking primitives, different snapshot semantics. Incus lets you treat the choice of container-vs-VM as a deployment detail rather than an architectural fork in your tooling.

This is also where the Proxmox comparison comes up a lot, and it’s worth being direct about it. If you want a polished web UI, clustering out of the box, and a ready-made backup/restore workflow, Proxmox is still the better starting point for most people — it’s opinionated in ways that save you decisions. Incus is the CLI-first, API-first alternative: it assumes you’re comfortable scripting infrastructure, and in exchange it gives you a lighter footprint, faster container start times, and a genuinely unified object model instead of Proxmox’s parallel LXC and QEMU/KVM subsystems bolted together under one UI.

Installing and getting oriented

Incus ships in Debian 13, most current Ubuntu releases, and has a Zabbly-maintained apt repo for older Debian/Ubuntu if your distro’s version lags.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Debian 13 / Ubuntu 24.04+: it's in the default repos
sudo apt install incus

# older releases: add the Zabbly repo first
curl -fsSL https://pkgs.zabbly.com/key.asc | sudo gpg --dearmor -o /etc/apt/keyrings/zabbly.gpg
echo "deb [signed-by=/etc/apt/keyrings/zabbly.gpg] https://pkgs.zabbly.com/incus/stable $(lsb_release -cs) main" | \
  sudo tee /etc/apt/sources.list.d/zabbly-incus-stable.list
sudo apt update && sudo apt install incus

# add yourself to the incus-admin group so you don't need sudo for every command
sudo usermod -aG incus-admin $USER
newgrp incus-admin

Then run incus admin init. It’s an interactive wizard: storage backend (ZFS or btrfs give you snapshots and copy-on-write clones; dir works but is slower and dumber), whether to configure a bridge network (say yes unless you already have one), and whether to expose the API on the network. For a single box, the defaults are sensible — accept ZFS if you have the disk to spare, since it’s what makes incus copy and incus snapshot fast.

Launching a container vs a VM

This is the party trick. Same command, one flag:

1
2
3
4
5
6
7
8

# a system container — shares the host kernel, starts in under a second
incus launch images:debian/13 web01

# a full VM — its own kernel, its own hardware boundary
incus launch images:debian/13 win-test --vm

# list everything, containers and VMs together
incus list

1
2
3
4
5
6
7

+---------+---------+----------------------+------+-----------------+-----------+
|  NAME   |  STATE  |         IPV4         | IPV6 |      TYPE       | SNAPSHOTS |
+---------+---------+----------------------+------+-----------------+-----------+
| web01   | RUNNING | 10.10.10.15 (eth0)   |      | CONTAINER       | 0         |
+---------+---------+----------------------+------+-----------------+-----------+
| win-test| RUNNING | 10.10.10.22 (eth0)   |      | VIRTUAL-MACHINE | 0         |
+---------+---------+----------------------+------+-----------------+-----------+

Both instances live in the same list, use the same network bridge, and get exec’d into the same way (incus exec web01 -- bash for the container; for a VM you’d use incus console win-test since there’s no shared kernel to break into). Snapshotting, cloning, and moving between hosts use identical syntax regardless of type:

1
2
3

incus snapshot create web01 pre-upgrade
incus copy web01 web01-clone
incus move win-test --target another-host.mylab.local

Profiles: the piece that actually saves you time

Profiles are reusable bundles of config — devices, limits, network attachments — that you apply to instances instead of repeating yourself. This is where Incus earns its keep in a homelab with more than a handful of instances.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# profile: web-tier — applied to every web-facing container
config:
  limits.cpu: "2"
  limits.memory: 1GiB
  security.nesting: "false"
devices:
  root:
    path: /
    pool: default
    type: disk
    size: 10GiB
  eth0:
    name: eth0
    network: incusbr0
    type: nic

1
2
3

incus profile create web-tier
incus profile edit web-tier   # opens $EDITOR with the YAML above
incus launch images:debian/13 web02 --profile default --profile web-tier

Change the CPU limit in the profile once, and every instance using it picks up the change on next restart. I run separate profiles for “web tier” (modest limits, no nesting), “build agents” (more CPU, security.nesting: true so they can run Docker-in-container for CI), and “gpu-passthrough” (a gpu device entry pointing at a specific PCI address) — and every new instance is one incus launch ... --profile away instead of a fresh config each time.

Storage and networking

Incus’s storage pools abstract over ZFS, btrfs, LVM, or plain directories, and every instance’s root disk is a dataset/subvolume in that pool. The practical upshot: incus copy and incus snapshot on ZFS or btrfs are copy-on-write and near-instant, even for a VM’s virtual disk. On dir storage they’re full copies, which is fine for a couple of test containers and painful for anything bigger.

Networking defaults to a bridged network (incusbr0) with NAT out to the host’s uplink, plus DHCP for instances. For anything you want reachable from the rest of your LAN, either add a second bridge tied to a physical interface, or use incus network to attach an instance directly to your existing VLAN — the same primitives from VLAN segmentation apply here; Incus just gives you a bridge.external_interfaces config key to hang a tagged interface off instead of hand-rolling it with ip link.

Backups and small clusters

Incus exports instances as portable images, which is the backup primitive worth knowing about even if you’re running a single box:

1
2

incus export web01 /mnt/backups/web01-$(date +%F).tar.gz
incus import /mnt/backups/web01-2025-12-01.tar.gz web01-restored

That export bundles the instance’s filesystem, config, and any local snapshots into one archive — good enough to stash on a NAS share or feed into whatever tool handles your actual 3-2-1 backup routine. For anything beyond a single host, incus cluster turns a handful of machines into one logical Incus deployment: instances get scheduled across members, and incus move between cluster members is a metadata operation rather than a full copy when the storage pool is shared. I run a two-node cluster at home mostly so a single host reboot for kernel updates doesn’t take every container offline at once — incus cluster list shows member health, and evacuate/restore move workloads off a node before you touch it and back on afterwards. It’s overkill for one box, but if you already have two machines sitting around, the clustering story is far less fiddly than the equivalent libvirt/Pacemaker setup would be.

Resource limits and the cgroup mechanics underneath

Containers and VMs enforce limits.cpu and limits.memory through completely different mechanisms, and understanding which one applies to your instance explains some behaviour that otherwise looks like a bug. A container’s limits are cgroups — the same kernel primitive Docker and Kubernetes use — applied directly to the container’s processes on the shared host kernel. Change limits.memory on a running container and the new ceiling applies immediately, because it’s just a cgroup value being rewritten. A VM’s limits are closer to a hardware spec: limits.memory sizes the virtual RAM handed to QEMU at boot, and limits.cpu sets the vCPU count, so changing either on a running VM doesn’t take effect until the next boot — you’re not adjusting a throttle, you’re respecifying the virtual hardware.

This distinction matters when you’re capacity-planning a host running both types side by side. Containers can be over-committed more aggressively than VMs, because cgroup limits are soft ceilings enforced by the kernel scheduler on shared resources, whereas a VM’s memory allocation is effectively reserved the moment it boots (Incus doesn’t do VM memory ballooning by default). A host with 32GB of RAM running six lightweight containers each capped at 1GB can comfortably run all six simultaneously with room to spare for burst usage; the same host running six VMs each configured for 4GB is committing 24GB whether or not any given VM is using it, because that RAM is handed to QEMU as if it were physical.

Nesting, security profiles, and where containers stop being suitable

security.nesting and the broader security.* namespace of config keys are worth understanding before you reach for a container as the default answer to everything. A container with security.privileged: true shares more of the host kernel’s capabilities than the default unprivileged mode, which is occasionally necessary — some workloads genuinely need raw device access or kernel module interaction — but it also meaningfully narrows the isolation boundary between the container and the host. Incus containers default to unprivileged, running as a mapped, unprivileged UID range on the host even though they look like root inside the container, and that default is doing real security work: a container escape from an unprivileged instance lands the attacker as an unprivileged user on the host, not root.

The practical rule I use: if a workload wants privileged mode or extensive nesting just to run at all, that’s usually a sign it should be a VM instead, not a signal to loosen the container’s security config. Docker-in-Incus-container (security.nesting: true) is a reasonable and common exception, because it’s a well-understood, narrow escalation rather than a blanket privilege grant — but anything asking for raw hardware access, custom kernel modules, or security.privileged is telling you it wants a kernel of its own, and Incus makes it a one-flag decision to just give it one rather than fighting the container model.

GPU passthrough

Passing a GPU through to a container or a VM uses different mechanics again, and it’s one of the clearer illustrations of why unifying the two under one tool is genuinely useful rather than just tidy. A container gets GPU access by adding a gpu device to its config, which bind-mounts the relevant device nodes into the container’s filesystem — the container shares the host’s GPU driver, so the driver version inside and outside must match, since there’s only one kernel and one driver involved. A VM instead needs the GPU (or a virtual function of it, if you’re using SR-IOV to split a card across multiple VMs) properly unbound from the host and passed through via VFIO, at which point the VM loads its own driver entirely independent of the host’s — more setup, but full isolation and no shared-driver version constraint. I run a gpu-passthrough profile of each kind: one for containers doing lightweight inference that’s happy sharing the host driver, one for a VM that needs an isolated environment for driver testing without risking the host’s own GPU stack.

Troubleshooting

incus admin init storage step fails on ZFS with “no pools available”. You need the zfsutils-linux package installed and a spare block device or a big-enough file-backed loop device. If you’re testing on a VM yourself, add a second virtual disk before running init rather than trying to carve a ZFS pool out of the root filesystem.

A VM won’t start, logs mention nested virtualisation or vmx/svm not available. Incus VMs use QEMU/KVM under the hood, so the host needs hardware virtualisation exposed. Check egrep -c '(vmx|svm)' /proc/cpuinfo — if it’s zero and you’re inside another VM (testing Incus inside a cloud VM, say), you need nested virtualisation enabled at the hypervisor level first.

Containers can’t reach the internet after a fresh init. Usually a missed iptables/nftables MASQUERADE rule on the bridge, or net.ipv4.ip_forward set to 0. Incus normally manages this itself via its bridge network config, but if you’ve got another firewall tool (firewalld, ufw) fighting for the same tables, check incus network show incusbr0 for the NAT setting and confirm nothing downstream is dropping the forwarded traffic.

Migrating an instance between hosts hangs or fails partway. incus move --target needs both hosts to trust each other’s certificates (incus remote add) and, for live migration of a running VM, needs matching-enough CPU features. For containers this is a non-issue since it’s just a filesystem and cgroup state; for VMs, stop the instance first if the hosts have different CPU generations — cold migration always works, live migration is pickier.

Instance stuck in “STOPPING” forever. Usually a process inside the container ignoring SIGTERM. incus stop <name> --force gets you unstuck immediately; longer term, check what’s swallowing the signal — misbehaved init systems inside minimal container images are the usual culprit.

Is it worth it

If you’re already happy with Proxmox’s UI and clustering, don’t switch — Incus doesn’t do anything Proxmox categorically can’t, and you’d be trading a mature web UI for a CLI-first workflow. But if you’re the kind of person who’d rather script incus launch in a loop than click through a wizard, or you’re running a mixed fleet of lightweight containers and the occasional full VM and you’re tired of maintaining two toolchains for it, Incus is genuinely pleasant. It’s also a lighter footprint than Proxmox if you’re running on modest hardware — no separate management stack, just one daemon.

It pairs well with Podman rather than competing with it: Podman is for running application containers (your Docker Hub images, your compose stacks), Incus is for running system containers and VMs (things that look and behave like a full machine). I run both on the same host — Incus for the machines, Podman inside one of those machines for the actual application containers.