NixOS for the Homelab: The Honest Trade-Off

I have run NixOS on part of my homelab for about two years. In that time I have rebuilt one machine from scratch in under fifteen minutes, rolled back a broken kernel upgrade from the bootloader while half-asleep, and spent an entire Saturday trying to make a piece of vendor software work that would have taken four minutes on Debian.

Both of those experiences are the real NixOS. The evangelism you read online covers the first kind and skips the second, and that asymmetry is why people arrive expecting a better Ubuntu and leave angry. This is my attempt at the honest version: what the trade actually is, priced properly, so you can decide before you have sunk three weekends into finding out.

The thing that makes it different

Every other Linux distribution is a mutable pile. You install packages, they write files into /usr and /etc, config files accumulate edits, and the current state of the machine is the sum of every command anyone has ever run on it. It works fine, right up until you need to reproduce it, at which point you discover that the machine is a historical artefact rather than a described thing.

NixOS builds the system from a function. You write configuration.nix, describing what the machine should be, and nixos-rebuild evaluates that into a complete system in /nix/store, then flips a symlink. Packages live at paths keyed by a hash of every input that produced them, so two versions of the same library coexist with no argument. /etc is generated. /usr barely exists — /usr/bin/env is the only thing in it, kept purely because a million shebang lines depend on it.

The consequence that matters day to day: the old system is still there, complete and bootable, until you garbage-collect it. An upgrade that breaks is a reboot and a menu selection away from being undone. That is meaningfully different from a snapshot rollback — the previous generation is a real, intact system that was built and tested, sitting on disk alongside the new one, with no filesystem-level cleverness involved.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

# /etc/nixos/configuration.nix — a small but real server
{ config, pkgs, ... }:

{
  imports = [ ./hardware-configuration.nix ];

  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  networking.hostName = "media01";
  networking.firewall.allowedTCPPorts = [ 22 8096 ];
  time.timeZone = "Europe/Copenhagen";

  users.users.smarc = {
    isNormalUser = true;
    extraGroups = [ "wheel" "docker" ];
    openssh.authorizedKeys.keys = [
      "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... smarc@workstation"
    ];
  };

  services.openssh = {
    enable = true;
    settings.PasswordAuthentication = false;
    settings.PermitRootLogin = "no";
  };

  services.jellyfin.enable = true;
  virtualisation.docker.enable = true;

  environment.systemPackages = with pkgs; [ git vim htop rsync ];

  system.autoUpgrade = {
    enable = true;
    allowReboot = false;
  };

  system.stateVersion = "25.05";
}

That is the entire machine, in the literal sense. Feed that file plus hardware-configuration.nix to a NixOS installer on identical hardware and you get a byte-identical system. The whole appeal is contained in that sentence, and if it does not make you feel anything, the rest of this article is going to be a hard sell.

What you actually gain

Rollback that works when you need it. Every nixos-rebuild switch writes a new generation and adds it to the boot menu. A kernel that does not boot, a driver update that kills your GPU, a service change that breaks networking — all recovered by rebooting and picking the previous entry. I have done this three times in two years, twice on a machine I could not physically reach, which is the case where it counts.

nixos-rebuild test before you commit. It builds and activates the new configuration without touching the bootloader. If it wedges, reboot and you are back. This one habit removes most of the fear from changing a running server.

Rebuilding a machine is a non-event. A dead SSD means: new SSD, boot the installer, copy in two files, nixos-install, wait. Fifteen minutes, and the result is the machine you had rather than an approximation you reconstructed from memory. Compare that against your last bare-metal restore rehearsal, assuming you have run one recently enough to trust it.

The module system is genuinely excellent. services.jellyfin.enable = true creates the user, the systemd unit, the data directories and the permissions. Someone has read the upstream documentation properly and encoded it. For the roughly two thousand services with NixOS modules, this is better than any Ansible role you will write, because it is maintained by people who use it and tested by the package build.

The store makes “what is installed” answerable. nix-store --query --requisites /run/current-system prints the complete transitive closure of everything on the machine. Every library, every version. No other distribution can tell you this honestly.

What it actually costs

The language. Nix the language is lazily evaluated, dynamically typed, and has an error reporting story that ranges from unhelpful to actively misleading. A typo in an attribute name produces a stack trace through the module system that mentions neither your file nor your typo. You will spend your first month unable to tell the difference between a syntax error and a semantic one. It gets better. It does not get good.

Documentation is scattered across three eras. The manual, the wiki, search.nixos.org, and a large body of Discourse threads and blog posts, many of which are three years stale and use pre-flakes idioms. Working out whether an answer applies to your setup is a skill you have to develop, and it is the single biggest time sink for beginners.

Anything outside nixpkgs is work. A binary from a vendor’s website expects a filesystem layout that does not exist on NixOS — no /lib64/ld-linux-x86-64.so.2, no /usr/lib. It will not run. The fixes (nix-ld, steam-run, buildFHSEnv, patching the interpreter with patchelf) all work, and every one of them is an hour you did not budget. Proprietary drivers, closed-source appliances, anything that ships an installer script: assume a lost evening each.

Secrets need a plan on day one. The Nix store is world-readable, so anything you put in a config file lands in /nix/store with mode 444 for every user on the box to read. The answer is sops-nix or agenix — I use the former, since I already run SOPS with age elsewhere. Discovering this after you have written your database password into configuration.nix is a rite of passage and a small incident.

Disk usage is comic. Every generation you keep holds its entire closure alive. A workstation with a few months of generations will happily eat 80 GB. nix-collect-garbage -d --delete-older-than 30d is a weekly timer you should set up on day one, and remember it deletes the rollback targets you were relying on.

Deploying to more than one machine

The single-machine story above is the one every tutorial tells, and it is also the point at which NixOS is least impressive: you have swapped apt install for a language, on one box, for a rollback you might use twice a year. The argument gets much stronger at three machines, because the configuration becomes shared code.

nixos-rebuild can build locally and push the result over SSH, which is the zero-dependency version and works fine:

1
2
3
4
5
6
7

# Build here, activate there. The target needs nothing but SSH and NixOS.
nixos-rebuild switch --flake .#media01 \
  --target-host [email protected] --use-remote-sudo

# Stage it for the next reboot instead of activating now
nixos-rebuild boot --flake .#media01 \
  --target-host [email protected] --use-remote-sudo

The important part is that the build happens on your workstation. A Raspberry Pi does not have to compile anything; it receives finished store paths. That inverts the usual homelab problem where the weakest machine does the most painful work, and it is the reason a repository of five machine configurations feels lighter than five machines each managed by themselves.

Once you have more than a couple, a deployment tool earns its keep. Colmena is the one I settled on: it reads a hive of node definitions, builds them in parallel, and pushes to all of them with one command, with tags so you can target a subset. deploy-rs does a similar job with a rollback-on-failure watchdog, which sounds better than it is in practice — the watchdog protects against the machine losing network, and losing network is exactly the case where you cannot tell it to roll back.

The shared-code effect is where the payoff actually shows up. My SSH hardening, my monitoring agent, my user account and my firewall defaults live in one module imported by every machine. Changing the SSH configuration on eight hosts is a two-line diff and one command, and there is no possibility that host number six silently missed it because a playbook failed halfway. Ansible can approximate this; it approximates it by running commands and hoping, whereas NixOS computes the answer and copies it.

The cost is a single repository that every machine depends on, and a bad commit that lands on all of them at once. nixos-rebuild boot rather than switch, plus staggered reboots, is the discipline that keeps a typo from being an outage across the whole rack.

Troubleshooting the first month

“error: attribute ‘foo’ missing” with no file or line. Almost always a misspelt option. Search the exact option name at search.nixos.org/options; that page is authoritative, so an option missing from it was never real. Adding --show-trace gives you fifty lines of module-system internals with your mistake somewhere in the middle; read from the bottom up.

“infinite recursion encountered”. You referenced config.something while defining that same something, usually via a helper you extracted to tidy things up. Untangle by inlining the value; the module system evaluates lazily and cannot see the loop coming.

A binary exits with “No such file or directory” and the file plainly exists. The dynamic linker is missing, and the error is about ld-linux. Enable programs.nix-ld.enable = true and set programs.nix-ld.libraries, or wrap the thing in steam-run to test quickly.

nixos-rebuild switch succeeds but the service does not start. Nix guarantees the unit file is correct; it guarantees nothing about the daemon’s opinion of your config. systemctl status and journalctl -u work exactly as they always did. This is the moment to remember that NixOS is Linux underneath, which is easy to forget in the first fortnight.

The build recompiles the world. You changed something low in the dependency graph — an overlay on glibc, a kernel patch — and invalidated every hash downstream. Nothing is wrong. It is going to take four hours. This is why services.hardware overrides deserve a moment’s thought before you type them.

Out of disk during a rebuild. Generations again. Garbage collect, then set the timer you meant to set.

Who should walk away

If your homelab works and you resent it when it takes attention, stay where you are. Debian with unattended-upgrades and a decent backup regime is a fine answer, and the NixOS payoff arrives too slowly to justify the disruption.

If your machines mostly run Docker containers, the case is thin. Compose files are already declarative, and NixOS gives you a beautifully reproducible host for a mutable pile of containers. That is a real benefit, and it is a fraction of the one you were promised. Podman without the Docker daemon or a tidy compose layout get you most of the same calm for a tenth of the cost.

If you want reproducible development environments and nothing else, use Nix the package manager on your existing distro. It works on Debian, Fedora and macOS, gives you nix develop shells, and costs no reinstall — I went through that route in reproducible dev environments with Nix, and for many people that is the whole of the value.

If you want a Kubernetes host with no state and no SSH, Talos Linux achieves the same immutability with none of the language.

The verdict

NixOS is worth it if you rebuild machines, if you have more than three of them, and if you find the idea of a system defined by a file genuinely appealing rather than merely sensible. The break-even in my experience is around six months of part-time use. Before that you are slower than you were on Debian and you will doubt the decision roughly weekly.

After it, something changes that is hard to convey. Config drift stops existing as a concept. Upgrades stop being events. You start treating servers as things you can throw away, which changes what you are willing to try. That shift is the product, and no feature list conveys it.

The failure mode I see most often is people adopting NixOS for the reproducibility and then discovering they never actually needed to reproduce anything. If you cannot name a machine you would rebuild this year, you are buying insurance against a risk you do not carry, and paying for it in weekends. Be honest about that first. The learning curve is real, the ceiling is high, and both of those are true at the same time.

Next up: the flakes question, which is where most of the modern documentation lives and where the remaining confusion is concentrated. I have written that one up separately in Nix flakes explained without the jargon, because it deserves the space.