Proxmox on a second-hand workstation

The machine running most of my home lab is a Lenovo ThinkStation P700 that cost less than a mid-range mini PC. It has a Xeon E5-2699 v3 in it, which is eighteen cores of 2014-era server silicon that nobody particularly wants any more, and it has been more useful than anything else I have bought for the lab.

This isn’t a build guide. Proxmox installs itself in about ten minutes and the official documentation is good. It’s about the three decisions that turned out to matter, most of which I got wrong the first time.

Why old workstation silicon

The instinct is to buy something small and new. A modern mini PC is quiet, sips power, and fits behind a monitor, and I’ve got one — it runs my always-on services. But it has four cores and one memory channel, and the moment you want to run several machines that each want real memory, you’re stuck.

Second-hand dual-socket workstations solve that in a way nothing else at the price does, and the reason isn’t really the core count, although eighteen cores that are genuinely spare rather than merely sufficient does change how you think about the box. It’s that registered ECC memory is worth almost nothing on the used market, because almost nothing consumer will take it, so 64 or 128GB costs less than 32GB of desktop DDR5. And the one people forget is PCIe lanes: a consumer CPU gives you sixteen usable lanes and a chipset arguing over the rest, where this class of chip gives you forty per socket, which is what lets you run a GPU and an HBA and a decent NIC all at full width without a spreadsheet.

The cost is electricity and noise. A dual-socket Haswell-EP box idles somewhere around 90 to 120W depending on what’s in it, and the stock cooling was designed for an office rather than a bedroom. If the machine has to live where you sleep, buy the mini PC. If it can live in a garage or a loft, I would take the workstation and spend the difference on memory.

Storage, decided before you install anything

Proxmox will happily install onto a single disk with LVM-thin and you will never think about it again, right up until you want snapshots, or a disk dies, or you want to replicate to another host.

What I would probably suggest is deciding up front whether you want ZFS, because converting later means rebuilding the host. ZFS gives you snapshots that are instant and nearly free, checksums that catch silent corruption, and send/receive for backups that are incremental at the block level. It wants roughly a gigabyte of RAM per terabyte and there’s no point fighting it on that — which is exactly the resource the second-hand workstation hands you for nothing.

The mistake I made was treating the boot disk as unimportant. It holds every VM definition, every firewall rule and the cluster configuration, and two cheap SSDs in a mirror cost almost nothing and remove an entire category of bad weekend.

Passthrough is a hardware decision

I run a GPU passed through to a machine that does image and video generation. Getting there is straightforward if the hardware cooperates and fairly miserable if it doesn’t, and you find out which after you’ve bought everything.

The thing to check before you buy is IOMMU grouping. Your motherboard decides which devices are isolated from each other, and if the GPU shares a group with, say, the USB controller, you can’t pass one through without the other. Older workstation boards are generally good at this because they were built to take several discrete cards; consumer boards frequently aren’t. It costs nothing to check the grouping on a machine you already own before committing to a plan that depends on it.

The second one would be that a card being passed through can’t also be the one the host is drawing its console on. On a workstation with onboard graphics that’s free. Without it you need a second card, or you need to be confident enough to run the host headless.

Backups are not snapshots

A ZFS snapshot protects you from your own mistakes. It doesn’t protect you from the machine catching fire, and it doesn’t protect you from deleting the pool.

I learned that one the hard way. I had a box holding the only copy of a project’s configuration — the automation that drove it, and the credentials it needed — and it died. Not corrupted, not recoverable. Dead. Everything that machine did had been written down nowhere except on the machine.

The site it published survived, because a static host kept serving the last build for nine months while nothing new appeared, and that’s luck rather than design. What changed afterwards isn’t really a Proxmox lesson, but it’s the one I’d actually hand to somebody else: anything that describes how a system works belongs in a repository, not on the host that runs it. Code, configuration, unit files, all of it. Secrets stay out and go somewhere they can be reissued. If the machine disappears overnight, the recovery should be boring.

Proxmox Backup Server is the natural other half of that, and to be honest it’s worth running even if the only target is a USB disk on the same desk. Deduplicated, incremental, verifiable — and restore-tested, which is the part everybody skips. A backup you have never restored is a hypothesis.

What I’d do differently

I’d put management on its own VLAN from day one, because retrofitting segmentation onto a working lab is tedious in a way that doing it up front isn’t. I’d also resist over-allocating vCPU early: it’s tempting to give every machine eight cores because there are plenty, and it makes scheduling worse rather than better. Start small and raise it when something is actually slow.

The other one would be naming. Name things for what they do, because six months later you won’t remember which of vm-101 and vm-104 is the one you’re not allowed to reboot.

And set the no-subscription repository up properly rather than scripting the nag dialog away, because the people who script it away are the people who then never patch. A hypervisor is a large piece of attack surface running as root, and it isn’t the thing to leave sitting on an eighteen-month-old kernel.

Is it worth it

For me, yes. The value isn’t really the hardware — it’s that provisioning a machine becomes a five-minute decision instead of a purchase, so wanting to try something no longer has a cost attached. That changes what you try.

The trade is noise, heat, about ten pounds a month in electricity, and a machine that looks like it was rescued from a skip.