I’m running three servers: one for home automation/NVR, one for NAS/media services, and one for network/firewall services.
Does this breakdown look doable based on the hardware? Should the services be ditributed differently for better efficiency?
Server 1 and 3 are already up and running. I just received my NAS, and am trying to decide where to run each service to best take advantage of my hardware.
I’m also considering UnRaid instead of Proxmox for a NAS OS. I just chose Proxmox because I’m familiar with it, and I like the ability to snapshot. I also intend to run Proxmox Backup Server offsite at some point, and I like the PVE/PBS integration.
Any advice would be much appreciated!
Just remember the KISS principal: Keep It Simple, Stupid
Keep the NAS as a NAS, and I would honestly trim down everything else into a clustered hypervisor setup (like Proxmox) with dedicated VMs to run each stack. That way if you need to take a machine down for whatever reason, you can migrate its VMs/containers to another machine, with minimal downtime, so you can do whatever it is you need to do with said machine.
Full disclosure: this is what I do. I was in your shoes before.
I wouldn’t do that unless you have lots of money to blow on crazy hardware. Running separate virtual machines is very inefficient. Instead, run a few virtual machines with a few services in each. I would separate it out into classes based on the load and use case.
Or just run them in containers and skip the need to run the VMs at all. You can do snapshots with Debian fine.
I personally would avoid LXC. That seems to be a hot take but in my experience it is better to run docker/podman in a few VMs.
…really? I run most of my services in an LXC, and have for a while without issue.
Maybe I’m doing it wrong then. I run LXC but has always been a much worse experience. Boot times are terrible and the controls that work for VMs don’t work as well for LXC. You also can live transfer which is problematic for me.
I think you’re doing it wrong. LXCs boot almost instantaneously on a hypervisor since they hijack the host kernel, I’d be surprised if my CTs take 5 seconds.
I would agree on the live migration issue but I guess you pick your services accordingly. I have a VM that runs docker and a LXC docker host, and I pick my containers for each accordingly.
How on earth are you getting 5 second boot time with LXC? My containers take around 10 minutes to boot while VMs take a few seconds. Also LXC networking seems to break randomly.
Edit: I went back and figured it out. It was that IPv6 was set to dhcp in Proxmox which caused everything to halt until timeout. I set it to static in Proxmox and now it boots instantly
I have no idea what you have going on, I’ve never seen LXCs take that long, even if I include the time it takes to down the containers and bring them up after a reboot.
What are you using for running them? I just tested my docker LXC and it took 16 seconds from when I typed “reboot” to having a login prompt. And that’s on an ancient R410 server running proxmox.
That is not my experience at all…
Are you running ZFS?
Yes, RAID 10 ZFS with no ARC, 6GB SAS drives.
@possiblylinux127 @ikidd Something sounds wrong there. I exclusively use LXC containers because I loathe docker and my containers boot basically instantly, and the networking is rock-solid.