Proxmox HA, Docker Swarm, Kubrenetes, or what? - eviltoast

I’ve gotten to the point where I have more than a few servers in my homelab and am looking for a way to increase reliability in case of an update. Two problems: 2 of the servers will be on Wifi and one is a Synology NAS. I can’t do any wiring but I can put together a WiFi 6E network for the servers only, That means buying 4 Wifi 6E devices in a mix of types. As for Synology, it’s container manager is a little odd so I expect to run a Linux VM and use that as my cluster node. That may mean buying more RAM as I haven’t upgraded it. Hardware ranges from a 6 core CPU on the NAS (with a few important docker containers), 8 core on my main SFF server (which also runs my OpnSense VM inside Proxmox), 16 core Ryzen on my old big server, and a 10 year old NUC for fun. So the question is what do I use to orchestrate all the services I have. My Vaulwarden runs reliability but only on one system. I want better reliability for Pihole that automatically syncs settings. The NAS’ docker implementation doesn’t support gravity sync. Since everything I do runs in docker besides storage it seems Proxmox clusters is not the best option. That puts me between K8s and Docker Swarm. I’d like something that is simple to administer but resilien when hardware fails.

  • Domi@lemmy.secnd.meEnglish
    81·
    10 months ago

    Definitely go with K3s instead of K8s if you want to go the Kubernetes route. K8s is a massive pain in the ass to setup. Unless you want to learn about it for work I would avoid it for homelab usage.

    I currently run Docker Swarm nodes on top of LXCs in Proxmox. Pretty happy with the setup except that I can’t get IPv6 to work in Docker overlay networks and the overlay network performance leaves things to be desired.

    I previously used Rancher to run Kubernetes but I didn’t like the complexity it adds for pretty much no benefit. I’m currently looking into switching to K3s to finally get my IPv6 stack working. I’m so used to docker-compose files that it’s hard to get used to the way Kubernetes does things though.