Just to add some more background on this: I used to work tightly with the Network Team in the website team of the biggest contender in its market (can’t disclose which one without people figuring out the company since the market is a bit niche).
We had 20’000 Users a day with a lot of images served.
The whole infrastructure consisted of 2 Firewall servers and the main DB (pSQL) on 2 self-hosted servers (think colo, it was sitting in a very remote location with 2 big diesel generators that would’ve ran the whole datacenter for a week iirc), with 14 Hetzner backend mirrors who ran the whole PHP code, served images and the angular + some weird custom Javascript. Scaling was done by simply throwing more Hetzners at it.
Given that Lemmy runs super performance efficient in comparison to 20 year deprecated PHP code that held together with ducttape, I feel like much less could make it work.
yeah, you pretty much described the use case for k8s. It allows for rapid horizontal scaling, since you can easily throw another machine into the cluster if you need it. It mostly makes sense if you actually have multiple machines sitting idle to begin with, so this technology is mostly used in combination with managed quick rent servers (think AWS).
Beyond that, k8s is kinda fancy for cluster management, but if you don’t have a cluster you kinda don’t need it to begin with. Using simple kernel VMs (think Proxmox) or just Docker works better there. You could still go for k8s since it’s pretty much docker with cluster functionalities, just in case you want to expand eventually (sidenote, docker allows for cluster functionalities too, but they put a price on it, while k8s is open source iirc).
In that company I worked, k8s was considered but ultimately not implemented since it was considered a bit overkill. We already had everything set up with a bunch of bash scripts anyway, so it didn’t matter too greatly to begin with.