Remote Server won't come back online after "sudo reboot" - eviltoast

I know I probably fucked something up, but still want some advice.

I have two houses, halfway across the world from each other. Whenever I am on holidays in my second home, I would like to still access my home network and vice versa. I have a Tailscale VPN setup for this and I regularly SSH into my server from other devices to configure it, rather than use the physical device. I tend to only access it whenever I need to turn it on or off.

TIFU by trying to reboot it. I was configuring some network stuff for my brand-new project with installing PiHole, and after debugging a little issue, having changed many configurations and being unsure about how to restart everything needed for the configs to take effect, the answer that I was following suggested rebooting to sort all issues in one go. Having tried `sudo reboot` on my local VM earlier today, I thought it couldn’t hurt, ran the command on my remote, and it hasn’t come back online yet.

It should be automatically connecting to tailscale on startup, it has worked like that in the past, but it hasn’t this time. It has been an hour since tailscale last connected with the device.

What did I do wrong, and how do y’all handle rebooting your bare metal when you don’t have access to the physical server atm?

  • mazowo@alien.topB
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    It can have many reasons as to why the server won’t connect again. Most likely one of the configurations changes caused it to loose internet connectivity. Don’t worry, it happens to everyone at some point.

    I can’t tell you what exactly you did wrong, but I can give you tips on how to prevent it in the future:

    Firstly, take the time to research about whichever service you want to change. Typically, there is a way to reboot any service, without rebooting the entire server. Once you figured that out, you can write a little script that backd up your original config, applies the new one and then reverts the old one back unless you manually cancel within 30 or more seconds.

    This ensures that, if your server does loose internet connectivity, your changes are reverted automatically. If everything works as exoected, you just CTRL + C to cancel the rest of the script. Also make sure to spawn the script in an independant tool like tmux, so it doesn’t end with your session.

    Only once you are sure that all your configs 100% work, you may or may not fully restart the server.