Tools and ideas for backup of many files - eviltoast

Hello everyone,

In a day or two, I am getting a motherboard with an N100 integrated CPU as a replacement to the Raspberry Pi 4 (2 GB Model). I want to run Jellyfin, the *arr stack and Immich on it. However, I have a lot of photos(for Immich) and movies(for Jellyfin) (in total about 400 GB) that I want to back up, just in case something happens. I have two 1TB drives, one will have the original files, and the second will be my boot drive and have the backup files.

How can I do that? Just copy the files? Do I need to compress them first? What tools do I need to use, and how would you do it?

Thanks in advance.

EDIT: I forgot to mention that I would prefer the backups to be local.

  • pe1uca@lemmy.pe1uca.dev
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    5 months ago

    For local backups I use this command

    $ rsync --update -ahr --no-i-r --info=progress2 /source /dest
    

    You could first compress them, but since I have the space for the important stuff, this is the only command I need.

    Recently I also made a migration similar to yours.

    I’ve read jellyfin is hard to migrate, so I just reinstalled it and manually recreated the libraries, I didn’t mind about the watch history and other stuff.
    IIRC there’s a post or github repo with a script to try to migrate jellyfin.

    For immich you just have to copy this database files with the same command above and that’s it (of course with the stack down, you don’t want to copy db files while the database is running).
    For the library I already had it in an external drive with a symlink, so I just had to mount it in the new machine and create a simlar symlink.

    I don’t run any *arr so I don’t know how they’d be handled.
    But I did do the migrarion of syncthing and duplicati.
    For syncthing I just had to find the config path and I copied it with the same command above.
    (You might need to run chown in the new machine).

    For duplicati it was easier since it provides a way to export and import the configurations.

    So depending on how the *arr programs handle their files it can be as easy as find their root directory and rsync it.
    Maybe this could also be done for jellyfin.
    Of course be sure to look for all config folders they need, some programs might split them into their working directory, into ~/.config, or ./.local, or /etc, or any other custom path.

    EDIT: for jellyfin data, evaluate how hard to find is, it might be difficult, but if it’s possible it doesn’t require the same level of backups as your immich data, because immich normally holds data you created and can’t be found anywhere else.

    Most series I have them in just the main jellyfin drive.
    But immich is backedup with 3-2-1, 3 copies of the data (I actually have 4), in at least 2 types of media (HDD and SSD), with 1 being offsite (rclone encrypted into e2 drive)

    • VitabytesDev@feddit.nlOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      5 months ago

      Thanks for responding. I actually don’t have Immich yet on the Raspberry Pi, so it’s the first time I will be installing it and then importing the photos. I don’t actually care a lot about the migration, since I can just reconfigure the services. I want to ensure that if a drive fails, I can restore the data. I would try RAID, but I read that “RAID is not backup”. Or I could just run the command you provided in a cronjob.

      • pe1uca@lemmy.pe1uca.dev
        link
        fedilink
        English
        arrow-up
        2
        ·
        5 months ago

        In that case I’d recommen you use immich-go to upload them and still backup only immich instead of your original folder, since if something happens to your immich library you’d have to manually recreate it because immich doesn’t update its db from the file system.
        There was a discussion in github about worries of data being compressed in immich, but it was clarified the uploaded files are saved as they are and only copies are modified, so you can safely backup its library.

        I’m not familiar with RAID, but yeah, I’ve also read its mostly about up time.

        I’d also recommend you look at restic and duplocati.
        Both are backup tools, restic is a CLI and duplocati is a service with an ui.
        So if you want to create the crons go for restic.
        Tho if you want to be able to read your backups manually maybe check how the data is stored, because I’m using duplicati and it saves it in files that need to be read by duplicati, I’m not sure if I could go and easily open them unlike the data copied with rsync.