Can We Nixify Lemmy's Dev Environment? - eviltoast

How easy would it be to flakify the Lemmy repo to add a fourth build option for those of us in the Nix world?

I had the (perhaps foolishly ambitious) idea of creating a rolling fork of Lemmy with the intent of modifying the codebase for use in an open source pub sub implementation of retail inventory. But I have to get standard Lemmy working first…and I like to use Nix for everything I do in the dev world (where feasible).

So, I forked the repo and was immediately brought into dev environment hell.

They only offer a choice between:

A.) Docker B.) Ansible C.) Building from scratch.

Two hours of fighting with the scratch build instructions and I eventually had to admit defeat due to some vague dependencies (and general malaise). Though I have completely flakified my Purescript and Haskell dev environments, I have found Rust to be a lot more challenging even on simple projects.

Anyway, I decided to come here to ask: **How easy would it be to flakify the Lemmy repo to add a fourth build option for those of us in the Nix world? **

Can I reference the build instructions from nixpkgs to get close to my intended goal? I need all of the help I can get. Be as pedantic or defeatist as you will. I currently have no skin in this game and merely want to help the Lemmy devs welcome people that are more nixy like myself (if nothing else).


edit: here’s a naive attempt. I haven’t tested anything other than it being a valid flake. Here’s the link to my rolling flakification fork.

{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
    systems.url = "github:nix-systems/default";

    # Dev tools
    treefmt-nix.url = "github:numtide/treefmt-nix";
  };

  outputs = inputs:
    inputs.flake-parts.lib.mkFlake { inherit inputs; } {
      systems = import inputs.systems;
      imports = [
        inputs.treefmt-nix.flakeModule
      ];
      perSystem = { config, self', pkgs, lib, system, ... }:
        let
          cargoToml = builtins.fromTOML (builtins.readFile ./Cargo.toml);
          nonRustDeps = [
            pkgs.libiconv
            # Derived from Documentation on Dev installation for Linux
            pkgs.cacert
            pkgs.postgresql
            pkgs.openssl
            pkgs.pkg-config
            pkgs.libpqxx
          ];
          rust-toolchain = pkgs.symlinkJoin {
            name = "rust-toolchain";
            paths = [ pkgs.rustc pkgs.cargo pkgs.cargo-watch pkgs.rust-analyzer pkgs.rustPlatform.rustcSrc pkgs.cacert pkgs.postgresql pkgs.openssl pkgs.pkg-config pkgs.libpqxx ];
          };
        in
        {
          # Rust package
          packages.default = pkgs.rustPlatform.buildRustPackage {
            inherit (cargoToml.package) name version;
            src = ./.;
            cargoLock.lockFile = ./Cargo.lock;
          };

          # Rust dev environment
          devShells.default = pkgs.mkShell {
            inputsFrom = [
              config.treefmt.build.devShell
            ];
            shellHook = ''
              # For rust-analyzer 'hover' tooltips to work.
              export RUST_SRC_PATH=${pkgs.rustPlatform.rustLibSrc}
              export CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG=true

              echo
              echo "🍎🍎 Run 'just <recipe>' to get started"
              just
            '';
            buildInputs = nonRustDeps;
            nativeBuildInputs = with pkgs; [
              just
              rust-toolchain
            ];
            RUST_BACKTRACE = 1;
          };

          # Add your auto-formatters here.
          # cf. https://numtide.github.io/treefmt/
          treefmt.config = {
            projectRootFile = "flake.nix";
            programs = {
              nixpkgs-fmt.enable = true;
              rustfmt.enable = true;
            };
          };
        };
    };
}

and here’s the lemmy-ui flake (which, judging by past forays into tsc with nix last summer, should be a lot easier to create and work with than the Rust library):

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    systems.url = "github:nix-systems/default";
  };

  outputs = {
    systems,
    nixpkgs,
    ...
  } @ inputs: let
    eachSystem = f:
      nixpkgs.lib.genAttrs (import systems) (
        system:
          f nixpkgs.legacyPackages.${system}
      );
  in {
    devShells = eachSystem (pkgs: {
      default = pkgs.mkShell {
        buildInputs = [
          pkgs.nodejs
          # You can set the major version of Node.js to a specific one instead
          # of the default version
          # pkgs.nodejs-19_x

          # You can choose pnpm, yarn, or none (npm).
          pkgs.nodePackages.pnpm
          # pkgs.yarn

          pkgs.nodePackages.typescript
          pkgs.nodePackages.typescript-language-server
        ];
      };
    });
  };
}
  • onlinepersona@programming.dev
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    8 months ago

    Hmm, do you need to contribute? I think there should be a way to have a shell.nix that uses the lemmy-server package as input.

    Damn, the nixpkgs manual is horrific. Ctrl+F “mkShell” requires hitting “next result” for ages to finally get to the “documentation”. Sheesh.

    Anyway, have you tried putting this into the repo

    shell.nix

    { pkgs ? import <nixpkgs> {} }:
    pkgs.mkShell {
      inputsFrom = [ pkgs.lemmy-server ];
    }
    

    then executing nix-shell ?

    If so, you could try to make a default.nix from the lemmy-server package and open a PR with it. Maybe they’ll accept it.

    Edit: I see you wrote “flakes”. Can’t help you with that. I find the documentation on that even worse than normal nix documentation since it’s cobbled together from so many different sources.

    CC BY-NC-SA 4.0

    • demesisx@infosec.pubOP
      link
      fedilink
      English
      arrow-up
      13
      ·
      8 months ago

      I’ll definitely give that a try.

      I was hoping to spell out everything in the flake. So, maybe I’d use the lemmy-server nix build instructions to help me get a proper flake working. I definitely want to contribute because, at least for me, it kept me from joining their dev efforts in any meaningful way.

  • pr06lefs@lemmy.ml
    link
    fedilink
    English
    arrow-up
    6
    ·
    edit-2
    8 months ago

    Are you just after a dev environment? Or are you looking to make lemmy a nix module, so you could make it a service on a nixos machine?

    I made some flakes that might be a starting point for nix develop.

    https://github.com/bburdette/lemmy/tree/flake.nix

    https://github.com/bburdette/lemmy-ui/tree/flake.nix

    Both repos require git submodule init and git submodule update .

    After nix develop, I was able to build the rust with cargo build.

    The lemmy-ui one you have to npm install pnpm so it makes a node_modules dir, then you can exit and reenter nix develop to get it to set the path. I used nodejs_20 but you might not need to. When I run it with pnpm dev, it complains about missing translation files. Some files got installed with a submodule, but maybe they’re in the wrong place? dunno, stopped there.

    Beyond that, I would try using their docker compose thing. Probably that will use the compiled typescript and/or rust code so you can work in a normal env and have those containers use your code. Haven’t tried it.

    • demesisx@infosec.pubOP
      link
      fedilink
      English
      arrow-up
      10
      ·
      8 months ago

      This looks great. I’ll try to incorporate some of what you did in a little while. Thanks!

    • einsteinx2@programming.dev
      link
      fedilink
      English
      arrow-up
      1
      ·
      8 months ago

      Heads up with modern git you can now just use git clone --recursive and it will clone all sub modules automatically.

  • ours@jlai.lu
    link
    fedilink
    English
    arrow-up
    1
    ·
    7 months ago

    If you’re using flakes and since lemmy is packaged in nixpkgs, you can spawn a dev shell with:

    nix develop nixpkg#lemmy-server
    
    • demesisx@infosec.pubOP
      link
      fedilink
      English
      arrow-up
      11
      ·
      edit-2
      7 months ago

      Thanks for the advice.

      I’m working on getting my own fork of Lemmy therefore I’m attempting to repackage it independently of Nixpkgs so I can get a flake in their actual repo that allows devs to spin up a Lemmy dev environment. I want my dev environment to reflect the changes being made on Lemmy in that dev environment when I make them. Unless I’m missing something, with your suggestion, I’d have no way to test a major change to Lemmy’s codebase in my fork (as I intend to).

      • ours@jlai.lu
        link
        fedilink
        English
        arrow-up
        1
        ·
        7 months ago

        This gives you a build environment for lemmy, see the example. You can clone upstream and use that shell to build lemmy. Quite convenient to test quick changes. However, that won’t help distributing your fork.