Nix 3rd party repos and Binary packages - eviltoast

I’ve read a lot, but didn’t get a simple answer for 3 topics I’m interested about:

  1. Are there any 3rd party repos for NixOS? It seems some people overlay one package, but I cannot find entire packagesets such as the one in Nixpkgs. Are channels used for that? Would I be able to have my own repo with several packages? Can I have more than 1 repo in my Nix system, and set which package install each program from?
  2. Would it be possible, for propietary software compiled directly for NixOS, to have a binary-only (as opposed to source, or binary+patched) repository? Everything I see is either source+cache or binary+patched, but nothing like binary especifically for NixOS.
  3. Let’s say I want to add some extra, propietary codecs unavailable to NixOS. How would I be able to change a dependency so that each package dependent on it automatically picked it? Could I switch a dependency for a certain repo from one of another, different repo?

Thank you for your support guys!

  • Laser@feddit.de
    1·
    1 year ago

    Thanks for the explanation. I didn’t make the connection that buildInputs is an attribute itself as it is an attribute of stdenv instead of the function describing the derivation directly. Or at least I think that’s where my confusion comes from.

    I tripped over some stuff back then when I found out that which is not part of the sandbox.

    That’s what buildInputs are for. Add which to buildInputs and it’s available inside the sandbox. The stdenv takes care of putting its binaries into $PATH and making its libraries discoverable.

    In the case of which, you’d probably need it in order to execute its binary during the build process though, so nativeBuildInputs is more appropriate but that only truly matters for cross-compilation.

    Small correction, it wasn’t which, but rather env, I had those mixed up. The “issue” is described here: https://github.com/NixOS/nixpkgs/issues/6227 What created more problems was that patchShebangs wouldn’t work here because it appeared in a configure script that was created and run during the actual build process (I think the build process is horrible, but here it is in case you’re inclined: https://github.com/BSI-Bund/TaSK/tree/master/tlstesttool the stuff in the 3rdparty directory gets downloaded, configured and linked against in the main program’s build phase so you have no opportunity to actually follow the solution in that issue, similar to what is described here https://github.com/NixOS/nixpkgs/issues/6227#issuecomment-73410536. I got it to work in the end and like to tell myself that it’s elegant but the project’s build process is just bad in my opinion.

    https://pastebin.com/0GwLk1wP if you want to see an example of my level of nix-fu. I have programming basics but the nix language can be confusing sometimes. I’d say I have a basic understanding of things but as said before the more intricate stuff still escapes me.

    • Atemu@lemmy.mlEnglish
      1·
      1 year ago

      it wasn’t which, but rather env, I had those mixed up. The “issue” is described here: https://github.com/NixOS/nixpkgs/issues/6227

      The problem there isn’t that env isn’t available (it is, we have coreutils in stdenv) but rather that /usr/bin/env (that specific path) does not exist in the sandbox.

      What created more problems was that patchShebangs wouldn’t work here because it appeared in a configure script that was created and run during the actual build process (I think the build process is horrible, but here it is in case you’re inclined: https://github.com/BSI-Bund/TaSK/tree/master/tlstesttool the stuff in the 3rdparty directory gets downloaded, configured and linked against in the main program’s build phase

      Yikes, you don’t want that.

      so you have no opportunity to actually follow the solution in that issue

      I didn’t read the issue in full but, since everything is pre-downloaded, you can always fix these scripts.

      For the auto-generated configure script for example, you’d have to patch the build step which generates the configure script to put in ${coreutils}/bin/env rather than /usr/bin/env. Simple as that.

      3rd party repositories can be patched during their respective fetches. You have to inject them anyways since there’s no way to download 3rd party repos during a build.

      https://pastebin.com/0GwLk1wP if you want to see an example of my level of nix-fu. I have programming basics but the nix language can be confusing sometimes. I’d say I have a basic understanding of things but as said before the more intricate stuff still escapes me.

      You have to differentiate between the Nix expression language and using Nixpkgs’ frameworks to define packages here. These are two entirely different skillsets with only a slight dependence on the former by the latter.

      If you take a look at your expression, it only required fairly basic Nix syntax: Simple function calls, declaring recursive attrsets, string interpolation and attribute access. Most packages are like this.
      Figuring out which attributes need to be set how and what that means is an entirely different skillset. Defining patchPhase in that attrset is trivial syntax-wise but figuring out what needs to be substituted using which stdenv “library” function is something else entirely.

      Looks pretty good btw. I’d look into whether the build could be coerced to link against Nixpkgs’ versions of those libraries though instead of vendoring dependencies. Especially security-critical stuff like openssl. That’d probably also save you the trouble with the interpreter.
      If you take a look at the build definition, there’s this handy dandy USE_3RDPARTYBUILD option.

      I’d wager if you did cmakeFlags = [ ... "-DUSE_3RDPARTYBUILD=OFF" ]; and buildInputs = [ asio zlib openssl ... ]; (from pkgs, not your fetched sources) it’d just work. (Might need pkg-config in nativeBuildInputs but I don’t think it uses that and will instead discover those deps via cmake.)

      If you can get rid of the vendoring, feel free to submit that to Nixpkgs ;)