Zombie Zen

Posts tagged "Software Development"

Bundling Scripts with Nix

Posted at by Ross Light

I write a lot of shell scripts. Many are one-offs or specific to a project, but every so often, I’ll have a script that transcends to become a part of my toolbelt. For example, nix-op-key is a script I wrote to generate new Nix signing keys and place them in 1Password. It’s not a task that requires a dedicated program, it just needs to glue two existing programs together: nix key generate-secret and op (the 1Password CLI). These sorts of scripts are great, but if you want to share them with someone else (or even just use it on a different computer), how do you do it? Scripts like these depend on specific programs (or maybe even specific versions) being installed and Bash does not have a package manager like pip or the go tool.

As it turns out, Nix is such a package manager. And with flakes, there’s built-in support for installing and running scripts with well-specified dependencies in a single command. For example, you can run my nix-op-key script I mentioned earlier (pinned to a specific version) with:

COMMIT=25e9bd52e977cca415df84ea91028efd92d3da92
nix run "github:zombiezen/dotfiles?dir=nix&rev=$COMMIT#nix-op-key" -- --help

Furthermore, you can install the script using the nix profile install command:

nix profile install "github:zombiezen/dotfiles?dir=nix&rev=$COMMIT#nix-op-key"

(If you try this out yourself, you can uninstall the script with nix profile remove '.*.nix-op-key').

In this blog post, I’ll show you how you can package your own shell scripts with Nix to make them more reliable and easier to share. This article assumes familiarity with Unix command line and Bash shell scripting. I’m using Nix 2.17.1. All of the source code in this post is released under the Unlicense and is available on GitHub.

Read more…
Posted at
Permalink

Impure Nix Derivations

Posted at by Ross Light

I’ve been writing about Nix for a few years now, and it has become a staple for all my personal projects. It solves the thorny problem of setting up reproducible development environments in a flexible and general way, while still allowing me to use my tooling the way I want to. I no longer have to worry about my GitHub Actions having different versions of tools from my local dev environment, which is a huge relief.

I’ve started making my CI jobs use nix flake check, which lets me write and run my CI checks using Nix syntax, further removing any differences between local development and CI. However, I sometimes have tests that want to use the network, which usually isn’t allowed. I recently discovered that you can use __impure = true; to remove the networking sandbox:1

# flake.nix
{
  inputs = {
    nixpkgs.url = "nixpkgs";
  };

  outputs = { nixpkgs, ... }: {
    checks.x86_64-linux.foo =
      let
        pkgs = import nixpkgs { system = "x86_64-linux"; };
      in pkgs.stdenvNoCC.mkDerivation {
        name = "foo-check";
        src = ./.;
        __impure = true;
        nativeBuildInputs = [
          pkgs.cacert
          pkgs.curl
        ];
        buildPhase = ''
          runHook preBuild
          curl -fsSL https://example.com/ > /dev/null
          runHook postBuild
        '';
        installPhase = ''
          runHook preInstall
          touch "$out"
          runHook postInstall
        '';
      };
  };
}

For this to work, you must also set extra-experimental-features = impure-derivations ca-derivations in /etc/nix/nix.conf.2 Just passing a command-line argument doesn’t seem to work in my usage with Nix 2.13.3 in a multi-user installation.

Since this feature is still experimental, it’s not widely advertised or documented. Hopefully this helps you use it for your own tests.

Posted at
Permalink

Connecting Bash to Nix

Posted at by Ross Light

Julia Evans wrote a toot asking:

are there any guides to nix that start from the bottom up (for example starting with this bash script https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh) and then working up the layers of abstraction) instead of from the top down?

I realized that despite the title, my blog post Nix From the Ground Up misses the mark on providing this type of explanation. While I do think the Nix language is the lowest abstraction layer to learn Nix, I wanted to zoom in on the core derivation abstraction through a tutorial. This way, we can better understand how Nix derivations relate to Bash scripts.

Read more…
Posted at
Permalink

Nix From the Ground Up

Posted at by Ross Light
Nix logo

I recently spent some time learning Nix after watching this talk by Xe. Nix is a package manager/build system for Linux and macOS. It does a number of things I really like:

  • Transparent handling of source and binary packages.
  • Includes a rich central package registry, but you can host your package descriptions or binaries anywhere.
  • Does not require root and runs alongside any Linux distribution.
  • Easy to pin or customize versions of individual packages.
  • Straightforward support for project-specific dependencies.

Nix is a cool piece of tech, but in my opinion, it’s pretty hard to learn (at least at time of writing). I think this is accidental complexity: I was able to be productive with Nix in my personal projects in a few days, but it took a fair amount of research from many different sources. I took a lot of notes, then realized I wanted to publish them to share this knowledge.

So here’s my guide! “Nix From the Ground Up” aims to help explain the concepts behind Nix with a hands-on approach.

Read more…
Posted at
Permalink

How I packaged a Go program for Windows and Linux

Posted at by Ross Light
Icon by Philipp Petzka, used under a Creative Commons license.

Icon by Philipp Petzka, used under a Creative Commons license.

In the two months since I published gg 1.0, a project to reduce the friction in working with Git, I’ve been working to make it more accessible and easier to install. To this end, I’ve made three big improvements:

  • A standalone Go library, gg-scm.io/pkg/git, allows any Go program to interact with Git repositories. (I may end up writing another blog post just about this — stay tuned!)
  • Windows support, complete with MSI installer.
  • An APT repository for Debian and Ubuntu users.

If you’re interested in trying out gg, it’s never been easier: see the instructions at gg-scm.io. Read on if you’re interested in how to package a Go program for Windows and Linux.

Read more…
Posted at
Permalink

gg 1.0 released!

Posted at by Ross Light
gg

I’m proud to announce the first stable release of gg, my alternative Git command-line interface! This has been a release over two years in the making: I’ve battle-tested gg across many different workflows and projects. It’s saved me tons of time every day, and I hope it can save time for others too. Download the latest release and try it out for yourself!

Read more…
Posted at
Permalink

How Structure Affects Git's UX

Posted at by Ross Light

It’s always interesting to me to compare different approaches to solving the same problem. Git and Mercurial are two version control systems that came out at similar times, trying to address very similar requirements. Git came from a very low-level systems perspective, whereas Mercurial spent a lot of effort on its user experience. Despite what you might think, their data models are remarkably similar. It’s from this observation I started my side project — gg. I found myself greatly missing the experience of Mercurial, but I’ve resigned myself to the fact that Git is here to stay.

I came across a rather interesting challenge today while working on gg. I am trying to replicate the behavior of hg pull, and even though I’ve worked on gg for over a year now, I still haven’t reached a behavior that I’m satisfied with. I’ve finally realized why, and it boils down to a very subtle difference in the data models of the two systems.

Read more…

Life of a Go Infrastructure Maintainer

Posted at by Ross Light

I originally gave this as a talk at the Seattle Go Meetup on 2017-02-16 (video). The following is a refined version of the talk, not just a verbatim transcript, based on my speaker notes.

Read more…