Today I’m making a broadly visible change to live more authentically and joyfully.
My name is Roxy Light. My pronouns are she/her or they/them. You may have known me by a different name, but I always have been — and always will be — zombiezen.
In my previous blog post, I talked about some of the shortcomings of Nix as a project and what I wish to see exist in its stead. Since I published the blog post, there have been some promising developments: a constitutional assembly has formed, a fork of the Nix interpreter has arisen, and an alternative Nixpkgs ecosystem is beginning to form. I am enheartened that Nix has a potential future, but it is too early to tell how these new efforts will play out.
In the meantime, I prototyped an idea I had kicking around while I wrote the blog post. zb is an experiment in using Lua as an build configuration language while still using the model described in The Purely Functional Software Deployment Model. I am quite pleased with the results. In my view, this is an existence proof for using a more conventional programming language to reduce the onboarding cost of a technology like Nix.
The README has a more in-depth description of what I built, but in short:
.drv files and uses nix-store --realise to build them.
This means that zb is fully interoperable with Nix derivations
(e.g. using nixpkgs derivations in zb derivations),
but also proves that an alternative build backend could be created in the future.I want to share this research prototype in the interest of showing others what is possible in the space, but I’m not planning on developing it to production-readiness. If you have feedback on zb, let me know on the Auxolotl forum, on the zb repository discussions, or privately.
(This is a long post. In the words of Blaise Pascal, “I have made this longer than usual because I have not had time to make it shorter.”)
I see the Nix project to have passed a critical moment. I think the Nix project has made great strides in solving some very hard problems, but my fear is that due to mismanagement, Nix will not succeed. In this post, I hope to share my perspective on what a better future for declarative, reproducible builds could look like, but sadly, I don’t think it involves Nix as it exists today. It’s hard to know what the future holds, and I am certain I do not hold the solutions to the problems that exist with Nix. However, I feel it would be a shame for the progress to be lost.
As background, on 2024-04-21, a group of concerned contributors and community members published an open letter to the NixOS foundation detailing Eelco Dolstra’s ill effects on the Nix project’s governance. On 2024-04-26, Eelco published a blog post that has caused many package maintainers to leave, including me. While I don’t have direct experience with the problems in the letter, I very much trust the folks who signed the letter. I am disappointed with how Eelco has chosen to handle the situation. I personally think this is a watershed moment for Nix, and not for the better. Such actions set the precedent for how an organization will act in the future, and given the power structure, there’s not much that can be done. In my mind, this endangers the health of Nixpkgs moving forward, and without healthy package maintenance, the Nix project is doomed. Consider this post a eulogy for a dream.
While the solution-that-is-Nix may evaporate, the problem it solves remains, at least for me. The rest of this blog post is my attempt at sketching what a suitable replacement would look like, in the hopes that it inspires someone to succeed where others have not. Perhaps a replacement is a fork, perhaps it is a wholly new project, perhaps it is the Nix project under better governance. I don’t know.
I’m happy to announce a project I’ve been working on for the last month: pql. pql is a pipelined query language inspired by the syntax of Microsoft’s KQL and Splunk’s Search Processing Language that I developed in collaboration with the fine folks at RunReveal. pql takes queries like:
logs
| where eventName != ''
| summarize AllEventNames=groupUniqArray(eventName) by sourceType
and translates them into SQL like:
WITH "__subquery0" AS (SELECT * FROM "logs" WHERE coalesce("eventName" <> '', FALSE))
SELECT "sourceType" AS "sourceType", groupUniqArray("eventName") AS "AllEventNames" FROM "__subquery0" GROUP BY "sourceType";
I personally find the pql query easier to read and modify, while still retaining the power of what a relational database has to offer. pql supports all the basics of filtering, sorting, and aggregating, as well as complex joins.
Under the hood, pql builds a full Abstract Syntax Tree (AST) of its input and transforms it into a sequence of chained subqueries (similar to Static Single-Assignment form). While the generated SQL is sometimes more verbose than hand-written SQL, the queries are straightforward for database query planners to optimize out. This means that the generated queries are just as efficient as the equivalent, more condensed queries.
pql can be used with nearly any SQL-based database. If you’re interested in learning more, check out pql.dev and the announcement on RunReveal’s blog. pql is packaged as a Go library and small CLI that’s available on GitHub under an Apache 2.0 license.
If you're looking for someone to help with a complex software problem like this for your business, I'm available for consulting and contract work! Read about how I can help you and contact me at consulting@zombiezen.com.
rsync is a wonderfully useful utility for copying files
both locally and between machines.
It makes a reasonable effort to copy only files that have changed,
which can save a tremendous amount of time in many scenarios.
However, I’ve always been a bit daunted by its plethora of command-line options,
and would always resort to its man page for every invocation.
Thus, it would never stick in my set of tools I would commonly use.
After doing some work where rsync really was the only tool for the job,
I decided to finally make a little cheatsheet on how to use rsync.