Zombie Zen

Posts tagged "programming"

A SQLite Notebook

Posted at by Ross Light

Similar to the Jupyter kernel for Ivy I hacked up a few months ago, I’ve created a SQLite kernel for Jupyter Notebook.

Screenshot of a SQLite Visual Studio Code notebook.

Screenshot of a SQLite Visual Studio Code notebook.

Check out the demo on GitHub! (Installation is a little DIY if you’re not using Nix, but there are instructions in the README on how to build.) Read on if you’re interested in how it works.

Read more…
Posted at
Permalink

An Ivy Notebook

Posted at by Ross Light

I hacked out a Jupyter kernel for Rob Pike’s ivy language, an APL-like desk calculator. This allows using Ivy expressions inside a Jupyter notebook and inside Visual Studio Code’s notebooks feature. It’s got rough edges, but it’s good enough for me to quickly do calculations without leaving Visual Studio Code.

Screenshot of a Jupyter notebook with the Ivy demo.

Screenshot of a Jupyter notebook with the Ivy demo.

The source is up on GitHub. (Installation is a little DIY if you’re not using Nix, but it should go build if you have libzmq installed locally.)

Read more…
Posted at
Permalink

Introducing postgrestest

Posted at by Ross Light

Today, I released a small library called postgrestest. It spins up an ephemeral PostgreSQL database in Go. I’ve found it quite useful for writing tests that use PostgreSQL while keeping the test hermetic and reasonably fast. In my benchmarks, starting a server takes roughly 650 milliseconds and creating a database takes roughly 20 milliseconds — a 70% improvement and 90% improvement, respectively, over a postgres Docker container in the default configuration.

You can install it with:

go get zombiezen.com/go/postgrestest
Read more…
Posted at
Permalink

Apollo Client Caching

Posted at by Ross Light

Since I’ve started working at Clutter, I’ve grown to enjoy the Apollo React Client library. However, I’ve noticed that the client’s caching behavior can be difficult to understand and cause surprising issues, so I’ve decided to collect my findings into one easily digestible post. This post assumes a basic understanding of GraphQL.

The major hidden component of Apollo Client is the cache, which sits between every query and the backend server.

The major hidden component of Apollo Client is the cache, which sits between every query and the backend server.

Read more…
Posted at
Permalink

Introducing: My GraphQL Go Server

Posted at by Ross Light
GraphQL logo

I’m excited to share my hobby project from the last few months: a GraphQL server library for Go. While it doesn’t implement the whole spec and might not be ready for production, I think it’s in a useful enough state to share more broadly. I’ve tested this library out with some toy projects and the results have been promising: I’m able to quickly publish a Go struct as a GraphQL endpoint.

Check it out now by reading the docs and adding it to your project with:

go get zombiezen.com/go/graphql-server/graphql
Read more…
Posted at
Permalink

How Structure Affects Git's UX

Posted at by Ross Light

It’s always interesting to me to compare different approaches to solving the same problem. Git and Mercurial are two version control systems that came out at similar times, trying to address very similar requirements. Git came from a very low-level systems perspective, whereas Mercurial spent a lot of effort on its user experience. Despite what you might think, their data models are remarkably similar. It’s from this observation I started my side project — gg. I found myself greatly missing the experience of Mercurial, but I’ve resigned myself to the fact that Git is here to stay.

I came across a rather interesting challenge today while working on gg. I am trying to replicate the behavior of hg pull, and even though I’ve worked on gg for over a year now, I still haven’t reached a behavior that I’m satisfied with. I’ve finally realized why, and it boils down to a very subtle difference in the data models of the two systems.

Read more…

The Design of Everyday Go APIs

Posted at by Ross Light

Frequently when people discuss what is a “good” Go library, they usually use terms like “idiomatic” or “the Go way”. These terms are imprecise and make it difficult to discuss the merits of different API designs. I recently re-read Don Norman’s The Design of Everyday Things and realized that the same principles of design discussed in the book can be used to evaluate the design of APIs.

Read more…

Canceling I/O in Go Cap'n Proto

Posted at by Ross Light

This report details an experience I had while writing an RPC system in Go. While Go’s standard I/O libraries make a great many things simple, I found cancellation to be more complex than I would have liked. Parts of this situation have improved in the last couple of Go releases (as I have noted below). I hope this positive trend continues in a way that allows the Go ecosystem to easily propagate cancellation, deadlines, and request values. My intent in this report — as well as the proposal I created back in May 2017 — is to give background and feedback to inform future design decisions. Suggestions for solutions welcome!

(Thanks to Ian Lance Taylor, Damien Neil, Cassandra Salisbury, and Andrew Bonventre for reviewing this report for accuracy and clarity.)

Read more…
Posted at
Permalink

cardcpx: a Go/AngularJS project

Posted at by Ross Light

I’ve posted a project I’ve been hacking on to my Bitbucket account: cardcpx. This has nothing to do with my day job, it’s just something I did to help out on a film shoot (which also had nothing to do with my day job).

From the README:

cardcpx is a specialized UI for copying files from a camera card to 1+ replicas. The replica copies happen concurrently, so if you are copying N bytes to R replicas, the time is O(N) instead of O(N * R).

The interface also has includes simple scene/take ingestion, which is stored in a CSV transaction log. Selects will be copied first, so you can do a proofing check on a fast disk while your import finishes.

cardcpx supports a flat directory structure as well as the RED camera directory structure. It assumes that your clip names do not overlap. Attempting to copy the same file name will not overwrite data.

It’s still a bit rough around the edges, but I’m happy with how Go allowed me to compose a fairly complex a tool out of easy-to-understand pieces. This was my first excursion into writing a frontend with AngularJS, but I’m also similarly pleased about the design.

Everything’s open source. Feel free to dig in and use. I’d love to know if other people find this program useful.

Chromebook Hacking: Developer Mode

Posted at by Ross Light

This is the second part in my series about creating a low-cost programming environment with an HP Chromebook 11. See Part 1 of Chromebook Hacking for the overview. This blog post assumes you have some basic knowledge of using a Linux terminal.

Enable Developer Mode

Danger: This will wipe out your entire local storage! Back up everything!

Enabling developer mode on a Chromebook is the blessed way of “rooting” your Chromebook: it gives you root shell access. This comes with the cost of losing parts of ChromeOS that depend on hardware-backed security systems (namely Netflix. Sad days).

Enabling developer mode is reversible (see below), but going in either direction will wipe your Chromebook’s SSD. Please back up your files first.

Instructions are on the Chromium OS website. For the HP Chromebook 11, the short version is: hold Esc+Refresh+Power button and when you see the boot screen, tap Ctrl+D. Follow the prompts and wait for roughly 10 minutes for the operating system to reinstall.

On the boot screen, you need to press Ctrl+D every time you boot to skip the “Danger, you don’t have verified boot” screen, or you can wait for 30 seconds. You can alternatively press the spacebar to perform a factory reset and disable developer mode.

Go through the standard Chrome OS “create an account” process to get to the desktop.

Install Secure Shell and set up Crosh

This step isn’t strictly necessary, but it gives you a very slick terminal interface. If you decide not to do this, you can always open crosh with Ctrl+Alt+T.

Install Secure Shell from the Chrome Web Store. Right-click on the app’s icon in the shelf and choose “Open as Window”. This allows you to use shortcuts like Ctrl+W inside the terminal without Chrome intercepting them. Click the shelf icon to open a new terminal. Create a new profile with a random non-empty user name (I picked foo) and a host of >crosh.

Secure Shell app with user "foo" and host "crosh"

Harden your install

Open a bash shell by running shell at the crosh prompt. Inside your bash shell, run this command:

chromeos-setdevpasswd

This sets your Chromebook’s UNIX password for the chronos user — the UNIX user used for any logged in Chrome profile. Now that you’ve enabled developer mode, chronos has the ability to run sudo. Since all Chrome profiles use the same UNIX user account, you will want to open chrome://settings/accounts and whitelist the users that can access this machine. Disable Guest browsing and restrict sign-in to yourself.

Chrome settings with guest browsing disabled and sign-in restricted to Ross's account

Finally, you should prevent your Chromebook from booting anything other than ChromeOS by running the following from your bash prompt:

crossystem dev_boot_usb=0 dev_boot_signed_only=1

The first parameter (dev_boot_usb) disables booting from an external USB stick. The second parameter (dev_boot_signed_only) forbids booting to operating systems that are not signed by Google.

Wrapping Up

Now you have a minimal Linux environment with bash and vim. After poking around a bit, you will quickly notice that only Downloads and /usr/local are writable. In the next blog post, I will introduce Crouton: a less restrictive Linux environment for ChromeOS.

Posted at
Permalink

Chromebook Hacking

Posted at by Ross Light
HP Chromebook 11 showing a command prompt

In my spare time, I’ve been working on Go-SDL2 and Camlistore. I’m usually not sitting in front of my home desktop, so I want to be able to code anywhere. My battle-worn MacBook Pro could do the job, but it’s heavy, bulky, and aging.

To fix this, I bought the HP Chromebook 11. The display is nice, the price is great, and the keyboard is comfortable. It accepts a microUSB charge, so I don’t even have to worry about packing an extra charger. The bummer is development: good luck trying to run anything besides JSFiddle.

For a while, I was using SSH into my desktop to code. Most of the time this works. The problem was me: some days I would forget to turn on my desktop, other days I left it in Windows, and every day I gritted my teeth if I was coding graphics. I fantasized about a machine that booted quickly and had all the programs I needed.

Turns out, the answer was right in front of me: the Chromebook.

Recipe for Success!

  • Buy the Chromebook
  • Enable developer mode
  • Install Secure Shell and set up with Crosh
  • Harden your install
  • Download and run Crouton
  • Install X11 and xmonad
  • Profit!

Total Cost: $279 + half a day’s worth of work.

Over the next few days, I’ll go into more detail on these steps.

Posted at
Permalink

A night of fun hacks

Posted at by Ross Light

I discovered that some of my code was still living on my old laptop from high school. The laptop hadn’t been booted in about 4 years, so the clock battery had died, causing the filesystem checks to fail (this is Ubuntu 9.10, which cared about these things). I reset the clock manually, boot up the machine, and decide that to be on the safe (but marginally inefficient) side, I’ll copy my entire multi-GB home directory to my more beefy desktop and filter out what I don’t want there.

Sadly, this laptop came before 802.11n and is limited to 54 Mbps. I have an AirPort Express with only one local port, which is connected to my desktop, but I don’t want to disrupt the network setup. I realize I can chain the Ethernet from my old laptop (named metroid) to my new laptop (named roran), which has 802.11n to connect to my desktop (named nasuada).

It's a UNIX system. I know this.
ross@nasuada$ nc -l 8080 > mybackup.tar.bz2
ross@roran$ nc -l 8080 | nc nasuada.local 8080
ross@metroid$ tar -jcf - $HOME | nc roran.local 8080



I love Unix.

Posted at
Permalink

Why Go Does Not Need Generics

Posted at by Ross Light

(This developed from a thread with some other Cal Poly students discussing Go.)

One of the frequently asked questions about Go is why are there no generics in the language. It’s a fair point from the perspective of other OOP languages, but idiomatic Go code un-asks the question.

First, it is important to understand how interfaces work in Go. The gist is that interfaces use virtual method lookups, but retain the underlying type, so a conversion back to the original type is still type-safe. An empty interface (interface{}) works as a type-safe equivalent of C’s void*: you can put any type inside of an empty-interface variable, and then you can un-box it later.

The simplest way to define a linked list is through the empty interface: interface{}.

type Node struct {
    Value interface{}
    Next *Node
}

And indeed, this is how the container/list package package works, give or take some complexity. However, my argument is that this is the wrong way to do this. It requires an explicit boxing/un-boxing hit for every access, and this is where (I think) most of the call for generics comes from. There’s a much more idiomatic way to do this. My two case studies here are the sort package and the container/heap package from the standard library.

We start with sort, which defines an interface, sort.Interface:

package sort

func Sort(data Interface)

type Interface interface {
    // Len is the number of elements in the collection.
    Len() int
    // Less returns whether the element with index i should sort
    // before the element with index j.
    Less(i, j int) bool
    // Swap swaps the elements with indexes i and j.
    Swap(i, j int)
}

This may not strike you as a generic container, but that is precisely the point. You can – with a bit of boilerplate – create a type that wraps a typed slice that implements sort.Interface. Thus, the algorithm can work on any type and only works uses indices. This is much cheaper than boxing/unboxing on access.

The second example is container/heap:

package heap

func Init(h Interface)
func Pop(h Interface) interface{}
func Push(h Interface, x interface{})
func Remove(h Interface, i int) interface{}

type Interface interface {
    sort.Interface
    Push(x interface{}) // add x as element Len()
    Pop() interface{}   // remove and return element Len() - 1.
}

Admittedly, there is a bit more boxing/un-boxing going on here, but the idea is still similar. We now have a data structure that requires very few box/unbox operations (two per push/pop). It is important to note that this is still type-safe.

You could implement a binary tree structure similarly:

package tree

func Search(root *Node, val interface{}) *Node

type Node struct {
    Index       int
    Left, Right *Node
}

type Interface interface {
    Len() int
    Less(i, j int) bool
    Push(x interface{})
    Pop() interface{}
}

The Search function would push the val interface and use it for comparisons against the values stored in the tree, then pop it once finished. Admittedly, this isn’t the prettiest interface, but it works in a pinch. It works out to be only one box/un-box round-trip.

As we’ve seen, by “flipping” around the algorithms and separating them from the data structures using interfaces, we can still reap the benefits of generic containers from other languages.

Posted at
Permalink

I’m finishing up my game, “Urge to Merge”, for my Intro to Computer Graphics course. This is the first time I’ve worked with OpenGL shaders; I’m pleased with the results. These are screenshots from the game.

Posted at
Permalink

goray: A Drop into BitBucket

Posted at by Ross Light

I’ve moved goray from SourceForge to BitBucket yesterday. This also signals a switch from Bazaar to Mercurial. Along with other changes in the goray project, I think this will make goray easier to work with and contribute to. Read on for more details.

Read more…

New Go Stable Release

Posted at by Ross Light
Go gopher

Yesterday, the Go programming language announced their second stable release, introducing language changes, better packages, and general speed-ups. I’m proud to say that I helped out in a small part by contributing a patch to the zlib package.

For those of you who haven’t heard me talk about Go, it’s a programming language that brings all of the pleasant features from scripting languages and mixes it with the simplicity of C. Its syntax and feel is unlike any other language, but after a week, it begins to feel like second nature, and it’s now one of the first languages I grab to solve a problem. I can’t say enough good things about it, and if you haven’t already, download it and try the tutorial. (And yes, it will feel weird at first. Give it a week if you’re coming in fresh.)

I’m also excited that the next weekly will include my implementation of the SPDY protocol — Google’s improvement to the HTTP protocol — as one of the standard packages! The Go development team is exceptionally friendly and open to contributions, and I must say, this is the most fun open source project I’ve worked on. If you want a fun open source project to help out with, start hanging out on the go-nuts mailing list.

Congratulations to Russ and everyone involved on a successful release!

Posted at
Permalink

Texture Mapping!

Posted at by Ross Light
Tree texture-mapped onto cube

It’s been a while since I’ve posted because of some events that required my attention, but I’m back with yet another goray rendered image! A lot has changed in the rendering infrastructure (refactoring and such), and now it supports spot lights and texture mapping! Texture mapping and Blender export have not been merged to the trunk yet, but they’re coming soon.

The other exciting news is, if you’re bold enough, you can play with goray, too! Check out the Sourceforge project!

Posted at
Permalink

Currently expanding the FIRST scouting project to generate PDFs with barcode IDs. Data entry will be epic.

Posted at
Permalink

After fixing my ray/triangle intersection code, I’ve moved to working on the material system for my raytracer. I’m starting on the simple material first: Lambert shading with recursive raytracing for reflection.

For those who don’t know that much about raytracing, the Wikipedia article is surprisingly informative. The cool part is that the same code that you use to compute the normal scene just gets called recursively to create the reflection. It also makes physics class fun when you can readily see that just by applying the Law of Reflection, you can make cool effects. Next stop, Snell’s law!

Posted at
Permalink

When you have a bug in an enterprise program, money is lost. When you have a bug in a graphics program, art is created.

I’ve been working on rewriting parts of the raytracer I wrote in Go in x86-64 assembly. The test model I’ve been using is Suzanne, one of the primitives in Blender. Needless to say, something in the assembly code is not right, but it’s the first actual image the raytracer has spit back since I removed the pure Go implementation of the ray/triangle intersection code.

Posted at
Permalink

FIRST Experiences with MongoDB

Posted at by Ross Light

I’m currently working on a FIRST scouting data management project in my spare time. The requirements I’m working with are fairly well-defined, but the statistics collected change from year to year. I want to be able to use the same software year after year and keep historical data intact. I’m using Django for the front-end, but instead of using a traditional SQL database, I’m experimenting with using MongoDB, one of those new, fancy No-SQL databases.

Read more…

I used this code in some systems programming work. It actually does something useful.

Posted at
Permalink

Autocomplete Considered Harmful

Posted at by Ross Light

Stop using autocomplete. People no longer think about what they say, what they type, and how they spell it. Especially for programming, autocomplete is a sin.

Read more…

Unix Command Line of the Day

Posted at by Ross Light

I wanted to paste a text file with my homework assignment into Gmail. Unfortunately, said text file was on the school server and I really didn’t want to permanently download the thing to my hard drive. So this is what I did:

scp $vogon:~/graph.txt /dev/stdout | pbcopy

This connects to the school server, retrieves graph.txt from my home directory, and copies it to standard out. I then piped the text into pbcopy, which is the Mac utility for manipulating the clipboard. After that, I switched to Gmail and did a paste.

Unix/bash is cool!

Posted at
Permalink

I find this rather amusing. If you write Hello, World in Python in the sand, does that make it sand-script? :P

Posted at
Permalink
Now I’ve got an 0xE to Valgrind.
— Me (after finally getting results from a black box test for an assignment)
Posted at
Permalink
$ grep -r CPU_FAMILY /Volumes/BOOTCAMP/WindRiver/vxworks-6.3/target/h/

Some days, you just have to use grep.

Posted at
Permalink

Patent Absurdity

Posted at by Ross Light

This is a 30-minute documentary about the patent process and how it applies to software. It’s a remarkably complicated and important issue, and it does a good job of explaining it understandable terms. Even if you’re not into computer science, this is an important issue that affects many aspects of technological progress.

Posted at
Permalink

Sass Stylesheets

Posted at by Ross Light

I just started using this a couple days ago. It’s essentially a CSS preprocessor that allows you to use variables, arithmetic, functions, and true inclusion. It’s also very good at compressing CSS for you. Zombie Zen is now using it; it shortened my development time significantly.

Posted at
Permalink

vi: I Love You.

Posted at by Ross Light

I’m reading a book about the vi editor right now. For those less UNIX-inclined, vi is a command-line based text editor for UNIX systems like Linux, FreeBSD, and Mac OS X. It’s installed on basically every system (in some flavor or another. I prefer strawberry Vim, but I’ve been told the cinnamon variety deserves a second glance) and is remarkably easy to use.

It’s also more powerful than I realized.

The problem is that you need to find a good guide to vi. Its built-in help is wonderful reference, but it can’t get you up and running. For years, I’ve been bound to only a fraction of the commands. No more. Why would I use anything else?

[Cue angry Emacs users]

If my readers are so inclined, I may start posting some vi tips.

Posted at
Permalink

100th Post!

Posted at by Ross Light
A DOS prompt makes me sad. A bash prompt elicits unbridled joy in my heart.

The above quote came out of a rather interesting discussion in my early morning programming lecture.

And as stated above, this is my 100th post! I’d like to thank my kind readers and followers for making Zombie Zen a wonderful experience. To celebrate the occasion, I thought I’d send along a picture!

100th post!

HOPE YOU LIKE FINGER CONTORTIONS. :P

Posted at
Permalink

After spending the entire day on it, it’s amazing how joyful it is to get a program to accept keyboard input.

Posted at
Permalink

Using Go

Posted at by Ross Light

angelsfancrc:

zombiezen:

After initially posting about Google’s programming language, and working with it for about three weeks, I think I’m ready to give it a review.

Just a quick overview of Go: type names on the right side, language-level support for concurrent processing, and objects are interface-based. If you’re interested, go check out http://golang.org/.

First of all, I applaud Google for coming up with a language that actually brings something new to the table, instead of the usual “You use this syntax to do the exact same thing you could have done more cleanly in Python!” Go has actually helped me make programs that are more multiprocessing-aware, and that’s something that’s pretty amazing.

I think my main problem with it right now is that the language is fluctuating so much that it’s hard to keep up with. I’ve met some unstable code bases before, but this breaks my code every three hours if I stay on the bleeding edge. Heaven forbid if I don’t update for three days, I have to recompile everything. It’s just ridiculous.

Yes, I know that Go is weeks old. I know that there isn’t an infrastructure for formalizing language changes yet. I further know that the poor developers are getting beaned over the head by trolls on the Google group who just want to whine about how different the language is, without even coding in it in some cases. Darn you, Internet trolls. I like Go’s potential, but I think I’m going to wait a month before trying to do any serious work in it yet.

Definitely wait. I havent been keeping up with it, but from your description I imagine they prematurely released any compiler for the language, if this is not some sort of Alpha or Beta at all.

You’re right, it is my own fault. I broke Rule #42 of Code-land: Any code that doesn’t have a version number is trouble. :)

Posted at
Permalink

Using Go

Posted at by Ross Light

After initially posting about Google’s programming language, and working with it for about three weeks, I think I’m ready to give it a review.

Read more…
Posted at
Permalink

Go: A New Programming Language

Posted at by Ross Light

Dude. Google built a programming language.

Posted at
Permalink

I find it ironic (Part 8)...

Posted at by Ross Light

I find it ironic that my prototype solver (written in Python) is less buggy than my production solver (written in Java).

Posted at
Permalink

Snow Leopard Install

Posted at by Ross Light

I’m finally installing Snow Leopard on my family’s Mac Pro. I’ll keep you posted.

Posted at
Permalink

Movie Golf

Posted at by Ross Light

This is my trivia game that I implemented. I still think this is going to catch on soon.

Posted at
Permalink