Zombie Zen

Posts tagged "Programming"

Canceling I/O in Go Cap'n Proto

Posted on

This report details an experience I had while writing an RPC system in Go. While Go’s standard I/O libraries make a great many things simple, I found cancellation to be more complex than I would have liked. Parts of this situation have improved in the last couple of Go releases (as I have noted below). I hope this positive trend continues in a way that allows the Go ecosystem to easily propagate cancellation, deadlines, and request values. My intent in this report — as well as the proposal I created back in May 2017 — is to give background and feedback to inform future design decisions. Suggestions for solutions welcome!

(Thanks to Ian Lance Taylor, Damien Neil, Cassandra Salisbury, and Andrew Bonventre for reviewing this report for accuracy and clarity.)

Read more…

cardcpx: a Go/AngularJS project

Posted on

I’ve posted a project I’ve been hacking on to my Bitbucket account: cardcpx. This has nothing to do with my day job, it’s just something I did to help out on a film shoot (which also had nothing to do with my day job).

From the README:

cardcpx is a specialized UI for copying files from a camera card to 1+ replicas. The replica copies happen concurrently, so if you are copying N bytes to R replicas, the time is O(N) instead of O(N * R).

The interface also has includes simple scene/take ingestion, which is stored in a CSV transaction log. Selects will be copied first, so you can do a proofing check on a fast disk while your import finishes.

cardcpx supports a flat directory structure as well as the RED camera directory structure. It assumes that your clip names do not overlap. Attempting to copy the same file name will not overwrite data.

It’s still a bit rough around the edges, but I’m happy with how Go allowed me to compose a fairly complex a tool out of easy-to-understand pieces. This was my first excursion into writing a frontend with AngularJS, but I’m also similarly pleased about the design.

Everything’s open source. Feel free to dig in and use. I’d love to know if other people find this program useful.

Chromebook Hacking: Developer Mode

Posted on

This is the second part in my series about creating a low-cost programming environment with an HP Chromebook 11. See Part 1 of Chromebook Hacking for the overview. This blog post assumes you have some basic knowledge of using a Linux terminal.

Enable Developer Mode

Danger: This will wipe out your entire local storage! Back up everything!

Enabling developer mode on a Chromebook is the blessed way of “rooting” your Chromebook: it gives you root shell access. This comes with the cost of losing parts of ChromeOS that depend on hardware-backed security systems (namely Netflix. Sad days).

Enabling developer mode is reversible (see below), but going in either direction will wipe your Chromebook’s SSD. Please back up your files first.

Instructions are on the Chromium OS website. For the HP Chromebook 11, the short version is: hold Esc+Refresh+Power button and when you see the boot screen, tap Ctrl+D. Follow the prompts and wait for roughly 10 minutes for the operating system to reinstall.

On the boot screen, you need to press Ctrl+D every time you boot to skip the “Danger, you don’t have verified boot” screen, or you can wait for 30 seconds. You can alternatively press the spacebar to perform a factory reset and disable developer mode.

Go through the standard Chrome OS “create an account” process to get to the desktop.

Install Secure Shell and set up Crosh

This step isn’t strictly necessary, but it gives you a very slick terminal interface. If you decide not to do this, you can always open crosh with Ctrl+Alt+T.

Install Secure Shell from the Chrome Web Store. Right-click on the app’s icon in the shelf and choose “Open as Window”. This allows you to use shortcuts like Ctrl+W inside the terminal without Chrome intercepting them. Click the shelf icon to open a new terminal. Create a new profile with a random non-empty user name (I picked foo) and a host of >crosh.

Secure Shell app with user "foo" and host "crosh"

Harden your install

Open a bash shell by running shell at the crosh prompt. Inside your bash shell, run this command:

chromeos-setdevpasswd

This sets your Chromebook’s UNIX password for the chronos user — the UNIX user used for any logged in Chrome profile. Now that you’ve enabled developer mode, chronos has the ability to run sudo. Since all Chrome profiles use the same UNIX user account, you will want to open chrome://settings/accounts and whitelist the users that can access this machine. Disable Guest browsing and restrict sign-in to yourself.

Chrome settings with guest browsing disabled and sign-in restricted to Ross's account

Finally, you should prevent your Chromebook from booting anything other than ChromeOS by running the following from your bash prompt:

crossystem dev_boot_usb=0 dev_boot_signed_only=1

The first parameter (dev_boot_usb) disables booting from an external USB stick. The second parameter (dev_boot_signed_only) forbids booting to operating systems that are not signed by Google.

Wrapping Up

Now you have a minimal Linux environment with bash and vim. After poking around a bit, you will quickly notice that only Downloads and /usr/local are writable. In the next blog post, I will introduce Crouton: a less restrictive Linux environment for ChromeOS.

Chromebook Hacking

Posted on
HP Chromebook 11 showing a command prompt

In my spare time, I’ve been working on Go-SDL2 and Camlistore. I’m usually not sitting in front of my home desktop, so I want to be able to code anywhere. My battle-worn MacBook Pro could do the job, but it’s heavy, bulky, and aging.

To fix this, I bought the HP Chromebook 11. The display is nice, the price is great, and the keyboard is comfortable. It accepts a microUSB charge, so I don’t even have to worry about packing an extra charger. The bummer is development: good luck trying to run anything besides JSFiddle.

For a while, I was using SSH into my desktop to code. Most of the time this works. The problem was me: some days I would forget to turn on my desktop, other days I left it in Windows, and every day I gritted my teeth if I was coding graphics. I fantasized about a machine that booted quickly and had all the programs I needed.

Turns out, the answer was right in front of me: the Chromebook.

Recipe for Success!

  • Buy the Chromebook
  • Enable developer mode
  • Install Secure Shell and set up with Crosh
  • Harden your install
  • Download and run Crouton
  • Install X11 and xmonad
  • Profit!

Total Cost: $279 + half a day’s worth of work.

Over the next few days, I’ll go into more detail on these steps.

A night of fun hacks

Posted on

I discovered that some of my code was still living on my old laptop from high school. The laptop hadn’t been booted in about 4 years, so the clock battery had died, causing the filesystem checks to fail (this is Ubuntu 9.10, which cared about these things). I reset the clock manually, boot up the machine, and decide that to be on the safe (but marginally inefficient) side, I’ll copy my entire multi-GB home directory to my more beefy desktop and filter out what I don’t want there.

Sadly, this laptop came before 802.11n and is limited to 54 Mbps. I have an AirPort Express with only one local port, which is connected to my desktop, but I don’t want to disrupt the network setup. I realize I can chain the Ethernet from my old laptop (named metroid) to my new laptop (named roran), which has 802.11n to connect to my desktop (named nasuada).

It's a UNIX system. I know this.
ross@nasuada$ nc -l 8080 > mybackup.tar.bz2
ross@roran$ nc -l 8080 | nc nasuada.local 8080
ross@metroid$ tar -jcf - $HOME | nc roran.local 8080



I love Unix.

Why Go Does Not Need Generics

Posted on

(This developed from a thread with some other Cal Poly students discussing Go.)

One of the frequently asked questions about Go is why are there no generics in the language. It’s a fair point from the perspective of other OOP languages, but idiomatic Go code un-asks the question.

First, it is important to understand how interfaces work in Go. The gist is that interfaces use virtual method lookups, but retain the underlying type, so a conversion back to the original type is still type-safe. An empty interface (interface{}) works as a type-safe equivalent of C’s void*: you can put any type inside of an empty-interface variable, and then you can un-box it later.

The simplest way to define a linked list is through the empty interface: interface{}.

type Node struct {
    Value interface{}
    Next *Node
}

And indeed, this is how the container/list package package works, give or take some complexity. However, my argument is that this is the wrong way to do this. It requires an explicit boxing/un-boxing hit for every access, and this is where (I think) most of the call for generics comes from. There’s a much more idiomatic way to do this. My two case studies here are the sort package and the container/heap package from the standard library.

We start with sort, which defines an interface, sort.Interface:

package sort

func Sort(data Interface)

type Interface interface {
    // Len is the number of elements in the collection.
    Len() int
    // Less returns whether the element with index i should sort
    // before the element with index j.
    Less(i, j int) bool
    // Swap swaps the elements with indexes i and j.
    Swap(i, j int)
}

This may not strike you as a generic container, but that is precisely the point. You can – with a bit of boilerplate – create a type that wraps a typed slice that implements sort.Interface. Thus, the algorithm can work on any type and only works uses indices. This is much cheaper than boxing/unboxing on access.

The second example is container/heap:

package heap

func Init(h Interface)
func Pop(h Interface) interface{}
func Push(h Interface, x interface{})
func Remove(h Interface, i int) interface{}

type Interface interface {
    sort.Interface
    Push(x interface{}) // add x as element Len()
    Pop() interface{}   // remove and return element Len() - 1.
}

Admittedly, there is a bit more boxing/un-boxing going on here, but the idea is still similar. We now have a data structure that requires very few box/unbox operations (two per push/pop). It is important to note that this is still type-safe.

You could implement a binary tree structure similarly:

package tree

func Search(root *Node, val interface{}) *Node

type Node struct {
    Index       int
    Left, Right *Node
}

type Interface interface {
    Len() int
    Less(i, j int) bool
    Push(x interface{})
    Pop() interface{}
}

The Search function would push the val interface and use it for comparisons against the values stored in the tree, then pop it once finished. Admittedly, this isn’t the prettiest interface, but it works in a pinch. It works out to be only one box/un-box round-trip.

As we’ve seen, by “flipping” around the algorithms and separating them from the data structures using interfaces, we can still reap the benefits of generic containers from other languages.

I’m finishing up my game, “Urge to Merge”, for my Intro to Computer Graphics course. This is the first time I’ve worked with OpenGL shaders; I’m pleased with the results. These are screenshots from the game.

New Go Stable Release

Posted on
Go gopher

Yesterday, the Go programming language announced their second stable release, introducing language changes, better packages, and general speed-ups. I’m proud to say that I helped out in a small part by contributing a patch to the zlib package.

For those of you who haven’t heard me talk about Go, it’s a programming language that brings all of the pleasant features from scripting languages and mixes it with the simplicity of C. Its syntax and feel is unlike any other language, but after a week, it begins to feel like second nature, and it’s now one of the first languages I grab to solve a problem. I can’t say enough good things about it, and if you haven’t already, download it and try the tutorial. (And yes, it will feel weird at first. Give it a week if you’re coming in fresh.)

I’m also excited that the next weekly will include my implementation of the SPDY protocol — Google’s improvement to the HTTP protocol — as one of the standard packages! The Go development team is exceptionally friendly and open to contributions, and I must say, this is the most fun open source project I’ve worked on. If you want a fun open source project to help out with, start hanging out on the go-nuts mailing list.

Congratulations to Russ and everyone involved on a successful release!

Texture Mapping!

Posted on
Tree texture-mapped onto cube

It’s been a while since I’ve posted because of some events that required my attention, but I’m back with yet another goray rendered image! A lot has changed in the rendering infrastructure (refactoring and such), and now it supports spot lights and texture mapping! Texture mapping and Blender export have not been merged to the trunk yet, but they’re coming soon.

The other exciting news is, if you’re bold enough, you can play with goray, too! Check out the Sourceforge project!

After fixing my ray/triangle intersection code, I’ve moved to working on the material system for my raytracer. I’m starting on the simple material first: Lambert shading with recursive raytracing for reflection.

For those who don’t know that much about raytracing, the Wikipedia article is surprisingly informative. The cool part is that the same code that you use to compute the normal scene just gets called recursively to create the reflection. It also makes physics class fun when you can readily see that just by applying the Law of Reflection, you can make cool effects. Next stop, Snell’s law!

When you have a bug in an enterprise program, money is lost. When you have a bug in a graphics program, art is created.

I’ve been working on rewriting parts of the raytracer I wrote in Go in x86-64 assembly. The test model I’ve been using is Suzanne, one of the primitives in Blender. Needless to say, something in the assembly code is not right, but it’s the first actual image the raytracer has spit back since I removed the pure Go implementation of the ray/triangle intersection code.

FIRST Experiences with MongoDB

Posted on

I’m currently working on a FIRST scouting data management project in my spare time. The requirements I’m working with are fairly well-defined, but the statistics collected change from year to year. I want to be able to use the same software year after year and keep historical data intact. I’m using Django for the front-end, but instead of using a traditional SQL database, I’m experimenting with using MongoDB, one of those new, fancy No-SQL databases.

Read more…

Unix Command Line of the Day

Posted on

I wanted to paste a text file with my homework assignment into Gmail. Unfortunately, said text file was on the school server and I really didn’t want to permanently download the thing to my hard drive. So this is what I did:

scp $vogon:~/graph.txt /dev/stdout | pbcopy

This connects to the school server, retrieves graph.txt from my home directory, and copies it to standard out. I then piped the text into pbcopy, which is the Mac utility for manipulating the clipboard. After that, I switched to Gmail and did a paste.

Unix/bash is cool!

Patent Absurdity

Posted on

This is a 30-minute documentary about the patent process and how it applies to software. It’s a remarkably complicated and important issue, and it does a good job of explaining it understandable terms. Even if you’re not into computer science, this is an important issue that affects many aspects of technological progress.

Sass Stylesheets

Posted on

I just started using this a couple days ago. It’s essentially a CSS preprocessor that allows you to use variables, arithmetic, functions, and true inclusion. It’s also very good at compressing CSS for you. Zombie Zen is now using it; it shortened my development time significantly.

vi: I Love You.

Posted on

I’m reading a book about the vi editor right now. For those less UNIX-inclined, vi is a command-line based text editor for UNIX systems like Linux, FreeBSD, and Mac OS X. It’s installed on basically every system (in some flavor or another. I prefer strawberry Vim, but I’ve been told the cinnamon variety deserves a second glance) and is remarkably easy to use.

It’s also more powerful than I realized.

The problem is that you need to find a good guide to vi. Its built-in help is wonderful reference, but it can’t get you up and running. For years, I’ve been bound to only a fraction of the commands. No more. Why would I use anything else?

[Cue angry Emacs users]

If my readers are so inclined, I may start posting some vi tips.

100th Post!

Posted on
A DOS prompt makes me sad. A bash prompt elicits unbridled joy in my heart.

The above quote came out of a rather interesting discussion in my early morning programming lecture.

And as stated above, this is my 100th post! I’d like to thank my kind readers and followers for making Zombie Zen a wonderful experience. To celebrate the occasion, I thought I’d send along a picture!

100th post!

HOPE YOU LIKE FINGER CONTORTIONS. :P

Using Go

Posted on

angelsfancrc:

zombiezen:

After initially posting about Google’s programming language, and working with it for about three weeks, I think I’m ready to give it a review.

Just a quick overview of Go: type names on the right side, language-level support for concurrent processing, and objects are interface-based. If you’re interested, go check out http://golang.org/.

First of all, I applaud Google for coming up with a language that actually brings something new to the table, instead of the usual “You use this syntax to do the exact same thing you could have done more cleanly in Python!” Go has actually helped me make programs that are more multiprocessing-aware, and that’s something that’s pretty amazing.

I think my main problem with it right now is that the language is fluctuating so much that it’s hard to keep up with. I’ve met some unstable code bases before, but this breaks my code every three hours if I stay on the bleeding edge. Heaven forbid if I don’t update for three days, I have to recompile everything. It’s just ridiculous.

Yes, I know that Go is weeks old. I know that there isn’t an infrastructure for formalizing language changes yet. I further know that the poor developers are getting beaned over the head by trolls on the Google group who just want to whine about how different the language is, without even coding in it in some cases. Darn you, Internet trolls. I like Go’s potential, but I think I’m going to wait a month before trying to do any serious work in it yet.

Definitely wait. I havent been keeping up with it, but from your description I imagine they prematurely released any compiler for the language, if this is not some sort of Alpha or Beta at all.

You’re right, it is my own fault. I broke Rule #42 of Code-land: Any code that doesn’t have a version number is trouble. :)

I find it ironic (Part 8)...

Posted on

I find it ironic that my prototype solver (written in Python) is less buggy than my production solver (written in Java).