Statistical Alchemy

A famous man once said, that while searching for gold, the alchemists discovered treasures of greater value. Who said that? What did he mean by that? Who knows. Doesn’t matter. They are famous words by an eminent person that justify my latest, most unexpected adventure in programming. Which is the Illumetrics project.

The project began after a debate with another programmer, on whether or not Illumos was seeing a rise in new developers, that weren’t contributors before the fall of Sun — and what effect this would have on Illumos’s future. It wasn’t too long before I realized that neither one of us had a clue what were talking about.

Knowing that it was a capital mistake to theorize before one had data, we set out to gather some data on the vital signs of the Illumos ecosystem. The first stop was the git repositories which contain the entire development history of the Illumos kernel and peripheral software. The Illumos community is more than the kernel. It’s everything from the various kernel branches to the software that leverages Illumos to get stuff done — like FIFO, Manta, SDC, and many others.

So I’ve developed a framework to automatically fetch and update all of those repositories, and to convert their commit-logs into something more machine-friendly — JSON. The framework is far from complete. It likely doesn’t have all the relevant repositories, yet. It is still a Frankenstein’s monster — Node, Perl, R, Awk, and Sed code stitched together with KSH. We’re working on it.

The point is, the project is still in the prenatal stages, and there are many sweeping changes likely to come. The topic of monitoring the Illumos community’s vital signs, is interesting to myself and the team, but like the alchemists and explorers of old we hope to find treasure — we just aren’t sure where to find it or what form it will take.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s