codefork.com

A few weeks ago, I switched from Ubuntu’s default Unity desktop to Xfce.

Why? Because hitting Alt-Tab to cycle through windows takes about a 1/4 sec for Unity to register. I cycle through windows literally hundreds of times a day. And that small but perceptible pause was driving me completely bonkers. Mind you, this is on my brand spanking new Core i5 laptop. Unacceptable!

What good are all the cute visual effects, glowing icons, and retracting launcher bars if Unity can’t smoothly perform the ONE function that defines what a desktop environment is supposed to be for: managing my application windows?!

In Xfce, Alt-Tab responds immediately.

There is also the small plus that the laptop battery seems to last a bit longer running Xfce than Unity. It’s not perfect, of course. Switching to and from an external display causes some issues. And I’d like a better volume control widget in the panel. But to me, these are minor annoyances.

The other day, I discovered this wonderful video from 2009. It’s an informal chat/whiteboard session with Rich Hickey, the creator of Clojure, on Microsoft’s Channel 9.

Watching it really helped solidify in my brain various concepts around Lisp and Clojure that I’ve been trying to learn recently in my spare time. I didn’t grasp every tangential reference to various debates in the world of languages and compilers (and there are many of them in this video!) but it’s still worth watching for the interested layperson who’s trying to delve into the world of functional programming.

The best part, for me, is Rich’s explanation of how data structures in Clojure are immutable yet very fast when it comes to copying. This seems counterintuitive, since, on the face of it, you would think that it’s very slow to have to copy whole data structures to make changes. The secret is that it doesn’t do a deep copy. The underlying implementation of lists and sequences uses shallow trees: “copies” of data only need to clone a short branch of the tree and reference a new root node, which also keeps the old data structure intact. The discussion is wonderfully elegant.

Having used only imperative languages my entire life, this recent foray into Lisps and functional programming has been really challenging and eye-opening. Putting aside its alleged strengths and advantages, I find the mathematical flavor of this style of coding very appealing. I don’t have a mathematically-oriented brain, but at the same time, oddly enough, something about Lisps seems truer to what I intuit the essence of computing to be. If I were formally trained in computational theory, I’m sure I’d have some fancy jargon to name this quality I’m trying to describe. As it stands, I can only say it feels close to the machine.

It’s that time when bloggers wax introspective about the past year. For me, the major personal revelation in 2011 was re-discovering something very old, and putting it to new use. For me, 2011 was the year of the Emacs IDE.

I’ve been using Emacs, on and off, for close to a decade now. What’s changed is that, in the past few months, I’ve been writing extensions for it. It started with a simple desire to better navigate files in a complex directory hierarchy that followed specific and somewhat convoluted conventions. At first, learning Emacs Lisp was simply a means to an end, but I ended up liking it so much that I started exploring Common Lisp (and more recently, Clojure, since I’ve worked with Java in the past).

What started as a small task has become a larger project of turning Emacs into an IDE.

To understand this, one needs to know some context about the system I work with. We developers edit tons of XML files and files in other text formats, which all drive our proprietary web application product. We have many command line tools that manipulate these files in various ways; the system was originally designed by folks who followed the UNIX philosophy of building orthogonal tools and chaining them together.

There are pros and cons to this system; for reasons I won’t get into, I don’t love it, but it’s what we work with right now. When I started the job, the vast majority of the developers used screen, vi, and the shell prompt. Typical workflows that involved working with only a few files could be extremely hard to keep track of, and usually required a lot of copying and pasting between screen and/or ssh sessions. Few people seemed to mind, but I found the workflow to contain too much extraneous cognitive load, and the state of the tools made development very prone to error.

Gradually, I’ve been integrating our tools into Emacs. Sometimes that simply means binding a key combination to running a diagnostics program and storing the output in a buffer. Sometimes it means accumulating that output for history’s sake. Sometimes it means parsing program output, processing it in Emacs Lisp, and inserting a result into the current buffer. Sometimes it means running external programs, even GUI applications, and tweaking them a bit to tell Emacs to open specific files you want to look at.

The productivity gains have been amazing. This is no reason to brag: managing several screen sessions with different vi and shell instances wasn’t exactly hard to improve upon. But Emacs made it fairly painless. Emacs Lisp has proved to be wonderful “glue” for integrating existing tools in our environment.

Writing tools that enable you to do other job tasks better is a really interesting experience; I’ve never done it to such an extensive degree. So far, one other person in my group is using this Emacs IDE, and she has been happy with how much it facilitates her work. Others who swing by my desk for something often watch me work for a moment, and ask, “how did you do that?! that’s not vi, is it?”

Getting more people to switch over means convincing them that the steep learning curve of Emacs is worth the gains of the integration I’ve done. I’m not sure how much that will happen, since a big part of it is cultural. But if there aren’t any more converts, I don’t really care. The best thing about this ongoing project is that I am the end user. The software I wrote is really for myself. It is something I use intensively every single day. And that makes it all the more gratifying.

I discovered the Solarized color palette about a month ago, and have been using it since. I’ve found it helpful for reducing eye strain.

Getting the palette colors to accurately display in terminal-based Emacs under Ubuntu 11.10 took a bit of work. There’s a lot of scattered info about this topic if you google around. I first tried to do it with as little modification to my system as possible (I’m a minimalist), but I found that I absolutely had to do these three things to get it working.

1) Set the color palette in gnome-terminal. No matter what I tried, I could not get Emacs colors to look right without this step, so it’s a crucial one. Don’t skip it.

Solarized comes with an Xdefaults file, but that didn’t work for me. This blog post had instructions for setting gnome-terminal colors using gconftool-2, and that worked very nicely.

I put the commands in a handy little shell script that allows you to choose either the light or dark version of the palette. You can get it here:

https://gist.github.com/1397104

2) Your TERM environment variable needs to be set to “xterm-256color.” The default “xterm” will cause the Solarized color theme to look completely crazy in emacs.

Add the following line to your ~/.bashrc line (if you already set TERM there or some elsewhere else, change it instead of adding this):

export TERM="xterm-256color"

3) Use this version of the Emacs color theme for Solarized:

https://github.com/sellout/emacs-color-theme-solarized

Note that unless you’re using Emacs 24 (pre-release source code tarballs are available here), you’ll also need the color-theme package if you don’t already have it installed.

Put the files in a directory. In your .emacs file, you’ll need to add the directory to your load-path; I also added a bit of conditional logic for how to load the color theme in Emacs 23 and 24, since I switch between them:

(add-to-list 'load-path "~/.emacs.d/emacs-color-theme-solarized")
(if
    (equal 0 (string-match "^24" emacs-version))
    ;; it's emacs24, so use built-in theme
    (require 'solarized-dark-theme)
  ;; it's NOT emacs24, so use color-theme
  (progn
    (require 'color-theme)
    (color-theme-initialize)
    (require 'color-theme-solarized)
    (color-theme-solarized-dark)))

That should do it.

For work projects, I often have to jump around a lot of XML files referenced in one another by name. The files follow a strict naming convention, but it’s one that looks like gibberish to human eyes. Typing them is difficult and prone to error; plus there are so many files in a given project that navigating a visual tree of the directory isn’t much easier.

So I wrote a small library for Emacs to find and open a file in the directory tree, if the cursor is over a string of text that matches a valid filename in the convention. I’m embarrassed to admit this took me two full days (a few other functions were in the library too). The only previous exposure to Emacs Lisp I had was tweaking my .emacs file, so I had to learn the language as well as hunt down the right functions to call for what I wanted. The work is paying off: it feels 10x’s easier to navigate project files now.

I experimented with trying to do this in Komodo Edit, a popular editor among coworkers, but it involved learning Mozilla’s XUL and Komodo’s own API, as well as writing javascript (yuck), so I abandoned that effort pretty quickly.

I didn’t think I would like Lisp, but the experience has been pretty fascinating, and I’m now making my way through the Practical Common Lisp book by Peter Seibel. It’s interesting learning a language that’s half a century old (!), and that’s influenced so many contemporary languages. One might think that there’s nothing there worth learning or revisiting, but that is so wrong. In particular, I’m trying to wrap my head around the power of Lisp macros and the way they allow you to create new syntactic abstractions. The idea of extending the very language itself, rather just adding new functions, is mind-boggling, to say the least. And, from what I understand, it remains fairly unique to Lisp in spite of the flood of new languages in the past few decades.

It’s disheartening not being able to find much info about who actually uses Lisp anymore, aside from hackers building modules for Emacs. Paul Graham has a cool essay, “Beating the Averages”, about using Lisp to build online store software that Yahoo eventually acquired. And ITA software, which makes an airfare search engine that powers the entire travel industry, uses Lisp. But aside from these bits of info, there isn’t much out there in the way of Lisp “success stories.”

It takes quite a bit of effort to decipher my work history. I’ve worked on enough diverse software projects that I’m not a “junior” programmer; but because of the time I spent in academia doing non-tech related studies, I don’t have a substantial enough career history to justify a “senior” status. Recruiters and prospective employers usually lack the patience to make sense of it all. Though my core software development skills are solid, I’m regarded as a misfit.

Three months ago, I landed a new full-time job. The company is full of highly intelligent oddballs with convoluted, non-linear professional histories—so I feel like I fit right in. I’m liking it a lot so far, and I feel lucky to have work in such an awful economy.

It’s precisely because of my weird background that I often forget exactly how much I know, and what I’m capable of. It occurred to me recently that I think of programmers as tradesmen. Most recent college grads with CS degrees are poorly prepared for real world software development. Genuine knowledge comes from hands-on experiences as “apprentices” to seasoned programmers. If you are lucky to work with good people, what you learn is not simply a particular programming language or technology, but a paradigm of core principles that will stay with you for life.

I’ve been lucky to work under some great “masters” of the trade. More than a decade after my first “real” programming job, for example, I still use the fundamentals I learned from coworkers on that project: proper object oriented design and modeling, what abstract data types really are and how they’re useful, the extreme importance of extensibility and maintainability in an “agile” world. These things have become such a part of me that I take them for granted.

When I meet or talk to other programmers, I listen carefully to how they talk about things, what concerns they raise, how they approach problems. These are what distinguish the tradesmen-programmers from those whose aim is just to “get it to work,” and who inevitably run into all sorts of problems because of that mindset. Tradesmen-programmers have to balance the practical and theoretical aspects of software design in order to create something of quality, in all measures of that word. And this is something that can only be learned through dedication, experience, and open-mindedness, not a degree, certificate, or book.

I’ve been doing a little work with Drupal for a personal project. Since I can code and have the time to work with it, I chose to go with the latest and greatest version, instead of the older stable release. Poking around in the innards of Drupal has been an interesting experience, to say the least.

I wish Drupal’s repositories were hosted on github. Sometimes module maintainers can’t keep up with all the patches submitted (I don’t blame them, it’s a ton of work), so fixes are very slow to make it into the source base. github would allow people to fork and send their patches back upstream, instead of having to hunt around on the Drupal bug reporting system.

I’ve been eager to get the comment_notify module working better for Drupal 7. I found fixes for some known issues, and also created, tested, and submitted patches of my own. I decided to make available my own fork of the module on github, for those who want to grab a patched version for their own use:

https://github.com/codeforkjeff/comment_notify

It’s meant to be a temporary solution until the fixes make it into the official module, but it’s better than nothing!

I’ve taken a hiatus from programming work the last few months in order to do some other things, both paying and non-paying. I did some teaching, spent a lot of time at the bicycle co-op, published some articles in a local periodical, and started a few side projects. It has felt very gratifying and healthy.

My erratic life nowadays makes me think back to the starry-eyed aspirations to be a professional programmer that I had, say, ten years ago. I have a much more casual relationship to computers these days. Gone is the anxiety of having to prove, both to myself and to the world (including employers), that I am skilled and capable. I don’t care much anymore for the company of obsessive coders who enjoy religious debates about languages, tools, and best practices. They don’t seem to realize that in their pursuit of mastery and professionalism, technology is actually controlling them, and not the other way around.

Technology has become truly “technical” for me, in the fundamental sense of being a means to an end. I’ve been writing and tweaking Javascript and PHP recently. I don’t like them much, but they are decent tools for the specific tasks I am trying to accomplish. And so it goes. Getting back into code has been a Zen-like experience: when I am doing it, I am fully engaged in all the particularities. But when I’m finished with it, that’s it. Programming, and software more generally, is instrumental. I dive into it; then I come out. And there is nothing more to it than that.

It is silly to fetishize a hammer when you should be focusing on building the house, but that’s exactly what a lot of the culture of coding often feels like on blogs, message forums, mailing lists, conference agendas, etc. How silly and ridiculous. One should, of course, be thoughtful about tools, when and how to best use them, and how they can be improved. I do appreciate and value that. But at the end of the day, code is simply code. There is just so much more to life. Remembering that helps us keep sight of the things that code is for in the first place.

The official Apple specs say that the 2.0Ghz Macbook can take up to 2 GB of memory. There’s a bit of information on the web–like this forum posting, for example–that says you can go up to 3 GB. The system board can address slightly more than 3 GB, so the 2 GB limit is reportedly lower than what the hardware is capable of. By chance, I noticed a 2 GB chip for a reasonable $40 on my local craigslist, so I decided to see for myself whether the stories were true.

It seems they are! I’ve been running the computer for a little over a week now with two chips: a 1 GB module, which had been in there before, and the new 2 GB module. It’s been put through its paces: on a given work day, I run Eclipse, Firefox, Thunderbird, Colloquy (an irc client), Adium (instant messaging client), Skype (a VoIP client), a java application server in development mode, a mysql server, and emacs, all at once.

The swap size has still been high, typically around ~500 MB, but there are no longer the delays that I used to experience with 2 GB of memory when I had a lot of apps open and switched between them. So far, there have been no problems with stability.

Here’s a screenshot from System Profiler:

Note that Apple released two different models with the 2.0 Ghz Core 2 Duo processor. You can find your machine’s model in System Profiler on the Hardware Overview screen. The 2 GB official limit applies to “Macbook2,1.” The later version, “Macbook3,1″ can take up to 4 GB.

With the additional memory, this machine will hopefully last me another two years.

The DSL at the house has been really flakey the past 3 days. The line seems to periodically drop and I also noticed that the voice line had a lot of static.

I suspected a line problem so I called Earthlink support last night to try to get it straightened out. Surprisingly, the line tested good up until the point where it came into the house. But the diagnostics on the DSL modem were showing low noise margin (signal-to-noise ratio) and high attenuation (degradation), so there was definitely a problem somewhere. The trouble had to be inside the house: either the DSL modem or some wiring somewhere had gone bad or both.

I tried changing out some cables and tweaking the way the phone and fax (my housemate runs her own business) were all hooked up into the line. That seems to have fixed the problem. The line’s been steady for almost 24 hours now. I’ve been watching the diagnostics and researching what the numbers mean, and they seem to be within acceptable-to-good ranges. So far so good.

It was annoying to have to load the web interface on the ZyXEL P-600 Series modem (it’s a P-660R-ELNK) and continually refresh the page to watch the diagnostics. Plus I had to remember if any of the numbers changed and by how much. So I whipped up a little python script to fetch the data from the web interface and do some logging.

There were some interesting peculiarities in the ZyXEL web interface. After authenticating with a password, only that computer’s IP can use the interface, apparently; requests from other hosts get locked out until a few minutes of inactivity. Also, the way the interface refreshed the diagnostics page was a bit odd: it used a form submission to set some state that would cause another page to update its contents.

The script output looks like this:

Sun Jan 17 19:15:37 2010 noise = 16 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:16:37 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:17:38 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:18:38 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:19:38 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:20:39 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:21:39 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:22:39 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:23:39 2010 noise = 18 (good) outputPower = 11 attenuation = 31 (very good)
Sun Jan 17 19:24:39 2010 noise = 17 (good) outputPower = 11 attenuation = 31 (very good)

A copy of the script is available here. You may need to do some tweaking to get it to work with your setup. It works with python 2.5 and 2.6.