Programming exercises can be just as addictive as some forms of well-marketed physical exercise. I have to actively resist writing solutions in Lisp to every silly problem I stumble upon online. But this morning, I came across this simple one (a blog post from 2007, but that happens on Hacker News) and, for some reason, couldn’t pass it up:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.

It’s so silly that I guess some folks, like the blogger linked above, have turned it into a problem of “what’s the most creative/strange way I can think of to do this?” Which is interesting and fun in its own right.

What I don’t understand is the attitude expressed in the original fizzbuzz blog post, “Using FizzBuzz to Find Developers who Grok Coding”. Does it really matter if it takes someone 5 or 20 minutes to write a solution? (FWIW, I took about 10 mins). A shorter time with such a goofy example means someone “groks” coding more than another? Ridiculous.

These types of puzzles are amusing precisely because a lot of real-world development (which is what the vast majority of people who code for a living do) is unfortunately pretty rote, and doesn’t require a lot of algorithmic thinking. Code monkeys learn to use libraries and frameworks, and spend a lot of time trying to use them correctly and cleanly, in order to implement straightforward requirements. So these exercises are a shift in mindset. In a tiny way, they put you back on the path of engineering. It’s why people find them interesting to do. It’s why I’m reading the Structure and Interpretation of Computer Programs.

Taking a few more minutes might mean you’re not accustomed to solving such problems every minute of your working life. But that’s not at all the same as gauging whether someone understands coding or not. That kind of measurement is naive at best.

I’ve been using Emacs, on and off, for close to a decade now. What’s changed is that, in the past few months, I’ve been writing extensions for it. It started with a simple desire to better navigate files in a complex directory hierarchy that followed specific and somewhat convoluted conventions. At first, learning Emacs Lisp was simply a means to an end, but I ended up liking it so much that I started exploring Common Lisp (and more recently, Clojure, since I’ve worked with Java in the past).

What started as a small task has become a larger project of turning Emacs into an IDE.

To understand this, one needs to know some context about the system I work with. We developers edit tons of XML files and files in other text formats, which all drive our proprietary web application product. We have many command line tools that manipulate these files in various ways; the system was originally designed by folks who followed the UNIX philosophy of building orthogonal tools and chaining them together.

There are pros and cons to this system; for reasons I won’t get into, I don’t love it, but it’s what we work with right now. When I started the job, the vast majority of the developers used screen, vi, and the shell prompt. Typical workflows that involved working with only a few files could be extremely hard to keep track of, and usually required a lot of copying and pasting between screen and/or ssh sessions. Few people seemed to mind, but I found the workflow to contain too much extraneous cognitive load, and the state of the tools made development very prone to error.

Gradually, I’ve been integrating our tools into Emacs. Sometimes that simply means binding a key combination to running a diagnostics program and storing the output in a buffer. Sometimes it means accumulating that output for history’s sake. Sometimes it means parsing program output, processing it in Emacs Lisp, and inserting a result into the current buffer. Sometimes it means running external programs, even GUI applications, and tweaking them a bit to tell Emacs to open specific files you want to look at.

The productivity gains have been amazing. This is no reason to brag: managing several screen sessions with different vi and shell instances wasn’t exactly hard to improve upon. But Emacs made it fairly painless. Emacs Lisp has proved to be wonderful “glue” for integrating existing tools in our environment.

Writing tools that enable you to do other job tasks better is a really interesting experience; I’ve never done it to such an extensive degree. So far, one other person in my group is using this Emacs IDE, and she has been happy with how much it facilitates her work. Others who swing by my desk for something often watch me work for a moment, and ask, “how did you do that?! that’s not vi, is it?”

Getting more people to switch over means convincing them that the steep learning curve of Emacs is worth the gains of the integration I’ve done. I’m not sure how much that will happen, since a big part of it is cultural. But if there aren’t any more converts, I don’t really care. The best thing about this ongoing project is that I am the end user. The software I wrote is really for myself. It is something I use intensively every single day. And that makes it all the more gratifying.

So I wrote a small library for Emacs to find and open a file in the directory tree, if the cursor is over a string of text that matches a valid filename in the convention. I’m embarrassed to admit this took me two full days (a few other functions were in the library too). The only previous exposure to Emacs Lisp I had was tweaking my .emacs file, so I had to learn the language as well as hunt down the right functions to call for what I wanted. The work is paying off: it feels 10x’s easier to navigate project files now.

I experimented with trying to do this in Komodo Edit, a popular editor among coworkers, but it involved learning Mozilla’s XUL and Komodo’s own API, as well as writing javascript (yuck), so I abandoned that effort pretty quickly.

I didn’t think I would like Lisp, but the experience has been pretty fascinating, and I’m now making my way through the Practical Common Lisp book by Peter Seibel. It’s interesting learning a language that’s half a century old (!), and that’s influenced so many contemporary languages. One might think that there’s nothing there worth learning or revisiting, but that is so wrong. In particular, I’m trying to wrap my head around the power of Lisp macros and the way they allow you to create new syntactic abstractions. The idea of extending the very language itself, rather just adding new functions, is mind-boggling, to say the least. And, from what I understand, it remains fairly unique to Lisp in spite of the flood of new languages in the past few decades.

It’s disheartening not being able to find much info about who actually uses Lisp anymore, aside from hackers building modules for Emacs. Paul Graham has a cool essay, “Beating the Averages”, about using Lisp to build online store software that Yahoo eventually acquired. And ITA software, which makes an airfare search engine that powers the entire travel industry, uses Lisp. But aside from these bits of info, there isn’t much out there in the way of Lisp “success stories.”

Three months ago, I landed a new full-time job. The company is full of highly intelligent oddballs with convoluted, non-linear professional histories—so I feel like I fit right in. I’m liking it a lot so far, and I feel lucky to have work in such an awful economy.

It’s precisely because of my weird background that I often forget exactly how much I know, and what I’m capable of. It occurred to me recently that I think of programmers as tradesmen. Most recent college grads with CS degrees are poorly prepared for real world software development. Genuine knowledge comes from hands-on experiences as “apprentices” to seasoned programmers. If you are lucky to work with good people, what you learn is not simply a particular programming language or technology, but a paradigm of core principles that will stay with you for life.

I’ve been lucky to work under some great “masters” of the trade. More than a decade after my first “real” programming job, for example, I still use the fundamentals I learned from coworkers on that project: proper object oriented design and modeling, what abstract data types really are and how they’re useful, the extreme importance of extensibility and maintainability in an “agile” world. These things have become such a part of me that I take them for granted.

When I meet or talk to other programmers, I listen carefully to how they talk about things, what concerns they raise, how they approach problems. These are what distinguish the tradesmen-programmers from those whose aim is just to “get it to work,” and who inevitably run into all sorts of problems because of that mindset. Tradesmen-programmers have to balance the practical and theoretical aspects of software design in order to create something of quality, in all measures of that word. And this is something that can only be learned through dedication, experience, and open-mindedness, not a degree, certificate, or book.

My erratic life nowadays makes me think back to the starry-eyed aspirations to be a professional programmer that I had, say, ten years ago. I have a much more casual relationship to computers these days. Gone is the anxiety of having to prove, both to myself and to the world (including employers), that I am skilled and capable. I don’t care much anymore for the company of obsessive coders who enjoy religious debates about languages, tools, and best practices. They don’t seem to realize that in their pursuit of mastery and professionalism, technology is actually controlling them, and not the other way around.

Technology has become truly “technical” for me, in the fundamental sense of being a means to an end. I’ve been writing and tweaking Javascript and PHP recently. I don’t like them much, but they are decent tools for the specific tasks I am trying to accomplish. And so it goes. Getting back into code has been a Zen-like experience: when I am doing it, I am fully engaged in all the particularities. But when I’m finished with it, that’s it. Programming, and software more generally, is instrumental. I dive into it; then I come out. And there is nothing more to it than that.

It is silly to fetishize a hammer when you should be focusing on building the house, but that’s exactly what a lot of the culture of coding often feels like on blogs, message forums, mailing lists, conference agendas, etc. How silly and ridiculous. One should, of course, be thoughtful about tools, when and how to best use them, and how they can be improved. I do appreciate and value that. But at the end of the day, code is simply code. There is just so much more to life. Remembering that helps us keep sight of the things that code is for in the first place.

It seems they are! I’ve been running the computer for a little over a week now with two chips: a 1 GB module, which had been in there before, and the new 2 GB module. It’s been put through its paces: on a given work day, I run Eclipse, Firefox, Thunderbird, Colloquy (an irc client), Adium (instant messaging client), Skype (a VoIP client), a java application server in development mode, a mysql server, and emacs, all at once.

The swap size has still been high, typically around ~500 MB, but there are no longer the delays that I used to experience with 2 GB of memory when I had a lot of apps open and switched between them. So far, there have been no problems with stability.

Here’s a screenshot from System Profiler:

Note that Apple released two different models with the 2.0 Ghz Core 2 Duo processor. You can find your machine’s model in System Profiler on the Hardware Overview screen. The 2 GB official limit applies to “Macbook2,1.” The later version, “Macbook3,1″ can take up to 4 GB.

With the additional memory, this machine will hopefully last me another two years.

It’s been nearly a year since I wrote about returning to grad school. So what’s happened? Well, I didn’t finish my dissertation. It felt nice to work on my research project again, but after a few months, I fell into the same depressive slump. I don’t want to go into it here, but it should suffice to say that, for me, the various burdens of academic life far outweigh its advantages. Though it would have been wonderful to get the psychological closure of finishing my degree before moving on to other things, that just hasn’t been possible. So “ABD” I will have to be….

There have been a lot of other changes in my life as well. I’ve moved to West Philadelphia, one of the most remarkable places I have ever lived. I am also writing code again to earn a living. It is a full-time position, and so far, it’s pretty great. Life is good, for now, anyway.

Working on software again has been sparking a lot of thoughts I haven’t had the time or energy to blog about, but hopefully that will change.

Working in a small business as the sole do-it-all technology person has its unique challenges. It can be very fulfilling to be the sole expert and “enabler,” if that turns you on. But the flip side is that management might not really understand or care that much about their technology. Is there a reasonable budget for what they’re trying to accomplish? Do they understand, at a high level, your projects and how they contribute to the mission? Are technology projects considered a burdensome mystery or something valuable and embraced by the company? Question the reasons why there’s only one tech guy/girl and whether that seems right.

Another thing to assess is whether you can deal with taking over the existing codebase. I’ve taken over other code before with success, retaining what was good and doing clean up as necessary. At this past gig, things looked reasonably tidy at a first glance, but as time progressed, I realized a ton of abstractions weren’t in place, and those that did exist didn’t make sense. Some refactoring might have been interesting to do, but this endeavor wasn’t valued when I proposed it as a project.

Lastly, I think it’s important to be wary of promises about the future. Even with the best of intentions, things change quickly at small businesses. The projects I was initially excited about got perpetually deferred for various reasons, and I found myself preoccupied with doing maintenance code fixes, making cosmetic tweaks, performing server administration, and providing support for third party software (which I really don’t like to do). The company needed these things done, so I did them with as much cheer as I could muster, hoping we’d eventually get to a place where some solid new development could occur (and I could sneak in some refactoring)—that’s what floats my boat. But it became to clear to me that wasn’t going to happen anytime soon.

So that’s that. It’s a shame it didn’t work out, especially since I actually liked everyone I worked with. At least it’s an amicable departure, and I hope to be involved in hiring a replacement who might be a better fit for their current needs than I am.

The new gig? Java. Been catching up on it, since it’s been a few years. Oh, it feels so nice to have package namespaces, real data types, full-featured APIs, and real object-orientedness again. Like coming home.

As I talked about this with friends who have had similar experiences, they observed that there’s often a common mistake made in small businesses: since every employee is precious, the organization tries to extract as much value as possible by encouraging everyone to contribute in as many ways as possible. It bills itself as a democracy, as an environment that genuinely listens to its employees.

The problem, of course, is that when people aren’t in sync about the mission, or about specific project goals, you end up with a frustrating mess of conflicting directions. The business (somewhat desperately?) tries to latch onto everything at once, and there’s a lack of decisiveness in moving forward. Projects get fragmented. People believe they’re collaborating but they’re actually not. Confusion ensues. Little gets done.

Leadership is even more key in small businesses than in larger ones, I’d wager. Because if you have few resources, you need to choose your projects very carefully, dedicate resources accordingly, and make absolutely sure they go to completion quickly. There’s not much room for failure. But instead, small businesses seem prone to being wishy-washy. The insidious and tragic aspect of this is that when projects move at a snail’s pace or even fail, the accountability falls back on the individuals. Because management failed to lead, it can remain blame-free. It’s a formula for endless frustration.

In this unfortunate strategy, the small business is structured like the subconscious and its conflicts. Its desire is vague and repressed, and though seemingly absent, it’s actually very much the motive force for its existence. Like a human being, the small business can’t decide upon its identity, which remains in flux. And just like a human being, this repressed desire can lash out violently when it isn’t fulfilled.

The piece makes his usual argument that copyright law stifles innovation in the age of new media. Most striking to me, though, was the part where he uses the phrase “amateur culture.” He explains, “…I don’t mean amateurish culture, I mean culture where people produce for the love of what they’re doing and not for the money.” He uses the term to describe the activity of “kids” (?) creating their own remixes from existing media.

I can remember another amateur culture that’s now largely disappeared. Back in my teens, modem-based bulletin board systems (BBSes) fostered a rich “read-write” culture for amateur programmers. Most of us did not work in technology; after all, the commercial Internet hadn’t been born yet, so the computing industry was much smaller and more obscure. A career as a programmer seemed like a mysterious and rarefied thing to me back then. The coders you met on BBSes were often people who simply liked to do programming in their spare time.

These systems allowed us to circulate public domain source code for fun games and useful applications written in BASIC, Pascal, C, even assembler. We hacked on existing code to get it to do what we wanted, trying to figure out ways to push the limits of our little 8086 processors and 640K of RAM. We mingled regardless of our level of knowledge, beginners and experts alike. We had friendly user meetings in diners in Brooklyn and Manhattan (I lived in NY at the time), where we chatted about home-grown upgrades and discussed how to link up to the nation-wide discussion networks that existed then.

It was amateur culture at its best: lots of exchange, circulation, and cooperation happened all the time. But it was definitely not amateurish. Many were extremely capable and knowledgeable coders.

Today, there are still people who code just because they enjoy it, but the amateur culture and its community hardly exist anymore. Beginners on web forums are more interested in what they need to know in order to land a job, rather than in coding itself. Even open source projects tend to be dominated by career professionals; read any public mailing list and you’ll see how unhelpful they often are to amateurs who want to get involved. One reason I like python is that the project makes a genuine effort to connect to the sensibilities of amateurs. But even its forums are littered with snarky individuals.

All of this is largely due, I think, to the ideology of professionalism, which convinces us that having a stable career is the pinnacle of achievement. It damagingly equates amateurs with dilettantes. That’s why one of the first things we ask in this country when meeting a stranger is, “So what do you do?” By which we really mean, “Tell me what you do for a living so I can know who you are and whether you’re worth talking to.”

In 2008, I resolve to be more wary of this ideology and its negative effects. I want to embrace being an amateur in the various things that I do. I want to think less about careers and focus more on how to best spend my time doing what’s important to me. And I want to find more amateurs to hang out with as well.