Archive for 2008

My Perl Hating Days are Over…

Thursday, November 13th, 2008

… as are my coding days in general. Since September, I’ve returned to grad school.

The decision wasn’t one I made lightly. And with the economic slump, I often wonder if I’ve made a smart choice in taking a 400% pay cut, only to return to teaching and writing my dissertation. In good times, the job prospects for my field were poor; in bad times, they may be practically non-existent.

So why do it? I miss having the kind of intellectual stimulation centered upon social impact. Application programming is challenging and fun, but I no longer find it as fulfilling as I once did or as others do. Increasingly, it is a thankless job where the headaches tend to outweigh personal growth and learning. In the past year, the best gig I had involved smart architecturally-minded people and a strong project lead. And that was pretty nice, though I still felt something missing. The worst gig involved passive aggressive people who mostly feigned their knowledge. It’s interesting that since I left that job, my replacement (a “CTO” who asked for double the salary) did many of the things I recommended, including a complete overhaul of the codebase. Oh well, I guess they needed to hear it from someone other than me.

So this post is a good-bye of sorts. I may still blog occasionally here about technology when the mood strikes. And I have an idea for a project I’d like to do once my time frees up, one that is more about today’s “culture of technology” than about coding.

Thanks to everyone for reading.

Perl Makes You Cry Harder

Wednesday, August 20th, 2008

Slashdot is linking to an ONLamp.com article, Why Corporates Hate Perl. I think it’s interesting that while zealots exist for every language, you rarely encounter the kind of vehement hatred for a language the way you do for perl. Which probably just fuels the antagonism in both directions.

Many Slashdot comments point out that the requirements specific to corporate environments are rather idiosyncratic. Some stress the “right tool for the job” philosophy, which I totally agree with.
And here’s one that makes a wonderful characterization about what perl is and is not:

The problem is, Perl is just a programming language, not a conceptual system. Arguably it is the antithesis of a conceptual system. Many teams then create their own application frameworks atop it (e.g. Mason, POE), and it’s rare for these frameworks to be compatible since Perl offers so many variations in the construction of even standard programming artifacts like classes & objects.

In addition, the level of expression (i.e. TMTOWTDI) means in practice that highly varying programming styles occur throughout large, long-lived bodies of code.

As a result, significant Perl-based business applications tend to become hard-to-maintain hairballs of divergent style and subtly variegated concept.

The root cause: as I started with; the absence of a standard conceptual framework for Perl means that during the early phases of a project, it’s much harder to reason meaningfully about the eventual form of the system than it is with, say, Java or .NET where many of the design patterns are explicitly standardised.

I wouldn’t say that “Corporates Hate Perl”. It’s just the Perl as an application language doesn’t suit the formal design & architecture process we’re seeing increasingly as IT departments start to grow up and realise that they’re not the most important people in the company.

That doesn’t disqualify Perl from being a useful tool, and it’ll always have a place in data transformation, but it does mean that Perl isn’t going to be one of the general-purpose application programming languages of the future.

Bravo. I’d add that what the author identifies as a “problem” is also Perl’s strength. There’s more than one way to do it, so do it as you please. That definitely has allure for many programmers. As a project scales up, though, I think this does in fact become a detriment, and not only for corporate projects.

In response to someone who wrote, “chomp is not ambiguous. RTFM and stop crying,” here’s another awesome comment:

http://perldoc.perl.org/functions/chomp.html [perl.org]
This safer version of “chop” removes any trailing string that corresponds to the current value of $/ (also known as $INPUT_RECORD_SEPARATOR in the English module). It returns the total number of characters removed from all its arguments. It’s often used to remove the newline from the end of an input record when you’re worried that the final record may be missing its newline. When in paragraph mode ($/ = “” ), it removes all trailing newlines from the string. When in slurp mode ($/ = undef ) or fixed-length record mode ($/ is a reference to an integer or the like, see perlvar) chomp() won’t remove anything. If VARIABLE is omitted, it chomps $_ . Example:

If anything I’m crying harder after reading that.

Amen, brother.

Comparing Documentation Methods

Tuesday, August 19th, 2008

I’ve always thought Javadoc was one of the best features of Java. The Javadoc pages for the core API are invaluable for finding what I need very quickly. The utility can be run on any Java source files to generate a nice set of HTML pages that gives you a thousand-foot view of packages, classes, members, and method signatures. Nothing special or extra is required. Of course, you’ll often want to add comments and descriptions and that’s done by following commenting conventions that Javadoc can recognize and insert automatically into its HTML output, but you don’t need to do this for Javadoc to work.

Python’s docstring conventions are not quite as elegant, in my opinion, but they work just as well. Documentation is so much more important in a dynamic language like Python because unlike Javadoc, the pydoc utility can’t determine types. So if a parameter for a function or object method is “user,” one needs to know whether to pass in a User object, a username string, an integer id… or whether any of those will work.

Both Python’s docstring and Javadoc let you document as-you-go, eliminating or reducing the need for documentation as a separate task. If you change something, the documentation is right there for you to update.

Perl’s POD format isn’t nearly as convenient. The markup is oriented more towards layout and formatting rather than following the structure of the code. You can write section headers, indent, and list items in the documentation, but you don’t really attach them to subroutines or methods. Well, you can, sort of, with “=item” but each item must be nested inside other markup, and it feels kludgey and weird. The consequence is that the documentation feels really much separate from the code, even if it resides in the same file. It doesn’t encourage documentation as you go.

In the perl project I worked on, I wrote some POD comments in the very beginning but it fell by the wayside. I should have kept up with it, but it felt like an extra thing to do. My client’s taken over the code, and he’s spending time reading a lot of code to figure out what the parameters should be for various calls. In a dynamic language, there’s no easy way around this if there’s no documentation. Plus perl’s subroutine syntax can make it very difficult to decipher parameter lists quickly. It’s frustrating. I can’t really blame Perl for my own failure to write extensive documentation, but I must say, the idiosyncrasies of POD don’t exactly make it easy.

Installing DBI on Leopard’s perl 5.8.8

Saturday, June 14th, 2008

I needed to get my perl installation updated to do some development locally. As usual, perl was a pain in the ass. Long story short: Install Xcode 3.0, copy the “reentr.inc” file from the 5.8.8 source distribution, and DBI should install.

Below is the long-winded log of my woes, offered in the hopes it might help someone.

First, DBI 1.604 wouldn’t install via cpan, so I tried installing it by hand. But I just got the same error when running “make”:

No rule to make target `/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE/config.h', needed by `Makefile'.

I found this blog post, “Leopard Perl 5.8.8 installation throws errors when compiling (makefile)” mentioning the exact message, which recommended copying the CORE directory from 5.8.6 instead the above location. I tried that, but then I got this error instead:

DBI.xs: In function ‘dbi_profile’:
DBI.xs:2398: warning: implicit declaration of function ‘GvSVn’
DBI.xs:2398: error: invalid lvalue in assignment
DBI.xs: In function ‘dbi_profile’:
DBI.xs:2398: warning: implicit declaration of function ‘GvSVn’
DBI.xs:2398: error: invalid lvalue in assignment
DBI.xs: In function ‘XS_DBI_dispatch’:
DBI.xs:2970: warning: assignment makes pointer from integer without a cast
DBI.xs:2972: error: invalid lvalue in assignment
DBI.xs:2985: error: invalid type argument of ‘->’
DBI.xs:2989: error: invalid lvalue in assignment
DBI.xs:3293: warning: unused variable ‘Perl___notused’
DBI.xs: In function ‘XS_DBI_dispatch’:
DBI.xs:2970: warning: assignment makes pointer from integer without a cast
DBI.xs:2972: error: invalid lvalue in assignment
DBI.xs:2985: error: invalid type argument of ‘->’
DBI.xs:2989: error: invalid lvalue in assignment

Googling these error messages turned up surprisingly little. On another blog with a post titled, “Mac OS 10.5: Leopard” that mentioned difficulties with DBI, commenters suggested various solutions, but none of them worked for me. The blog author got an older version of DBI to install but I couldn’t get that to work either.

I discovered that Xcode 3.0 (developer tools for Leopard) contains the 5.8.8 files that belong in that CORE directory. This seemed like a better option than copying the probably outdated 5.8.6 files. You can get the gigantic Xcode disk image, a whopping 1.1 gigabytes, from the Apple Developer Connection site. Registration is required through the “Member” link, and once you’re in, go to Downloads and search for Xcode.

Before installing Xcode, I cleared out the hosed CORE directory I’d been mucking with. “DeveloperTools.pkg” is what contains the perl headers, so you can probably get away with just installing that (double-click it), instead of the entire XcodeTools.pkg. It did the trick: the compiler was now finding the “GvSVn” symbol it couldn’t before. But now I got this message during make:

In file included from DBIXS.h:19,
from Perl.xs:6:
/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE/perl.h:3993:22: error: reentr.inc: No such file or directory
In file included from DBIXS.h:19,
from Perl.xs:6:
/System/Library/Perl/5.8.8/darwin-thread-multi-2level/CORE/perl.h:3993:22: error: reentr.inc: No such file or directory
lipo: can't open input file: /var/tmp//ccQ8vbDU.out (No such file or directory)
make: *** [Perl.o] Error 1

In desperation, I downloaded the perl 5.8.8 source distribution tarball, and simply copied reentr.inc into the CORE directory. Voila! Make went to completion and I could install the module. From there, I went back into cpan to install DBD::mysql without any problems (you need mysql installed in the default location, /usr/local/mysql, of course).

On Programmer Insecurity: Is it Personality or the Market?

Friday, June 13th, 2008

Here’s a wonderful blog post by Ben Sussman-Collins, “Programmer Insecurity”, to which Jesse Noller has responded with “Programmer Insecurity and Mea Culpa”. (I don’t know either of these folks, I just follow their blogs in my RSS reader.) Ben talks about the need for more transparency, communication, and iterative growth in a programmer’s development:

Be transparent. Share your work constantly. Solicit feedback. Appreciate critiques. Let other people point out your mistakes. You are not your code. Do not be afraid of day-to-day failures — learn from them. (As they say at Google, “don’t run from failure — fail often, fail quickly, and learn.”) Cherish your history, both the successes and mistakes. All of these behaviors are the way to get better at programming. If you don’t follow them, you’re cheating your own personal development.

At the moment, I’m lucky to have fairly down-to-earth colleagues who generally foster these principles, but overall, this sort of perspective is sadly all too rare.

I don’t think it’s purely a matter of personality peculiar to programmers, or as Ben suggests, just “human nature” to fear embarrassment. I mean, sure, to an extent… but the fear is also fostered by a competitive labor market that values personal marketing over personal growth.

That’s why there are so many “best practices” blogs, vanity websites boasting of track records, and heated religious arguments about almost anything pertaining to code. The market has created a culture of showing off. And if you can demonstrate you are more “perfect” than the next guy or gal, you’ll impress the interviewer and land the job or the gig. One might argue, rightfully, that these are not great places to work. But places like Google where there is a generous philosophy of employee growth are probably the exception rather than the rule.

I can remember a time when things were different.

Making It Happen

Monday, June 9th, 2008

A while ago, I had an idea for a cool website. It involved grouping blog posts under “debate” questions. There was a simple mechanism for auto-detection if you linked a blog post to a debate page. The point was to be able to group postings with more semantic richness than simple tags or categories.

It fell by the wayside, so I took it down after a few months. Recently I’ve been seeing sites pop up based on similar ideas. AllVoices is one, and Debategraph.org is another. It’s nice to see the idea of information richness continuing to develop in interesting ways.

And it makes me wish I had stuck with my site idea, though to be honest, it wasn’t realistically feasible. The hardest part wasn’t coding the functionality, which only took 2-3 weeks of the large pool of free time I had back then. (Side goals were to get a working knowledge of CherryPy and sqlalchemy, so at least those were accomplished!) No, the real difficulty was “selling” it to users: publicizing the site, making it visually attractive and user-friendly, and getting people to use it in their own blogs. I didn’t have the skills or resources to make those things happen.

There’s an adage that says success on the web largely depends upon execution, not the concept. That’s so true. I feel like I’ve known so many smart, talented technology people who excel at what they do but haven’t been able to pull off their interesting side projects. I think it’s because we often underestimate the non-technical challenges in getting a website off the ground. In many ways, those are more important to do well than solving the technological problems.

The oxymoron of perl best practices

Sunday, May 18th, 2008

I’ve been browsing Damien Conway’s excellent book, Perl Best Practices. His description of “best practices” in Chapter 1 is quite good, bestowing meaning upon an otherwise empty buzzword for marketing departments:

Rules, conventions, standards, and practices help programmers communicate and coordinate with one another. They provide a uniform and predictable framework for thinking about problems, and a common language for expressing solutions. This is especially critical in Perl, where the language itself is deliberately designed to offer many ways to accomplish the same task, and consequently supports many incompatible dialects in which to express any solution.

Simply put, best practices are the recognizably tried-and-true. The key word is recognizable: others should be able to identify the problem being solved as well as the specific type of solution. In a way, it’s redundant to speak of best practices when talking about the programming: unless no one will ever see your code, the foremost consideration of the programmer should be, Can other people understand what I’ve done easily and quickly, and also make changes and improvements easily and quickly?

That question is really at the heart of coding as a craft of logical elegance, I feel. Giving it due care and consideration is what distinguishes artisans from those who just “get it done” (they always seem to run into perpetual headaches down the line).

I find the philosophy of perl to be highly antithetical to the sort of shared understanding and communication described above. The motto of “there’s more than one way to do it” is self-serving and individualistic, to the detriment of maintainability. Perl seems to foster stubbornness and isolation: I’ll do things MY preferred way, and you do it YOURS. The codebase quickly devolves into baffling inconsistency. Of course, skilled perl programmers pride themselves on being able to know all the different ways of doing something, but at that point, the strength of the language isn’t the issue anymore; the ego of the coder is.

Anyway, back to Conway’s book. What’s interesting, but perhaps not surprising, is the number of “don’t do X” recommendations it makes. Leaving off parentheses on subroutine calls? Convenient, but ugly and confusing when a bunch are these are combined, so don’t do it. Tricks with dereferencing syntax? Save everyone headaches and stick to arrows. Pseudohashes? Stay away from them. Indirect object syntax? Potentially conflicts with subroutine names, so don’t use it.

Take away all the crazy tricks that perl devotees love, and what’s left? The only truly great thing about perl is its regular expression integration and text processing. It’s amazing stuff. But otherwise, perl doesn’t particularly excel at much else.

Which is not to say large complex projects in perl are impossible. The University of Washington’s Catalyst tools use a custom web framework written in perl, which they’ve open sourced. I only cite that particular example because I’ve used some of the tools and I know they’re nicely done. So sure, it’s possible, but it takes a lot of deliberate restraint and discipline, more so than with other languages and tools that are oriented towards helping you get your job done.

Seems to me that if you must willfully refrain from using what a language offers, and constantly fight the temptation to write mangled code that costs time, money, and frustration for others to decipher, perl isn’t a particularly strong choice. In the case of perl, best practices aren’t a way to get the most out of the language; they’re a stopgap measure to stave off its natural tendency towards chaos.

Eclipse and JDK 1.6.0_05 on Mac OS

Friday, May 9th, 2008

Last week, Java 1.6 went out of Developer Preview and became an “official” release for Mac OS 10.5.2. (You still can’t get 1.6 for 10.5.1, sadly.) I’ve been fiddling with 1.6 and Eclipse, trying to get them to play well together, and here’s what I’ve found so far.

Eclipse itself needs to run on 1.5. There’s a great blog post, “Running Eclipse on MacBooks with Java 6″, written by one “rkischuk,” that explains why: 1.6 doesn’t support 32-bit SWT-Cocoa bindings, so Eclipse will bomb. The error I got was a mysterious “JVM Terminated. Exit code=-1″ and a list of run-time options. If you run Eclipse from a shell, you might see this:

2008-05-09 10:53:55.443 eclipse[257:10b] Cannot find executable for CFBundle 0×116030 (not loaded)

or maybe this:

_NSJVMLoadLibrary: NSAddLibrary failed for /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Libraries/libjvm.dylib
JavaVM FATAL: Failed to load the jvm library.

When I installed 1.6, I had messed around with /System/Library/Frameworks/JavaVM.framework/Versions, trying to get 1.6 to run as the system default. But the cleanest solution for me was to KEEP 1.5 as the default. So make sure that directory looks like this:

drwxr-xr-x 11 root wheel 374 May 9 10:49 ..
lrwxr-xr-x 1 root wheel 5 May 5 22:41 1.3 -> 1.3.1
drwxr-xr-x 3 root wheel 102 Nov 2 2007 1.3.1
lrwxr-xr-x 1 root wheel 5 Apr 18 13:07 1.4 -> 1.4.2
lrwxr-xr-x 1 root wheel 3 May 5 22:41 1.4.1 -> 1.4
drwxr-xr-x 8 root wheel 272 Apr 27 2007 1.4.2
lrwxr-xr-x 1 root wheel 5 Apr 18 13:07 1.5 -> 1.5.0
drwxr-xr-x 8 root wheel 272 Apr 27 2007 1.5.0
lrwxr-xr-x 1 root wheel 5 May 5 22:41 1.6 -> 1.6.0
drwxr-xr-x 8 root wheel 272 Apr 18 14:03 1.6.0
drwxr-xr-x 9 root wheel 306 May 9 10:50 A
lrwxr-xr-x 1 root wheel 1 May 9 11:13 Current -> A
lrwxr-xr-x 1 root wheel 3 May 9 11:12 CurrentJDK -> 1.5

Eclipse should run as it normally does.

If your code project(s) don’t require SWT, you can use 1.6 as an Installed JRE within Eclipse. Go to Preferences -> Java -> Installed JREs -> Add…. Select “Mac OS VM” and point it to:

/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home

Building projects and running JBoss using the 1.6 seems to work just fine.

Lessons Learned

Thursday, April 17th, 2008

After about 5 months, I’ve decided that it’s time to move on to another gig. I’ve learned a few things, and I’m posting them here in the hopes that the lessons might be helpful to other programmers and techies-at-large.

Working in a small business as the sole do-it-all technology person has its unique challenges. It can be very fulfilling to be the sole expert and “enabler,” if that turns you on. But the flip side is that management might not really understand or care that much about their technology. Is there a reasonable budget for what they’re trying to accomplish? Do they understand, at a high level, your projects and how they contribute to the mission? Are technology projects considered a burdensome mystery or something valuable and embraced by the company? Question the reasons why there’s only one tech guy/girl and whether that seems right.

Another thing to assess is whether you can deal with taking over the existing codebase. I’ve taken over other code before with success, retaining what was good and doing clean up as necessary. At this past gig, things looked reasonably tidy at a first glance, but as time progressed, I realized a ton of abstractions weren’t in place, and those that did exist didn’t make sense. Some refactoring might have been interesting to do, but this endeavor wasn’t valued when I proposed it as a project.

Lastly, I think it’s important to be wary of promises about the future. Even with the best of intentions, things change quickly at small businesses. The projects I was initially excited about got perpetually deferred for various reasons, and I found myself preoccupied with doing maintenance code fixes, making cosmetic tweaks, performing server administration, and providing support for third party software (which I really don’t like to do). The company needed these things done, so I did them with as much cheer as I could muster, hoping we’d eventually get to a place where some solid new development could occur (and I could sneak in some refactoring)—that’s what floats my boat. But it became to clear to me that wasn’t going to happen anytime soon.

So that’s that. It’s a shame it didn’t work out, especially since I actually liked everyone I worked with. At least it’s an amicable departure, and I hope to be involved in hiring a replacement who might be a better fit for their current needs than I am.

The new gig? Java. Been catching up on it, since it’s been a few years. Oh, it feels so nice to have package namespaces, real data types, full-featured APIs, and real object-orientedness again. Like coming home.

The Small Business Subconscious

Thursday, April 10th, 2008

Small businesses can be challenging places to work. You often have to make do with few resources, play several roles at once, and be flexible enough to deal with loose/nonexistent company organization. If everything magically clicks, as it sometimes does, it can be a beautiful thing. But more often than not, that simply doesn’t happen.

As I talked about this with friends who have had similar experiences, they observed that there’s often a common mistake made in small businesses: since every employee is precious, the organization tries to extract as much value as possible by encouraging everyone to contribute in as many ways as possible. It bills itself as a democracy, as an environment that genuinely listens to its employees.

The problem, of course, is that when people aren’t in sync about the mission, or about specific project goals, you end up with a frustrating mess of conflicting directions. The business (somewhat desperately?) tries to latch onto everything at once, and there’s a lack of decisiveness in moving forward. Projects get fragmented. People believe they’re collaborating but they’re actually not. Confusion ensues. Little gets done.

Leadership is even more key in small businesses than in larger ones, I’d wager. Because if you have few resources, you need to choose your projects very carefully, dedicate resources accordingly, and make absolutely sure they go to completion quickly. There’s not much room for failure. But instead, small businesses seem prone to being wishy-washy. The insidious and tragic aspect of this is that when projects move at a snail’s pace or even fail, the accountability falls back on the individuals. Because management failed to lead, it can remain blame-free. It’s a formula for endless frustration.

In this unfortunate strategy, the small business is structured like the subconscious and its conflicts. Its desire is vague and repressed, and though seemingly absent, it’s actually very much the motive force for its existence. Like a human being, the small business can’t decide upon its identity, which remains in flux. And just like a human being, this repressed desire can lash out violently when it isn’t fulfilled.