Author: jeff

My new web project: DebateWire

A big reason that I read blogs is because there are great writers out there who raise questions that I hadn’t considered before. Often they make me look at things I care about in a new light.

It’d be cool, I thought, to be able to solidify questions of debate as a way to organize blog entries. It’s more specific than categories or tags, and if someone sees a provocative question that really makes them think, they could chime in with their own opinion or argument. I’m particularly interested in politics, but it would work for issues of debate on any controversial topic.

For the past few weeks, I’ve been sort of obsessed with that idea. The project is called DebateWire, and it lets you do what I described above. It’s got some rough edges, but I decided it’s ready for 1st release. Please take a look, and spread the word if you like it.

My next post is an example of how to use it.

The Uses of Function Attributes

A Python function is an object of type “function.” One of the things you can do, as a result, is set attributes on the function object, like so:

def say_hello():
	print "hi!"
say_hello.x = 1

(You can also do that on a function that’s an object method.)

This is pretty odd. In some ways, this feature interestingly blurs the distinction among classes, objects, and functions, since all of them can have their own attributes. It strikes me that this illustrates one of the key philosophical differences between Python and a language like Java, where object-oriented principles are more rigidly enforced, In Java, a class is a class, an object is an object, and a method is a method. We shall not even speak of functions!

For a while, I understood this language feature, but not its real utility, which is this: you don’t have to write entire classes for small pieces of functionality.

cherrypy is a good example. It uses the attribute name “exposed” to indicate whether a method should be accessible to the URL-to-object mapping mechanism. Combined with some other clever design, this allows an HTTP request handler to be an ordinary method in a tree structure that corresponds to the web application’s URL scheme. By contrast, in Java, you have to subclass Servlet for each handler, and then manually map those Servlets to URLs. Ugh.

Another use is that decorators can manipulate the attributes of functions they modify. If the decorator’s behavior should change based on state, or it wants to track statistics or debugging information, it can use the underlying function’s attributes for storage. (Some examples of this are strewn throughout the PythonDecoratorLibrary page at the Python Wiki.) Again, this avoids extra class definitions and objects, which would typically be necessary for separately storing that state information.

The last use I can think of right now is that you can easily and quickly create singletons. Because classes are meant for instantiation into objects, it can be inelegant to enforce a singleton pattern. With function attributes, you can define a function in a module, set some defaults, and allow callers to manipulate them, which will remain a single set application-wide.

(Note that you CAN create multiple instances of a function, as this interesting example shows. But the singleton idea is still sound in the context of module functions.)

The Right Metaphor

If you haven’t caught Kyle Wilson’s recent piece, “Software is Hard,” I highly recommend it. The essay moves elegantly from book review, to musings on knowing when the code is “done,” to issues of measuring quality, to the ever-present problems of lateness and going over budget, to the potential inadequacy of “engineering” as the metaphor for writing software.

It’s the last topic that’s the most fascinating to me. Kyle points out that new software is written only in response to new problems (otherwise, you’d just use existing software). As such, new code ventures into the unknown, where you can, at best, only guess at the challenges you’ll encounter. We always try our best to assess what we’ll face, but by their very nature, these are imperfect assessments. As Kyle puts it, “The only way to avoid that is to have your design go all the way down to specifying individual lines of code, in which case you aren’t designing at all, you’re just programming.”

Which is not to say, of course, we should simply give up engineering. Without some sort of plan for design and advance assessment, we’d be utterly lost. Businesses couldn’t function and programmers couldn’t make a living. For better or worse, the smooth functioning of our society is founded on the arrogance of making accurate predictions, not just about business and software, but about everything from politics and law, to human behavior and psychology, to weather. Such hubris…

No surprise, then, that even real-world traditional engineering often fails to be predictable. Kyle mentions the Oakland Bay Bridge as a project that’s hugely over time and budget. Just yesterday, Boeing announced its much-anticipated Dreamliner would be six months late.

So maybe software engineering IS the right term after all.

A Lesson in Software Development

Slashdot reported on an opinion piece by David Sivers entitled, “7 reasons I switched back to PHP after 2 years on Rails.” It’s sparked heated discussion about languages, but if you read the article carefully, it’s not really anti-Ruby or pro-PHP, though David does imply serious shortcomings to Rails. He writes, “at every step, it seemed our needs clashed with Rails’ preferences.”

The problem with the short piece is that he’s not very specific about what these “preferences” are. The main criticisms seem to be that Rails is too complicated, too slow, and doesn’t allow direct SQL. Complexity and performance are often the cost of using a large web framework; these don’t seem like Rails problems, but issues you’d have to deal with when using any big framework (in fact, there’s an apples-and-oranges dimension here: Ruby on Rails is a full web stack and architecture, whereas PHP is a scripting language with built-in access to tons of different libraries. But we’ll put that aside for now.). As for direct SQL, a lot of folks have pointed out that David is just plain mistaken about not being able to do what he wanted.

The real reasons he switched back actually have little to do with PHP per se: read the piece carefully and you’ll see he’s much more comfortable thinking in terms of libraries than frameworks. Also, he needed to integrate tightly with an existing codebase. Those two things are the real reasons why switching back to PHP worked for him. They don’t have anything to do with either the strengths or failures of Ruby on Rails itself.

David doesn’t quite do justice to his main point, which is actually about software development, not language features: don’t expect a much touted language or tool to work magic for you. Of course, this should apply as much to PHP as to Rails! I’m not trying to defend Rails, which I know zilch about. There’s a bigger lesson here: the strengths and weaknesses of languages have less to do with success than the overall environment, the functional and integration requirements, and the coders’ facility with a toolset.

The Virtues of Simplicity

In this age of bloated software, it’s hard to find solutions that do what you need without a ton of unnecessary complexity. Software that’s more complex than the problem at hand makes it costly to learn, maintain, and troubleshoot. For companies and individuals with limited resources, that’s a real challenge. Glenn (my current client) and I wanted a solution to replace some slow performing CGIs but mod_perl was too much. Plus the idea of embedding a perl interpreter in Apache seemed scary.

I looked for perl web application server options that could be proxied through Apache. That architecture would give us the performance benefits of code preloaded into memory, persistent database connections, and precompiled templates. The only thing I found was OpenInteract, but its module list is quite large, and the project hasn’t been updated in a while. I didn’t want to have to dig around a ton of foreign code if I needed to squash a bug. So I decided to write my own.

The result is a neat little thing I call “perlserver,” a heavily modified version of this piece of public domain code for a preforking HTTP server. I deliberately kept it simple but gave it all the needed features: URL-to-handler mapping, safeguards against memory leaks, maximum configurability. It’s only 450 lines of code and is tightly integrated with the LWP modules. To convert the existing set of perl CGIs, the code was simply wrapped in packages that conform to a simple API understood by perlserver. Existing URLs were proxied to perlserver by clever RewriteRules in Apache.

Before, a CGI that built a complex page typically took 900 – 1500ms. Now, the same page served from perlserver by proxy takes around 300 – 400ms.

I’m thinking about releasing the code as open source. It’s not fancy, but that’s why it’s great: it’s a good option for sites that want better perl performance without the tedious complexity of other existing solutions.