misc – codefork.com

Engineering

Engineering is about making tradeoffs. You consider all the factors, optimize for the ones that are most important, and make the best choices based on that.

Put in that way, engineering is pretty damn boring. It’s not easy. But being difficult doesn’t necessarily make something interesting.

In my younger days, I put a lot of energy into exploring the latest fads and keeping up with “best practices,” all in the hopes of becoming a better software engineer. I do that a lot less these days–only as much as I need to for work. Anyone who genuinely gets off on “best practices” is a bit suspect in my book.

At some point, I think my interest in coding changed from “is this well written software?” to something like “what kind of thinking does this code imply about how it solves the problem?”

This is way more interesting! Even so-called “bad” code can be very interesting and teach you something about a unique way to approach the problem it solves.

Obviously, we can’t spend all our time appreciating various aspects of less-than-optimal code. There’s work to be done, efficiency to be measured, best practices to follow. It’s not like we got into coding because we enjoyed thinking or anything.

Generator Expressions

I discovered Python generator expressions this week. They’ve been around since v2.4, way back in 2004, and yet, somehow, they managed to escape my notice until now. I don’t think I’ve ever seen them used in any codebase I’ve worked on.

Generator expressions are almost the same as list comprehension syntax, except instead of square brackets, you use parentheses. And instead of returning a list, the comprehension returns a generator object.

This is quite a nice bit of syntactic sugar that helps to keep things more succinct than writing iterators and generator functions. But it’s not as succinct as, say, the chained method syntax you find in Scala or Ruby, which I think is vastly superior in terms of clarity. Several years ago, I wrote up some notes about Ruby streams in a github repo for my co-workers because it was a valuable technique in some data processing scripts we were working on. (Since then, I haven’t done much Ruby.)

Here’s a bit of Python code that duplicates the simple Ruby example in the linked repo above, with the identical output. Again, it’s not nearly as nice to look at as chained method calls, but still better than having to write separate generator functions or classes.

nums = list(range(1,6))

add10 = lambda i: print(f"adding 10 to {i}") or i + 10
filter_even = lambda i: print(f"filtering {i} on evenness") or i % 2 == 0
output = lambda i: print(f"in each: {i}")

# list comprehensions

r1 = [ add10(x) for x in nums ]
r2 = [ x for x in r1 if filter_even(x) ]
r3 = [ output(x) for x in r2 ]

# generator expressions

r1 = ( add10(x) for x in nums )
r2 = ( x for x in r1 if filter_even(x) )
r3 = ( output(x) for x in r2 )
list(r3)

Lumen: A port of Blacklight to Scala and the Play framework

I’ve done some work the past two years using Blacklight, a great discovery interface for Solr with a lot of library catalog features. It’s quality software with years of work invested in it by some very smart people.

Over time, “the Ruby way” of doing things, as well as “the Rails way,” has bugged me more and more. Things like the use of naming conventions for hooks, passing arrays and hashes around in lieu of actual data structures, the varying use of hashes and OpenStructs, the ability to monkey patch, the difficulty of looking at a method and not being able to tell what its arguments are or can be, the need to go digging around in the source code of a gem to figure out how certain APIs are dynamically created because those methods can’t get automatically documented by tools like rubydoc or yard. These things often make life easier when you’re writing new code and trying to do it quickly, but they create nightmares when you try to refactor stuff or upgrade gem dependencies.

The last few months, I’ve been slowly porting Blacklight to Scala and the Play framework. I’m calling this new project Lumen.

Scala is a powerful statically typed, compiled language that permits you to mix object-oriented and functional paradigms, and it allows you to take advantage of the enormous ecosystem of existing Java libraries. It also incorporates some of the innovations of the last two decades of dynamic languages that make programmers happy. I think it’s a language that privileges software quality over rapid development.

So far, Lumen has been a hobby project to learn Scala, so I’ve approached it in a less disciplined fashion than I otherwise might. This means there are lots of TODOs scattered throughout, and style/design inconsistencies as I’ve learned better ways to do things but haven’t always gone back to change things everywhere. That’s life when you’re noodling around in your spare time. Lumen is largely an experiment right now, but I hope it will eventually grow into a full-featured, production-quality piece of software. We’ll see.

You can check out a demo here: http://lumen-demo.codefork.com.

Upgrading the Touchpad on a Thinkpad x240

This is a stock photo of a Thinkpad x240, stolen from the interwebs:

This is my own x240, which I bought back in January 2015.

If you know about Thinkpads, you probably noticed the difference right away. The x240 (and other models that year) suffered from an incredibly crappy buttonless touchpad. It’s so bad that it’s barely usable. Clicking is ridiculously inaccurate: there’s so much travel that the mouse pointer moves during a click, and there are no buttons to use instead. There were so many complaints that Lenovo replaced it with a better one in the next year’s lineup.

This weekend I finally got around to upgrading it with a touchpad replacement part for the x250. It cost $32 on ebay. This modification is popular, so you can find info about it scattered around in forums and such. I followed the instructions on this page, How to change an x240 trackpad, as it’s one of the clearest ones out there. I couldn’t find much info about the author, whose name appears only as “Michael” on that blog.

Some notes and tips from my experience:

1) Michael’s picture shows a set of wires connected to the touchpad along its side, but mine didn’t have them.

2) The touchpad sits in a well, held in place with adhesive tape, so to remove it, you just pry it off. The problem is that it’s hard to reach “under” the entire touchpad assembly, which is sort of like a sandwich with layers. I ended up partially prying off the top layer before I could get to the bottom and pry the whole thing from the case. Needless to say, this bent the touchpad.

I couldn’t figure out a way to avoid effectively destroying the old touchpad. But since it was so crappy, it was also somewhat satisfying.

3) Detaching and re-attaching the small ribbon cable from/to the underside of the touchpad is VERY tricky. The end of the ribbon is held in place to the connector on the touchpad by a thin black “latch” sitting just behind it. You CANNOT just yank the ribbon out. (This took me a while to figure out!) Lift the black latch, and the ribbon will slide from the connector easily. When connecting it to the new touchpad, tuck the ribbon end securely into the connector, then flip the latch down to lock it in place.

4) At first, the new touchpad wasn’t being recognized by the machine. It worked after I re-seated the ribbon in the connector and also reset the BIOS (as shown in this video: stick a paper clip end into the tiny hole beside the battery and press for 20 seconds). I should have tried those things separately, but got a bit too excited. So you may or may not have to do a BIOS reset.

So far so good. The new touchpad is definitely a big improvement. There’s much less click travel using the pad, and it feels snappier. I really like having the buttons.

The only quirk is that the surface of the touchpad now sits just a hair higher than the palm rest. It’s probably slightly more likely for my hand to accidentally brush it while typing, as compared to the original touchpad, but only time will tell for sure.

It feels like a totally different computer.

“Proxy mode” added in refine_viaf 1.4

A refine_viaf user recently commented that she would like to get Library of Congress IDs for the name candidates in OpenRefine, instead of VIAF IDs.

It would be ideal if the name IDs for LC and other sources could be additional fields in the JSON data returned from refine_viaf, which you could then extract using some GREL code. Unfortunately, OpenRefine doesn’t allow you to access additional fields on name candidate objects.

So I’ve created a separate “proxy mode” that returns IDs used by source institutions themselves, rather than the VIAF IDs. To use proxy mode, add a reconciliation service in OpenRefine using this URL format instead of the usual URL:

http://refine.codefork.com/reconcile/viafproxy/LC

One quirk is that OpenRefine will create broken hyperlinks for a few sources (at the moment, these are BNC, BNF, DBC, and NUKAT). This is due to the fact that the IDs in these URLs don’t match the name record IDs, which is a requirement for the hyperlinking mechanism to work properly.

In short, you can now use refine_viaf to reconcile “directly” against the name authority records of VIAF’s source institutions, which should be useful to many people.