closing the loop

March 16, 2006

A blast from my past. When I wrote that code I didn’t think anyone would ever read it, more less try to run it. I wrote the paper so that I could visit Barcelona, and thought that fleshing it out with code would impress the reviewers. It seemed to work. Barcelona was incredible. Folks danced to a crazy band playing on the plaza outside the cathedral after Easter mass. Traditional Catalan music, I guess. The “Flying Norwegians” kept me out all night at strange, unmarked clubs. Two men were fighting at 4am on the Ramblas. One hit the other over the head and he fell down, covered in blood. I thought he was dead until he rose up from the ambulance stretcher, swinging his fists, screaming, and ran off into the night. The culinary academy’s menu was only in Catalan, every dish a delectable surprise. Now someone’s actually read the code!

I’m now a “Yahoo!”.

March 13, 2006

On the first of this year, after four years as an independent contractor, I accepted a full-time job with Yahoo!. This isn’t as big of a change as it sounds. For much of the past four years my work on Nutch had been in-part funded by Yahoo! (and Overture before they were acquired by Yahoo!). I’m still primarily working from home, and, so far, entirely working on open-source stuff: Lucene, Hadoop and Nutch. The biggest change is that I don’t have to draft contracts, submit invoices, etc. I can now instead better focus on the technology and the open-source process.


March 13, 2006

We’ve split the distributed computing parts of Nutch into a new project named Hadoop. This includes a filesystem modelled after GFS and a distributed computing system modelled after Google’s MapReduce. So far a few folks are using Hadoop on tens of machines, and we’re testing it on clusters with hundreds of machines. Next stop, thousands!

Is hacking a capital offense now?

March 13, 2006

Someone suggested that I should be shot.

Mental Models For Search Are Getting Firmer (Jakob Nielsen’s Alertbox)

May 12, 2005

Jakob Nielsen, in an article titled Mental Models For Search Are Getting Firmer, provides more fuel for my claim that web search is a commodity. He warns against trying to change the search user interface. This argues that search engines should not try to distinguish themselves with fancy front ends. That leaves the backend, where innovation seems to have slowed as well…