Hamster Food

Update: I've ported this blog from my own platform to Lektor, but I haven't changed my mind about this article: building my own blog platform with Motor was worth it.

If you aren't using your own libraries as you build them, you're skipping an essential test: not mainly for correctness or performance but for usability.

(Using your software as you develop it is normally called "eating your own dogfood", but I don't have any dogs. Only hamsters. This is my dwarf hamster Rhoda.)

I develop Motor, my asynchronous driver for Tornado and MongoDB, mainly with test-driven development: I think of an API Motor should implement, I write the test, and I make Motor pass the test. But I also use Motor in the blog platform that runs this site. By using Motor, I discovered a few features that are absolutely essential for building a real application with it, which I never would have thought of otherwise:

• Opening a MotorConnection. My initial API for opening a connection to MongoDB with Motor was asynchronous:

connection = motor.MotorConnection()

That's fine for unittests. But as soon as I started building my blog it was clear it's a pain in the ass. There's no place in a Tornado application's usual startup sequence to do this step. So I made Motor open connections on demand, when you first use them.

GridFSHandler. I recently completed Motor's methods for accessing GridFS, MongoDB's binary blob-storage system. Then I updated my blog to serve images from GridFS. And even though all the functionality I needed was complete, it was horribly inconvenient. So I wrote a stream_to_handler method to pipe a GridFS file into a Tornado RequestHandler. Once I started using it, I figured it was still too low-level, so I reimplemented Tornado's StaticFileHandler on top of GridFS. Now serving static files straight from MongoDB is as easy as serving them from the file system.

I've sunk a lot of hours into building this site. I wondered if all the time was worth it. It's not like it has any special features I couldn't get from Nikola or Pelican. Building a capable blog platform with code syntax highlighting, drafts, media, Disqus, Google Analytics, and so on took longer than I expected, and I'm still tinkering with it. But the investment pays off marvelously. By using Motor in a real-world application, even a small one, I've discovered serious usability problems my testing wouldn't reveal.