Porting From PyMongo To Motor
Distressingly often, I see good programmers building applications with a flawed stack: PyMongo plus Tornado. Tornado is an asynchronous Python web framework, and PyMongo is a blocking driver for MongoDB. Obviously, any lengthy PyMongo operation blocks the event loop and hobbles its throughput.
I'm the author of Motor, a non-blocking driver for MongoDB and Tornado. I wrote it as an async alternative to PyMongo. If you have a Tornado application and want to add MongoDB to your toolset, I hope you'll connect to it with Motor. Of course, most of your MongoDB operations still need to be well-tuned, but if you're using Motor, a handful of slow queries won't destroy your application's total throughput.
But what if you've already written a large application with Tornado and PyMongo? At every layer of your code, you've relied on PyMongo's simple blocking interface:
# Using PyMongo.
def get_document():
document = db.collection.find_one()
return document
To port a function from PyMongo to Motor, you either have to switch to a callback style:
# Using Motor with a callback.
def get_document(callback):
db.collection.find_one(callback=callback)
...or use a Tornado coroutine:
# Using Motor with a coroutine.
@gen.coroutine
def get_document():
document = yield motor.Op(db.collection.find_one)
raise gen.Return(document)
Either way, the caller of your function must also be ported to a callback or coroutine style. So must its caller, and the caller above it.... Is there any way to update your code incrementally, or must it be rewritten all at once?
David Stainton, an acquaintance of mine on the Tornado mailing list, has recently ported a 10,000-line Tornado app from PyMongo to Motor. He tried a few angles of attack and learned which approach is best for incrementally rewriting a large application. He wrote a thorough and pragmatic article on porting a Tornado app from PyMongo to Motor; you should go read it.