A. Jesse Jiryu Davis

Announcing PyMongo 3.0 Beta

You have no idea how it thrills me to tell you this. We have just shipped PyMongo 3.0 Beta. You need to read the changelog and install it in your dev environment now: pip install [...]

Adder

You have no idea how it thrills me to tell you this. We have just shipped PyMongo 3.0 Beta. You need to read the changelog and install it in your dev environment now:

pip install https://github.com/mongodb/mongo-python-driver/archive/3.0b0.tar.gz

It's not common among Python folk, in my experience, to test each others' betas. But this month I really need your help: Please give PyMongo 3 a spin.

Tell me if the new interfaces work for you. Tell me if our API changes make sense, if they require more updates to your code than they're worth, if we documented them all properly in the changelog. And naturally, tell me if you found any bugs.

Vastly more information will come in the next few weeks. I'll write to you about how much more conformant, responsive, robust, and modern the new PyMongo is. Besides that, my colleagues and I at MongoDB are drafting articles on the specifications PyMongo 3 implements:

  • The new CRUD API spec that gives you a clean new interface for basic operations on your data.
  • My Server Discovery And Monitoring Spec defines how drivers connect to the servers in your replica set or sharded cluster and stay abreast of changes.
  • The Server Selection Spec is a straightforward new expression of how to choose replica set members for reads, or how to load balance among mongoses.

Stay tuned.

Image: Pearson Scott Foresman.

MongoDB Driver Issues Are Now Public

For the last year, the MongoDB drivers team has focused on standardizing our eleven official drivers. Gradually, we're cleaning up inconsistent behaviors and APIs. Ideally, users and our customer-support team need to learn MongoDB [...]

Forest

For the last year, the MongoDB drivers team has focused on standardizing our eleven official drivers. Gradually, we're cleaning up inconsistent behaviors and APIs. Ideally, users and our customer-support team need to learn MongoDB once, instead of re-learning MongoDB in each programming language. We're also sharing our bugfixes among drivers: if I made a mistake, there's a good chance someone else made the same mistake in another language. We publish our bugfixes to the team so everyone can check their code.

We've tracked these cross-language bugs and standards in a private Jira project, "DRIVERS". Last week, that project went public. Anyone can comment, browse tickets, and create new ones.

In the past you could see issues in each driver's project, like the "PYTHON" or "JAVA" projects. Those were always public. But not the umbrella DRIVERS tickets. We felt that we needed a private place to propose and debate features, for a few reasons: We wanted to talk about specific customers' requirements in confidence, and we wanted to discuss controversial features frankly among ourselves, before getting the community's opinion. We worried that if we didn't have a private place to discuss them in Jira, we'd use email and lose the advantages of a ticket-tracking system. The cross-language bugfixes didn't need to be private but they were by default, since they were in the same project as the cross-language features.

But we changed our minds about these private discussions. The MongoDB server's tickets have always been public, after all. Our drivers are open source and we develop in the open, so the DRIVERS project should be open too. We'll still use private comments when we discuss particular customers' needs, of course. But now the default for DRIVERS tickets is open, not closed.

My First Zen Dharma Talk, March 29

The morning of March 29, I will give my first Zen talk. I will be challenged by my community in dharma combat, a snappy give-and-take where my insight is tested. If I pass the test, I'll be a senior Zen student, with the authority to talk about [...]

Village Zendo

The morning of March 29, I will give my first Zen talk. I will be challenged by my community in dharma combat, a snappy give-and-take where my insight is tested. If I pass the test, I'll be a senior Zen student, with the authority to talk about the dharma, so watch out! I'll spew even more bullshit than I do now.

You should come to my talk, and the wild Japanese Buddhist ceremony around it called "shuso hossen." The show starts 9:30 am. If you feel generous you can leave $20 in the pot.

Event information is on villagezendo.org.

March To Triumph As A Mentor

John Flaxman (1755–1826) Good engineers write good code. But the best engineers cultivate and inspire their junior colleagues. If you are a senior engineer, you must learn to be a mentor. Especially if you are committed to [...]

Telemachus and Mentor

John Flaxman (1755–1826)

Good engineers write good code. But the best engineers cultivate and inspire their junior colleagues. If you are a senior engineer, you must learn to be a mentor. Especially if you are committed to diversity: mentorship is critical to the careers of women and minorities in tech. I have failed at mentoring, then succeeded. Learn from me and march to mentorship triumph.

A Mandatory Skill

This essay, and my talk from which I adapted it, is influenced by John Allspaw's On Being A Senior Engineer:

Mature engineers lift the skills and expertise of those around them.

They recognize that at some point, their individual contribution and potential cannot be exercised singularly. They recognize that there is only so much that can be produced by a single person, and the world's best engineering feats are executed by teams, not singularly brilliant and lone engineers....

At Etsy we call this a "generosity of spirit." Generosity of spirit is one of our core engineering values, but also a primary responsibility of our Staff Engineer position, a career-level position. These engineers spend the time to make sure that more junior or new engineers unfamiliar with the tech or processes we have not only understand what they are doing, but also why they are doing it. "Teaching to fish" is a mandatory skill at this level, and that requires having both patience and a perspective of investment in the rest of the organization.

Allspaw is Senior Vice President of Operations and Infrastructure at Etsy. He manages people like you and me, and decides if we are promoted. So if you are ambitious, pay attention when people like him say it is mandatory to demonstrate skill at mentoring.

Telemachus

The word "mentor" is from the Odyssey.

The story goes that the warrior Odysseus left his wife Penelope and his infant son to fight in the Trojan War. He spent ten years at the siege of Troy, and, cursed with atrocious luck, he took ten more years to sail home. Unfathered, his son Telemachus did not mature into a warrior. His home was invaded by frat boys who drank all the wine, ate all the food, and sexually harassed Penelope. (The translations conventionally call them "suitors".) Telemachus was too weak to throw them out. He literally could not draw his father's bow against them.

Athena, goddess of wisdom, craft, and war, saw the pickle he was in, and came to earth to help. They traveled the country, met kings and soldiers, and Athena taught Telemachus the courage he needed to be a man. But she did not appear as a goddess: she disguised herself as an old friend of his father's, named Mentor.

Mentor and Telemachus

So a mentor is not one who "ments". The word is from a person named Mentor. The young person coached is not a "mentee". Luckily there is no need to use an ugly non-word, because this is an ancient role with many names: apprentice, protégé, pupil. I call someone I mentor my telemachus.

These days, Athena does not descend to guide young coders. Not unless you embody her: you must be Mentor.

Why Mentor?

Your Career, Their Careers

John Allspaw says mentoring is mandatory for your career advancement. And junior engineers' careers depend on your generous mentorship, too. More than college, more than GPA, perhaps more than talent, what matters to an engineer's career is who mentors her. It certainly was so for my career: I am a moderately talented coder with a bachelor's in computer science, but I was a sorry engineer in my first job. It was not until I worked closely with Olivier Pomel and Alexis Lê-Quôc, then at Wireless Generation, that I learned how to be a professional.

Your Company's Future, Our Industry's Future

Your company's future depends on you, too. You need senior engineers and there are not enough to go around. Bringing up young coders is a huge, risky investment, but it's the only way we have to make senior engineers.

The same holds for our whole discipline: mentoring is how we raise the next generation of software engineers, who will have the next ideas, found the next companies, invent the next programming languages. Besides, at many companies, including mine, there are few women and minorities, and men hold most of the senior engineering roles. We must mentor women and minorities so they can grow into senior roles; it is our best weapon to fight for a fairer future.

As the National Center for Women and IT says, mentoring and sponsoring women is a top strategy to promote gender diversity:

Men recognized the need for women to be mentored by senior leaders, noting from their own experiences that all employees—regardless of gender or cultural background—can benefit from a leader who advises them and looks out for them. Similarly, the men also talked about the importance of sponsorship, which goes further than mentoring. As one leader explained: "A sponsor is really looking much longer-term for the person, helping them get to new positions in the company, looking out for them over a longish period of time."

Disaster

When I began guiding junior programmers, I failed immediately. I shirked my responsibilities and pretended things were going fine. I failed, everyone was mad at me, and I vowed never to mentor again.

Icarus Falling

Icarus Falling, Jacob Peter Gowy (1615–1661)

In retrospect there were warnings, circumstances that set me up to fail.

Absent Visionary

It was the summer of 2013, and MongoDB had a cohort of interns arriving in need of projects and supervisors. A colleague said, "We should do something cool with big data." But he would not lead it himself. Reluctantly, I agreed to supervise two interns on this project. Without the original visionary present, the internship was rudderless. There is no substitute for the person who had the idea. I will not again agree to lead a project with an absent visionary.

Unclear Goals

The original idea, "something cool with big data," had no clear goal and no steps. Since I was not the visionary, I could not come up with any. I thought, "I'll run the internship agilely. We'll just choose a big data set and see what happens, make it up as we go." But agile is not the same as no goals. We never arrived at a rigorous description of the project nor measures of success, so I could not evaluate whether my interns were productive.

Inexpert Mentor

I am expert at some things, but I am not a big data man. My interns were actually quite skilled: one was a double major in computer science and finance, and they were both competent with Hadoop. So what they did impressed me, but I did not know if it was impressive because they were doing well, or if it was just because I could not understand what they were doing. I hesitated to question them. They moved fast but no one was steering.

Feeling Bad

I hid from my interns. I avoided them in the office, or worked from home. We met barely once a week. I procrastinated answering their messages. Time and again they complained they were unguided, but I did not change. I knew I was failing but I pretended not to know. I hoped to muddle through the summer and never oversee interns again.

My managers were watching me founder and they issued ultimata: get involved, make goals, get the project on track. I never did. With only weeks left in the summer, Intern Protective Services reassigned my apprentices to Mike O'Brien, who maintained the MongoDB Connector for Hadoop at the time. He canceled the remainder of our inchoate project and had my interns add features to the Connector instead. He gave them clear goals, supervised them expertly, and saved the internship from total loss.

The Turnaround

It is a great blessing of my career that MongoDB's management gave me another shot at mentoring.

I did not know it was a blessing then. When my boss told me, right after my humiliating summer, that I would have to oversee a new hire, I was furious. And the reason for my fury was my fear. I could handle an apprentice no better than a baby handles a python.

Baby Hercules with a snake

Baby Hercules with a snake, Roman, 2nd Century CE

But this second apprenticeship was a turnaround. The summer's circumstances had set me up to fail, but now the circumstances led me and my telemachus to triumph.

Expert Mentor

When new hires join MongoDB they rotate through four teams for about six weeks each, feeling out where they fit. My apprentice was a gifted young woman named Amalia Hawkins, who had just graduated. She worked with me on PyMongo, the code I know best. I could expertly evaluate her patches and guide her as she learned the code.

Small, Clear Goals

Since PyMongo was an established project with a backlog, its maintainer Bernie Hackett and I could start Amalia with easy features and bugs until she found her footing, then challenge her with more complex tasks. At each stage we had a clear problem statement and criteria. I always knew when Amalia was productive or stuck.

Guiding Visionary

I know PyMongo's roadmap, and Bernie has been its guiding visionary for years. Together we ensured that Amalia's contributions kept PyMongo on track. It is not enough to simply complete tasks. The visionary knows whether a bug is fixed or a feature is implemented in a way that advances a project toward its long-term goals.

Simpatico

Amalia and I like each other and became friends. Do not underestimate the importance of personal comfort and compatibility. We are professionals who program machines, so we sometimes think it does not matter how it feels to work with another person. But I must be simpatico with my pupil or the relationship fails.

I give you permission to mentor only the junior engineers you click with. Tell your boss whom you want to mentor and stick with the people you enjoy. Your organization should have a graceful way to reassign mentors. There is no point trying to develop a close relationship with someone who doesn't appeal to you.

Lessons

It was Amalia's idea that I should speak and write about mentoring, so I asked her what she observed me learn when she was my apprentice. How did I improve, why did we succeed? She identified the following:

Be Eager To Help

Or at least always appear eager.

I grouch when my peers interrupt me. But an apprentice is different, I cannot discourage him from asking for help when he is stuck. Stuckness is a total waste. Teachers and pupils alike overvalue "being independent" and "figuring it out on one's own." A stuck apprentice produces no new information: he is not learning, and I am not learning how skilled he is. Nothing happens until he is unstuck. There is not enough time in a summer internship, or a six-week rotation, or a life to waste on stuckness. My apprentice must not be afraid to come to me immediately.

Know What Your Apprentice Does Not Know

There are just as many questions that your telemachus does not know to ask. She does not know what she does not know. In my experience, new professionals do not know how to submit clean patches that fix one bug or implement one feature. They do not really get how to use git for code reviews and pull requests. They do not know what maintainable and legible code is. They do not know how to approach a very large old codebase, nor how to obey semantic versioning. In other words, university has taught them computer programming, not software engineering.

Now I anticipate these lacks and teach my apprentices the principles they need, just before they need them.

Admit You Do Not Know

Amalia told me that "most people just pretend omniscience, which is super frustrating." A humble mentor is approachable. Not knowing gives the apprentice permission not to know, either. I model being comfortable showing my knowledge gaps.

Besides, whenever I suffer imposter syndrome, it cripples me: I pretend to know more than I do, and I would rather hide from my apprentice than be caught out. It is not until I drop that act that I am no longer afraid.

But the real advantage of admitting you do not know is, it brings on the most valuable hour of the apprenticeship.

Teach Problem-Solving

I wish it were different, but there is only an hour or so, in the course of a summer or a six-week rotation, when we solve a problem together. Almost the whole value of the apprenticeship is in this hour. After all, when my protégé asks me a question and I know the answer, it is not much different from Stack Overflow or a textbook. But when I cannot answer, we have to solve the problem together.

Problem-solving cannot be explained. Not in general. But you and I, if we have been debugging software for years, we know how to solve problems in general. We cannot really explain how, but this knowledge is transmissible. It is precisely this knowledge that apprenticeships are designed to transmit. So seek problems to solve together and when it happens, relish it. Soak together in it. Do not waste it.

Ask Your Apprentice's Help

Find ways for your apprentice to mentor you. It requires a conscious effort, because you are so much more experienced. But seek her advice on your work. I ask Amalia and other protégés to critique my talks and articles.

Have your telemachus code-review your patches. He might not have useful comments, but when you trade places he sees what it is like to review a diff: he sees why it matters that a patch is clean, that it focuses on a single bug or feature, that it is well-described and coded simply. When you swap positions again, he will give you patches that are easy to review.

Success

I failed when I first supervised interns and I announced I would not supervise again. But the fall I spent working with Amalia changed my view: I can be an effective mentor, given the right conditions. So I designed an internship for the next summer with conditions optimized for success. I proposed that two interns revive Monary, a specialized driver for NumPy and MongoDB.

Monary was invented by David Beach, who wanted to load MongoDB data into NumPy arrays faster than PyMongo can do it. Once Monary did what David needed, he put the project down. I thought it was a nifty piece of software and I wanted to update it, fix bugs, add features, and make it easier to use. My proposal was accepted and I was assigned Matt Cotter and Kyle Suarez as interns for the summer.

Expert Mentor

I had most of the expertise required to supervise the project: Monary is a driver for MongoDB and Python, and I am an expert on that. But Monary is part-written in C. So I recruited Jason Carey, a master C programmer at MongoDB, to help.

Small Clear Goals

Monary is an established project of a well-known type: it is a MongoDB driver. Drivers have a list of features they can support, so Jason and I picked features from the list and added them to the project. Having small goals brought two advantages: it kept Matt and Kyle on track and made it quick to diagnose when they were stuck, and it also let us divide the work so we could evaluate their performance separately.

Guiding Visionary

Jason and I had ideas for Monary's future, but the visionary was its inventor, David Beach. David video-conferenced with our team regularly to guide our contributions. He was the ultimate reviewer of patches: Once Jason and I approved a patch, our interns submitted it to David.

Triumph

Winged Victory

Nike, goddess of victory, Kharkiv, Ukraine

Matt and Kyle triumphed. They made exceptional progress that summer adding features, improving the code, and writing documentation. They gave a talk about Monary at a Python conference. The interns' contributions enhanced MongoDB's edge among scientific Python programmers: MongoDB has the fastest path between the database and NumPy. David was grateful to have his project revived and promoted. We made full-time offers to both interns and they accepted.

My investment in Matt and Kyle paid me huge dividends. They rated my mentorship very highly, which contributed to my promotion the next year. My bosses were confident in my mentorship now, so I was assigned another talented new grad, Anna Herlihy. Just this week Anna completed her rotations and returned to work on our Python code long-term. Having Anna working with me gives me the time and flexibility to direct my future at MongoDB, and I am directing it toward taking over our C Driver. Because I worked on a C project with Jason Carey over the summer, I am better prepared to make this transition.

I had failed just a year earlier, but the Monary project was as successful as an internship can be. It paid off not just for the interns, but for everyone involved, especially me. No innate quality of mine changed: I simply learned to recognize success's prerequisites, and I put them in place.

Excuses

What if you believe that mentoring is important and you can succeed at it, but you still have excuses not to?

I Am An Introvert

Me too. Many programmers are. But introversion has nuances; it is not simply disliking people. In fact, introverts make good mentors. We prefer one-on-one interactions, we like predictability, we like to work toward goals with people instead of just hanging out. We are better listeners than gabby brainless extroverts are: when a novice joins a large company he needs someone who listens and understands him, who looks out for him. So our traits favor success. We just have to overcome our reluctance and make the effort.

I Cannot Explain What I Do

This is the implicit knowledge that apprenticeships transmit. Your telemachus has spent years in college learning everything that can be explained. It is up to you to teach what cannot be explained, by working together tightly.

I Am A Bad Mentor

It takes practice. You are prone to early failures, like me, and to improve slowly. These early failures are very personal and embarrassing, but forgive yourself. Next time, ensure the prerequisites for success.

Get a meta-mentor. I learned the hard way, so I want to make it easier for others: with future internships I plan to train a colleague in mentoring at the same time as I train interns. Shadow another mentor first, and make sure your company sets up apprentices with both an experienced and a first-time supervisor.

I Still Don't Care, Just Let Me Code

I thought the same. When I first agreed to supervise interns at MongoDB, it was reluctantly. Only because we had such a need for supervisors, and because my bosses told me it was required of senior staff, did I agree. My impatience with mentoring is one reason I let everyone down that summer. My view of myself was that writing code was the only part of my job that gave me joy. Everything else was a distraction I had to reject, to guard my job satisfaction.

Mentoring does take time from coding, it is an investment, and you must commit to it if you are to succeed. But when you succeed you may transform your self-image as I did.

Masters And Apprentices

Craftspeople have always learned by apprenticing with masters. The master-apprentice relationship is older than professions—older than humanity, even. Universities are a recent innovation, and they are no substitute for the original training method.

If you commit to developing yourself as a mentor, you will advance your career, help your young colleagues and your company succeed, and lay the foundation for a fairer future. For people who think they love only code, it can be a sacrifice to spend time guiding junior engineers, but the success of your apprentices will be among the great satisfactions of your professional life.

Athena building the Argo

Athena helps build the Argo, Roman, 1st Century CE

I'm PyDev Of The Week!

On "Mouse versus Python", Mike Driscoll profiles a Python developer each week; this week he features me. In the interview I talk about my rocky college education and my first days with Stackless Python 2.3, and I claim you're wasting your [...]

Mouse versus Python logo

On "Mouse versus Python", Mike Driscoll profiles a Python developer each week; this week he features me. In the interview I talk about my rocky college education and my first days with Stackless Python 2.3, and I claim you're wasting your life using vim.

Read "PyDev of the Week: A. Jesse Jiryu Davis" on Mouse Versus Python.

Zen at Sing Sing

February 7 marks the 10th anniversary of the Village Zendo's program at Sing Sing Correctional Facility, which I help coordinate. On the 7th, the Sing Sing group will hold an all-day meditation retreat, and practitioners at the Village [...]

Sing Sing

February 7 marks the 10th anniversary of the Village Zendo's program at Sing Sing Correctional Facility, which I help coordinate. On the 7th, the Sing Sing group will hold an all-day meditation retreat, and practitioners at the Village Zendo will be sitting in the Zendo in New York. Ryotan Sensei leads the program. He wrote the following reminiscence to mark the event:


In the Fall of 2004, the Village Zendo received a request from inmates at Sing Sing Correctional Facility for a teacher to come in and lead their Zen group. The group had disbanded in 2000 when they had lost their previous teacher, a priest sent out by Dai Bosatsu monastery in Livingston Manor, New York.

When we arrived at the facility, we discovered a zendo that had been frozen in time. Cushions, mats, musical instruments, and sutra books had been locked away in cabinets and had not been touched or moved for the past five years. When we walked in that day in February 2005, all we had to do was to lay down the mats, set up the altar, dust off the kesu, and strike the gong to begin the first period.

We sat in a room with a view of the Hudson River. Each week the floor was swept and mopped. The altar was a school desk with a couple of books piled up on it. We lay an altar cloth over the books and set a statue of Shakyamuni on top of them. The Catholics lent us a candle with a cross inscribed in its plastic covering. The flowers were artificial flowers that had been crafted by an inmate while waiting on death row.

A few months later Roshi arrived and we had an "Eye-Opening" ceremony at the new temple. We called it "Dharma Lotus Temple", combining the names of the two prison temples where our members had previously practiced: "Dharma Song", which was the old Sing Sing sangha of the '90s, and "Fire Lotus", the temple at Greenhaven Correctional Facility sponsored by Zen Mountain Monastery.

It is now 10 years since we walked into Sing Sing and unlocked the cabinets. Only one of the original sangha members remains but new aspirants have come and joined their spirit to the flowing stream of practice. Our Village Zendo volunteers have also changed over the years, each leaving their trace in that empty room looking out on the Hudson River.

When I think of their sincere practice and the aspiring men of Dharma Lotus Temple I get a catch in my throat. I pray that our practice may continue to flow with the Hudson that lies beyond these walls.


Ryotan

Ryotan Sensei.

Nashville

I'm visiting Nashville Tennessee for the first time! I'm going to eat hot chicken and do everything I highlighted in the Wildsam Guide. My girlfriend Jennifer is coming with me; she wants to be an extra on the "Nashville" show. I'm speaking [...]

Nashville

I'm visiting Nashville Tennessee for the first time! I'm going to eat hot chicken and do everything I highlighted in the Wildsam Guide. My girlfriend Jennifer is coming with me; she wants to be an extra on the "Nashville" show.

I'm speaking Thursday night, February 5, at the Nashville MongoDB User Group. I'll demo an application I built with MongoDB, Python, and NumPy to visualize historical weather data.

Saturday afternoon, Feburary 7, I'm giving a talk I'm excited about: "Dodge Disasters and March to Triumph as a Mentor" at the PyTennessee conference.

What's the best thing in Nashville? Tell me in the comments what I cannot omit from my trip.

Announcing PyMongo 2.8 and Motor 0.4

I'm delighted to announce that Bernie Hackett released PyMongo 2.8 yesterday, and I released Motor 0.4 today. They support features in the upcoming MongoDB release, which will be named MongoDB 3.0. PyMongo's changelog. Motor's [...]

Motor

I'm delighted to announce that Bernie Hackett released PyMongo 2.8 yesterday, and I released Motor 0.4 today. They support features in the upcoming MongoDB release, which will be named MongoDB 3.0.

More information about PyMongo 2.8 is in my post about the release candidate from November.

William Zola

I was organizing old negatives and came across these of William Zola, a brilliant MongoDB Support Engineer, in Palo Alto in 2013. William died this August. He taught me, and everyone else who worked for 10gen back then, how to think [...]

I was organizing old negatives and came across these of William Zola, a brilliant MongoDB Support Engineer, in Palo Alto in 2013. William died this August. He taught me, and everyone else who worked for 10gen back then, how to think critically about customer support issues, how to generate hypotheses and cull them efficiently. And not just that, but how to debug a computer problem over the phone, and how to ease a panicking customer's heart.

I think of him every time I hit a bug that I think might be too hard for me. He calms me down and shows me the next step.

We established The William Zola Outstanding Contributor Award to recognize our community members who support MongoDB users in the same spirit.


William Zola


William Zola and Bernie Hackett


William Zola