The CEO of Automattic on Holding “Auditions” to Build a Strong Team

Photography: Liz Caruana The Idea: In its early years, Automattic relied on traditional résumé screening and interviews to do its hiring. But over time Mullenweg came to focus on tryouts, in which final candidates are paid to spend several weeks working on a project. Automattic employs 230 people. We’re located in 170 cities all over the world. Although we maintain an office in San Francisco that functions like a coworking space, most of our employees work somewhere other than our home base. Some are even nomads, traveling from place to place and couch to couch. This might seem like a recipe for disaster—low productivity, poor management, and abuse. But Automattic is profitable and fast-growing. With an applicant-screening process that gives candidates genuine job responsibilities and relationships, we can hire smarter, retain strong employees longer, and reduce terminations and turnover. The key? Tryouts. Where 9 to 5 Fails It all starts with the way we think about work. In my experience, people in lots of companies are just going through the motions. If someone shows up in the morning dressed appropriately and isn’t drunk or asleep at his desk, we assume he’s working. If he’s making spreadsheets and to-do lists, we assume he’s working really hard. Unfortunately, none of this gets at what an employee actually creates during the day. It’s possible (and, sadly, not uncommon) for someone to sit at a desk for eight hours, moving papers and sending e-mail, without producing any results. At Automattic we focus on what you create, not whether you live up to some ideal of the “good employee.” We measure work according to outputs. I don’t care what hours you work. I don’t care if you sleep late or if you pick up a child at school in the afternoon. I don’t care if you spend the afternoon on the golf course and then work from 2 to 5 AM . What do you actually produce? Many people create great things without sitting at a desk in an office all day, including all the people at Automattic. (One employee recently wrote a book about his time with us. Its title— The Year Without Pants —should tell you something about our lack of emphasis on professional dress.) We work in open-source software, which is a decentralized product, so uncoupling our workforce from the 9-to-5 workday makes sense. We can do it well because we have the right team. We used to hire people the way most other companies do. We’d screen résumés and conduct interviews. Sometimes we asked candidates brainteaser-style questions, as Microsoft and Google famously did. We focused on their past experiences and paid special attention to what other start-ups they’d worked for. A candidate might interview with five employees, and we’d all take him or her out to lunch. We invested a lot of energy in the process, and we believed we were being as rigorous as we could. Inevitably, some of those hires didn’t work out, which was a big disappointment. When you hire someone, you see all the potential in the world. If things break down, you have to do some soul-searching. Is it my fault? Is it something I did as a manager? How did we miss the less-than-positive signs? Managers at other companies appeared not to care, assuming that it was normal for a third of new hires to wash out. But it seemed so inefficient. As we considered the situation, it became clear that we were being influenced by aspects of an interview—such as someone’s manner of speaking or behavior in a restaurant—that have no bearing on how a candidate will actually perform. Some people are amazing interviewees and charm everyone they talk to. But if the job isn’t going to involve charming others, their interview skills don’t predict how well they’ll do as employees. Just like work, interviews can be “performed” without real productivity. Tryouts Trump Interviews The more we thought about why some hires succeeded and some didn’t, the more we recognized that there is no substitute for working alongside someone in the trenches. So we gradually changed our approach. I still personally review most of the résumés we receive, and we screen out about 85% of candidates immediately, for lack of relevant experience, lack of technical skills, or mistakes on the application. (One thing I don’t pay much attention to is education: I’m a University of Houston dropout, so it would be hypocritical of me to obsess over where or whether someone went to college.) Candidates then have initial interviews, usually via online chat. Since March 2013 Automattic has received 314 applications, interviewed 155 candidates, run 94 tryouts, and hired 38 people The most significant shift we’ve made is requiring every final candidate to work with us for three to eight weeks on a contract basis. Candidates do real tasks alongside the people they would actually be working with if they had the job. They can work at night or on weekends, so they don’t have to leave their current jobs; most spend 10 to 20 hours a week working with Automattic, although that’s flexible. (Some people take a week’s vacation in order to focus on the tryout, which is another viable option.) The goal is not to have them finish a product or do a set amount of work; it’s to allow us to quickly and efficiently assess whether this would be a mutually beneficial relationship. They can size up Automattic while we evaluate them. Paying them for their efforts is important; this isn’t about getting work done free. Originally we tried to set hourly pay rates based on what they might earn if they were hired, but that became too complicated. We were almost negotiating what we would pay someone who hadn’t yet received an offer, which didn’t make any sense. To keep it simple, we decided to pay a standard $25 an hour, whether the candidate was hoping to be an engineer or the chief financial officer. If you’re applying to work in customer support, you’ll be talking directly to customers. If you’re an engineer, you’ll be writing real code. If you’re a designer, you’ll design. In some cases the nature of the job means that we have to get creative. If you’re looking for a role in business development, we can’t send you out to negotiate deals with potential partners, so we come up with something as closely related as possible to the actual work. A business-side hire might prepare presentations, or do an analysis of a business problem, or run numbers on a potential project. Tryouts may not offer 100% overlap with the job in question, but they give us a better window on someone’s skills and cultural fit than a lunch meeting would. We’re especially interested in how well candidates self-motivate, how well they communicate in writing (because most of us work remotely, we rely heavily on instant messaging), and how they deal with mistakes. We don’t expect perfection—we care more about how quickly they identify an error, how they communicate about it, and what they learn from it. One Manager’s Experience When Matt Mullenweg asked Scott Berkun, a veteran Microsoft executive and management author, to become a “team leader” at Automattic, he said yes—on one condition: He wanted to write a book about working in its unique corporate culture. Berkun, the author of The Year Without Pants: WordPress.com and the Future of Work, talked with HBR about the experience. What did you think of the job tryout process? I’m a huge fan, because it makes sense. Somehow hiring managers have come to believe that having people talk about work in a job interview is a reliable indicator of their ability to do work. It’s not. If you want a cake for your wedding, you’re going to ask the baker for a sample. You don’t ask him to talk about his cake. Tryouts are far better predictors of who can do the job and be a good fit. Did the tryouts give you, as a manager, more confidence in making hiring decisions? To some extent, yes. I like to talk about hiring decisions with my team. At Automattic, instead of talking about how a candidate replied to interview questions, we could give firsthand reports of how he responded to a particular problem. Firsthand data gave us more confidence. Should other companies use tryouts? Many already do, albeit under different names. An internship is very similar—it’s limited-term employment that you hope will lead to a long-term job. Apprenticeships have existed in artistic endeavors since the Renaissance. What in the culture was difficult to adapt to? I adjusted well to working remotely. Communicating digitally was easy: Even if you work in the same building, most of your dealings with colleagues are done via computer anyway. In general I really liked working at Automattic. My frustrations had to do with the level of ambition. The culture of the company is to collaborate and to make a lot of incremental changes. People had so much freedom to take risks, and I was disappointed by how few employees took advantage of that to make really big bets. What should other companies learn from Automattic? They could learn from its commitment to innovation. Executives like to talk about being innovative, but most people are really conservative. Matt Mullenweg is willing to try stuff out. He’s happy to see stuff not work—he’ll say, “Now we know a little more about how this problem can be solved.” If other companies adopted a model in which the CEO was the chief experimenter, they’d be more successful. The Final Interview (via Text) Only very rarely do people object to a tryout or say that they’re too busy to undertake one. The tryout may feel like an extraordinary obligation, but that makes it a filter: We want people who are willing to do what’s necessary to succeed and who are passionate enough about Automattic to make the process a priority. Tryouts aren’t like a bake-off; if we audition 10 people and they’re all strong, we may hire all of them. Applicants are competing against our standards of quality, not against one another. During a tryout, we try to provide a lot of feedback. If we conclude that an applicant isn’t going to succeed, we call an end to the process as quickly as possible, out of respect for everyone’s time. Sometimes a candidate decides to end the trial. We also prioritize coordinating the system. It’s a huge investment of time, and we’ve found that any single Automattic employee can oversee only two or three auditions at once. We now have four engineers who perform that service for their department. In our culture, overseeing a tryout takes higher priority than doing your regular job; it’s perfectly acceptable for people to temporarily step back from their usual work for that reason. When the process is over, all the people involved have a great sense of whether they want to work together going forward. At that point the final step is an interview with me. (Even as our company has grown, I continue to spend at least a third of my time on hiring.) I’ve changed my interview style dramatically over the years, to stay in the spirit of the tryout process. I conduct interviews via text-only Skype chats or instant messaging. I don’t know the gender or ethnicity of anyone I interview; I see only the words on the screen. It’s as close to a double-blind process as you can get. I’m looking mainly for passion and cultural fit. Of the people who make it to the final interview, 95% get a job offer—a testament to the effectiveness of our approach. Even when we don’t ultimately offer someone a position, he has obtained valuable information about what we viewed as his strengths and weaknesses. If we see potential, we’ll encourage him to continue developing his skills and reapply. Repeat applicants tend to be strong, because they’ve taken the feedback to heart and worked to improve; we’ve hired many of them. We’re In It for the Long Haul Trials don’t just help us better determine who will succeed at Automattic; they help potential employees avoid big problems down the road. If a working relationship sours after a few months, the company can just let the person go. It’s not the best outcome for anyone—we still have to deal with turnover costs—but for the employee, it’s terrible. She may have quit another job to accept the one she’s now losing. Maybe she has moved across the country. She’s left with a short-term job on her résumé, which can be a red flag for future employers. Trials also help potential employees avoid big problems down the road. When we hire someone at Automattic, we want the relationship to last for decades. That’s not the way most Silicon Valley companies operate. (See, for instance, “Tours of Duty: The New Employer-Employee Compact,” HBR June 2013.) But we feel strongly that the most productive relationships are formed over years, not months. If you’re making a movie, it might be reasonable to bring people together for the time required to film it and then move on. But if you’re trying to build products that will change the world, it’s going to take much longer, and there’s value in retaining people for a large chunk of their careers. Some people say that younger workers have a different notion of job tenure and want to move around more frequently. That may be, but I think companies, too, have changed. If a company treats people as if they’ll be around for a while, managers develop a different sensibility when it comes to investing in and retaining people. Loyalty is a two-way street. Overall, we end up hiring about 40% of the people who try out with us. In 2013 we hired 101 people, and only two of them didn’t work out. In the entire history of the company we’ve hired 270 people, and only 40 of them are no longer here. Our flexible-work philosophy is one reason we use this unique hiring process: We have to go to great lengths to identify people who will thrive in our culture. Outsiders have always questioned whether our system of limited supervision would be effective as we grew. “It’s fine when you’re at 10 or 15 people, but when you get to 30, it falls apart,” they’d say. After we passed 30 people, we started hearing that the magic number was 100—and then 150. Now that we’re approaching 250 employees, we still have no significant issues. There are some quirks in the way we work that tryouts help us address. For instance, we tend not to hire people with little professional experience, because our lack of structure may be too unfamiliar to them. But that doesn’t mean we never hire young people—we’ve acquired more than one talented teenager. (And I was 19 when I cofounded WordPress.) Many of our workers were self-employed or freelance at some point in their careers, which helps them understand how to be self-directed. Auditions may not be the right process for your workplace, but you probably could augment a traditional interview process in order to move beyond appearance to real potential. A candidate assessment that mimics the actual work the candidate must do to succeed will give you the best hiring ROI. Maybe it’s a trial; maybe it’s a presentation; maybe it’s a short-term assignment that can be done in an afternoon. Nothing you do for your company has as much impact as putting the right people around the table. The aphorism is true: You can’t manage your way out of a bad team. An approach to hiring that genuinely identifies real skill and fit will give you the best shot at assembling the right team.

The CEO of Automattic on Holding “Auditions” to Build a Strong Team

Summary.

Partner Center