Hiring and collective intelligence

Home - About » Industry Blog - November 22, 2005 « Previous Entry - Next Entry
Computer Science
Research, Industry Work,
Programming
Community Service
Hillside Group, CHOOSE,
Stanford GSA
The Serious Side
Business School,
Learning Chinese
Humorous Takes
Switzerland, United States,
Software, Fun Photos
Travel Stories
Europe, United States, Asia
  
Living Places
Berlin (+ Gallery), Zürich
Boston, S.F. + Bay Area

Deciding to hire a person, at most companies, is a group decision process. Usually, the potential co-workers interview the candidate and get together afterwards to make up their collective mind. Most of the smarter companies in the software industry just sign off on the group's decision to make an offer. As such, the hiring decision is subject to common mistakes that can make a group substantially smarter or dumber than any single person.

There are different types of decision processes, some more complicated than others. Hiring, fortunately, often falls into the less complicated category. That's because you can aggregate the opinions of the different interviewers quite effectively, for example, by voting. Turning a decision process into an aggregation process requires independence, diversity, and decentralization of opinion. If these three conditions are met, a group decision is usually smarter than any particular individual's decision, as expert a person as that individual may be. At least statistically speaking.

If you find this surprising, just do the math. Let's use the well-known jelly bean counting example. You've seen it before: a competition to guess most accurately how many jelly beans there are in a jar. In making their guesses, some people will be way off, and some will be close to the real number, with the person coming closest winning the competition's prize.

The group decision here is the average of all people's guesses. If the aforementioned conditions of independence, diversity, and decentralization are in place, the group decision will be close to the real number. The error will get smaller and smaller as more participants enter the competition. Repeat the competition often enough, and the group decision will consistently outperform even the most experienced jelly bean guesser.

Why is that so? Well, mathematically speaking, increasing the number of people's guesses (the random sample) makes the margin of error smaller, or the "confidence interval" larger. The real question, however, is: Why is the guesses' average zeroing in on the real number rather than some other (random) number? This is where the preconditions of independence, diversity, and decentralization come in.

These three preconditions ensure that there is no systematic bias (or correlation) in the participants' guesses that might thwart the final result. Decentralization ensures that participants draw on widely differing local knowledge, diversity ensures that participants draw on widely different backgrounds of personal experience, and independence means that participants don't interfere with each others guesses.

Without a systematic bias, the average of the guesses will be (close to) the real number. While that may stand here as a pure assertion, observation in the real world has it that exactly this is what is happening. Experiments like the jelly bean guessing competition are examples of where these conditions hold up nicely. There are, of course, many situations, where we make group decisions, but where these conditions may not be in place. Which brings me back to hiring people.

How do you select for and set up a group of people to interview a candidate such that these preconditions are met and collective intelligence rather than collective stupidity emerges?

In terms of selecting the right people, I believe that decentralization means you need to get people with widely differing knowledge about the work situation; thus, rather than just choosing immediate peers, you should have substantially senior and junior people interview the candidate, ideally from different types of organizational units from within the company. This tails nicely with diversity, where you want very different kinds of people to interview the candidate so you can draw on their diverse body of experience.

In terms of the process, I believe it is important to decouple the individual interviews of an interview day so that no bias from a previous interviewer spills over to the next interviewer, sending the candidate too early on the winning or loosing street, just because the first interviewer got a particular impression. This is the independence criterion.

Also, it is rather smart, in my opinion, to pair up interviewers so that from the same interview slot, two different people can learn about the candidate. With one interviewer asking questions and interacting with the candidate and one interviewer observing, differing local knowledge is created and decentralization is improved.

The terms decentralization, independence, and diversity may seem overly academic here, but in their application to a given situation, they are not. And that application, I believe, is essential to an effective hiring process.

Copyright (©) 2007 Dirk Riehle. Some rights reserved. (Creative Commons License BY-NC-SA.) Original Web Location: http://www.riehle.org