Activity: Interview Analysis

Iris Bohnet thinks firms are wasting their money on diversity training. The problem is, most programs just don’t work. Rather than run more workshops or try to eradicate the biases that cause discrimination, she says, companies need to redesign their processes to prevent biased choices in the first place.

Bohnet directs the Women and Public Policy Program at the Harvard Kennedy School and cochairs its Behavioral Insights Group. Her new book, What Works, describes how simple changes—from eliminating the practice of sharing self-evaluations to rewarding office volunteerism—can reduce the biased behaviors that undermine organizational performance. In this edited interview with HBR senior editor Gardiner Morse, Bohnet describes how behavioral design can neutralize our biases and unleash untapped talent.

HBR: Organizations put a huge amount of effort into improving diversity and equality but are still falling short. Are they doing the wrong things, not trying hard enough, or both?

Bohnet: There is some of each going on. Frankly, right now I am most concerned with companies that want to do the right thing but don’t know how to get there, or worse, throw money at the problem without its making much of a difference. Many U.S. corporations, for example, conduct diversity training programs without ever measuring whether they work. My colleague Frank Dobbin at Harvard and many others have done excellent research on the effectiveness of these programs, and unfortunately it looks like they largely don’t change attitudes, let alone behavior. (See “Why Diversity Programs Fail” by Frank Dobbin.) I encourage anyone who thinks they have a program that works to actually evaluate and document its impact. This would be a huge service. I’m a bit on a mission to convince corporations, NGOs, and government agencies to bring the same rigor they apply to their financial decision making and marketing strategies to their people management. Marketers have been running A/B tests for a long time, measuring what works and what doesn’t. HR departments should be doing the same.

HBR: What would a diversity evaluation look like?

Bohnet: There’s a great classroom experiment that’s a good model. John Dovidio and his colleagues at Yale evaluated the effect of an antibias training program on first and second graders in 61 classrooms. About half the classrooms were randomly assigned to get four weeks of sessions on gender, race, and body type with the goal of making the children more accepting of others who were different from them. The other half didn’t get the training. The program had virtually no impact on the children’s willingness to share or play with others. This doesn’t mean you can’t ever teach kids to be more accepting—just that improving people’s inclination to be inclusive is incredibly hard. We need to keep collecting data to learn what works best.  So the point for corporations is to adopt this same methodology for any program they try. Offer the training to a randomly selected group of employees and compare their behaviors afterward with a control group. Of course, this would also mean defining success beforehand. For diversity training programs to go beyond just checking the box, organizations have to be serious about what they want to change and how they plan to evaluate whether their change program worked.  Diversity training programs largely don’t change attitudes, let alone behavior.

HBR: What does behavioral science tell us about what to do, aside from measuring success?

Bohnet: Start by accepting that our minds are stubborn beasts. It’s very hard to eliminate our biases, but we can design organizations to make it easier for our biased minds to get things right. HBR readers may know the story about how orchestras began using blind auditions in the 1970s. It’s a great example of behavioral design that makes it easier to do the unbiased thing. The issue was that fewer than 10% of players in major U.S. orchestras were women. Why was that? Not because women are worse musicians than men but because they were perceived that way by auditioners. So orchestras started having musicians audition behind a curtain, making gender invisible. My Harvard colleague Claudia Goldin and Cecilia Rouse of Princeton showed that this simple change played an important role in increasing the fraction of women in orchestras to almost 40% today. Note that this didn’t result from changing mindsets. In fact, some of the most famous orchestra directors at the time were convinced that they didn’t need curtains because they, of all people, certainly focused on the quality of the music and not whether somebody looked the part. The evidence told a different story.

HBR: So this is good news. Behavioral design works.

Bohnet: Yes, it does. The curtains made it easier for the directors to detect talent, independent of what it looked like. On the one hand, I find it liberating to know that bias affects everyone, regardless of their awareness and good intentions. This work is not about pointing fingers at bad people. On the other hand, it is of course also depressing that even those of us who are committed to equality and promoting diversity fall prey to these biases. I am one of those people. When I took my baby boy to a Harvard day care center for the first time a few years back, one of the first teachers I saw was a man. I wanted to turn and run. This man didn’t conform to my expectations of what a preschool teacher looked like. Of course, he turned out to be a wonderful caregiver who later became a trusted babysitter at our house—but I couldn’t help my initial gut reaction. I was sexist for only a few seconds, but it bothers me to this day. Seeing is believing. That is, we need to actually see counterstereotypical examples if we are to change our minds. Until we see more male kindergarten teachers or female engineers, we need behavioral designs to make it easier for our biased minds to get things right and break the link between our gut reactions and our actions.

HBR: What are examples of good behavioral design in organizations?

Bohnet: Well, let’s look at recruitment and talent management, where biases are rampant. You can’t easily put job candidates behind a curtain, but you can do a version of that with software. I am a big fan of tools such as Applied, GapJumpers, and Unitive that allow employers to blind themselves to applicants’ demographic characteristics. The software allows hiring managers to strip age, gender, educational and socioeconomic background, and other information out of résumés so they can focus on talent only. There’s also a robust literature on how to take bias out of the interview process, which boils down to this: Stop going with your gut. Those unstructured interviews where managers think they’re getting a feel for a candidate’s fit or potential are basically a waste of time. Use structured interviews where every candidate gets the same questions in the same order, and score their answers in order in real time. You should also be thinking about how your recruitment approach can skew who even applies. For instance, you should scrutinize your job ads for language that unconsciously discourages either men or women from applying. A school interested in attracting the best teachers, for instance, should avoid characterizing the ideal candidate as “nurturing” or “supportive” in the ad copy, because research shows that can discourage men from applying. Likewise, a firm that wants to attract men and women equally should avoid describing the preferred candidate as “competitive” or “assertive,” as research finds that those characterizations can discourage female applicants. The point is that if you want to attract the best candidates and access 100% of the talent pool, start by being conscious about the recruitment language you use.

HBR: What about once you’ve hired someone? How do you design around managers’ biases then?

Bohnet: The same principle applies: Do whatever you can to take instinct out of consideration and rely on hard data. That means, for instance, basing promotions on someone’s objectively measured performance rather than the boss’s feeling about them. That seems obvious, but it’s still surprisingly rare. Be careful about the data you use, however. Using the wrong data can be as bad as using no data. Let me give you an example. Many managers ask their reports to do self-evaluations, which they then use as part of their performance appraisal. But if employees differ in how self-confident they are—in how comfortable they are with bragging—this will bias the manager’s evaluations. The more self-promoting ones will give themselves better ratings. There’s a lot of research on the anchoring effect, which shows that we can’t help but be influenced by numbers thrown at us, whether in negotiations or performance appraisals. So if managers see inflated ratings on a self-evaluation, they tend to unconsciously adjust their appraisal up a bit. Likewise, poorer self-appraisals, even if they’re inaccurate, skew managers’ ratings downward. This is a real problem, because there are clear gender (and also cross-cultural) differences in self-confidence. To put it bluntly, men tend to be more overconfident than women—more likely to sing their own praises. One meta-analysis involving nearly 100 independent samples found that men perceived themselves as significantly more effective leaders than women did when, actually, they were rated by others as significantly less effective. Women, on the other hand, are more likely to underestimate their capabilities. For example, in studies, they underestimate how good they are at math and think they need to be better than they are to succeed in higher-level math courses. And female students are more likely than male students to drop courses in which their grades don’t meet their own expectations. The point is, do not share self-evaluations with managers before they have made up their minds. They’re likely to be skewed, and I don’t know of any evidence that having people share self-ratings yields any benefits for employees or their organizations.

HBR: But it’s probably not possible to just eliminate all managerial activities that allow biased thinking.

Bohnet: Right. But you can change how managers do these things. One message here is to examine whether practices that we thought were gender-neutral in fact lead to biased outcomes. Take the SAT, for example. Your score shouldn’t have been affected by whether you’re male or female. But it turns out it was. The test once penalized students for incorrect answers in multiple-choice questions. That meant it was risky to guess. Research by Katie Baldiga Coffman of Ohio State University shows that this matters, especially for women. Among equally able test takers, male students are more likely to guess, while female students are more likely to skip questions, fearing the penalty and thus ending up with lower scores. Katie’s research reveals that gender differences in willingness to take risk account for about half of the gender gap in guessing. An analysis of the fall 2001 mathematics SAT scores suggests that this phenomenon alone explains up to 40% of the gap between male and female students in SAT scores. The 2016 SAT has been redesigned so that it doesn’t penalize for incorrect answers. Taking risk out of guessing means that different appetites for risk taking will no longer affect students’ final scores. This can be expected to level the playing field for male and female students. Notice that the new SAT doesn’t focus on changing the students’ mindsets about risk but instead corrects for different risk tolerances. After all, the test is meant to measure aptitude, not willingness to take risk. Organizations should take a page from this book: Look around and see whether your practices by design favor one gender over the other and discourage some people’s ability to do their best work. Do meetings, for example, reward those most willing to hold forth? If so, are there meeting formats you can use that put everyone on an equal footing?

HBR: How can firms get started?

Bohnet: Begin by collecting data. When I was academic dean at the Harvard Kennedy School, one day I came to the office to find a group of students camped out in front of my door. They were concerned about the lack of women on the faculty. Or so I thought. Much to my surprise, I realized that it was not primarily the number of female faculty that concerned them but the lack of role models for female students. They wanted to see more female leaders—in the classroom, on panels, behind the podium, teaching, researching, and advising. It turns out we had never paid attention to—or measured—the gender breakdown of the people visiting the Kennedy School. So we did. And our findings resembled those of most organizations that collect such data for the first time: The numbers weren’t pretty. Here’s the good news. Once you collect and study the data, you can make changes and measure progress. In 1999, MIT acknowledged that it had been unintentionally discriminating against female faculty. An examination of data had revealed gender differences in salary, space, resources, awards, and responses to outside offers. The data had real consequences. A follow-up study, published in 2011, showed that the number of female faculty in science and engineering had almost doubled, and several women held senior leadership positions. Companies can do their own research or turn to consultants for help. EDGE, where I serve as a scientific adviser, is a Swiss foundation and private company that helps organizations across the sectors measure how well they do in terms of gender equality. A firm named Paradigm is another. I came across it when I was speaking with tech firms in Silicon Valley and San Francisco. It helps companies diagnose where the problems are, starting by collecting data, and then come up with possible solutions, often based on behavioral designs.

HBR: You said that “seeing is believing.” But given the lack of senior female role models in organizations, what else can we do?

Bohnet: About a decade ago we noticed that of all the portraits of leaders on the walls of the Kennedy School, exactly zero were of women. The portraits we display affect what our employees and our students believe possible for themselves. I can attest that it was not our intention to signal to fully half of our students that they were not made to be leaders. Rather, this was done unthinkingly. Since then we have added new portraits, including Ida B. Wells, the U.S. civil rights activist and suffragist, and Ellen Johnson Sirleaf, the president of Liberia, winner of the Nobel Peace Prize, and a graduate of the Kennedy School.

HBR: You argue that it’s often a waste of time to try to debias people—but hanging portraits of women seems like a strategy to actually change individuals’ perceptions.

Bohnet:I am not arguing that mindsets can never change. But what we generally find is for beliefs to change, people’s experiences have to change first. Being surrounded by role models who look like you can affect what you think is possible for people like you. Sapna Cheryan of the University of Washington, for example, has shown that decorations in a computer science classroom can affect performance. Replacing the male-dominated Star Wars and Star Trek images with gender-neutral art and nature pictures strengthened female students’ associations between women and careers in computer science. In another study, women who were shown a picture of Hillary Clinton or Angela Merkel before giving a public speech did objectively better than those who were shown a picture of Bill Clinton or no picture at all. So what do we do with our boardrooms and hallways that celebrate our (male focused) history? When asked this question at a recent talk I gave at the Organization for Economic Cooperation and Development, I answered that, sometimes, we have to “hurry history.” I think that presidents John and John Quincy Adams, spouse and son of the thought leader and First Lady, Abigail Adams, would be proud that her portrait now is on Harvard’s walls—and of course, its presence makes a big difference to our female students.

HBR: Men may resist organizational changes favoring women because they view gender equality as zero sum—if women win, men lose. How then do you enlist men as agents of change?

Bohnet: Few men oppose the idea of benefiting from the entire talent pool—at least in theory. But some are concerned about actually leveling the playing field. In practice, of course, the blind auditions in orchestras have increased competition for male musicians. And the inclusion of women affects competition for men in all jobs. I understand that increased competition can be painful, but I am too much of an economist to not believe in the value of competition. There is no evidence that protectionism has served the world well. Enlisting men is partly about helping them to see the benefits of equality. Fathers of daughters are some of the strongest proponents of gender equality, for obvious reasons, so they can be particularly powerful voices when it comes to bringing other men along. Research on male CEOs, politicians, and judges shows that fathers of daughters care more about gender equality than men without children or with only sons. I would urge fathers of daughters to be outspoken in their own organizations and to advocate for equality not just as a broad goal, but to actively help drive the changes I describe here—collecting baseline organizational data, promoting experiments, measuring what works, changing processes to limit the impact of our biased minds and level the playing field, and so on. A big part is, simply, continued awareness building—not just of the problem but also of the solutions available to organizations. I recently gave a talk on Wall Street to an audience that was male. I started by inviting people with children to raise their hands. Then I asked those with daughters to raise their hands. Many hands were up. I told them that this made my job easy as some of my biggest allies were in the room. It broke the ice, especially when I told the audience that my husband and I only have sons—who are great feminists, I might add, and in small ways have already brought behavioral insights to their school by reminding the principal to refer to teachers in general as both “he” and “she.”