Confirmation bias in an article about gender bias

Harvard Business Review reports on a study on gender bias in the workplace. Here’s the experimental design:

We decided to investigate whether gender differences in behavior drive gender differences in outcomes at one of our client organizations, a large multinational business strategy firm, where women were underrepresented in upper management. In this company, women made up roughly 35%–40% of the entry-level workforce but a smaller percentage at each subsequent level. Women made up only 20% of people at the fourth level (the second highest at this organization).

We collected email communication and meeting schedule data for 500 employees in one office, across all five levels of seniority, over the course of four months. We then gave 100 of these individuals sociometric badges, which allowed us to track in-person behavior. These badges, which look like large ID badges and are worn by all employees, record communication patterns using sensors that measure movement, proximity to other badges, and speech (volume and tone of voice but not content). They can tell us who talks with whom, where people communicate, and who dominates conversations.

We collected this data, anonymized it, and analyzed it. Although we were not able to see the identity of individuals, we still had data on gender, position, and tenure at the office, so we could control for these factors. To retain privacy, we did not collect the content of any communications, only the metadata (that is, who communicated with whom, at what time, and for how long).

It sounds like an interesting approach, and N=100 is respectable for what they were trying to accomplish.

A “large business strategy firm” might be representative of one type of work environment, but there are many others where women are allegedly discriminated against. I’m curious why the researchers chose this type of workplace over others, and whether they think discrimination against women happens in various types of workplaces for the same reasons.

Our analysis suggests that the difference in promotion rates between men and women in this company was due not to their behavior but to how they were treated.

I wonder if they did interviews with participants after the study was over, to generate qualitative data that would have supported the analysis. Nothing like that was reported so I guess they didn’t.

Bias, as we define it, occurs when two groups of people act identically but are treated differently.

This strikes me as a very flawed definition of bias. Bias connotes an unfair treatment, but people can be treated differently for more than just how they act, as the authors flat out admit:

Bias is not only about how behavior is perceived in the office, but also includes out-of-office expectations. At this company, women tend to leave the workforce between the third and fourth level of seniority, after having been at the company for four to 10 years. This timing presents another possible hypothesis: Perhaps women decide to leave the workplace for other reasons, such as wanting to raise a family. Our data can’t determine whether this is true or not, but we don’t think this changes the argument for reducing bias.

I agree this doesn’t change the argument for reducing bias as commonly understood, but it does change the argument for reducing bias as the authors define it.

If men and women are equal stakeholders in a family, they should presumably be leaving the workforce at the same rate. But this isn’t happening.

Here the authors confuse “equal” with “identical.” A man and woman can be equal stakeholders in a family but the husband fulfills his role by working hard to put food on the table and keep the bills paid, while the wife fulfills her role by keeping house and doing most of the day-to-day child-rearing. Their roles are not identical. The aforementioned pattern is in fact so established that it’s a cliche, and I’m puzzled why the authors feign ignorance about it. Maybe it’s because that pattern grows out of our natural sexual dimorphism, fighting against which is the essence of feminism.

Previous research has also shown that men are perceived as more responsible when they have children, while women are seen as being less committed to work.

Left unsaid is whether those perceptions are based in fact. It would be inconvenient to let facts get in the way of an agenda:

One way to [reduce bias in the workplace] is to make promotions and hiring more equal.

And there it is. Gotta love the circular logic there. In related advice, one way to be the top chess player in the world to make Magnus Carlsen knock over his king whenever the two of you play each other.

Significant research suggests that mandating a diverse slate of candidates helps companies make better decisions. A study by Iris Bohnet of Harvard Kennedy School showed that thinking about candidates in groups helped managers compare individuals by performance — but when managers evaluated candidates individually they fell back on gendered heuristics.

If you have a mixed barrel of apples and oranges, and you’re going through looking for the tastiest piece of fruit, it might be easier to systematically compare one apple to one orange rather than to just grab random pieces of fruit and evaluate their flavor one after the other. It seems like that’s what the study basically found.

Another potential problem lies in workload. In this company, we measured higher workloads as individuals advanced to higher levels of seniority. This isn’t intrinsically gendered, but many social pressures push women around this age to simultaneously balance work, family, and a disproportionate amount of housework. Companies may consider how to modify expectations and better support working parents so that they don’t force women to make a “family or work” decision.

Am I misunderstanding, or are the authors calling for companies to give women less work than men at the same seniority level? And they call this a solution to gender bias??

Companies need to approach gender inequality as they would any business problem: with hard data.

The problem is, anyone can find the hard data they need to support their argument. The important part isn’t just having the data, it’s in what data you collect, how you evaluate it, and whether you’re open to updating your initial views after you and the person arguing against you agree on the source of the data and the method of analysis.

These researchers collected data about the tone of conversations and people’s physical proximity to one another, but they didn’t cross-reference it with data from interviews that might have suggested whether the bias they thought they were seeing was really there. They also didn’t disclose whether their hypotheses changed as a result of the experiment. Just because you’re collecting data doesn’t mean you’re doing science.

Most programs created to combat gender inequality are based on anecdotal evidence or cursory surveys. But to tailor a solution to a company’s specific problems, you need to seek data to answer fundamental questions such as “When are women dropping out?” and “Are women acting differently than men in the office?” and “What about our company culture has limited women’s growth?” When organizations implement a solution, they need to measure the outcomes of both behavior and advancement in the office. Only then can they transition from the debate about the causes of gender inequality (bias versus behavior) and advance to the needed stage of a solution.

I like that these researchers have introduced another approach to measuring bias, and I like that they talk about reducing bias on an organizational rather than an institutional level. But I wish they’d have used multiple approaches together to get more reliable findings, and I wish the article had resisted the clickbaity impulse to give the impression, especially in the headline, that the findings they got were universally applicable.

By the way, can’t gender inequality be caused by bias and behavior? Pitting the two against each other as mutually exclusive seems extremely disingenuous to me. We actually can’t advance to the needed stage of a solution so long as people–including even professional researchers!–are engaging in this kind of bad-faith false-dichotomizing.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s