If a picture is worth a thousand words, what are those words saying about gender and age in the workplace when women are overwhelmingly portrayed as younger than men in online images? This skewing of reality can lead to bias in hiring, applying for jobs, and evaluating people and their roles. Solène Delecourt is a UC Berkeley Haas assistant professor of entrepreneurship & innovation whose work sits at the intersection of strategy and organizational behavior. Solène’s recently published paper in Nature has unsettling findings on the distortion of gender and age in online media and large language models. On this episode of The Culture Kit, Solène joins organizational culture experts Jenny Chatman and Sameer Srivastava to discuss why reliance on AI models and online searching might affect our expectations about how someone should look. They also discuss why her research has particular implications for hiring, the phenomenon’s potential to erode progress on gender and pay equality, and how leaders can actively counter it. *The Culture Kit with Jenny & Sameer is a production of Haas School of Business and is produced by University FM.*
If a picture is worth a thousand words, what are those words saying about gender and age in the workplace when women are overwhelmingly portrayed as younger than men in online images? This skewing of reality can lead to bias in hiring, applying for jobs, and evaluating people and their roles.
Solène Delecourt is a UC Berkeley Haas assistant professor of entrepreneurship & innovation whose work sits at the intersection of strategy and organizational behavior. Solène’s recently published paper in Nature has unsettling findings on the distortion of gender and age in online media and large language models.
On this episode of The Culture Kit, Solène joins organizational culture experts Jenny Chatman and Sameer Srivastava to discuss why reliance on AI models and online searching might affect our expectations about how someone should look. They also discuss why her research has particular implications for hiring, the phenomenon’s potential to erode progress on gender and pay equality, and how leaders can actively counter it.
*The Culture Kit with Jenny & Sameer is a production of Haas School of Business and is produced by University FM.*
[00:00:00] Jennifer Chatman: Hi, Sameer!
[00:00:00] Sameer Srivastava: Hey, Jenny!
[00:00:01] Jennifer Chatman: So, to start off today's episode, I want to ask you a question about perceptions of online content. You know, it's so incredibly pervasive in our lives these days. So, have you ever seen something online, maybe a restaurant review or a vacation rental, and then gotten to the place and the reality is completely different than the online image?
[00:00:23] Sameer Srivastava: Hmm. Vacation rental comes to mind. So, last summer, my family was traveling to Iceland, and we were getting an Airbnb, and we found one that looked great in the photos. Everything looked exactly as we wanted. Until we actually showed up at the place and realized that there were no window treatments on the many skylights in the property. And of course, this is the land of the midnight sun, so we got exactly no sleep that night that we stayed in that place.
[00:00:50] Jennifer Chatman: Oh, that sounds rough.
[00:00:52] Sameer Srivastava: It was.
[00:00:53] Jennifer Chatman: Well, today, we're going to explore how the images we see online can influence how people behave at work. Only, this time, the stakes are much higher than having a bad night's sleep or a crummy vacation rental. We're going to be talking about distortions around gender and age and the impact these distortions have on organizations.
[00:01:17] Sameer Srivastava: I'm Sameer Srivastava, a professor at UC Berkeley's Haas School of Business and co-founder of the Berkeley Center for Workplace Culture and Innovation.
[00:01:25] Jennifer Chatman: And I'm Jenny Chatman, a professor and dean at the Haas School. And this is The Culture Kit.
[00:01:33] Sameer Srivastava: Today, we're joined by Solène Delecourt, Assistant Professor of Entrepreneurship and Innovation here at UC Berkeley's Haas School of Business. Solène has recently published a paper in Nature, of all places, on the distortion of gender and age in online media and large language models. Her earlier research focuses on what drives variation in profits across firms and how we can reduce inequality in business performance among entrepreneurs in different market settings.
This more recent research identified many areas where reliance on AI models, or simply even our online searching, might affect our expectations about how someone should look, or perhaps even influence how we think about pursuing a new role if everyone appears different from us in age or gender, say a doctor or an astronaut.
We'll be talking about how this can affect organizations and erode some of the progress we've made towards gender equality and how leaders can actively counter it.
Solène, welcome to The Culture Kit.
[00:02:34] Solène Delecourt: Thank you so much. I'm so excited to be here today.
[00:02:37] Sameer Srivastava: Solène, I know gender inequality is a theme that runs through a lot of your research, but this particular study starts with a fundamental question, which is, are stereotypes, particularly those we find online, accurate reflections of reality, or are they distortions in some way? And you chose age and gender as the place to dig into that question. Why was that the right lens, in your mind? And what made it possible to actually study this question rigorously?
[00:03:05] Solène Delecourt: This is a very timeless question, whether stereotypes are accurate or if they're distorting reality. And what's very cool about age, in particular, is that it has an objective reality. It has an objective truth to it. If you know someone's birth date, you can measure their age. And there is no subjectivity about that. It's, sort of, very agreed upon.
So, the feature of being able to look at age in particular is that we can know if the representation is accurate or if it is distorted. And in that particular paper that was published in Nature that you were talking about, we look at how men and women are represented online in terms of age, and we find that women are over-represented as younger compared to men.
And that is the case for the average woman online, but that's not an accurate representation of reality at all. That is a distortion because we know that the average woman in the real world is, in fact, older than the average man. Women have a higher life expectancy than men, and so we know for a fact that the average woman in the real world is older. But online, if you look at text or images or videos, you would think that the average woman is younger than the average man, if you were just looking at the online content and trying to predict reality. So, that is a very cool feature of the data that allows us to look at this distortion.
[00:04:49] Sameer Srivastava: And to get a representation, you also looked across industries and years using census data. Can you say a little bit more about that?
[00:04:55] Solène Delecourt: Yes, absolutely. So, what I was just talking about is the average age for a woman, sort of, in the world or in society. But you could say, well, how about if we look at a given occupation? Maybe in a given occupation it is true that the average man is older than the average woman. And so, what we do in this research is we dig into very specific roles and occupations, and we can use census data to quantify for a given occupation, for example, what is the average age for a man in this occupation? What is the average age for a woman in this occupation? And we can compare that to the representation of that occupation online. And we, again, often find that there is a distortion, even looking within a given occupation.
[00:05:45] Jennifer Chatman: That's fascinating, Solène. I mean, one of the things that really jumped out in your finding is that the age and gender gap isn't evenly distributed across all jobs. Instead, it's most pronounced in certain kinds of roles, like those with higher status and higher earners. So, in other words, it appears that the distortion gets worse as you move up the ladder. So, what did you find there? And why does that pattern matter?
[00:06:12] Solène Delecourt: Yeah, that's a very good question. Initially, when I started working on this topic, I thought, "Oh, isn't it interesting how women face a lot of pressure to look a certain way, or maybe women face a lot of pressure to look younger?" And this was part of the impetus for studying this topic. And that's certainly a part of it.
But if it was purely a cosmetic issue, I don't think it would be very or as important as what we find. In fact, it goes way beyond, sort of, the cosmetic issue. What we find is that women appear younger, especially for the highest-status occupations. And so, it, kind of, in a way, primes people to think about who belongs in a given occupation, maybe who looks a part, you know, who has the authority and expertise to be in a certain high-status occupation. In addition to that, we find that the gap in representation of men and women in terms of age is greatest when the occupations have the largest pay gap. So, it's not just about status, it's also about pay inequality.
If you think about it, it's like, in the occupations where there is a highest stake for women to be represented a certain way, that's where we find the largest gap. And we'll probably talk about that more later, but we also ran an experiment, and long story short, we find that ChatGPT produces resumes for a female name as being younger and less experienced, everything else being held constant.
So, it's not just about status or pay, it's also about in the context of hiring, this inequality is most likely to play out.
[00:08:01] Jennifer Chatman: Wow. I mean, in some industries, the census actually shows women are older than men, but Google Images inverts that. So, what does it mean when the visual record contradicts the demographic reality?
[00:08:17] Solène Delecourt: Well, what that means is that it's not simply that the online content is, sort of, amplifying the reality, but it's just totally flipping the relationship, you know. It's not that maybe men are a little bit older than women in some occupations and the gap appears largest online. It is that it flips, you know, that sometimes women are actually older on average than men in a given occupation, but the online content just flips up that correlation, that relationship.
[00:08:49] Sameer Srivastava: So, give us a sense, Solène, of how problematic this is. So, how do tools like Google Image Search or ChatGPT amplify age-gender stereotypes? And how seriously do people seem to take these images? And what's the impact of consistently looking at images of women who are younger and images of men who are older for very similar kinds of roles?
[00:09:13] Solène Delecourt: We were very curious about that as well, you know, because at first we showed that there is this difference, but then we wondered, you know, who cares? Like, why does it matter that this difference exists? And so, we ran multiple experiments to really be able to show and quantify the effect of this difference.
The first experiment we ran with hundreds of people online, we showed them either Google images of a given occupation or we asked them to look for images of a given occupation or of unrelated images. And if people were searching for Google images of a given occupation, that shifted their personal beliefs in terms of, you know, we asked them to estimate the average age of a given woman in that occupation, and if people had searched for images on Google, they were more likely to estimate the average age of women to be younger than reality. And that was compared to a control group where people had looked at unrelated images.
So, you know, if you think about it, looking at these images is actually shifting personal beliefs, and that maybe relates to who belongs to that particular occupation. In addition, we asked them about hiring judgments, and we also found a correlation that older women might face a penalty in that case. And that was just after a few minutes. I mean, that was just an experiment where we asked people to search for Google images. And immediately after, that had already shifted their personal beliefs and hiring judgments.
[00:10:53] Sameer Srivastava: Yeah. So, I was going to say that this idea that you've described in the paper of gendered ageism, Jenny, it, kind of, reminds me of the paper that you and Laura Kray have on the evaluations of male and female faculty as they get older. Could you say a little bit about that paper?
[00:11:08] Jennifer Chatman: Well, the findings are actually consistent in many ways with Solène's work. We basically found that women were penalized as they rose in the hierarchy of organizations, that they were most discriminated against, I guess you could say, when they were in middle age rather than when they were younger or older. They were viewed as less threatening at those times.
And we actually found this both in archival data. So, women had the lowest teaching ratings of their careers when they were in middle age compared to themselves when they were both younger and older, which is a pretty crazy thing.
[00:11:53] Sameer Srivastava: You mean, they didn't get worse? They didn't get worse in middle age?
[00:11:55] Jennifer Chatman: No, you don't get worse at teaching over time. You actually get better. So, yeah, it was quite a robust effect. We found it in an experiment and we found it in a survey, as well as in this within-person archival data set.
So, Solène, I mean, one thing that makes this research particularly unsettling, I mean, there's a lot of things that are unsettling about the work, except that it's excellent work pointing out an unsettling problem, is the suggestion that this isn't just, like, static, that the bias is, kind of, self-reinforcing. And I wonder if you can help us understand how that self-reinforcing loop works.
[00:12:35] Solène Delecourt: Yes. And I don't have the full window and the full context on that. I can only tell you what I have observed and experienced. But I think this is a very, very widespread problem that we don't even fully grasp.
So, if you think about the bias online, I mean, first of all, it starts from us, humans. I mean, we are the ones putting up content that goes online, like, you know, we take pictures or we create images. We are the ones writing text and uploading text, and also the ones consuming it. So, if you think about it, the bias that's present is reflecting humans' behaviors and preferences. And once people have put content online, they are also the ones who consume the content.
Now, what happens in between, sort of, the production of content and the consumption of content is this black box of algorithms that would include search rankings, that would include language models. Like, there is a lot of filtering that happens, say, maybe humans think that for a high-status occupation an older man is a better representation than an older woman, for example.
Then maybe people are more likely to click on that content or to download that image that they're trying to use for a presentation. And as a result, maybe the search rankings updates and, sort of, amplifies that, oh, yes, when we're talking about a high-status occupation, we better have a picture of an older man because that would be what humans are later more likely to use or consume.
I mean, this is a little bit of an oversimplification, but the filtering by algorithm is not neutral. And the bias, sort of, gets encoded in algorithms and tools. As a result, the content itself becomes more biased. And you could even add more parts to that loop. You know, you could think, "Oh, well, now that the online content is more biased, that could also shift reality," you know. Like in the experiment, you know, we have shown that, immediately, after looking at images on Google, people shift their personal beliefs. So, maybe younger women think, "Well, I'll never be that person. You know, I'll never be this older man in a high-status occupation, so maybe this is not an occupation for me." And so, then maybe the reality becomes even more biased that, down the line, also affects the content.
[00:15:08] Sameer Srivastava: Yeah.
[00:15:08] Jennifer Chatman: Right. Well, and with organizations using AI tools, like ChatGPT, in their hiring processes, you know, they're using it to write job descriptions, generate candidate profiles, and even screen resumes, so, if a company is using AI to screen resumes or recruiters are constantly being fed these youthful images for a lower-level role or experienced images for senior roles, what does that mean?
[00:15:36] Solène Delecourt: Yes. Well, in fact, we were curious about this exact question, and we ran an experiment to really quantify what is the effect of using AI to screen resumes, which a lot of companies use these days. And we found that, when evaluating resumes, ChatGPT was assuming not just that women were younger or less experienced, but it also rated older male applicants as being higher quality.
And this is in an experiment where everything else was held constant about the resume. So, this is the most direct consequential application that we document in this paper, that in the context of hiring, if people rely on AI tools to screen resumes, that is likely to further impact the pattern we've shown, that older men get, sort of, an age premium that doesn't apply for women.
[00:16:31] Sameer Srivastava: So, given that the AI tools available to us, at least in the short term, are biased in the ways that you're describing, what advice do you have for managers, for leaders in companies who want to combat that bias and still take advantage of automation where possible in their evaluation processes?
[00:16:50] Solène Delecourt: Well, I don't have a magic bullet here that's going to solve that.
[00:16:56] Jennifer Chatman: Oh, darn, we were hoping for one.
[00:16:59] Solène Delecourt: I think the first step is awareness. And we show that in our paper. We quantify that. We measure it. And awareness not just that the tools are biased but that the bias is multimodal. It's not just in text. It's not just in images. It's also in videos. It's everywhere in online content. And all the content that people use online is not just text, but it's everywhere, and everywhere there is bias.
If anything, we show that gender bias might be stronger in images than in text. And in addition to that, bias is multifaceted. You know, what this paper shows is that, really, if you just look at gender, you're missing out the intersection between age and gender. And there are some very important implications in this case, especially for older women.
So, that would be the very first step. And by the way, in our paper, we, unfortunately, don't really have a bright spot, you know. Because we look at five different image platforms. We look at billions of words, millions of images. We use nine language models. If anything, the bias is remarkably consistent across these. So, it's not like it's one company's fault or that one platform is, you know, especially biased compared to everyone else. Like, it just seems to be the case everywhere.
Now, in addition to awareness, and this is not a solved problem, but there is this work that's starting to be done to try to de-bias algorithms. Now, that's very difficult. And part of that comes from the training dataset, or, like, what data do you even feed in the algorithm?
And at least, what we show in this paper is the training dataset is extremely biased. It's skewed and it has all that I talked about. It has multimodal bias. It has multifaceted bias. But I know actually at the Berkeley Artificial Intelligence Research Lab, they're trying to do research on de-biasing algorithms, and they have made some optimistic progress.
Again, it's not solved, but I know that there is a community of talented people, at least at UC Berkeley and elsewhere, doing some very important work and progress on this topic.
[00:19:11] Sameer Srivastava: It's nice to end on a note of optimism. And with that, we like to conclude our sessions, Solène, with some practical takeaways for organizational leaders. So, what do you want leaders to walk away with from this session?
[00:19:24] Solène Delecourt: The first thing that I would like to emphasize is that AI is not going to solve the bias issue by itself. If anything, bias is baked into AI, and it could even amplify the bias if left to its own devices. So, that would be the first thing.
Second, I think there is some important work that leaders can do in terms of not just awareness of bias, but auditing the bias. It's possible to measure and track who is adopting AI tools and what are the outcome of using these tools. So, for example, in the context of hiring, can you measure who is using what tool, and how does that affect the pool of people who are selected for a given position, for example? So, I think looking at tool usage and the outcome of using the tools would be the second step.
And finally, I think in this age where everything is changing very quickly and new tools are coming up, there is, maybe, more than ever, this need to design organizational policies around AI and bias. So, these would be my three takeaways from this paper.
[00:20:41] Jennifer Chatman: Those are incredibly helpful, Solène. Thank you so much for sharing your work with us. It's genuinely eye-opening. And we appreciate the importance of this research. Thank you.
[00:20:52] Sameer Srivastava: Thanks, Solène.
[00:20:53] Solène Delecourt: Thank you for having me. Thank you so much.
[00:20:57] Jennifer Chatman: Thanks for listening to The Culture Kit with Jenny & Sameer, a podcast dedicated to helping you build a strong and effective workplace culture.
[00:21:05] Sameer Srivastava: The Culture Kit Podcast is a production of the Berkeley Center for Workplace Culture and Innovation at the Haas School of Business and it’s produced by University FM. If you enjoyed the show, be sure to subscribe, leave us a review, and share this episode online so others who have workplace culture questions can find us, too!
[00:21:24] Jennifer Chatman: I'm Jenny.
[00:21:26] Sameer Srivastava: And I'm Sameer.
[00:21:27] Jennifer Chatman: We'll be back soon with more tools for your culture kit.