Algorithms are Propagating Bias—Are We Complicit?
As algorithms continue to determine more and more of what we interact with on the Internet every day, questions arise as to the effectiveness, ethicality, and impartiality of these same algorithms. It has been well documented to this point that since algorithms draw from large banks of information generated by humans, they tend to propagate the very same biases that humans do, especially when it comes to gender stereotypes.
What is less clear in this process, however, is how people play a complicit role in accepting and reinforcing the algorithmic biases presented to them in the forms of targeted advertising, content recommendations, and more on the Internet. In our research, we undertake a set of studies to determine to what extent algorithms are presenting biased recommendations and further, to what extent people are going along with those biased choices and reinforcing the algorithms’ bias.
The Study
In the first of our studies, we undertook to uncover the extent to which gender bias is propagated by online algorithms. To do this, we compiled a list of 59 customer psychographic attributes, separated into positive (helpful, reasonable, independent, etc.) and negative (unfriendly, unkind, rigid, etc.) groups. Equipped with this set of words, we referenced an algorithm using word embeddings pre-trained on the Common Crawl text corpus, which consists of billions of webpages. By using this word embeddings algorithm, we were able to create a similarity/dissimilarity vector when compared to gendered terms (e.g. he, she, her, him, etc.) for each of the words in our list, which revealed that algorithms uncover gender bias associated with the psychographic attributes in the body of text on which they were trained.
Our first exploratory study revealed that algorithms learning from the Common Crawl text corpus learn significant biases, associating men with positive psychographic attributes at a significantly higher rate than women, and associating women with negative attributes at a significantly higher rate than men.
Our second study sought to understand how gender bias within computer algorithms affects the set of options that consumers are presented with when engaging online, especially while shopping or being targeted by ads. In our study, we focused on product search recommendations on shopping portals to measure the effect. We recruited 87 participants from TurkPrime (46 women, 41 men) who went on to create entirely new accounts on an online shopping platform. From there, we had them search for a list of different products. Participants then uploaded screenshots of the first 20 product recommendations that were served to them and picked one item they liked the most.
In the second phase of this study, we had a separate set of 140 raters go through each of the consideration sets for the men and women and evaluate the degree to which each product recommendation aligned with positive or negative psychographic (such as rational, emotional, industrious, lazy, etc.) characteristics. This was used to create an average of the positivity or negativity of products served to our male and female participants.
Our second study confirmed this finding, with the product set delivered to our female participants being negatively biased compared to that delivered to our male participants. We found through this study as well that Google delivered more negatively biased results than Bing through the same methods. The choices made by the participants as to their preferred product from the selection set also served to reinforce the bias, with women choosing more negatively associated products.
Our third study aimed to address the degree to which consumers play a role in confirming and perpetuating the gender stereotypes found in algorithmic recommendations. To test for this, we partnered with an astrologer who was looking to expand online marketing efforts. We ran ads on Facebook for the astrologer, with two versions reflecting either a positive or negative psychographic attribute, but otherwise not targeted towards either gender in any other way. We then looked at the delivery rate for each ad to men and women, as well as the clickthrough rate for each among men and women.
Our third study revealed much the same as the other two, while expanding on the role that consumers play in perpetuating online biases. We found that the ads with negative psychographic attributes were delivered to women at a higher rate (38.9%) than the positive ad (36.5%). Furthermore, when running the campaign under clickthrough optimization objectives, the negative ad was delivered to a much greater proportion of women (70.5%) than the positive counterpart (54.8%). This suggests that men and women are cooperating with the gender biases presented to them by algorithms and furthering their effects through online interactions.
Real Estate Implications
While our research was not specifically focused on the field of real estate, our findings are applicable and important for anyone advertising online or engaging with advertisements online. If you are using social media, paid search advertising, or any other means of algorithm-driven content delivery to promote your business, it is helpful to be aware of how your choice of words and phrases can affect who your message reaches. Using more negative psychographic attributes in marketing has been shown to reach more women than men, while using more positive psychographic attributes has been shown to reach more men than women. In order to reduce the bias present in your ads’ deliveries, consider using neutral language whenever possible or targeting women specifically with positive attributes in your ads. In addition, from a consumer perspective, it is important to be aware of these trends in order to reject biases that are presented to us and mitigate content to shift algorithms away from continuing to propagate bias.
. . . . . . . . . . . . . . . . . . .
Recommended Reading
Rathee, Shelly, Sachin Banker, Arul Mishra, and Himanshu Mishra (2023), “Algorithms Propagate Gender Bias in the Marketplace – with Consumers’ Cooperation,” Journal of Consumer Psychology, 33, 621-631. https://doi.org/10.1002/jcpy.1351
. . . . . . . . . . . . . . . . . . .
About the Authors
Shelly Rathee, PhD
Director of Laboratory for the Advancement of Interdisciplinary Research and The Diana & James Yacobucci ’73 Assistant Professor, Marketing & Business Law, Villanova University
Dr. Shelly Rathee (PhD – University of Utah) teaches in the areas of marketing analytics. Her research interests include numerosity and magnitude perception, gender bias and demographic perception, marketing analytics, and language and linguistic content. Her research has been published in the Journal of Consumer Psychology, Management Science, Psychology & Marketing, International Journal of Advertising, Journal of Business Research, and more. In addition to her own research and teaching endeavors, Dr. Rathee has served on the editorial review boards of several publications, including the Journal of Advertising Research and Journal of Public Policy & Marketing.
Sachin Banker, PhD
Assistant Professor in Marketing, David Eccles Emerging Scholar, University of Utah
Dr. Sachin Banker (PhD – MIT Sloan School of Management) teaches courses in marketing research. His research interests focus on understanding how emerging technologies influence consumer decision making by applying methods from behavioral economics, neuroeconomics, and computational social science. His research has been published in journals such as the Journal of Consumer Psychology, Frontiers in Psychology, Journal of Business Research, and many others. Prior to joining the University of Utah, Dr. Banker completed postdoctoral research at Princeton University’s Woodrow Wilson School of Public and International Affairs.
Arul Mishra, PhD
Emma Eccles Jones Presidential Chair Professor, University of Utah
Dr. Arul Mishra (PhD – University of Iowa) focuses her research on understanding different aspects of a person’s decision-making process, specifically diving into the topics of consumer decision-making, behavioral promotions, risk perception and financial decision-making. Her research has appeared in Journal of Marketing Research, Journal of Consumer Research, Journal of Marketing, Marketing Science, Management Science, Organizational Behavior and Human Decision Processes, and Psychological Science. In addition, Dr. Mishra has received awards such as the Journal of Consumer Research’s Outstanding Reviewer Award.
Himanshu Mishra, PhD
David Eccles Professor of Marketing, University of Utah
Dr. Himanshu Mishra (PhD – University of Iowa) current research interests are broadly in the area of computational social science. He uses machine learning techniques to understand social and marketplace decisions made by people. The insights he derives from unstructured data are applied in his many collaborations with firms. His research has implications for consumer decision-making, AI and fairness, risk assessment, and well-being. Dr. Mishra’s research has appeared in numerous leading journals of marketing, business, and psychology such as the Journal of Marketing Research, Journal of Consumer Research, Psychological Science, Journal of Personality and Social Psychology, Journal of Marketing, Marketing Science, Management Science, Organizational Behavior and Human Decision Processes etc. His work has been featured in a variety of media outlets such as MSNBC, Wall Street Journal, Scientific America, NPR, SmartMoney, CBS, The New York Times, Washington Post, Newsweek.
. . . . . . . . . . . . . . . . . . .
Download this Article
KCRR 2025 March - Algorithms are Propagating Bias—Are We Complicit? (Rathee)
