Sorry Ladies… This Is Still a MAN’s World!
How algorithms feed gender bias
Fairness and Bias
So, by now you must have heard and perhaps forgotten about Amazon’s automated hiring tool that discriminates based on gender (Dastin, 2018). But just how deep does the gender bias rabbit hole really go within artificial intelligence? What other consequences exist besides women’s CVs being overlooked by the dreaded algorithm of a Fortune 500 company? This article will quickly probe a series of unfortunate events involving gender issues, with a view to exploring how incongruity within datasets influence algorithmic bias and what these implications might suggest for a particular group.
Rehashing the Amazon Story
A properly trained AI model is one that has learned to predict based on high-quality datasets that are both representative and inclusive (Buolamwini et al., 2018). However, this is clearly not the case in all circumstances as the data that trains an algorithm can overrepresent one group. This is evidenced in the Amazon debacle where their hiring model could identify patterns and make predictions based on a decade of applications that came largely from men and therefore made recommendations in their favour.
This incident highlights the importance of data gathering to the algorithmic training process: if training data is circumstantially and systematically grounded in bias while deep learning is occurring, then the outcome will negatively impact inclinations toward participation. In other words, if algorithms are fed a certain type of data over and over again, it will not miraculously and automatically include what it has not learned, thus resulting in exclusivity and a buttressing of bias amplifications within artificial intelligence, as evidenced in the above scenario – how many talented women were denied a chance based on Amazon’s sexist hiring tool?
To take it a step further, adaptive algorithms possess the ability to adjust its learning to facilitate new developments so as to make optimal decisions; this might therefore explain why Amazon’s hiring model remained in its biased loop (Lavanchy 2018). But what does this mean for the average Joe? I mean, Jill?
Algorithms are the result of human undertaking, from data gathering all the way up to model training, and this is the crux of the matter: a mere 22 percent of the professionals comprising this field are women (Young et al., 2021), which reinforces why underrepresentation can gravely impact outcomes. If those governing the machine learning process do not consider equitable and ethical features pertaining to gender dynamics when determining what and how much training data is needed to improve accuracy, the end-result will be outcomes that are unrepresentative and undesirable.
The Forbidden Fruit – Women Vs Apple
Now, imagine just getting over the fact that Amazon’s hiring tool did not even consider your application on the basis that you are a woman, only to be slapped with Apple’s gender discriminatory credit limits. In 2019, several complaints were made regarding Apple card’s seemingly misogynistic predilections; while investigations cleared the tech giant of any algorithmic ‘foul play’ – a thorough examination of nearly 400,000 applicants’ underwriting data was conducted (Shahien et al., 2021) – some very important questions are yet to be answered.
Who probed the algorithm? Was the training dataset examined for possible undetected bias? How was the data that taught the model collected and enriched? These are surely the most fundamental steps toward ensuring that the algorithm used to determine credit worthiness at Apple is actually gender-neutral and not merely sifting through some records to quickly placate the masses: this has nothing to do with the machine-driven program in question that prejudices one gender over another.
If the decision-making process for the development of algorithms is largely male-governed (D’Ignazio et al., 2020), then to what extent are the architects of machine intelligence systematically and voluntarily dismantling inherited bias present within the models they create so as to eliminate gender prejudice? What is the impetus to safeguard against the use of subjective data labels based on facile male female dichotomy? Training data that is replete with traditional notions of gender will forever produce statistical models that augment harmful ideological asymmetries, and the implications suggest the perpetuation of gender inequality (Leavy, 2018).
Bowdlerizing Bias – Equity in Gender Dynamics
One does not need to be exceedingly cerebral to perceive "the elephant in the server room" (D’Ignazio et al., 2020): biased AI models result from biased historical training data which results from biased human characteristics. To that end, if gender equity is to be achieved within algorithmic programs, then historical data that are representative and standardised should be used for model training purposes.
This necessitates a re-evaluation of the ways in which algorithms are developed to include thoughtful scrutiny of the data gathering process, careful curation of the data, as well as a thorough examination of human interactions, interventions and interference with training data before, during and after the deep learning process. Gebru et al. (2018) further propose that datasets be accompanied by specifications detailing their application, design, dissemination, rationale, framework, and continuation, among other things, with a view to augmenting accountability and transparency.
Bias greatly influences normative behaviour and societal principles; it must thus be eliminated in order to stymie the continued reinforcement of ideologies that serve to subjugate – social justice will never be enjoyed otherwise. Therefore, the conversation on gender bias in machine learning should be continually broadened so that best practices can be developed and features of artificial intelligence that support hegemonic power can be eliminated, in an effort to transition to a more democratic, inclusive and equitable digital economy.
Bibliography
BBC (2019) Apple’s ‘sexist’ credit card investigated by US regulator. Available online: https://www.bbc.co.uk/news/business-50365609 [Accessed 28/12/ 2021].
Buolamwini, J. & Gebru, T. (2018). Proceedings of Machine Learning Research 81:1–15. 2018 Conference on Fairness, Accountability, and Transparency Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. New York, 23–24 February, 1–15.
Dastin, Jeffrey (2018). Amazon scraps secret AI recruiting tool that showed bias against women. Available online: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G [Accessed 28/12/ 2021].
D’Ignazio, C. & Klein, L. (2020). Data Feminism. Cambridge: The MIT Press.
Gebru, T., Morgenstern, J., Vecchione, B., Wortman Vaughan, J., Wallach, H., Daumé III, H. & Crawford, K. (2018). Datasheets for Datasets. Available online: https://www.microsoft.com/en-us/research/uploads/prod/2019/01/1803.09010.pdf. [Accessed 2/1/2022].
Lambrecht, A and Tucker, C E (2019). Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads. Management Science, 65 (7). pp. 2966–2981.
Lavanchy, M (2018). Amazon’s sexist hiring algorithm could still be better than a human Expecting algorithms to perform perfectly might be asking too much of ourselves. Available online: https://www.imd.org/research-knowledge/articles/amazons-sexist-hiring-algorithm-could-still-be-better-than-a-human/ [Accessed 28/12/ 2021].
Leavy, S. (2018) Gender Bias in Artificial Intelligence: The Need for Diversity and Gender Theory in Machine Learning Susan Leavy. Available online: https://dl.acm.org/doi/pdf/10.1145/3195570.3195580?casa_token=01YpH_LOxi4AAAAA:G1bWwkOywH4OVoZgbvHtGgm_W5CxDRoT5UJGPzjPMBO0livdnxXDJpoOjlzzghvE-faHIehyMkd2eg. [Accessed 1/1/2022].
Nasiripour, S. & Farrel, G. (2021) Goldman Cleared of Bias in New York Review of Apple Card. Available online: https://www.bloomberg.com/news/articles/2021-03-23/goldman-didn-t-discriminate-with-apple-card-n-y-regulator-says. Accessed [1/1/2022].
Smith, G., & Rustagi, I. (2021). When Good Algorithms Go Sexist: Why and How to Advance AI Gender Equity. Stanford Social Innovation Review. https://doi.org/10.48558/A179-B138
Young, E., Wajcman, J. and Sprejer, L. (2021). Where are the Women? Mapping the Gender Job Gap in AI. Policy Briefing: Full Report. The Alan Turing Institute.
Share This Article
Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.
Write for TDS
