Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • COMMENT

AI can be sexist and racist — it’s time to make it fair

Computer scientists must identify sources of bias, de-bias training data and develop artificial-intelligence algorithms that are robust to skews in the data, argue James Zou and Londa Schiebinger.
  1. James Zou

    1. James Zou is assistant professor of biomedical data science and (by courtesy) of computer science and of electrical engineering, Stanford University, California, USA.

    View author publications

    You can also search for this author in PubMed  Google Scholar

  2. Londa Schiebinger

    1. Londa Schiebinger is the John L. Hinds Professor of History of Science and director of Gendered Innovations in Science, Health & Medicine, Engineering, and Environment, Stanford University, California, USA.

    View author publications

    You can also search for this author in PubMed  Google Scholar

Algorithms trained on biased data sets often recognize only the left-hand image as a bride. Credit: Left: iStock/Getty; Right: Prakash Singh/AFP/Getty

When Google Translate converts news articles written in Spanish into English, phrases referring to women often become ‘he said’ or ‘he wrote’. Software designed to warn people using Nikon cameras when the person they are photographing seems to be blinking tends to interpret Asians as always blinking. Word embedding, a popular algorithm used to process and analyse large amounts of natural-language data, characterizes European American names as pleasant and African American ones as unpleasant.

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Nature 559, 324-326 (2018)

doi: https://doi.org/10.1038/d41586-018-05707-8

References

  1. Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V. & Kalai, A. Adv. Neural Inf. Proc. Syst. 2016, 4349–4357 (2016).

    Google Scholar 

  2. Shankar, S. et al. Preprint at https://arxiv.org/abs/1711.08536 (2017).

  3. Esteva, A. et al. Nature 542, 115–118 (2017).

    Article  PubMed  Google Scholar 

  4. Schiebinger, L. et al. (eds) Gendered Innovations in Science, Health & Medicine, Engineering and Environment, Engineering, Machine Translation (2011–2015); see go.nature.com/2mopnkg

  5. Wagner, C., Garcia, D., Jadidi, M. & Strohmaier, M. Proc. 9th Int. AAAI Conf. Web Soc. Media 454–463 (2015).

    Google Scholar 

  6. Buolamwini, J. & Gebru, T. Proc. Mach. Learn. Res. 81, 1–15 (2018).

    Google Scholar 

  7. Gebru, T. et al. Preprint at https://arxiv.org/abs/1803.09010 (2018).

  8. Dwork, C., Hardt, M., Pitassi, T., Reingold, O. & Zemel, R. Proc. 3rd Innov. Theor. Comp. Sci. Conf. 2012, 214–226 (2012).

    Article  Google Scholar 

  9. Zemel, R., Wu, Y., Swersky, K., Pitassi, T. & Dwork, C. Proc. 30th Int. Conf. Int. Conf. Mach. Learn. 28, III-325–III-333 (2013).

    Google Scholar 

  10. Garg, N., Schiebinger, L., Jurafsky, D. & Zou, J. Proc. Natl Acad. Sci. USA 115, E3635–E3644 (2018).

    Article  PubMed  Google Scholar 

Download references

Subjects

Latest on:

Nature Careers

Jobs

  • Postdoctoral Position

    We are seeking highly motivated and skilled candidates for postdoctoral fellow positions

    Boston, Massachusetts (US)

    Boston Children's Hospital (BCH)

  • Qiushi Chair Professor

    Distinguished scholars with notable achievements and extensive international influence.

    Hangzhou, Zhejiang, China

    Zhejiang University

  • ZJU 100 Young Professor

    Promising young scholars who can independently establish and develop a research direction.

    Hangzhou, Zhejiang, China

    Zhejiang University

  • Head of the Thrust of Robotics and Autonomous Systems

    Reporting to the Dean of Systems Hub, the Head of ROAS is an executive assuming overall responsibility for the academic, student, human resources...

    Guangzhou, Guangdong, China

    The Hong Kong University of Science and Technology (Guangzhou)

  • Head of Biology, Bio-island

    Head of Biology to lead the discovery biology group.

    Guangzhou, Guangdong, China

    BeiGene Ltd.

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing

Search

Advanced search

Quick links