Vitaly Sharovatov

Generative AI is powerful. It’s also powered by bias that can be incredibly harmful.

A study of a text-to-image model revealed results that perpetuate troubling stereotypes. Take a look at this image. Prompts for images of workers in high-paying positions overwhelmingly showed light-skinned subjects, while the low-paying positions were dominated by darker-skinned subjects.

Keep in mind that these predictions are not based on actual people in these roles — they are based on what the AI thinks a person in those roles would look like.

This is just one of many data visualizations that showcase extreme bias from an AI-powered tool.

What’s perplexing is that when designing something as common as a work chair, creators consider how the chair could potentially impact the health of the end-user. Yet, when it comes to designing software that could be used by millions and whose data could be used by millions more, there seems to be less consideration for how that tool could negatively affect individuals or society at large.