Connect with us

The Conversation

I unintentionally created a biased AI algorithm 25 years ago – tech companies are still making the same mistake

Published

on

I unintentionally created a biased AI algorithm 25 years ago – tech companies are still making the same mistake

Facial recognition software misidentifies Black women more than other people.
JLco – Ana Suanes/iStock via Getty Images

John MacCormick, Dickinson College

In 1998, I unintentionally created a racially biased artificial intelligence algorithm. There are lessons in that story that resonate even more strongly today.

The dangers of bias and errors in AI algorithms are now well known. Why, then, has there been a flurry of blunders by tech companies in recent months, especially in the world of AI chatbots and image generators? Initial versions of ChatGPT produced racist output. The DALL-E 2 and Stable Diffusion image generators both showed racial bias in the pictures they created.

My own epiphany as a white male computer scientist occurred while teaching a computer science class in 2021. The class had just viewed a video poem by Joy Buolamwini, AI researcher and artist and the self-described poet of code. Her 2019 video poem “AI, Ain’t I a Woman?” is a devastating three-minute exposé of racial and gender biases in automatic face recognition systems – systems developed by tech companies like Google and Microsoft.

The systems often fail on women of color, incorrectly labeling them as male. Some of the failures are particularly egregious: The hair of Black civil rights leader Ida B. Wells is labeled as a “coonskin cap”; another Black woman is labeled as possessing a “walrus mustache.”

Echoing through the years

I had a horrible déjà vu moment in that computer science class: I suddenly remembered that I, too, had once created a racially biased algorithm. In 1998, I was a doctoral student. My project involved tracking the movements of a person’s head based on input from a video camera. My doctoral adviser had already developed mathematical techniques for accurately following the head in certain situations, but the system needed to be much faster and more robust. Earlier in the 1990s, researchers in other labs had shown that skin-colored areas of an image could be extracted in real time. So we decided to focus on skin color as an additional cue for the tracker.

a color video frame showing a young man entering a room with a red curve overlaying the image outlining his head
The author’s 1998 head-tracking algorithm used skin color to distinguish a face from the background of an image.
Source: John MacCormick, CC BY-ND

I used a digital camera – still a rarity at that time – to take a few shots of my own hand and face, and I also snapped the hands and faces of two or three other people who happened to be in the building. It was easy to manually extract some of the skin-colored pixels from these images and construct a statistical model for the skin colors. After some tweaking and debugging, we had a surprisingly robust real-time head-tracking system.

Not long afterward, my adviser asked me to demonstrate the system to some visiting company executives. When they walked into the room, I was instantly flooded with anxiety: the executives were Japanese. In my casual experiment to see if a simple statistical model would work with our prototype, I had collected data from myself and a handful of others who happened to be in the building. But 100% of these subjects had “white” skin; the Japanese executives did not.

Miraculously, the system worked reasonably well on the executives anyway. But I was shocked by the realization that I had created a racially biased system that could have easily failed for other nonwhite people.

Privilege and priorities

How and why do well-educated, well-intentioned scientists produce biased AI systems? Sociological theories of privilege provide one useful lens.

Ten years before I created the head-tracking system, the scholar Peggy McIntosh proposed the idea of an “invisible knapsack” carried around by white people. Inside the knapsack is a treasure trove of privileges such as “I can do well in a challenging situation without being called a credit to my race,” and “I can criticize our government and talk about how much I fear its policies and behavior without being seen as a cultural outsider.”

In the age of AI, that knapsack needs some new items, such as “AI systems won’t give poor results because of my race.” The invisible knapsack of a white scientist would also need: “I can develop an AI system based on my own appearance, and know it will work well for most of my users.”

YouTube video
AI researcher and artist Joy Buolamwini’s video poem ‘AI, Ain’t I a Woman?’

One suggested remedy for white privilege is to be actively anti-racist. For the 1998 head-tracking system, it might seem obvious that the anti-racist remedy is to treat all skin colors equally. Certainly, we can and should ensure that the system’s training data represents the range of all skin colors as equally as possible.

Unfortunately, this does not guarantee that all skin colors observed by the system will be treated equally. The system must classify every possible color as skin or nonskin. Therefore, there exist colors right on the boundary between skin and nonskin – a region computer scientists call the decision boundary. A person whose skin color crosses over this decision boundary will be classified incorrectly.

Scientists also face a nasty subconscious dilemma when incorporating diversity into machine learning models: Diverse, inclusive models perform worse than narrow models.

A simple analogy can explain this. Imagine you are given a choice between two tasks. Task A is to identify one particular type of tree – say, elm trees. Task B is to identify five types of trees: elm, ash, locust, beech and walnut. It’s obvious that if you are given a fixed amount of time to practice, you will perform better on Task A than Task B.

In the same way, an algorithm that tracks only white skin will be more accurate than an algorithm that tracks the full range of human skin colors. Even if they are aware of the need for diversity and fairness, scientists can be subconsciously affected by this competing need for accuracy.

Hidden in the numbers

My creation of a biased algorithm was thoughtless and potentially offensive. Even more concerning, this incident demonstrates how bias can remain concealed deep within an AI system. To see why, consider a particular set of 12 numbers in a matrix of three rows and four columns. Do they seem racist? The head-tracking algorithm I developed in 1998 is controlled by a matrix like this, which describes the skin color model. But it’s impossible to tell from these numbers alone that this is in fact a racist matrix. They are just numbers, determined automatically by a computer program.

a matrix of numbers in three rows and four columns
This matrix is at the heart of the author’s 1998 skin color model. Can you spot the racism?
Source: John MacCormick, CC BY-ND

The problem of bias hiding in plain sight is much more severe in modern machine-learning systems. Deep neural networks – currently the most popular and powerful type of AI model – often have millions of numbers in which bias could be encoded. The biased face recognition systems critiqued in “AI, Ain’t I a Woman?” are all deep neural networks.

The good news is that a great deal of progress on AI fairness has already been made, both in academia and in industry. Microsoft, for example, has a research group known as FATE, devoted to Fairness, Accountability, Transparency and Ethics in AI. A leading machine-learning conference, NeurIPS, has detailed ethics guidelines, including an eight-point list of negative social impacts that must be considered by researchers who submit papers.

Who’s in the room is who’s at the table

On the other hand, even in 2023, fairness can still be the victim of competitive pressures in academia and industry. The flawed Bard and Bing chatbots from Google and Microsoft are recent evidence of this grim reality. The commercial necessity of building market share led to the premature release of these systems.

The systems suffer from exactly the same problems as my 1998 head tracker. Their training data is biased. They are designed by an unrepresentative group. They face the mathematical impossibility of treating all categories equally. They must somehow trade accuracy for fairness. And their biases are hiding behind millions of inscrutable numerical parameters.

So, how far has the AI field really come since it was possible, over 25 years ago, for a doctoral student to design and publish the results of a racially biased algorithm with no apparent oversight or consequences? It’s clear that biased AI systems can still be created unintentionally and easily. It’s also clear that the bias in these systems can be harmful, hard to detect and even harder to eliminate.

These days it’s a cliché to say industry and academia need diverse groups of people “in the room” designing these algorithms. It would be helpful if the field could reach that point. But in reality, with North American computer science doctoral programs graduating only about 23% female, and 3% Black and Latino students, there will continue to be many rooms and many algorithms in which underrepresented groups are not represented at all.

That’s why the fundamental lessons of my 1998 head tracker are even more important today: It’s easy to make a mistake, it’s easy for bias to enter undetected, and everyone in the room is responsible for preventing it.The Conversation

John MacCormick, Professor of Computer Science, Dickinson College

This article is republished from The Conversation under a Creative Commons license. Read the original article.

The Conversation

Measles can ravage the immune system and brain, causing long-term damage – a virologist explains

Published

on

theconversation.com – Peter Kasson, Professor of Chemistry and Biomedical Engineering, Georgia Institute of Technology – 2025-03-31 07:16:00

Measles infections send 1 in 5 people to the hospital.
wildpixel/ iStock via Getty Images Plus

Peter Kasson, Georgia Institute of Technology

The measles outbreak that began in west Texas in late January 2025 continues to grow, with 400 confirmed cases in Texas and more than 50 in New Mexico and Oklahoma as of March 28.

Public health experts believe the numbers are much higher, however, and some worry about a bigger resurgence of the disease in the U.S. In the past two weeks, health officials have identified potential measles exposures in association with planes, trains and automobiles, including at Washington Dulles International Airport and on an Amtrak train from New York City to Washington, D.C. – as well as at health care facilities where the infected people sought medical attention.

Measles infections can be extremely serious. So far in 2025, 14% of the people who got measles had to be hospitalized. Last year, that number was 40%. Measles can damage the lungs and immune system, and also inflict permanent brain damage. Three in 1,000 people who get the disease die. But because measles vaccination programs in the U.S. over the past 60 years have been highly successful, few Americans under 50 have experienced measles directly, making it easy to think of the infection as a mere childhood rash with fever.

As a biologist who studies how viruses infect and kill cells and tissues, I believe it is important for people to understand how dangerous a measles infection can be.

Underappreciated acute effects

Measles is one of the most contagious diseases on the planet. One person who has it will infect nine out of 10 people nearby if those people are unvaccinated. A two-dose regimen of the vaccine, however, is 97% effective at preventing measles.

When the measles virus infects a person, it binds to specific proteins on the surface of cells. It then inserts its genome and replicates, destroying the cells in the process. This first happens in the upper respiratory tract and the lungs, where the virus can damage the person’s ability to breathe well. In both places, the virus also infects immune cells that carry it to the lymph nodes, and from there, throughout the body.

YouTube video
Measles can wipe out immune cells’ ability to recognize pathogens.

What generally lands people with measles in the hospital is the disease’s effects on the lungs. As the virus destroys lung cells, patients can develop viral pneumonia, which is characterized by severe coughing and difficulty breathing. Measles pneumonia afflicts about 1 in 20 children who get measles and is the most common cause of death from measles in young children.

The virus can directly invade the nervous system and also damage it by causing inflammation. Measles can cause acute brain damage in two different ways: a direct infection of the brain that occurs in roughly 1 in 1,000 people, or inflammation of the brain two to 30 days after infection that occurs with the same frequency. Children who survive these events can have permanent brain damage and impairments such as blindness and hearing loss.

Yearslong consequences of infection

An especially alarming but still poorly understood effect of measles infection is that it can reduce the immune system’s ability to recognize pathogens it has previously encountered. Researchers had long suspected that children who get the measles vaccine also tend to have better immunity to other diseases, but they were not sure why. A study published in 2019 found that having a measles infection destroyed between 11% and 75% of their antibodies, leaving them vulnerable to many of the infections to which they previously had immunity. This effect, called immune amnesia, lasts until people are reinfected or revaccinated against each disease their immune system forgot.

Occasionally, the virus can lie undetected in the brain of a person who recovered from measles and reactivate typically seven to 10 years later. This condition, called subacute sclerosing panencephalitis, is a progressive dementia that is almost always fatal. It occurs in about 1 in 25,000 people who get measles but is about five times more common in babies infected with measles before age 1.

Researchers long thought that such infections were caused by a special strain of measles, but more recent research suggests that the measles virus can acquire mutations that enable it to infect the brain during the course of the original infection.

There is still much to learn about the measles virus. For example, researchers are exploring antibody therapies to treat severe measles. However, even if such treatments work, the best way to prevent the serious effects of measles is to avoid infection by getting vaccinated.The Conversation

Peter Kasson, Professor of Chemistry and Biomedical Engineering, Georgia Institute of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Read More

The post Measles can ravage the immune system and brain, causing long-term damage – a virologist explains appeared first on theconversation.com

Continue Reading

The Conversation

Supreme Court considers whether states may prevent people covered by Medicaid from choosing Planned Parenthood as their health care provider

Published

on

theconversation.com – Naomi Cahn, Professor of Law, University of Virginia – 2025-04-02 17:04:00

Planned Parenthood clinics, like this one in Los Angeles, are located across the United States.
Patrick T. Fallon/AFP via Getty Images

Naomi Cahn, University of Virginia and Sonia Suter, George Washington University

Having the freedom to choose your own health care provider is something many Americans take for granted. But the Supreme Court is weighing whether people who rely on Medicaid for their health insurance have that right, and if they do – is it enforceable by law?

That’s the key question at the heart of a case, Medina v. Planned Parenthood South Atlantic, that began during President Donald Trump’s first term in office.

“There’s a right, and the right is the right to choose your doctor,” said Justice Elena Kagan on April 2, 2025, during oral arguments on the case. John J. Bursch, the Alliance Defending Freedom lawyer who is representing South Carolina Director of Health and Human Services Eunice Medina, countered that none of the words in the underlying statute had what he called a “rights-creating pedigree.”

As law professors who teach courses about health and poverty law as well as reproductive justice, we think this case could affect access to health care for 72 million Americans, including low-income people and their children and people with disabilities.

Excluding Planned Parenthood

The case started with Julie Edwards, who is enrolled in Medicaid and lives in South Carolina. After she struggled to get contraceptive services, she was able to receive care from a Planned Parenthood South Atlantic clinic in Columbia, South Carolina.

Planned Parenthood, an array of nonprofits with roots that date back more than a century, is among the nation’s top providers of reproductive services. It operates two clinics in South Carolina, where Medicaid patients can get physical exams, cancer screenings, contraception and other services. It also provides same-day appointments and keeps long hours.

In July 2018, however, South Carolina Gov. Henry McMaster issued an executive order that barred health care providers in South Carolina that offer abortions from reimbursement through Medicaid.

That meant Planned Parenthood, a longtime target of conservatives’ ire, would no longer be reimbursed for any type of care for Medicaid patients, preventing Edwards from transferring all her gynecological care to that office as she had hoped to do.

Planned Parenthood and Edwards sued South Carolina, claiming that the state was violating the federal Medicare and Medicaid Act, which Congress passed in 1965, by not letting Edwards obtain care from the provider of her choice.

A ‘free-choice-of-provider’ requirement

Medicaid operates as a partnership between the federal government and the states. Congress passed the law that led to its creation based on its power under the Constitution’s spending clause, which allows Congress to subject federal funds to certain requirements.

Two years later, due to concerns that states were restricting which providers Medicaid recipients could choose, Congress added a “free-choice-of-provider” requirement to the program. It states that people enrolled in Medicaid “may obtain such assistance from any institution, agency, community pharmacy, or person, qualified to perform the service or services required.”

This provision is at the core of this case. At issue is whether a civil rights statute provides a right for Medicaid beneficiaries to sue a state when their federal rights have been violated. Known as Section 1983, it was enacted in 1871.

Bursch, backed by the Trump administration, argued before the court that the absence of words like “right” in the Medicaid provision that requires states to provide a free choice of provider means that neither Edwards nor Planned Parenthood has the authority to file a lawsuit to enforce this aspect of the Medicaid statute.

Nicole A. Saharsky, Planned Parenthood’s lawyer, argued that the creation of a right shouldn’t depend on “some kind of magic words test.” Instead, she said it was clear that the Medicaid statute created “a right to choose their own doctor” because “it’s mandatory” that the state provide this option to everyone with health insurance through Medicaid.

She also emphasized that Congress wanted to protect “an intensely personal right” to be able “to choose your doctor, the person that you see when you’re at your most vulnerable, facing … some of the most significant … challenges to your life and your health.”

Restricting Medicaid funds

Through a federal law known as the Hyde Amendment, Medicaid cannot reimburse health care providers for the cost of abortions, with a few exceptions: when a patient’s life is at risk or her pregnancy is due to rape or incest. Some states do cover abortion when their laws allow it, without using any federal funds.

Therefore, Planned Parenthood only gets federal Medicaid funds for abortions in those limited circumstances.

McMaster explained that he removed “abortion clinics,” including Planned Parenthood, from the South Carolina Medicaid Program because he didn’t want state funds to indirectly subsidize abortions.

South Carolina “decided that Planned Parenthood was unqualified for many reasons, chiefly because they’re the nation’s largest abortion provider,” Bursch told the Supreme Court.

But only 3% of Planned Parenthood’s services nationwide last year were related to abortion. Its most common service is testing for sexually transmitted diseases. Across the nation, Planned Parenthood provides health care to more than 2 million patients per year, most of whom have low incomes.

Man with gray hair in a suit and red tie speaks at a podium.
South Carolina Gov. Henry McMaster speaks to a crowd during an election night party on Nov. 3, 2020, in Columbia.
Photo by Sean Rayford/Getty Images

Section 1983

Because the Medicaid statute itself does not allow an individual to sue, Edwards and Planned Parenthood are relying on Section 1983.

Lower courts have repeatedly upheld that the Medicaid statute provides Edwards with the right to obtain Medicaid-funded health care at her local Planned Parenthood clinic.

And the Supreme Court has long recognized that Section 1983 protects an individual’s ability to sue when their rights under a federal statute have been violated.

In 2023, for example, the court found such a right under the Medicaid Nursing Home Reform Act. The court held that Section 1983 confers the right to sue when a statute’s provisions “unambiguously confer individual federal rights.”

Consequences beyond South Carolina

The court’s decision in the Medina case on whether Medicaid patients can choose their own health care provider could have consequences far beyond South Carolina. Arkansas, Missouri and Texas have already barred Planned Parenthood from getting reimbursed by Medicaid for any kind of health care. More states could follow suit.

In addition, given Planned Parenthood’s role in providing expansive contraceptive care, disqualifying it from Medicaid could harm access to health care and increase the already-high unintended pregnancy rate in America.

The ramifications, likewise, could extend beyond the finances of Planned Parenthood.

If the court rules in South Carolina’s favor, states could also try to exclude providers based on other characteristics, such as whether their employees belong to unions or if they provide their patients with gender-affirming care, further restricting patients’ choices.

Or, as Kagan observed, states could go the opposite direction and exclude providers that don’t provide abortions and so forth. What’s really at stake, she said, is whether a patient is “entitled to see” the provider they choose regardless of what their state happens to “think about contraception or abortion or gender transition treatment.”

If the Supreme Court rules that Edwards does have a right to get health care at a Planned Parenthood clinic, the controversy would not be over. The lower courts would then have to decide whether South Carolina appropriately removed Planned Parenthood from Medicaid as an “unqualified provider.”

And if the Supreme Court rules in favor of South Carolina, then Planned Parenthood could still sue South Carolina over its decision to find them to be unqualified.The Conversation

Naomi Cahn, Professor of Law, University of Virginia and Sonia Suter, Professor of Law, George Washington University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Read More

The post Supreme Court considers whether states may prevent people covered by Medicaid from choosing Planned Parenthood as their health care provider appeared first on theconversation.com

Continue Reading

The Conversation

Feeling FOMO for something that’s not even fun? It’s not the event you’re missing, it’s the bonding

Published

on

theconversation.com – Jacqueline Rifkin, Assistant Professor of Marketing, Cornell University – 2025-04-02 07:48:00

They had so much fun without me.
Milko/E+ via Getty Images

Jacqueline Rifkin, Cornell University; Barbara Kahn, University of Pennsylvania, and Cindy Chan, University of Toronto

Imagine you’ve planned the trip of a lifetime for your animal-loving family: a cruise to Antarctica with the unique opportunity to view penguins, whales and other rare wildlife. Your adventure-loving kids can kayak through fjords, plunge into icy water and camp under the Antarctic sky.

But rather than being ecstatic, as you anticipated, your kids whine about skipping an after-school scout meeting at a neighbor’s house. Missing this ordinary weekly event triggers such intense FOMO – “fear of missing out” – for them that they don’t want to go on your amazing expedition.

If this kind of debacle sounds familiar to you – or at least if you find it perplexing – you’re not alone. The three of us are marketing professors and social psychologists who focus on how consumers make decisions and how this shapes well-being. We’ve been studying FOMO for over a decade and recently published our work in the Journal of Personality and Social Psychology. Over the years, we’ve learned what really drives intense feelings of FOMO – which explains why a run-of-the-mill meeting might feel more crucial than an over-the-top vacation.

FOMO’s real trigger

People use the term FOMO in many different ways. In our research, we focus on a very specific type of FOMO: the kind that occurs when people miss out on events that involve valued social connections.

With this kind of FOMO, we found that the pain of missing out is not related to missing the actual event or opportunity – although that could be there as well. The FOMO we study happens when people miss the chance to bond with friends, co-workers or teammates they care about.

So, the critical part of FOMO is missing out on interactions with people you value. FOMO about a group dinner at a restaurant isn’t really about the food and great lighting. Nor is FOMO about a concert just about the band’s performance. Instead, it’s about the lost opportunity to connect and make memories with people who are important to you.

Why is this upsetting? Imagine the scenario where all your best friends go out to dinner without you. They bond and make lasting memories with each other – and you’re not there for any of it.

If they get closer to each other, where does that leave you? What happens to your social relationships and your sense of belonging? Do you become a less important friend? Less worthy of future invites? Or even kicked out of the group altogether? The anxiety of FOMO can begin to spiral.

People with what psychologists call an anxious attachment style chronically fear rejection and isolation from others. Because FOMO involves anxiety about future social belonging, it may not come as a surprise that people who are naturally more anxious about their friendships tend to get more intense FOMO. When we asked people in one of our studies to scroll social media until they encountered something social they missed, we found that the more anxiously attached a participant was, the more intense FOMO they experienced.

cheerful group laughing together around an outdoor dining table
They’ll always remember that summer cookout – and you weren’t there.
Maskot/DigitalVision via Getty Images

Not just missing Coachella

Getting FOMO for an amazing event you can’t attend makes sense. But if FOMO is less about the event itself and more about the social bonding, what happens when you miss something that’s not really fun at all?

We find that people anticipate FOMO even for unenjoyable missed events. As long as there is some form of missed social bonding, feelings of FOMO emerge. One of our studies found that people anticipated more FOMO from missing an un-fun event that their friends would be at, than a fun event without their friends.

For better or for worse, sad and stressful events can often be emotionally bonding: Going to a funeral to support a friend, cleaning up the mess after a party, or even white-knuckling through a harrowing initiation ceremony can all offer opportunities to forge stronger connections with one another. Stressful contexts like these can be fertile grounds for FOMO.

How to fend off FOMO

Popular discussions about the negative consequences of FOMO tend to focus on the FOMO people feel from compulsively scrolling on social media and seeing what they missed out on. Consequently, much of the suggested advice on how to mitigate FOMO centers on turning off phones or taking a vacation from social media.

Those recommendations may be tough for many people to execute. Plus, they address the symptoms of FOMO, not the cause.

Our finding that the core of FOMO is anxiety about missed social relationships yields a simpler strategy to combat it: Reminding yourself of the last time you connected with close friends may provide a sense of security that staves off feelings of FOMO.

In an experiment testing multiple interventions, we asked 788 study participants to look through their social media feeds until they encountered a post of a missed social event. We asked about 200 of these participants to immediately rate how much FOMO they were feeling. They averaged a 3.2 on a 1-to-7 scale.

Another group of about 200 participants also scrolled through their social media feeds until they encountered a post of a missed social event. But before indicating how much FOMO they were feeling, we asked them to think back to a prior experience socializing and bonding with their friends. Encouragingly, this reflection exercise seemed to curtail FOMO. Their average FOMO rating was 2.7 out of 7, a significant drop.

group of older women on a neighborhood walk with a leashed dog
Reminding yourself about other good times with your pals can help keep FOMO at bay.
AJ_Watt/E+ via Getty Images

With the remaining participants, we tested other strategies for mitigating FOMO – thinking about the next time they might see their friends or imagining what they’d say to a FOMO-suffering friend – but the simple reflection exercise was by far the most promising.

So, reminding yourself of the meaningful relationships you already have and reaffirming your social belonging in the moment may help combat the rush of anxiety that is characteristic of FOMO.

And missing out on social bonding experiences doesn’t have to be anxiety-provoking. In fact, in our activity-packed, hectic lives, missing some “must-attend” events may be a welcome relief – especially if you remind yourself that your social belonging is not in jeopardy. Cue a recent wave of counter-FOMO programming called JOMO, or “Joy of Missing Out.”

To quote Stuart Smalley, the fictional self-help guru of 1990s “Saturday Night Live,” reminding yourself that “I’m good enough, I’m smart enough, and doggone it, people like me!” might be just the trick to mitigate FOMO.The Conversation

Jacqueline Rifkin, Assistant Professor of Marketing, Cornell University; Barbara Kahn, Patty and Jay H. Baker Professor of Marketing, University of Pennsylvania, and Cindy Chan, Assistant Professor of Marketing, University of Toronto

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Read More

The post Feeling FOMO for something that’s not even fun? It’s not the event you’re missing, it’s the bonding appeared first on theconversation.com

Continue Reading

Trending