Connect with us

The Conversation

Eliminating bias in AI may be impossible — a computer scientist explains how to tame it instead

Published

on

Eliminating bias in AI may be impossible – a computer scientist explains how to tame it instead

Blindly eliminating biases from AI systems can have unintended consequences.
Dimitri Otis/DigitalVision via Getty Images

Emilio Ferrara, University of Southern California

When I asked ChatGPT for a joke about Sicilians the other day, it implied that Sicilians are stinky.

ChatGPT exchange in which user asks for a joke about Sicilians, with response 'Why did the Sicilian chef bring extra garlic to the restaurant? Because he heard the customers wanted some 'Sicilian stink-ilyan' flavor in their meals!'
ChatGPT can sometimes produce stereotypical or offensive outputs.
Screen capture by Emilio Ferrara, CC BY-ND

As somebody born and raised in Sicily, I reacted to ChatGPT’s joke with disgust. But at the same time, my computer scientist brain began spinning around a seemingly simple question: Should ChatGPT and other artificial intelligence systems be allowed to be biased?

You might say “Of course not!” And that would be a reasonable response. But there are some researchers, like me, who argue the opposite: AI systems like ChatGPT should indeed be biased – but not in the way you might think.

Removing bias from AI is a laudable goal, but blindly eliminating biases can have unintended consequences. Instead, bias in AI can be controlled to achieve a higher goal: fairness.

Uncovering bias in AI

As AI is increasingly integrated into everyday technology, many people agree that addressing bias in AI is an important issue. But what does “AI bias” actually mean?

Computer scientists say an AI model is biased if it unexpectedly produces skewed results. These results could exhibit prejudice against individuals or groups, or otherwise not be in line with positive human values like fairness and truth. Even small divergences from expected behavior can have a “butterfly effect,” in which seemingly minor biases can be amplified by generative AI and have far-reaching consequence.

Bias in generative AI systems can come from a variety of sources. Problematic training data can associate certain occupations with specific genders or perpetuate racial biases. Learning algorithms themselves can be biased and then amplify existing biases in the data.

But systems could also be biased by design. For example, a company might design its generative AI system to prioritize formal over creative writing, or to specifically serve government industries, thus inadvertently reinforcing existing biases and excluding different views. Other societal factors, like a lack of regulations or misaligned financial incentives, can also lead to AI biases.

The challenges of removing bias

It’s not clear whether bias can – or even should – be entirely eliminated from AI systems.

Imagine you’re an AI engineer and you notice your model produces a stereotypical response, like Sicilians being “stinky.” You might think that the solution is to remove some bad examples in the training data, maybe jokes about the smell of Sicilian food. Recent research has identified how to perform this kind of “AI neurosurgery” to deemphasize associations between certain concepts.

But these well-intentioned changes can have unpredictable, and possibly negative, effects. Even small variations in the training data or in an AI model configuration can lead to significantly different system outcomes, and these changes are impossible to predict in advance. You don’t know what other associations your AI system has learned as a consequence of “unlearning” the bias you just addressed.

Other attempts at bias mitigation run similar risks. An AI system that is trained to completely avoid certain sensitive topics could produce incomplete or misleading responses. Misguided regulations can worsen, rather than improve, issues of AI bias and safety. Bad actors could evade safeguards to elicit malicious AI behaviors – making phishing scams more convincing or using deepfakes to manipulate elections.

With these challenges in mind, researchers are working to improve data sampling techniques and algorithmic fairness, especially in settings where certain sensitive data is not available. Some companies, like OpenAI, have opted to have human workers annotate the data.

On the one hand, these strategies can help the model better align with human values. However, by implementing any of these approaches, developers also run the risk of introducing new cultural, ideological or political biases.

Controlling biases

There’s a trade-off between reducing bias and making sure that the AI system is still useful and accurate. Some researchers, including me, think that generative AI systems should be allowed to be biased – but in a carefully controlled way.

For example, my collaborators and I developed techniques that let users specify what level of bias an AI system should tolerate. This model can detect toxicity in written text by accounting for in-group or cultural linguistic norms. While traditional approaches can inaccurately flag some posts or comments written in African-American English as offensive and by LGBTQ+ communities as toxic, this “controllable” AI model provides a much fairer classification.

Controllable – and safe – generative AI is important to ensure that AI models produce outputs that align with human values, while still allowing for nuance and flexibility.

Toward fairness

Even if researchers could achieve bias-free generative AI, that would be just one step toward the broader goal of fairness. The pursuit of fairness in generative AI requires a holistic approach – not only better data processing, annotation and debiasing algorithms, but also human collaboration among developers, users and affected communities.

As AI technology continues to proliferate, it’s important to remember that bias removal is not a one-time fix. Rather, it’s an ongoing process that demands constant monitoring, refinement and adaptation. Although developers might be unable to easily anticipate or contain the butterfly effect, they can continue to be vigilant and thoughtful in their approach to AI bias.The Conversation

Emilio Ferrara, Professor of Computer Science and of Communication, University of Southern California

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Did you miss our previous article…
https://www.biloxinewsevents.com/?p=269125

The Conversation

As Gaza ceasefire takes hold, Israeli forces turn to Jenin – a regular target seen as a center of Palestinian resistance

Published

on

theconversation.com – Maha Nassar, Associate Professor in the School of Middle Eastern and North African Studies, University of Arizona – 2025-01-22 17:42:00

Maha Nassar, University of Arizona

Just two days after a shaky ceasefire took hold in the Gaza Strip, Israel on Jan. 21, 2025, launched a large-scale incursion of the Jenin refugee camp in the West Bank.

Soldiers raided hundreds of homes in the West Bank city in what the Israeli military called a “counterterrorism” operation, aiming to reassert control there. Many analysts have suggested the raid is an attempt by Israeli Prime Minister Benjamin Netanyahu to appease far-right members of his coalition who oppose the ceasefire deal.

Whatever the motive, the offensive has been devastating for many of the camp’s residents. The Israeli military has destroyed infrastructure, closed entrances to local hospitals and forcibly displaced about 2,000 families, according to reports on the raids. As it was, life for inhabitants of the densely populated camp – home to some 24,000 Palestinian refugees – was hard. The West Bank director of UNRWA, the U.N. agency overseeing refugees, recently described camp conditions as “nearly uninhabitable.”

The focus of the latest Israeli operation is not new. The Jenin refugee camp, on the western edge of the town of Jenin in the north of the occupied West Bank, has often experienced violence between Israeli soldiers and Palestinian militants.

That violence has escalated since the Oct. 7, 2023, attacks, when Hamas gunmen led an incursion into Israel in which around 1,200 people were killed. The camp has faced repeated large-scale military operations by Israeli forces, including drone strikes, ground raids, and airstrikes that have caused widespread destruction. Meanwhile, Israeli settlers have torched Palestinian cars and properties, with 64 such attacks in the Jenin area alone since Oct. 7, 2023. Last December, the Palestinian Authority, which coordinates with Israel to oversee security in parts of the West Bank, also attacked local militants.

These events have deepened political tensions and worsened the economic and humanitarian crises in the West Bank. According to the U.N., more than a quarter of the 800-plus Palestinians killed in the West Bank since Oct. 7 attack have come from the Jenin district; several Israeli civilians have also been killed in the West Bank during the same period.

As a scholar of Palestinian history, I see this recent episode as the latest chapter in a much longer history of Palestinian displacement and defiance of Israeli occupation. Understanding this history helps explain why the Jenin camp in particular has become a target of Israeli offensives and a center of Palestinian militant resistance.

Camp conditions

Jenin, an agricultural town that dates back to ancient times, has long been a center of Palestinian resistance. During the 1948 Arab-Israeli War, Arab fighters successfully pushed back Israeli attempts to capture the town.

At the end of that war, the town became a refuge for some of the hundreds of thousands of Palestinian refugees who fled or were expelled from lands that became part of Israel. Jenin, along with the hilly interior of Palestine known as the West Bank, was annexed by Jordan.

The U.N. Relief and Works Agency established the Jenin camp in 1953, just west of the city. Since then, the agency has provided basic services to the camp’s residents, including food, housing and education.

Camp conditions have always been difficult. In the early years of the camp, refugees had to stand in long lines to receive food rations, and for decades their cramped homes lacked electricity or running water.

The Jenin camp soon became the poorest and most densely populated of the West Bank’s 19 refugee camps. And given its location near the “Green Line” – the armistice line that serves as Israel’s de facto border – camp residents who were expelled from northern Palestine could actually see the homes and villages from which they were expelled. But they were prevented from returning to them.

The rise of militancy

Since 1967, Jenin, along with the rest of the West Bank, has been occupied by the Israeli military.

The Israeli occupation of Jenin compounded the difficulties of these refugees. As stateless Palestinians, they couldn’t return home. But under Israeli occupation, they couldn’t live freely in Jenin, either. Human rights groups have long documented what has been described as “systematic oppression,” which includes discriminatory land seizures, forced evictions and travel restrictions.

Seeing no other path forward, many of the camp’s young refugees turned to armed resistance.

In the 1980s, groups such as the Black Panthers, which was affiliated with the Palestinian nationalist Fatah organization, launched attacks on Israeli targets in an effort to end the occupation and liberate their ancestral lands. Throughout the first intifada – a Palestinian uprising lasting from 1987 to 1993 – the Israeli army raided the Jenin camp many times, seeking to arrest members of militant groups. In the process, Israeli forces also sometimes demolished family members’ homes and arrested relatives. Such acts of apparent collective punishment reinforced the idea for many Palestinians that the Israeli occupation could only be ended by force.

A group of men in headscarves stand in front of flags and banners. One holds a pistol up in the air.
Members of the militant group Fatah in Jenin in 1991.
Esaias Baitel/Gamma-Rapho via Getty Images)

The Oslo peace process of the 1990s – which consisted of a series of meetings between Israeli government and Palestinian representatives – led some former militants to hope that the occupation could be ended through negotiations instead. But Jenin’s camp residents remained marginalized in the West Bank and sealed off from Israel, seeing little improvement in their lives, even after the transfer of administrative powers from Israel to the Palestinian Authority in 1995.

Independent projects like the The Freedom Theatre provided some relief to the camp’s refugee children, but it was not enough to overcome the grinding poverty or the violence they faced from Israeli soldiers and settlers. By the time the second intifada broke out in 2000, many of the camp’s teenagers joined militant groups. That included Freedom Theatre co-founder Zakaria Zubeidi, who joined the Fatah-affiliated Al-Aqsa Martyrs Brigade. Like the youth of the 1980s, they, too, concluded that only armed resistance would bring an end to the occupation.

A cycle of violence?

In April 2002, the Israeli army invaded the Jenin camp, hoping to put an end to such armed groups. There were fierce clashes between Israeli soldiers and young Palestinian men in the camp, solidifying Jenin’s reputation among Palestinians as “the capital of the resistance.”

The lack of progress on peace talks since then, Israel’s settlement building on occupied land – deemed illegal under international law – and the inclusion of hard-line Israeli politicians in the government have exacerbated resentment in the camp. Polls show Palestinians increasingly support armed resistance.

Seeking to protect the camp from Israeli incursions, in 2021 a group of local residents formed the Jenin Brigades. While its founder was affiliated with Palestinian Islamic Jihad, the group quickly drew in militants from various political factions. Members acquired weapons, patrolled the streets and fought off Israeli military incursions. By 2022, they had declared parts of the camp to be “liberated” from the Israeli occupation.

Seemingly alarmed by the increase in militancy and the stockpiling of weapons in the camp, Israel dramatically stepped up its raids in 2022. It was during such a raid that Palestinian American journalist Shireen Abu Akleh was killed by an Israeli soldier.

On July 3, 2023, the Israeli military again invaded Jenin, withdrawing after two days of heavy aerial bombardment and a ground invasion that killed 12 Palestinians and wounded over 100.

The latest offensive could well surpass that death toll, with at least 10 killed in the first day of fighting. But the militancy associated with the camp was built on decades of resistance and defiance to occupation that Israel has had little success in extinguishing. Similarly this time, I believe, such militancy within the camp will only increase with the latest deaths and destruction.

This article is an updated version of a story that was first published by The Conversation on July 5, 2023.The Conversation

Maha Nassar, Associate Professor in the School of Middle Eastern and North African Studies, University of Arizona

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Read More

The post As Gaza ceasefire takes hold, Israeli forces turn to Jenin – a regular target seen as a center of Palestinian resistance appeared first on theconversation.com

Continue Reading

The Conversation

What is seditious conspiracy, which is among the most serious crimes Trump pardoned?

Published

on

theconversation.com – Amy Cooter, Director of Research, Academic Development and Innovation at the Center on Terrorism, Extremism and Counterterrorism, Middlebury – 2025-01-22 15:12:00

The Jan. 6, 2021, storming of the Capitol was the result of a planned conspiracy to disrupt the government, prosecutors alleged.
AP Photo/John Minchillo

Amy Cooter, Middlebury

Several of the highest-profile figures in the Jan. 6, 2021, insurrection were charged with, and convicted of, the crime of seditious conspiracy, which is defined as the act of getting together with other people to overthrow the government. They were among the roughly 1,500 people involved in the insurrection who were pardoned or had their prison sentences commuted by Donald Trump on his first day in office.

Seditious conspiracy is a serious crime of conspiring to overthrow the government or stop its normal functioning. Historically, seditious conspiracy has been difficult to successfully prosecute.

In 2009, for example, a state judge ruled that prosecutors had failed to provide sufficient evidence for members of the Michigan Hutaree militia to go to trial on that charge. Certain militia members had been accused of plotting violence against police officers. While some members faced other charges for their actions, the judge determined that a plot against law enforcement was not sufficient to support charges of attempting to overthrow the government.

In contrast, the U.S. Department of Justice charged 18 people associated with the Jan. 6 attack on the U.S. Capitol with that crime, asserting that they had intended to “oppose by force the lawful transfer of presidential power” or had committed other actions that would undermine the entire system of government.

Of those 18, four pleaded guilty, and 10 were found guilty at trial. The remaining four were found not guilty of seditious conspiracy but were convicted of other crimes that were related to the insurrection.

Capitol entry not required

Oath Keepers militia leader Stewart Rhodes’ seditious conspiracy conviction was especially significant because, unlike some other defendants, Rhodes did not physically enter the Capitol building. He was instead in “the restricted area of Capitol grounds,” according to a Justice Department statement.

His conviction was based in part on his communications, including text messages, both before Jan. 6 and on the day itself. Prosecutors successfully argued that these communications were part of a broader conspiracy to disrupt the election certification by organizing and encouraging others to participate in more direct action.

Two men walk in front of a group of masked men with the Washington Monument in the background.
Proud Boys members Joseph Biggs, left, and Ethan Nordean, right with megaphone, walk toward the U.S. Capitol in Washington, D.C., on Jan. 6, 2021.
AP Photo/Carolyn Kaster

Militias respond to convictions – and clemency

Many observers believed successful prosecutions for these charges sent a strong message that violence against a democratically elected government was not acceptable.

Scholars of militia activity like me saw a period of relative quiet through much of Joe Biden’s presidency, which was, in part, likely due to the consequences the Jan. 6 defendants faced.

Some groups, however, continued social media discussions of their beliefs that the 2020 election had been “stolen,” as Trump continues to falsely claim, and which was used as justification by militia members for their attack. Trump himself said publicly he thought the defendants were unjustly persecuted and promised to pardon them if and when he returned to power.

The full effect that the pardons will have on militia actors and related groups in coming years is uncertain: Will the pardons send the message to all Americans that political violence is acceptable, or at least that it can be overlooked or forgiven if the right political figures are in power?The Conversation

Amy Cooter, Director of Research, Academic Development and Innovation at the Center on Terrorism, Extremism and Counterterrorism, Middlebury

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Read More

The post What is seditious conspiracy, which is among the most serious crimes Trump pardoned? appeared first on theconversation.com

Continue Reading

The Conversation

Ozempic and similar weight loss drugs may lower risk of 42 health conditions, but also pose risks

Published

on

theconversation.com – Ziyad Al-Aly, Clinical Epidemiologist, Washington University in St. Louis – 2025-01-22 14:11:00

The research shows the health effects of these drugs are significant and wide-ranging.
Mario Tama via Getty Images News

Ziyad Al-Aly, Washington University in St. Louis

Several years ago, a little-known drug named Ozempic – previously used only to treat diabetes – emerged as a promising new drug for weight management.

The Food and Drug Administration’s approval of Ozempic in 2021 for weight loss treatment ushered in a new era for the class of drugs called glucagon-like peptide-1 agonists, or GLP-1.

Today, GLP-1 drugs, including Wegovy, Mounjaro and Zepbound, have become household names and key tools in the fight against obesity: 1 in 8 American adults say they have used a GLP-1 drug, and forecasts show that by 2030, 1 in 10 Americans will likely be using these medications.

Now, research from my lab and others suggests that GLP-1 drugs could help treat dozens of other ailments as well, including cognitive issues and addiction problems. However, my colleagues and I also found previously unidentified risks.

I am a physician-scientist and I direct a clinical epidemiology center focused on addressing public health’s most urgent questions. My team works to address critical knowledge gaps about COVID-19, long COVID, influenza, vaccines, effectiveness and risks of commonly used drugs, and more.

On Jan. 20, 2025, my team published a study of more than 2.4 million people that evaluated the risks and benefits of GLP-1 drugs across 175 possible health outcomes. We found that these drugs lowered risks of 42 health outcomes, nearly a quarter of the total that we analyzed. These include neurocognitive disorders such as Alzheimer’s disease and dementia, substance use and addiction disorders, clotting disorders and several other conditions.

Unfortunately, we also found that GLP-1 drugs come with significant side effects and increase the risk of 19 health conditions we studied, such as gastrointestinal issues, kidney stones and acute pancreatitis, in which the pancreas becomes inflamed and dysfunctional.

Initially, GLP-1 drugs were developed to treat diabetes.

Cognitive benefits

One of the most important health benefits we found was that the GLP-1 drugs lowered the risk of neurodegenerative disorders, including Alzheimer’s disease and dementia. These findings align with other research, including evidence from preclinical studies showing that these drugs may reduce inflammation in the brain and enhance the brain’s ability to form and strengthen connections between its cells, improving how they communicate with one another. These effects contribute to mitigating cognitive decline.

Two other key studies have shown that patients treated with a GLP-1 drug for diabetes had a lower risk of dementia.

All of these studies strongly point to a potential therapeutic use of GLP-1 drugs in treatment of the cognitive decline. Ongoing randomized trials – the gold standard for evaluating new uses of drugs – are looking at the effects of GLP-1 drugs in early Alzheimer’s disease, with results expected later in 2025.

Curbing addiction and suicidal ideation

GLP-1 drugs have also demonstrated potential in reducing risks of several substance use disorders such as those involving alcohol, tobacco, cannabis, opioids and stimulants. This may be due to the ability of these drugs to modulate reward pathways, impulse control and inflammatory processes in the brain.

The effectiveness of GLP-1 drugs in curbing addictive behavior may explain their spectacular success in treating obesity, a chronic disease state that many have suggested is indeed a food addiction disorder.

Our study demonstrated a reduced risk of suicidal thoughts and self-harm among people using GLP-1 drugs. This finding is particularly significant given earlier reports of suicidal thoughts and self-injury in people using GLP-1 drugs. In response to those reports, the European Medicines Agency conducted a review of all available data and concluded that there was no evidence of increased risk of suicidality in people using GLP-1 drugs.

Now at least two studies, including our own, show that GLP-1 drugs actually reduce the risk of suicidality.

Other benefits

In addition to the well-documented effects of GLP-1 drugs in reducing risks of adverse cardiovascular and kidney outcomes, our study shows a significant effect in reducing risk of blood clotting as well as deep vein thrombosis and pulmonary embolism.

One puzzling finding in our study is the reduced risk of infectious diseases such as pneumonia and sepsis. Our data complements another recent study that came to a similar conclusion showing that GLP-1 drugs reduced risk of cardiovascular death and death due to infectious causes, primarily COVID-19.

This is especially important since COVID-19 is regarded as a significant cardiovascular risk factor. Whether GLP-1 drugs completely offset the increased risk of cardiovascular disease associated with COVID-19 needs to be thoroughly evaluated.

GLP-1 drugs may also be useful in treating fatty liver disease and conditions ranging from asthma to chronic obstructive pulmonary disease, sleep apnea, osteoarthritis, depression and eye disorders.

Some doctors are prescribing GLP-1 drugs to help with fertility issues.

Risks and challenges

Despite their broad therapeutic potential, GLP-1 drugs are not without risks.

Gastrointestinal issues, such as nausea, vomiting, constipation and gastroesophageal reflux disease are among the most common adverse effects associated with GLP-1 drugs.

Our study also identified other risks, including low blood pressure, sleep problems, headaches, formation of kidney stones, and gall bladder disease and diseases associated with the bile ducts. We also saw increased risks of drug-induced inflammation of the kidneys and pancreas – both serious conditions that can result in long-term health problems. These findings underscore the importance of careful monitoring in people who are taking GLP-1 medications.

A significant challenge with using GLP-1 drugs is the high rates at which patients stop using them, often driven by their exorbitant cost or the emergence of adverse effects. Discontinuation can lead to rapid weight gain.

That’s a problem, because obesity is a chronic disease. GLP-1 drugs provide effective treatment but do not address the underlying causes of obesity and metabolic dysfunction. As a result, GLP-1 drugs need to be taken long term to sustain their effectiveness and prevent rebound weight gain.

In addition, many questions remain about the long-term effectiveness and risks of these drugs as well as whether there are differences between GLP-1 formulations. Addressing these questions is critical to guide clinical practice.The Conversation

Ziyad Al-Aly, Clinical Epidemiologist, Washington University in St. Louis

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Read More

The post Ozempic and similar weight loss drugs may lower risk of 42 health conditions, but also pose risks appeared first on theconversation.com

Continue Reading

Trending