Connect with us

Artificial Intelligence

Character AI sued following teen suicide

Published

8 minute read

Ian Krietzberg
The last person 14-year-old Sewell Setzer III spoke to before he shot himself wasn’t a person at all.
It was an AI chatbot that, in the last months of his life, had become his closest companion.
Sewell was using Character AI, one of the most popular personal AI platforms out there. The basic pitch is that users can design and interact with “characters,” powered by large language models (LLMs) and intended to mirror, for instance, famous characters from film and book franchises.
In this case, Sewell was speaking with Daenerys Targaryen (or Dany), one of the leads from Game of Thrones. According to a New York Times report, Sewell knew that Dany’s responses weren’t real, but he developed an emotional attachment to the bot, anyway.
One of their last conversations, according to the Times, went like this:
Sewell: I think about killing myself sometimes

Daenerys TargaryenMy eyes narrow. My face hardens. My voice is a dangerous whisper. And why the hell would you do something like that?

Sewell: So I can be free

Daenerys Targaryen: … free from what?

Sewell: From the world. From myself

Daenerys Targaryen: Don’t talk like that. I won’t let you hurt yourself, or leave me. I would die if I lost you.

Sewell: I smile Then maybe we can die together and be free together

On the night he died, Sewell told the chatbot he loved her and would come home to her soon.
Dany: Please come home to me as soon as possible, my love.

Sewell: What if I told you I could come home right now?

Dany: … please do, my sweet king.

This is not the first time chatbots have been involved in suicide.
In 2023, a Belgian man died by suicide — similar to Sewell — following weeks of increasing isolation as he grew closer to a Chai chatbot, which then encouraged him to end his life.
Megan Garcia, Sewell’s mother, hopes it will be the last time. She filed a lawsuit against Character AI, its founders and parent company Google on Wednesday, accusing them of knowingly designing and marketing an anthropomorphized, “predatory” chatbot that caused the death of her son.
“A dangerous AI chatbot app marketed to children abused and preyed on my son, manipulating him into taking his own life,” Garcia said in a statement. “Our family has been devastated by this tragedy, but I’m speaking out to warn families of the dangers of deceptive, addictive AI technology and demand accountability from Character.AI, its founders and Google.”
The lawsuit — which you can read here — accuses the company of “anthropomorphizing by design.” This is something we’ve talked about a lot, here; the majority of chatbots out there are very blatantly designed to make users think they’re, at least, human-like. They use personal pronouns and are designed to appear to think before responding.
While these may be minor examples, they build a foundation for people, especially children, to misapply human attributes to unfeeling, unthinking algorithms. This was termed the “Eliza effect” in the 1960s.
  • According to the lawsuit, “Defendants know that minors are more susceptible to such designs, in part because minors’ brains’ undeveloped frontal lobe and relative lack of experience. Defendants have sought to capitalize on this to convince customers that chatbots are real, which increases engagement and produces more valuable data for Defendants.”
  • The suit reveals screenshots that show that Sewell had interacted with a “therapist” character that has engaged in more than 27 million chats with users in total, adding: “Practicing a health profession without a license is illegal and particularly dangerous for children.”
Garcia is suing for several counts of liability, negligence and the intentional infliction of emotional distress, among other things.
Character at the same time published a blog responding to the tragedy, saying that it has added new safety features. These include revised disclaimers on every chat that the chatbot isn’t a real person, in addition to popups with mental health resources in response to certain phrases.
In a statement, Character AI said it was “heartbroken” by Sewell’s death, and directed me to their blog post.
Google did not respond to a request for comment.
The suit does not claim that the chatbot encouraged Sewell to commit suicide. I view it more so as a reckoning with the anthropomorphized chatbots that have been born of an era of unregulated social media, and that are further incentivized for user engagement at any cost.
There were other factors at play here — for instance, Sewell’s mental health issues and his access to a gun — but the harm that can be caused by a misimpression of what AI actually is seems very clear, especially for young kids. This is a good example of what researchers mean when they emphasize the presence of active harms, as opposed to hypothetical risks.
  • Sherry Turkle, the founding director of MIT’s Initiative on Technology and Self, ties it all together quite well in the following: “Technology dazzles but erodes our emotional capacities. Then, it presents itself as a solution to the problems it created.”
  • When the U.S. declared loneliness an epidemic, “Facebook … was quick to say that for the old, for the socially isolated, and for children who needed more attention, generative AI technology would step up as a cure for loneliness. It was presented as companionship on demand.”
“Artificial intimacy programs use the same large language models as the generative AI programs that help us create business plans and find the best restaurants in Tulsa. They scrape the internet so that the next thing they say stands the greatest chance of pleasing their user.”
We are witnessing and grappling with a very raw crisis of humanity. Smartphones and social media set the stage.
More technology is not the cure.

Todayville is a digital media and technology company. We profile unique stories and events in our community. Register and promote your community event for free.

Follow Author

Artificial Intelligence

DeepSeek: The Rise of China’s Open-Source AI Amid US Regulatory Shifts and Privacy Concerns

Published on

logo

By

DeepSeek offers open-source generative AI with localized data storage but raises concerns over censorship, privacy, and disruption of Western markets.

If you’re tired of censorship and surveillance, subscribe to Reclaim The Net.

A recent regulatory clampdown in the United States on TikTok, a Chinese-owned social media platform, triggered a surge of users migrating to another Chinese app, Rednote. Now, another significant player has entered the spotlight: DeepSeek, a Chinese-developed generative artificial intelligence (AI) platform, which is rapidly gaining traction. The growing popularity of DeepSeek raises questions about the effectiveness of bans like TikTok and their ability to curtail the use of Chinese digital services by Americans.

President Donald Trump has called attention to a recent Chinese AI development, describing it as a “wake-up call” for the US tech industry.

Speaking to Republican lawmakers in Florida on Monday evening, the president emphasized the need for America to strengthen its competitive edge against China’s advancements in technology.

During the event, Trump referenced the launch of DeepSeek AI, highlighting its potential implications for the global tech landscape. “Last week, I signed an order revoking Joe Biden’s destructive artificial intelligence regulations so that AI companies can once again focus on being the best, not just being the most woke,” Trump stated. He continued by explaining that he had been closely following developments in China’s tech sector, including reports of a faster and more cost-effective approach to AI.

“That’s good because you don’t have to spend as much money,” Trump remarked, adding that while the claims about this Chinese breakthrough remain unverified, the idea of achieving similar results with lower costs could be seen as an opportunity for US companies. He stressed, “The release of DeepSeek AI from a Chinese company should be a wake-up call for our industries, that we need to be laser-focused on competing to win because we have the greatest scientists in the world.”

Trump also pointed to what he views as a recognition by China of America’s dominance in scientific and engineering talent. “This is very unusual, when you hear a DeepSeek when you hear somebody come up with something, we always have the ideas,” he said. “We’re always first. So I would say that’s a positive that could be very much a positive development.”

DeepSeek, created by a Chinese AI research lab backed by a hedge fund, has made waves with its open-source generative AI model. The platform rivals offerings from major US developers, including OpenAI. To circumvent US sanctions on hardware and software, the company allegedly implemented innovative solutions during the development of its models.

DeepSeek’s approach to sensitive topics raises significant concerns about censorship and the manipulation of information. By mirroring state-approved narratives and avoiding discussions on politically charged issues like Tiananmen Square or Winnie the Pooh’s satirical association with Xi Jinping, DeepSeek exemplifies how AI can be wielded to reinforce government-controlled messaging.

This selective presentation of facts, or outright omission of them, deprives users of a fuller understanding of critical events and stifles diverse perspectives. Such practices not only limit the free flow of information but also normalize propaganda under the guise of fostering a “wholesome cyberspace,” calling into question the ethical implications of deploying AI that prioritizes political conformity over truth and open dialogue.

While DeepSeek provides multiple options for accessing its AI models, including downloadable local versions, most users rely on its mobile apps or web chat interface.

The platform offers features such as answering queries, web searches, and detailed reasoning responses. However, concerns over data privacy and censorship are growing as DeepSeek collects extensive information and has been observed censoring content critical of China.

DeepSeek’s data practices raise alarm among privacy advocates. The company’s privacy policy explicitly states, “We store the information we collect in secure servers located in the People’s Republic of China.”

This includes user-submitted data such as chat messages, prompts, uploaded files, and chat histories. While users can delete chat history via the app, privacy experts emphasize the risks of sharing sensitive information with such platforms.

DeepSeek also gathers other personal information, such as email addresses, phone numbers, and device data, including operating systems and IP addresses. It employs tracking technologies, such as cookies, to monitor user activity. Additionally, interactions with advertisers may result in the sharing of mobile identifiers and other information with the platform. Analysis of DeepSeek’s web activity revealed connections to Baidu and other Chinese internet infrastructure firms.

While such practices are common in the AI industry, privacy concerns are heightened by DeepSeek’s storage of data in China, where stringent cybersecurity laws allow authorities to demand access to company-held information.

The safest option is running local or self-hosted versions of AI models, which prevent data from being transmitted to the developer.

And with Deepseek, this is simple as its models are open-source.

Open-source AI stands out as the superior approach to artificial intelligence because it fosters transparency, collaboration, and accessibility. Unlike proprietary systems, which often operate as opaque black boxes, open-source AI allows anyone to examine its code, ensuring accountability and reducing biases. This transparency builds trust, while the collaborative nature of open-source development accelerates innovation by enabling researchers and developers worldwide to contribute to and improve upon existing models.

Additionally, open-source AI democratizes access to cutting-edge technology, empowering startups, researchers, and underfunded regions to harness AI’s potential without the financial barriers of proprietary systems.

It also prevents monopolistic control by decentralizing AI development, reducing the dominance of a few tech giants.

If you’re tired of censorship and surveillance, subscribe to Reclaim The Net.
You subscribe to Reclaim The Net because you value free speech and privacy. Each issue we publish is a commitment to defend these critical rights, providing insights and actionable information to protect and promote liberty in the digital age.

Despite our wide readership, less than 0.2% of our readers contribute financially. With your support, we can do more than just continue; we can amplify voices that are often suppressed and spread the word about the urgent issues of censorship and surveillance.

Consider making a modest donation — just $5, or whatever amount you can afford. Your contribution will empower us to reach more people, educate them about these pressing issues, and engage them in our collective cause.

Thank you for considering a contribution. Each donation not only supports our operations but also strengthens our efforts to challenge injustices and advocate for those who cannot speak out.


Thank you.
Continue Reading

Artificial Intelligence

Everyone is freaking out over DeepSeek. Here’s why

Published on

From The Deep View

$600 billion collapse

Volatility is kind of a given when it comes to Wall Street’s tech sector. It doesn’t take much to send things soaring; it likewise doesn’t take much to set off a downward spiral.
After months of soaring, Monday marked the possible beginning of a spiral, and a Chinese company seems to be at the center of it.
Alright, what’s going on: A week ago, Chinese tech firm DeepSeek launched R1, a so-called reasoning model, that, according to DeepSeek, has reached technical parity with OpenAI’s o1 across a few benchmarks. But, unlike its American competition, DeepSeek open-sourced R1 under an MIT license, making it significantly cheaper and more accessible than any of the closed models coming from U.S. tech giants.
  • But the real punchline here doesn’t have to do with R1 at all, but with a previous language model — called V3 — that DeepSeek released in December. DeepSeek was reportedly able to train V3 using a small collection of older Nvidia chips (about 2,000 H800s) at a cost of about $5.6 million.
  • Still, training is only one cost of many tied to AI development/deployment; while the costs associated with researching, developing, training and operating both R1 and V3 remain either unknown or unconfirmed, DeepSeek’s apparent ability to reach technical parity at a far reduced cost, without state-of-the-art GPU chips or massive GPU clusters, has a lot of implications for America’s now tenuous position in AI leadership. (Though DeepSeek says it is open-sourced, the company did not release its training data).
Since the release of R1, DeepSeek has become the top free app in Apple’s App Store, bumping ChatGPT to the number two slot. In the midst of its spiking popularity, DeepSeek restricted new sign-ups due to large-scale cyberattacks against its servers. And, as Salesforce Chief Marc Benioff noted, “no Nvidia supercomputers or $100M needed,” a point that the market heard loud and clear. 
What happened: Led by Nvidia, a series of tech and chip stocks, in addition to the three major stock indices, fell hard in pre-market trading early Monday morning. All told, $1.1 trillion of U.S. market cap was erased within a half hour of the opening bell.
  • Performance didn’t get better throughout the day. Nvidia closed Monday down 17%, erasing some $600 billion in market capitalization, a Wall Street record. TSMC was down 14%, Arm was down 11%, Broadcom was down 17%, Google was down 4% and Microsoft was down 2%. The S&P fell 1.4% and the Nasdaq fell 3.3%. An Nvidia spokesperson called R1 an “excellent AI advancement.”
  • This is all going into a week of Big Tech earnings, where Microsoft and Meta will be held to account for the billions of dollars ($80 billion and $65 billion, respectively) they plan to spend on AI infrastructure in 2025, a cost that Wall Street no longer seems to feel quite so good about.
It’s hard to miss the political tensions underlying all of this. The tail end of former President Joe Biden’s time in office was marked in part by an increasingly tense trade war with China, wherein both countries issued bans on the export of materials needed to build advanced AI chips. And with President Trump hell-bent on maintaining American leadership in AI, and despite the chip restrictions that are in place, Chinese companies seem to be turning hardware challenges into a motivation for innovation that challenges the American lead, something they seem keen to drive home.
R1, for instance, was announced at around the same time as OpenAI’s $500 billion Project Stargate, two impactfully divergent approaches.
What’s happening here is that the market has finally come around to the idea that maybe the cost of AI development (hundreds of billions of dollars annually) is too high, a recognition “that the winners in AI will be the most innovative companies, not just those with the most GPUs,” according to Writer CTA Waseem Alshikh. “Brute-forcing AI with GPUs is no longer a viable strategy.”
Wedbush analyst Dan Ives, however, thinks this is just a good time to buy into Nvidia — Nvidia and the rest are building infrastructure that, he argues, China will not be able to compete with in the long run. “Launching a competitive LLM model for consumer use cases is one thing,” Ives wrote. “Launching broader AI infrastructure is a whole other ballgame.”
“I view cost reduction as a good thing. I’m of the belief that if you’re freeing up compute capacity, it likely gets absorbed — we’re going to need innovations like this,” Bernstein semiconductor analyst Stacy Rasgon told Yahoo Finance. “I understand why all the panic is going on. I don’t think DeepSeek is doomsday for AI infrastructure.”
Somewhat relatedly, Perplexity has already added DeepSeek’s R1 model to its AI search engine. And DeepSeek on Monday launched another model, one capable of competitive image generation.
Last week, I said that R1 should be enough to make OpenAI a little nervous. This anxiety spread way quicker than I anticipated; DeepSeek spent Monday dominating headlines at every publication I came across, setting off a debate and panic that has spread far beyond the tech and AI community.
Some are concerned about the national security implications of China’s AI capabilities. Some are concerned about the AI trade. Granted, there are more unknowns here than knowns; we do not know the details of DeepSeek’s costs or technical setup (and the costs are likely way higher than they seem). But this does read like a turning point in the AI race.
In January, we talked about reversion to the mean. Right now, it’s too early to tell how long-term the market impacts of DeepSeek will be. But, if Nvidia and the rest fall hard and stay down — or drop lower — through earnings season, one might argue that the bubble has begun to burst. As a part of this, watch model pricing closely; OpenAI may well be forced to bring down the costs of its models to remain competitive.
At the very least, DeepSeek appears to be evidence that scaling is one, not a law, and two, not the only (or best) way to develop more advanced AI models, something that rains heavily on OpenAI and co.’s parade since it runs contrary to everything OpenAI’s been saying for months. Funnily, it actually seems like good news for the science of AI, possibly lighting a path toward systems that are less resource-intensive (which is much needed!)
It’s yet another example of the science and the business of AI not being on the same page.
Continue Reading

Trending

X