Who is responsible for AI’s output? Artificial intelligence (AI) companies like OpenAI maintain that they are not. In fact, their terms and conditions in 2023 stated that responsibility lies solely with the user. A German court disagrees.
On June 9, a Munich court (subject to appeal) ruled that Google can be liable for false claims produced by its AI summaries, drawing a sharp line between ordinary search results and machine-generated assertions. In other words, AI companies must be held legally responsible for the output that is created by their systems and pushed to users.
The court’s logic was simple but profound: Search results point outward to sources, while AI summaries speak in Google’s own voice. That distinction matters because it goes to the heart of what kind of speech deserves protection — and what kind is subject to legal scrutiny. The U.S. should follow the German court’s lead. In the absence of such provisions, the entire burden of discerning truth from falsehood falls on the reader.
Akhil Bhardwaj
Akhil Bhardwaj studies extreme events, which range from organizational disasters to radical innovation. Akhil is interested in the epistemological problem of understanding the underlying dynamics that lead up to these events. He also studies how thinking can be improved as well as the implications of AI adoption in the context of strategic management, entrepreneurship, and high-risk systems.
In the U.S., the First Amendment is intended to protect the right to speak, argue, persuade and offend. But freedom of speech is not free of caveats. It does not allow people to incite others to commit crimes, to threaten or to defame, for example. And if speech causes material harm, speakers can be held liable for those harms. When a company chooses to put a synthetic answer engine between users and the web, it is no longer merely hosting speech; it is producing an amalgamation of complex mathematical expressions that, outputted as text, resemble human speech. AI companies want this text to enjoy the same protections user-generated text has, while simultaneously dodging all the responsibility associated with being a speaker.
The roots of this dilemma go back to the 1990s, when the advent of online forums and social media created a new problem. Unlike traditional publishers, forum hosts needed to provide a platform for their users’ voices, without being liable for what their users were saying. This problem was addressed with Section 230 of the Communications Decency Act, enacted in 1996. Section 230 was a bipartisan amendment written to preserve the internet as a space where ordinary people could speak (or post) without the forum host becoming liable for every third-party post.
That broad immunity reflected a democratic judgment: If the law made platforms responsible for all user content, many would censor aggressively or stop hosting speech altogether. This would limit free speech. Section 230 was meant to protect the ecosystem of human expression. In this sense, hosts of online spaces can be seen as providing a public square where speech occurs.
Free speech is a human right — it protects people as speakers and listeners in a democratic public sphere.
The lawmakers who passed Section 230 three decades ago could not have foreseen a world populated by chatbot-generated text. As such text increasingly leads to real-world harms, lawsuits are proliferating and tech companies are deploying a number of often-contradictory legal strategies to avoid culpability. In some cases, they are arguing that AI-generated text is not speech, but rather simply a tool, and that companies are therefore protected as “carriers,” not “publishers” by Section 230’s protection of a public forum for free expression.
Get the world’s most fascinating discoveries delivered straight to your inbox.
But the companies deploy this argument only when it suits them.
In other cases, they are increasingly reaching for free-speech language to defend AI-generated text because free-speech protections provide broad legal immunity. For example, in a Florida wrongful-death lawsuit against Open AI (maker of ChatGPT), a plaintiff has alleged that the company’s chatbot pushed a 14-year-old to take his own life. OpenAI argued that the chatbot was protected by the First Amendment, though the judge dismissed that defense and allowed the case to proceed.
Neither of these arguments is convincing. AI companies are not merely providers of a public forum, as the words produced by their AI summaries and chatbots are generated by the company’s products.
Similarly dubious is the claim that bots should be seen as equal participants in a public square. This is a category error. Free speech is a human right — it protects people as speakers and listeners in a democratic public sphere. Bots do not vote, deliberate, dissent, worship or participate in civic life. They generate text, but they do not possess a moral and political standing. Bots have no skin in the game.
What, then, justifies constitutional protection in the first place? Extending the strongest speech protections to machines would not defend liberty; it would confuse “botput” with free expression. It would, in actuality, extend the strongest free-speech protection to companies. But that requires a separate line of argumentation that ought to be agreed upon by society.
Open AI, the maker of ChatGPT, argued the chatbot has First Amendment protections.
(Image credit: Jaque Silva/NurPhoto via Getty Images)
The Munich court’s limited and nuanced way of governing “botput”provides a clear way forward.
Given its history with Nazism, Germany does not enshrine free speech quite the way the U.S. does. But the German court’s arguments still provide a useful template for a future U.S. ruling.
The Munich court held that if a system simply points users to sources, it resembles traditional search and should continue to enjoy broad protection afforded to aggregators. If it synthesizes claims, imitates the tone of authority, and offers a single authoritative answer generated by an AI, it should carry corresponding duties of care that entail liability for the company.
The need for such safeguards is only growing. AI-generated summaries can be copied instantly, scaled globally, and repeated across interfaces until a falsehood becomes regarded as “truth.” That is not a hypothetical concern; it is already happening.
Moreover, it is important to remember that the original intention of Section 230 was to insulate platforms from liability for third-party posts, not their own text.
This is not an anti-innovation argument. AI can be helpful, efficient and genuinely transformative. The law should encourage useful tools while insisting that the companies deploying them remain responsible for the foreseeable harms of their products.
We need clearer rules that keep the internet free for people while preventing machines from laundering falsehood into authority. The German ruling points toward that future. The sooner U.S. law and policy follow, the better chance we have of preserving our shared reality and a healthy democracy.
Opinion on Live Science gives you insight on the most important issues in science that affect you and the world around you today, written by experts and leading scientists in their field.
