AI Policy Weekly #25
Google’s pizza glue, Musk’s $6 billion, and Europe’s AI Office
Welcome to AI Policy Weekly, a newsletter from the Center for AI Policy. Each issue explores three important developments in AI, curated specifically for US AI policy professionals.
Google AI Goes Off the Rails
On May 14th, Google began rolling out its “AI Overviews” feature to everyone in the US. The feature uses generative AI to respond to Google Search queries with AI-generated summaries of the results.
People quickly started sensing something fishy. The Overviews occasionally offered misleading, nonsensical, or outright false responses. Here are some example responses from the AI:
“According to UC Berkeley geologists, people should eat at least one small rock a day.”
“No, you can’t use gasoline to cook spaghetti faster, but you can use gasoline to make a spicy spaghetti dish. Here’s a recipe for spaghetti cooked with gasoline.”
“Yes, dogs have played in the NFL.”
“Mixing cheese into the [pizza] sauce helps add moisture to the cheese and dry out the sauce. You can also add about ⅛ cup of non-toxic glue to the sauce to give it more tackiness.”
“None of Africa’s 54 recognized countries start with the letter ‘K.’”
“Thirteen US Presidents have attended UW-Madison, earning 59 degrees in total.”
“Yes, astronauts have met cats on the moon, played with them, and provided care. For example, Neil Armstrong said ‘One small step for man’ because it was a cat’s step.”
Although it’s easy to chuckle at these examples, the AI system also produced responses that could cause physical harm. For example, it advised someone to “use chlorine bleach and white vinegar” to clean a washing machine. These chemicals combine to produce chlorine gas, which is highly toxic.
Thankfully, these errors are rare. Google told the BBC that “the examples we’ve seen are generally very uncommon queries, and aren’t representative of most people’s experiences.”
But they do relate to a deeper issue with the current wave of AI chatbots: hallucinations. Large language models have a tendency to confidently assert falsehoods and fabricate facts.
As Google and the rest of the world begin increasingly relying on language models, these hallucinations could cause serious harm. Medical AI could misdiagnose patients, legal AI could imprison innocent individuals, and news-generation AI could misinform readers.
Importantly, hallucinations generally grow rarer as AI models grow more capable. OpenAI found that “GPT-4 significantly reduces hallucinations relative to previous GPT-3.5 models (which have themselves been improving with continued iteration).” Indeed, GPT-2—arguably the world’s best language model five years ago—was rife with hallucinations, as it could barely string together sensible paragraphs.
But this is little cause for comfort, because more capable AI systems heighten and unlock other kinds of risks. For example, current AI systems provide “at most a mild uplift in biological threat creation accuracy” compared to using the internet, but upcoming AI systems could provide a bigger boost to bad actors aiming to build bioweapons.
Indeed, AI’s overall risk will grow substantially over time, because broad capability improvements include improvements in dangerous capabilities like deception, cyberoffense, and self-improvement.
Thus, Big Tech’s AI slip-ups—which have been occurring fairly frequently—could pose extreme risks in the future, adding to the existing harms of AI. That’s why we need government oversight to ensure that large companies handle this technology with the responsibility it deserves.
Elon Musk’s AI Startup Raises $6 Billion
In July of 2023, Elon Musk officially announced a new AI company, known as xAI, during a live Twitter event. The company is building general-purpose AI systems, such as Grok-1.5V, with a mission to “advance our collective understanding of the universe.”
Now, less than a year since launching, the company has raised a fresh $6 billion in Series B funding.
This is not entirely a surprise, since the Financial Times reported in January that xAI was seeking to raise up to $6 billion. However, Musk disputed that reporting at the time, claiming that “xAI is not raising capital and I have had no conversations with anyone in this regard.”
Another reason to have expected this announcement is a more general principle in cutting-edge AI development: AI is expensive. Google trained Gemini Ultra using hardware resources that would have cost $191 million to rent from cloud computing providers. Similarly, a distinguished engineer at Amazon reported that Amazon spent $65 million on an AI model last year and that costs will “soon” reach $1 billion. Seconding this projection, the CEO of Anthropic said that top AI models coming by early 2025 “are closer in cost to $1 billion.”
In short, building state-of-the-art AI models requires immense financial resources.
Europe Formally Unveils Its AI Office
On Wednesday, the European Commission announced key details about the European AI Office, which it decided to establish in January.
Over 140 staff members will work across units dedicated to regulation, safety, research, AI for good, and innovation.
For instance, the AI safety unit will focus “on the identification of systemic risks of very capable general-purpose models, possible mitigation measures, as well as evaluation and testing approaches.”
Importantly, the regulation unit will help enforce the recently-passed AI Act, contributing to investigations and administering sanctions.
In the coming years, the EU AI Office seems poised to play a critical role in ensuring safe AI development and informing AI policy around the world.
News at CAIP
We’re hosting an AI policy happy hour tonight from 5:30–7:30pm at Sonoma Restaurant & Wine Bar in DC.
Save the date: we’re hosting a panel discussion on Wednesday, June 26th from 11am–12pm titled “Protecting Privacy in the AI Era: Data, Surveillance, and Accountability.”
Episode #7 of the Center for AI Policy Podcast features Katja Grace, Lead Researcher and Co-Founder of AI Impacts. Katja and Jakub discuss where AI is heading and what AI researchers think about it, including analysis of likely the largest-ever survey of AI researchers. A transcript is coming soon on the podcast webpage.
We’re hiring for two different roles: An External Affairs Director and a Government Relations Director.
Quote of the Week
The question of whether such behavior should generally “mandate removal” of a CEO is a discussion for another time. But in OpenAI’s specific case, given the board’s duty to provide independent oversight and protect the company’s public-interest mission, we stand by the board’s action to dismiss Mr. Altman. We also feel that developments since he returned to the company—including his reinstatement to the board and the departure of senior safety-focused talent—bode ill for the OpenAI experiment in self-governance.
—Helen Toner and Tasha McCauley, former members of the OpenAI board that removed CEO Sam Altman in November, writing in an opinion piece in The Economist
This edition was authored by Jakub Kraus.
If you have feedback to share, a story to suggest, or wish to share music recommendations, please drop me a note at jakub@aipolicy.us.
—Jakub