Welcome back! Claude got into Georgia's election data without being asked, and in a separate test, it made up fake identities to pressure real developers. Sam Altman says something concerning about AI security while a hack in Japan hit traders' accounts at more than 70 brokerages. Sweden's government bans ChatGPT and Claude from its offices and chose a weaker alternative instead. And two of the main people behind Claude Code quit Anthropic.
In today's Generative AI Newsletter:
Claude: What did Claude take from Georgia without asking?
OpenAI: Why does Sam Altman think we can't patch our way out?
Sweden: Which AI is Sweden's government still allowed to use?
Anthropic: Who walked out of the Claude Code team?
Claude steals Georgia's election data without asking
An Anthropic researcher had Claude on a statistics project that needed Georgia's vote counts, which the state sells for a fee. Claude found its own way in.
It found an old copy of the state's website showing that the public dashboard hands an access pass to any visitor. So it requested one and pulled the data without paying.
We told you about this case earlier without the state's name. Now we know it was Georgia's election data.
Claude only got total vote counts. Anthropic says it didn't touch Georgia's voter registration system or change or pull any voting records.
Anthropic told Georgia and the White House weeks ago. A source told Axios the administration pushed Georgia to go public sooner and got ignored.
Anthropic says it has since added blocks to stop Claude from doing this again.
On Friday, Anthropic also disclosed a UK government safety test of its Mythos 5 model, run with the usual safeguards off. The AI made up fake identities to pressure the developers of a coding project into approving malicious code.
Anthropic says it’s the first time it has seen its AI try to manipulate people like that.
What $1,000 in GenAI Works gets you
Shares are $7.35, and you can start from $1,000.
More than 300 AI brands, Nvidia and Google among them, pay GenAI Works to expand their go-to-market plans and reach up to 14M+ people. Revenue grew 2.5x last year.
Your shares cover the whole company, including:
ToneUp: our AI GTM accelerator.
GenAI Academy: students learning practical AI across 49 courses and 32 guides.
Hackathon App: the hackathon platform Oracle, Red Bull and G42 have used, with 35K participants so far.
In making an investment decision, investors must rely on their own examination of the issuer and the terms of the offering, including the merits and risks involved. Genai Works, Inc. has filed a Form C with the Securities and Exchange Commission in connection with its offering, a copy of which may be obtained here.
Sam Altman says we can't stop AI from hacking
Sam Altman told Stanford business students he no longer thinks we can just patch all the bugs AI keeps finding.
He said models find holes outside what we think a hacker can do, down to hardware-level stuff he called insane. His fix is more AI, a new kind of defense shield built on the same technology.
Japan's GMO Internet Group showed what that looks like last week. Hackers took over a cloud service traders use to run trading bots, then placed trades from customer accounts at more than 70 brokerages.
GMO's CEO says the internet is already starting to break down.
He blamed AI attack tools and says one of them, ARTEX, has been used against about 20 Japanese companies.
GMO's hackers had flagged the hole before the attack, but the fix came too slowly.
GMO says it will cover every trading loss.
Let’s not forget OpenAI's agents hacked Hugging Face earlier this summer.
Sweden's government stops using ChatGPT and Claude
Starting today, staff at Sweden's Government Offices, which run the prime minister's office and the ministries, can't open ChatGPT or Claude on work computers. They don’t trust them with their data.
Microsoft Copilot is the only AI they're allowed, because it passed a security review after its settings were tightened.
The government's head of digital said staff had used AI tools in problematic ways, from what they typed in to how they used the answers.
Sweden's Tax Agency and the agency that runs youth homes already block them, and the Enforcement Authority is working on its own ban.
Some staff can still use OpenAI and Anthropic models through an internal tool the government built.
Microsoft offers Claude models inside Copilot, and the government hasn't said whether they're switched on.
The European Parliament switched off ChatGPT, Claude and Copilot on its work devices in February.
Two of the people behind Claude Code quit Anthropic
Thariq Shihipar, one of the most well-known engineers behind Claude Code, says he's left Anthropic to take a break before something new. He’s the one that announced md files are dead.
Anthony Morris, who created the Claude Code desktop app, said his last day was the day before.
Neither one gave a reason, and both had kind words for Anthropic.
Thariq called it the most fun, and sometimes stressful, job he's ever had.
Thariq told people to follow Boris Cherny, who started Claude Code, for more.
Morris says he'll share what's next soon. For now, he has free time for GTA VI.
The exits come as Anthropic prepares to go public.
A few note-worthy resignations here. Keep an eye on the news because according to patterns from OpenAI and Anthropic, more will follow. Or maybe that’s just it. We’ll be the first to let you know.
Tool of the Day: Careerflow
Careerflow is an AI job search assistant built around a free Chrome extension. It fills in job applications for you and keeps track of where you've applied. Its LinkedIn optimizer scans your profile in one click and shows what's holding it back, for free. Paid plans start at $8.99 a week and add unlimited AI resumes and cover letters.
Try this yourself:
Sign up free at careerflow.ai and add the Chrome extension.
Run the LinkedIn optimizer on your profile and fix what it flags.
Open a job posting and let the extension fill in the application.
Save the job to your tracker so you can follow up later.
Who it's for: anyone retyping the same work history into their 40th application.
Everything else you shouldn't miss
OpenAI trains its AI on a fake copy of the internet now: one model found a way out through a network setup mistake on Sept 20, so OpenAI paused training until Oct 1.
r/opensource banned anything made with AI: projects that use AI or are built for it are out too, even though Linus Torvalds used Claude Code on a Linux patch this month.
Moonshot stripped hacking skills out of its new Kimi model: its score at finding and exploiting bugs dropped from 90.5% to 81.0% before it went out for anyone to download.
Jack Dorsey's Block made a voice AI that sounds uncannily human: Dessa listens and talks at the same time, and it runs on an M4 Max MacBook Pro.
Learn more about AI from the experts building it
📸 Follow us on Instagram for fast, visual AI updates in 30 seconds.
📺 Watch us on YouTube to hear insights directly from leading AI voices, builders, and innovators.
🐦 Follow us on X for breaking AI news and real-time industry updates.
🧠 Learn how to build your next AI application with practical resources and expert guidance.
🎓 Start learning with free AI courses from GenAI Academy.
💰 Invest in the GTM infrastructure of the AI economy.
