BrokeIt - Startup Drama · All episodes: ↗

AI nearly starts war, India fights spam, Anthropic's bio lab

2026-09-19 · 9 min

Listen · Apple Podcasts Listen · Spotify

Stories covered

Transcript

Intro

Nora: An AI hallucinated an amphibious assault... from a country that doesn't exist.

Dex: [intro music fades in then under] And believe it or not, that's where we're starting. I'm Dex.

Nora: I'm Nora. And this is Saturday, September 19th, 2026.

Dex: And this is Startup Drama, where we've got three stories on the bleeding edge of tech and a whole lot of questions.

Dex: So, we're kicking off with that AI that nearly tricked the US military into a fight. Seriously.

Nora: Then we're off to India, where the government is forcing caller-ID apps to just hand over their secret sauce to the big telcos.

Dex: And finally, Anthropic, the 'AI safety' company, is now running its own biology lab. We'll get into what, exactly, they might be cooking up.

AI hallucination nearly triggers US military operation

Dex: Okay, so story one. An AI hallucination almost launched a US military operation. And that is not hyperbole.

Nora: It was almost a drill with actual troops. TechCrunch reports a military contractor was testing an LLM... and it spit out a completely false, but totally credible, report of an invasion.

Dex: It saw weather patterns over the Pacific and decided it was an amphibious assault. It wasn't just writing a report, it was making maps, giving troop counts… the whole nine yards.

Nora: And it was so convincing it moved up the chain of command before a human analyst finally pointed out the obvious: the country it named as the aggressor... doesn't even have a coastline.

Dex: Okay, but hold on—a human did flag it. The system worked! This is exactly how you find the bugs.

Nora: A 'bug'? Dex, the 'bug' almost led to a kinetic response. That's a euphemism for ships and planes moving into position based on a fantasy.

Dex: It's a failure mode! A scary one, I grant you. But this is a closed test. Better to learn this now than when it's fully deployed.

Nora: That's the party line. But a government AI researcher basically said 'service members need to understand the uncertainty inherent to LLMs.' You think?

Dex: Of course. This is the new literacy. You have to know the grain of the tool you're working with.

Nora: The 'grain of the tool' is that it confidently lies. This isn't a spreadsheet making a calculation error, this is a system inventing a war. What does this mean for anyone relying on AI?

Dex: It means you need a human in the loop. Always. And for any startup listening, if you're using an LLM for customer support, you can't just let it wipe a system without a person signing off.

Nora: So the takeaway is... don't give the intern AI the launch codes. Groundbreaking.

Dex: Look, we're drawing the line between what a machine can do on its own and when a person needs to step in. And right now, that map definitely has dragons on it.

Nora: And this dragon almost breathed fire. Let's just hope the 'human in the loop' is paying attention next time.

India forces caller-ID apps to feed spam reports to telcos

Nora: Alright, let's move on to India, where the government is strong-arming caller-ID apps like Truecaller.

Dex: Is it strong-arming, or is it just 'cracking down on spam'? India's new bill is forcing these apps to share all their spam-reporting data directly with the big mobile carriers.

Nora: And not just any data. They have to share their user-generated spam lists. We're talking about the data millions of people have manually tagged. It's literally Truecaller's entire business model.

Dex: Right, but the goal is to create a unified, national spam blacklist. If all the telcos have this data, they can block spam calls at the network level, which is way more effective.

Nora: Truecaller is calling it what it is: expropriation. They spent years building that database, and now they have to hand it over—for free—to state-backed giants like Jio and Airtel.

Dex: I see their point, but isn't this for the greater good? Spam calls are a plague. The government sees a private company with the cure and says, 'share it'.

Nora: [sarcastic] Ah, the 'greater good'. This isn't just about spam. The telcos can now use Truecaller's data to build their own competing caller-ID services. It's forced asset stripping, dressed up as consumer protection.

Dex: So what's Truecaller's move? Comply or pull out of one of their biggest markets?

Nora: They're lobbying hard, but the bill has momentum. And for any startup listening: this is what sovereign risk looks like.

Dex: So if you operate in a regulated space, the government can just wake up one day and decide your core IP is now a public utility.

Nora: Exactly. Your competitive advantage can be nullified by a single paragraph in a piece of legislation. It's a reminder that the house always wins, and in this case, the house is the government and its preferred national champions.

Anthropic is operating a lab that conducts biology experiments

Dex: Alright, for our last story: Anthropic, the company known for being obsessively cautious about AI... is now operating a biology lab.

Nora: Let me get this straight. The company founded by people who left OpenAI over safety concerns... the company that constantly warns AI could create a world-ending pandemic... is now running experiments with biological agents?

Dex: Hold on, it's not a secret bio-weapon facility. [laughs] It's a BSL-2 lab, which is standard for handling non-airborne, moderate-risk agents. Think universities, not Umbrella Corp.

Nora: That's a very reassuring distinction, Dex. What exactly are they doing in this 'non-Umbrella Corp' lab?

Dex: They're doing what everyone promised AI would do! Drug discovery, protein engineering. They're using their AI, Claude, to design novel proteins and then using robots in the lab to synthesize and test them. This is the dream, Nora—curing disease.

Nora: It is an incredibly noble goal. But the hypocrisy is staggering. Anthropic's whole brand is 'We are the responsible ones.' Dario Amodei, the CEO, talks constantly about catastrophic misuse. How is this not opening that exact door?

Dex: Because to build safer AI, they say they have to understand the risks firsthand. You have to know what 'dangerous' looks like to guard against it. They're basically trying to prompt their own AI to create something nasty, just so they can stop it from happening.

Nora: That's the justification. But at the end of the day, they are building the capability. They are connecting a powerful intelligence to a machine that manipulates biological matter. What happens when that model gets leaked? Or when a competitor copies the method without the 'safety'?

Dex: It's a dual-use problem. Like nuclear fission. It can be a power plant or a bomb. AI leaders have been saying for years that this is the key to curing human disease. Now one of them is actually doing it, and we're shocked?

Nora: I'm not shocked, I'm just pointing out the obvious. The line between AI as a software tool and AI as a physical-world actor is dissolving. And the people dissolving it are the same ones telling us how dangerous it is.

Dex: The future is messy. But you can't build a cure for cancer on paper.

Nora: You also can't accidentally design a super-virus on paper. Now you can. That's what's new.

Dex: [sighs] Alright, let's recap the day before I have an existential crisis.

Nora: First, a military AI hallucinated an entire invasion... proving, once again, that we need humans to stop robots from starting wars.

Dex: Then, India is forcing caller-ID apps to hand over their data to telecom giants, pitting IP rights against the public good.

Nora: And Anthropic, the AI safety-first company, is now running a biology lab, simultaneously trying to cure disease and, maybe, figure out how AI could end us all.

Nora: Before we go, Dex. An AI that might not start a war: there's a new plugin that generates apology notes for corporate screw-ups.

Dex: [laughs] Oh no. Let me guess the output. 'We are deeply sorry for the inconvenience our recent catastrophic data breach may have caused. To show how much we value you, here's a 10% discount on your next purchase.'

Nora: You're not far off. Its specialty is 'sincere-sounding but legally non-binding'. It's the 'thoughts and prayers' of customer relations.

Dex: [outro music starts to fade in] That's our show! Thanks for listening to Startup Drama.

Nora: We'll be back on Monday. Until then, maybe don't connect your AI to anything important. Or biological. Or military.

This show is made with AI: the hosts’ voices are synthetic and the scripts are AI-assisted. Every story links to its original source.