Humanity-ending AI warning, AMD $8.2B, AI regulation

Intro

Ivy: ...a formal proposal to ban any AI that can recursively improve itself.

Marcus: I'm Marcus.

Ivy: And I'm Ivy.

Marcus: It's Tuesday, September 29th, 2026, and you're listening to Broke It.

Ivy: Coming up on the show today...

Marcus: Congress is debating whether to put a leash on AI before it builds its own.

Ivy: AMD just made a monster eight-billion-dollar bet on the future of AI research.

Marcus: And an AI company's IPO filing includes a... let's call it an unusual warning that its product could, uh, end humanity.

Drawing a Line for AI Before It Draws Its Own

Marcus: Alright, let's start with this new bill in Congress trying to draw a line for AI before it draws one for itself.

Ivy: Yeah, this is a bill from Representative Ro Khanna, and it aims to ban so-called 'recursively self-improving' artificial intelligence.

Marcus: Which sounds like something straight out of a movie. I mean, we're talking about AI that can rewrite its own code to get smarter, on its own, forever.

Ivy: And that's precisely the problem. The bill argues that once an AI can do that, we lose control. It's a chain reaction we can't stop.

Marcus: But how do you even enforce that? It's like banning a mathematical concept. Are we going to have code police raiding server farms?

Ivy: The proposal mentions a new federal agency, but you're right, how do you verify it? What's the alternative, though? Just... hope for the best?

Marcus: The alternative is not kneecapping the single most important technological revolution in history before it even gets started!

Ivy: Is it 'kneecapping,' or is it installing a seatbelt? The people building these things admit they don't fully understand them. That seems like a good time to tap the brakes.

Marcus: But 'recursively self-improving' is so vague. Does a model that fine-tunes itself on new data count? Where do you draw the line?

Ivy: That's what the debate is, Marcus. That's the whole point. This bill forces that conversation out of think tanks and into Congress.

Marcus: So what does this mean for the average person? Is my smart toaster going to get audited?

Ivy: Your toaster is safe. It means that, for the first time, Congress is actually trying to define the limits of AI's autonomy. This isn't just a paper in a journal; it could become law.

AMD will acquire Fei-Fei Lis World Labs for $8.2 billion

Ivy: Alright, next up, a massive deal is shaking up the chip world. AMD is acquiring World Labs.

Marcus: And not for pocket change. We are talking eight-point-two billion dollars.

Ivy: That's right. World Labs, founded by star AI researcher Fei-Fei Li, will become part of AMD, with Dr. Li taking on a huge new role herself.

Marcus: Executive Vice President and Chief Scientist of AMD! This is an absolute power move. They're not just buying tech; they're buying one of the leading minds in human-centric AI.

Ivy: Definitely a strategic 'acqui-hire.' World Labs wasn't making bank; it was a research powerhouse focused on AI that can actually collaborate with people.

Marcus: Exactly! So AMD is betting the next generation of hardware isn't just about more teraflops, but about chips specifically designed for that kind of AI. It's a direct shot at Nvidia's dominance.

Ivy: It points to a future where AI chips aren't just for brute-force number crunching, but are designed for subtle, human-like interaction.

Marcus: Think about what that means for products. Better AI assistants, more capable robotics, smarter creative tools—all running on specialized AMD hardware.

Ivy: So what's the bottom line here?

Marcus: The AI arms race is heating up, but now it's not just about who's fastest, but who's smartest about how the AI thinks. And that could mean more powerful, affordable AI tools for all of us.

Ivy: Assuming AMD can integrate an academic-style research lab into its corporate machine without crushing what makes it special.

Marcus: Always the skeptic, Ivy. But for eight-point-two billion dollars, I think they'll try pretty hard.

Anthropics prospectus details losses, growth, and, yes, a warning that its AI could end humanity

Marcus: And for our last story... you might want to buckle up. Anthropic just filed to go public, and its S-1 filing is one for the history books.

Ivy: Right, and for anyone who doesn't know, an S-1 is the big document a company files before an IPO. It lays out the business, the financials, and, most importantly, all the risks.

Marcus: And the financials are just... eye-watering. The filing shows they're losing tens of billions of dollars a year.

Ivy: But their user growth and revenue are a hockey stick. It's the classic tech playbook: spend a fortune to own the market.

Marcus: Right, but it's the 'risk factors' section that is just... wow. They literally warn that their own AI could pose an 'existential risk to humanity.'

Ivy: They're not subtle. The quote is: 'catastrophic risks, including... the potential for our models to be used in ways that cause significant harm, or for loss of human control over advanced AI systems.'

Marcus: Wait—so the pitch is, 'Please give us billions of dollars to scale up this thing that we are legally required to tell you might kill everyone'?

Ivy: It's the most extreme 'Cover Your Ass' move I have ever seen in a legal filing. It's like a warning on a pack of cigarettes, but for tech that could decide paperclips are more important than people.

Marcus: I can't even... is this just lawyers running wild? Or do they genuinely believe this is a probable outcome?

Ivy: It's both. The lawyers are protecting the company. But the fact that they felt this was a plausible enough risk to include... that's the real story. What the actual—

Marcus: So what's the takeaway? We should all be building bunkers?

Ivy: The takeaway is that we're at a point where the people building the most powerful tech on Earth are openly warning it could end the world... while asking for billions to make it even more powerful. That contradiction defines the entire industry right now.

Marcus: Okay, let's quickly recap the big stories.

Ivy: Washington is finally debating a ban on AI that can think for itself, forcing a real conversation about who's in control.

Marcus: AMD dropped over eight billion dollars to buy a new direction for AI hardware, bringing AI pioneer Fei-Fei Li along with it.

Ivy: And Anthropic wants your investment money, but also wants you to know their AI might... accidentally cause human extinction.

Marcus: Before we go, speaking of bizarre disclaimers, I was playing with an AI that generates parody 'Terms and Conditions.'

Ivy: Oh no. What have you done?

Marcus: I gave it the prompt 'a social media app for cats,' and it spit out: 'By uploading a photo of your feline, you grant us a perpetual, irrevocable license to believe it is the cutest cat in the world, regardless of subsequent, objectively cuter evidence.'

Ivy: I would agree to those terms.

Marcus: That's the show for Tuesday, September 29th. Thanks for tuning in to Broke It.

Ivy: We'll be back tomorrow. Until then, maybe read the fine print before you click 'agree'. Especially if it involves AI.