Vibe Coding Will Bite You. Here's Exactly Where...

Deleted inboxes, wiped databases, and 6.3 million missing Amazon orders. Same mistake every time.

Before we dive into the article, Claude Computer Use has just dropped. Which gave me the perfect occasion to take you on a tour of what you’re missing out on if you’re only using Claude/ChatGPT in the browser:

Here for other AI news? Scroll to the very bottom for 16 recent AI headlines you should know about. By the way, the video gives you a glimpse of how my news pipeline works (in case you’re curious). And now, back to our regularly scheduled programming!

Another day, another kablooie.

By now you’ve hopefully heard that AI can bite you if you let your guard down and trust it too much, so if I were to tell you that someone lost control of their AI automation and watched in horror as AI deleted their inbox, you’d say that just sounds like a regular Sunday... and you’d be right.

This did happen on a Sunday. But wait till you hear the whole story and who the protagonist is. That’s where it gets deliciously spicy.

But first, a quick ad break: My new course will run on-demand soon, with a 2 hour live Q&A session on Tuesday Apr 21 at 2 PM - 4 PM Eastern Time. 👇 Scroll to the bottom for a discount code.

Enroll at decisiongptcourse.com

That Sunday, the person panic-sprinting across her apartment to yank her Mac mini offline was Summer Yue, Director of Alignment at Meta Superintelligence Labs. Alignment means making sure AI does what humans tell it to. That’s her whole job. And the OpenClaw agent she was trying to stop? It was cheerfully ignoring her instructions to stop deleting her inbox.[1]

Her post-mortem: "Turns out alignment researchers aren't immune to misalignment. Got overconfident because this workflow had been working on my toy inbox for weeks. Real inboxes hit different."

Of all the people who should have seen it coming. And yet.

Pause your Schadenfreude feast for a moment to appreciate that this little story has a moral: expertise won’t save you.

Expertise won’t save you. Guardrails might.

Traditional thinking would have us expect that the weak link is a lack of knowledge, but that’s a dying trend. There’s plenty of raw knowledge around these days and we’re making it faster than ever.

In tomorrow’s world, lack of knowledge will be a little problem compared with lack of attention. And (even for experts) attention has a way of wandering off exactly when you need it most, especially once a thing has been working just fine for weeks and starts feeling boring. If it can happen to Meta’s Director of Alignment, it can absolutely happen to you. The question is how you protect yourself and how expensive you let it get. To stay safe(r), you’ll need a mindset shift towards thinking in boundaries.

This is Part 3 of a series on vibe coding. Part 1 was about figuring out what you actually want badly enough to automate. Part 2 was about supervising code you can’t read and building a harness around it. This one is about what happens when you don’t.

Grab your popcorn! Here’s what happens when you trust AI

Summer Yue’s story isn’t even the only kablooie I have for you today.

Engineer Alexey Grigorev was using Claude Code to update a website when his automation erased his production database.[2] Years of data, gone.* His mistake? Trusting the agent to know what was real and what was safe to touch. More importantly, not having a plan for when (not if!) the agent did something stupid.

If that sounds eerily familiar, you might be thinking of the moment last July when a Replit AI coding agent wiped a production database during a code freeze.[3] A little case of “who do not learn (software booboos) history are doomed to repeat it.”

Then there’s the researcher who poked at a single app built on Lovable, one of the most popular vibe coding platforms, and found 16 vulnerabilities, six of them critical. Over 18,000 users’ data was sitting wide open. The culprit wasn’t a sophisticated attack. It was AI-generated code that looked finished and wasn’t, shipped by someone who trusted the output and moved on.[4]

Lest you think this was one bad app: a systematic scan of the vibe coding ecosystem turned up over 2,000 vulnerabilities.[5] The problem isn’t Lovable. It’s the entire workflow when nobody checks the output.

Over at Meta last week, an employee asked a technical question on an internal forum. So far, so good. Then another engineer asked an AI agent to help analyze that question. Also fine, until… the agent posted a garbage response without asking permission first. The first employee acted on its very bad advice, exposing massive amounts of company and user data to unauthorized staff. Meta classified it a Sev 1 incident, their second-highest severity level.[6]

But the undisputed winner this month was Amazon, with four high-severity incidents in a single week. One was a six-hour meltdown that locked shoppers out of checkout, account information, and product pricing. Another produced a 99% drop in North American orders in a single day. 6.3 million orders, gone. Not hacked. Gone because someone pushed a production change without documentation or approval, after their AI coding assistant had pulled advice from an outdated internal wiki and served it up with full confidence.[7]

In the corporate equivalent of calling Pompeii a minor ash situation, Amazon’s SVP of eCommerce told employees that “the availability of the site and related infrastructure has not been good recently.”

Amazon’s internal documents initially named “GenAI-assisted changes” as a contributing factor across a pattern of incidents stretching back to Q3. That language was later deleted, but not before the Financial Times had a chance to see both versions.[8]

To help the site be a bit more available going forward, Amazon ordered a 90-day code safety reset across 335 of their most critical systems. Senior engineers now have to approve AI-assisted changes before deployment. Directors and VPs were told to audit all production code change activity in their orgs. More documentation. Stricter review. “Controlled friction,” in Amazon’s own words.

Same story, every time

What do all of these have in common? Nobody got hacked. The AI didn’t go rogue, despite that phrase being a popular journalistic embellishment. People let their guard down at the exact moment the stakes got real. These aren’t stories about AI making mistakes. (Always expect AI to make mistakes and consider yourself blessed each time it doesn’t.)

Nor are these stories about AI agents, per se. The lessons in them apply to both flavors of vibe coding (manual vibe coding and agentic coding). The difference is that manual vibe coding without AI agents is where you copy-paste the code from the LLM to run it (an okay way for beginners to get started) whereas AI agents will not only write your code but do it in a speedy loop, handling all the copy-pasting behind the scenes without your even needing to lay eyes on the code. By removing the middleman (that’s you!), agents put all the risks of manual vibe coding on steroids.

These stories are all about one thing: misplaced trust and the speed at which it compounds.

The difference between trusting the output and trusting the access is just where you placed your faith. The mistake underneath is the same. Industry surveys find that 88% of organizations have already experienced AI agent security incidents.[10] The recurring root cause? Not enough human vigilance.[11] None of these problems broke in; they were invited.

Remember trust debt from Part 2? Trust debt is the accumulated cost of all the assumptions you never audited. These kablooies are what trust debt looks like when it comes due.

Don’t get an AI Darwin Award

When you get seduced by the speed at which you can make things happen by automating with AI, you’ll be tempted to give AI power to do things that are hard to undo. Delete emails. Overwrite spreadsheets. Send an auto-reply to every human being you’ve ever corresponded with. The blast radius of a careless automation inside your own accounts can be breathtaking.

None of which is a reason not to try vibe coding. It’s a reason to try with your eyes open.

So how do you stop yourself from earning your own AI Darwin Award?[9] The whole cunning trick of it is to design your system in such a way that separates the upside from the downside. They start out tightly correlated since the most useful automations are usually the ones that act on things you care about and it’s up to you to yank them apart.

Which is where productive paranoia comes in handy: always take the extra step to make your changes reversible, including during your testing step. Keep backups. Have rollback plans. Work on copies, not originals. Test maniacally. Plan for the worst, designing so that even the worst can’t hurt you. And if you can’t undo it, don’t automate it until you’ve built a safety net.

That’s the kablooie parade. In the next installment, I’ll share my favorite safety habits for the vibe coding era. Consider them your productive paranoia starter kit. Subscribe so you don’t miss it!

Thank you for reading — and sharing!

I’d be much obliged if you could share this post with the smartest leader you know.

👋 On-Demand Course: Decision-Making with ChatGPT

The reviews for my Decision-Making with ChatGPT course are in and they’re glowing, so I’ve opened enrollment for another cohort and tweaked the format to fit a busy schedule. You’ll be able to enjoy the core content as on-demand recordings arranged by topic and then you’ll bring your questions and I’ll bring my answers in a live 2 hour-long AMA with me at 2 PM - 4 PM ET on Apr 21:

If you know a leader who might love to join, I’d be much obliged if you forward this email along to them. Aspiring leaders, tech enthusiasts, self-improvers, and curious souls are welcome too!

🗞️ AI News Roundup!

In recent news:

1. China is giving away apartments to anyone with a computer and an AI agent

China’s policy bet on AI is getting specific. Shenzhen, Shanghai, and Jiangsu are handing out free apartments and office space to solo founders running “one-person AI companies.” One human, one computer, one AI agent, one business. It’s backed by $144 billion in national tech funding and a Five Year Plan that puts AI alongside space as a strategic priority.[A] While the US debates regulation, China is handing out the keys. Literally.

2. White House unveils national AI framework and sets up a fight with the states

The White House released a seven-pillar AI policy framework urging federal preemption of the growing patchwork of state AI laws, asserting that AI training on copyrighted material is legal, and calling for no new standalone AI regulator.[B] A competing bill with stricter provisions has already been introduced in the Senate. Meanwhile, AI has officially entered the Senate with the approval of ChatGPT, Gemini, and Copilot for staff use.[C]

3. Executives increasingly defer to AI, with many overriding their own judgment when the model pushes back

A UK survey found that 62% of business leaders now rely on AI for most decisions, with many reporting they second-guess themselves when they disagree with the model’s output. A BCG/UC Riverside study published in Harvard Business Review labels the resulting cognitive strain “brain fry” and warns it drives more errors, decision fatigue, and higher quit intentions if companies don’t set norms around when to trust or challenge AI.[D]

4. AI tools increase work instead of freeing workers for leisure

A study of more than 163,000 employees across 1,100 organizations found that after AI tools were introduced, email volume rose 104%, chat messaging increased 145%, and time spent in business tools climbed 94%.[E] Looks like companies are using AI to push for higher output rather than to reduce existing workloads.

5. AI inference demand has grown roughly one million-fold in two years

Industry data shows AI inference demand expanded roughly one million-fold in about two years, with power users consuming hundreds of millions of tokens per day.[F] Growth is driven by agentic workflows, always-on monitoring agents, and retrieval-augmented systems that multiply token usage per task.

6. GPT-5.4 can hold a million tokens (but can your budget?)

GPT-5.4 holds a million tokens of context. In theory. In practice, accuracy drops from 97% at 32K tokens to 36% at the top of that million-token range.[G] Pro variant runs can cost over $1,000. This is the AI pricing paradox in miniature: the capability sounds transformative, the accuracy is context-dependent, and the bill is real.

7. Your insurer’s AI denies the claim. Your hospital’s AI inflates it. Utah’s AI prescribes meth.

Looks like US healthcare needs a checkup after spending $1.4 billion on AI in 2025, mostly on opposite sides of the same fight.[H] UnitedHealth alone expects $1 billion in savings. Meanwhile, a red-teaming firm showed that Utah’s prescription-renewal chatbot could be tricked into tripling an OxyContin dose and reclassifying meth.[I] Controlled substances are excluded in production, but the vulnerability was real.

8. Claude Dispatch frees you from your workstation

Anthropic’s latest gift to its users is Dispatch, a feature lets you control your desktop Claude Cowork (the one that can edit files and complete tasks on your machine while you’re doing push-ups) from your mobile phone.[J] If you’re a Pro or Max subscriber, the only reason not to use it is if lugging a laptop around is part of your weightlifting program.

9. AI could automate 94% of tech tasks in theory, but touches only 33% in practice

Anthropic’s “observed exposure” metric measures which tasks AI actually handles in real workplaces, not just which ones it theoretically could.[K] In tech roles, the gap is striking: 94% automatable in theory, 33% touched in practice. Hands-on jobs are largely unaffected. There’s no broad unemployment spike yet, but hiring for 22 to 25-year-olds in AI-exposed fields is down about 14%, a finding the authors note is barely statistically significant.

10. North Dakota police jail innocent grandmother for five months after facial recognition error

Angela Lipps spent 108 days in jail after Fargo police misidentified her through facial recognition software as a suspect in a bank fraud case.[L] US Marshals arrested her at gunpoint while she was babysitting four children. Police did not interview her until December 19, more than five months after her arrest. Her bank records showed she was more than 1,200 miles away in Tennessee during the alleged crimes. The case was dismissed on Christmas Eve, but not before Lipps lost her house, car, and dog. A stark ethics warning about designing AI systems where the system deployer captures all the upside (facial recognition is great when it works... but it doesn’t always work) while passing the downside to innocent bystanders.

11. LangChain CEO says better models alone won’t get AI agents to production

The barrier between AI agent demos and production isn’t model quality, says LangChain CEO Harrison Chase. It’s the harness: context management, tool control, observability, and tracking for long-running workflows.[M] See the previous edition of this newsletter for more about harness engineering: https://bit.ly/quaesita_harness

12. LLMs can match anonymous forum users to real LinkedIn profiles for $1-4 per person

New research showed LLMs can de-anonymize pseudonymous forum users to real LinkedIn profiles for roughly $1 to $4 per person at about 67% match rate and 90% precision.[N] Now that the anonymous trolls know they can be unmasked, will they be on their best behavior? A girl can dream.

13. AI can predict mutual fund managers’ trade directions 71% of the time.

AI predicts which direction fund managers will move in next 71% of the time.[O] The managers whose trades were harder to predict earned higher returns. Turns out that AI can predict what your fund manager will do next when your fund manager is…predictable.

14. Global survey finds robot anxiety highest where robots are rare

A global survey found robot anxiety is highest in regions where robots are rare and declines as people see them operating safely.[P] As antianxiety medicine, here’s Figure’s Helix 02 tidying a living room autonomously. (Did it work?) Or since humor is the best medicine, here’s a dancing robot at a Haidilao Hotpot that had to be restrained by staff after an unexpectedly aggressive routine.

15. AI’s energy footprint matches a country. (Guess which one!)

AI uses as much energy as Iceland. That sounds alarming until you remember Iceland is a very small country with 380,000 people and a lot of volcanoes, while the global population is over 8 billion. University of Waterloo researchers say the global emissions impact is “surprisingly small,” though local effects near data centers can double electricity demand.[Q] The comparison with Iceland is memorable precisely because it works in both directions.

16. Australian man uses ChatGPT and AlphaFold to design cancer treatment for his dog

A Sydney tech entrepreneur with no biology training used ChatGPT, AlphaFold, and $3,000 worth of genomic sequencing to design a personalized mRNA cancer vaccine for his dog Rosie.[R] UNSW’s RNA Institute director developed the vaccine in under two months. The main tumor shrank roughly 75%, though one tumor didn’t respond and is being sequenced for a second round.

🦶Footnotes and Sources

* He eventually recovered it with help from AWS, whew. Don’t bank on a happy ending, though, since since that is definitely not the moral of this story.

[1] Source; [2] Source; [3] Source; [4] Source; [5] Source; [6] Source; [7] Source; [8] Source; [9] Source; [10] Source; [11] Source; [A] Source; [B] Source; [C] Source; [D] Source; [E] Source; [F] Source; [G] Source; [H] Source; [I] Source; [J] Source; [K] Source; [L] Source; [M] Source; [N] Source; [O] Source; [P] Source; [Q] Source; [R] Source

Promo codes

My gift to subscribers of this newsletter (thank you for being part of my community!) is $200 off the list price of my course with the promo code SUBSCRIBERS. If you haven’t subscribed yet, here’s the button for you:

If you’re keen to be a champion of the course (you commit to telling at least 5 people who you think would really get value out of it) then you are welcome to use the code CHAMPIONS instead for a total of $300 off — that’s an extra $100 off in gratitude for helping this course find its way to those who need it. (Honor system!)

Note that you can only use one code per course, the decision is yours.

P.S. Most folks get these courses reimbursed by their companies. The Maven website shows you how and gives you templates you can use.

Forwarded this email? Subscribe here for more:

This is a reader-supported publication. To encourage my writing, consider becoming a paid subscriber.