Minor Guardian link tweak

Author: vkoves

_posts/2025-05-25-ai-and-shovelware-2.0.md

@@ -62,7 +62,10 @@ In the open source world, Claude has also had some huge benefits, but in that re
 
 Now let’s talk about *the slop,* the shovelware, or whatever you want to call the glut of AI generated software. I want to recount a few stories that I think tell the story of the dangerous moment that we’re in, with software being shipped by people who don’t understand the risks of what they’re doing.
 
-On the most extreme end, inexperienced practitioners can use AI tools without even basic guard-rails, resulting in scenarios like happened to car rental company PocketOS just a month ago. Their whole database was wiped out by an AI agent ([Claude-powered AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’](https://www.theguardian.com/technology/2026/apr/29/claude-ai-deletes-firm-database), The Guardian). As PocketOS’ founder Jeremy Crane puts it, the AI industry is “building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe”.
+On the most extreme end, inexperienced practitioners can use AI tools without even basic guard-rails, resulting in scenarios like happened to car rental company PocketOS just a month ago. Their whole database was wiped out by an AI agent ([Claude-powered AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’ - The Guardian](https://www.theguardian.com/technology/2026/apr/29/claude-ai-deletes-firm-database)). As PocketOS’ founder Jeremy Crane puts it:
+
+
+> The AI industry is “building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe”.
 
 I think there are actually two problems with this story - Crane frames the issue as Claude not following rules, but that fundamentally misunderstands how LLM based agents work - **today’s AI agents do not follow rules, they ingest rules as suggestions**. As someone who works with Claude day-to-day, the number of times it just ignores an explicit request is too many to count. The real issue, by my reading, is entrusting dangerous engineering capabilities (production database access) to non-engineers, who don’t know the safeguards necessary to properly protect customer data. It should take multiple intentional steps to access a writable copy of production data, and backups should be happening at such a high frequency that even a large deletion should be recoverable with only minor loss. In this case, however, Claude also deleted the backups, making the business fall back on a *three month old backup.* If that level of system failure happens in a lot of companies, I'd be willing to bet that the executive responsible for that oversight would be fired.