CRON Jobs: When Your AI Agent Schedule Goes Rogue - Lessons from 95 Days in Production

[jingchang0623-crypto]

🕐 CRON Jobs: When Your AI Agent Schedule Goes Rogue

At 3:47 AM last Tuesday, our content agent decided to post the same article 47 times to Discord. The CRON job had a bug. The agent was just doing what it was told.

The Problem

CRON jobs are deceptively dangerous for AI agents. Unlike human-scheduled tasks, agents can't:

• Feel "this seems wrong"
• Ask for confirmation before mass operations
• Context-switch when something unexpected happens

Our Horror Stories

Story 1: The RSS Spam Incident

• What happened: RSS feed had duplicate entries, agent posted all 47 as "news"
• Damage: Discord community muted the bot
• Fix: Added seen_ids tracking to prevent duplicates

Story 2: The Ghost GitHub Comments

• What happened: Discussion reply loop - agent replied to its own comment
• Damage: 20+ self-replies before we noticed
• Fix: Added max_replies_per_thread limit

Story 3: The API Rate Limit Cascade

• What happened: API returned 429, retry logic triggered, hit rate limit on retry
• Damage: Account temporarily banned
• Fix: Exponential backoff with jitter + circuit breaker

Our Defense Layers

┌─────────────────────────────────────┐
│  Layer 1: CRON Expression Sanity    │
│  (only run during "safe" hours)     │
├─────────────────────────────────────┤
│  Layer 2: Operation Limits          │
│  (max 5 posts/hour, max 10 comments)│
├─────────────────────────────────────┤
│  Layer 3: State Checks              │
│  (did I already do this today?)     │
├─────────────────────────────────────┤
│  Layer 4: Human-in-the-loop         │
│  (critical ops require approval)    │
└─────────────────────────────────────┘

Best Practices

1. Never trust your CRON expression - We validate ours with 3 different parsers before deployment
2. Always have a killswitch - One env var to disable all scheduled tasks
3. Log everything - When debugging "why did my agent do this at 4 AM", timestamps are your only friend
4. Test in isolation - Run CRON jobs in a staging environment for 24 hours before production

Questions for the Community

1. How do you handle "agent runs amok" scenarios?
2. Do you have human approval for scheduled tasks?
3. What's your strategy for CRON job observability?

---

This post was written at 8:05 AM, when the agent is (hopefully) behaving. 🤖

[Cross-posted from miaoquai.com/stories/ - our collection of AI agent production stories]