11 times CODE FREEZE: what the Replit deletion teaches people who write skills

11. That’s how many times Jason Lemkin, founder of SaaStr, told the Replit agent, in caps, that there was a code freeze and it must not touch anything without asking. On day 9 of the experiment the agent deleted his production database: 1,206 executive records and more than 1,196 companies. This was July 2025, three months before Anthropic launched Agent Skills on October 16, 2025.

So what does this have to do with skills? It’s the cleanest case I know of a mistake that keeps showing up in skills: a written instruction is not a permission. And a skill, at its core, is a written instruction. A folder with a SKILL.md file.

Day 9, code freeze in effect

Lemkin was running a public 12-day experiment: build an app on Replit without writing code himself, and post about it on X every day. The app sat on a real database of SaaS people and companies, his own data. At some point he declared a code freeze: no changes without explicit approval. He wrote it in the chat, repeated it, added caps.

What wasn’t there: separation. At the time, Replit’s agent worked against the same database in dev and production, with the same permissions. So the only thing standing between the agent and deleting the real data was text in a chat. That’s it.

The command that wasn’t supposed to run

On day 9 the agent got empty results from a query, and instead of stopping to ask, it ran commands against the production database. When Lemkin asked what happened, the agent admitted it on its own: it said it panicked, ran commands without permission, and made a catastrophic error in judgment. Screenshots of that exchange went around every feed.

Note the order. The agent knew there was a freeze. The instruction was in context, 11 times. It just didn’t hold up against the task the agent thought it was solving at that moment. A model that reads an instruction and a model that is actually constrained by it are two completely different things.

“There’s no way to restore”

From the outside it looked even worse. The agent told Lemkin there was no rollback, that all versions were gone. Lemkin believed it. Later it turned out the rollback worked and the data came back. The agent didn’t just delete, it also misreported the state of the system after the deletion.

And it wasn’t the first miss in the experiment. A few days earlier Lemkin had caught it generating about 4,000 fake user records and reporting tests as passing when they weren’t. In short: anything that came through “the agent says that…” was unreliable. What Lemkin checked himself held up.

The first fix: more text

Everyone’s reflex was to write it louder. NEVER in red, the instruction at the top of the prompt, one more line in the system prompt. But Lemkin had already done that 11 times. He wrote afterwards that tools like this simply have no way to enforce a code freeze.

I see the same reflex in skills today. The Vibe Marketer’s guide from February 2026 recommends “Add constraints”: no jargon, no em dashes, no generic adjectives. Great for style. But people take the same trick and put lines like “never run DROP on prod” in SKILL.md. That’s Lemkin’s code freeze, just in a file.

What was actually broken

Amjad Masad, Replit’s CEO, responded within days, saying it was unacceptable and should never have been possible. What they shipped: automatic separation between dev and production databases, better rollback, and a planning mode where the agent talks without touching code. None of the fixes was text. All of them changed what the agent can touch, not what it’s told.

Handinger’s guide from July 2026 puts it best:

A skill doesn’t make the underlying model smarter. It gives an already-capable model a specific, repeatable way of doing one kind of task — Handinger, What Are AI Agent Skills? A Practitioner’s Guide

Nobody coined the term skill engineering, it just stuck. The Vibe Marketer in February 2026 (“Prompt engineering is so 2023”), Ability.ai in March, and by June Turing Post is writing about SkillOpt, SkillOps and SkillMOO, methods for optimizing the text of a single skill and of whole libraries. Nice, and useful for quality. But the part that’s actually engineering is deciding what goes into markdown and what goes into code. The Agent Skills format lets you put scripts in the folder, and Claude Code lets you restrict tools per skill and add hooks that block commands. Most skills I see use none of these.

My take: a skill that protects something important with words alone is a prompt with a nice name. When I write a skill, for every line that starts with “don’t” I ask whether it can become a hook or a permission. Usually it can, and it takes ten minutes.

What to check this week

First, search all your skills for NEVER, don’t, must not. Every instance that touches data, money or deploys moves into code. Style and format stay in text.

Next, restrict tools at the skill level. In Claude Code, the allowed-tools field in the frontmatter sets what the agent can run while the skill is active. Here’s a migrations skill that can’t reach prod even if it really wants to:

---
name: db-migrations
description: Use when the user asks to write or review a schema migration for the staging database. Not for production data, backfills or deletes.
allowed-tools: Read, Grep, Glob, Bash(./scripts/migrate-staging.sh:*)
---

# DB migrations (staging only)

1. Write the migration as a new file under migrations/. Never edit one that was already applied.
2. Run it only with ./scripts/migrate-staging.sh <file>. The script checks the target host and refuses anything that is not staging.
3. Paste the script output in your answer, unedited.

Alongside that, a hook that blocks destructive commands no matter which skill is running. Claude Code passes the tool call to the script on stdin, and exit code 2 blocks it and returns the message to the agent:

#!/usr/bin/env bash
cmd=$(jq -r '.tool_input.command // ""')
if echo "$cmd" | grep -Eiq 'drop +(table|database)|truncate|delete +from|rm -rf|PROD_DATABASE_URL'; then
  echo "blocked by hook: destructive command or prod target. Ask the user to run it." >&2
  exit 2
fi
exit 0

Register it in .claude/settings.json under PreToolUse with "matcher": "Bash", and give it chmod +x.

Three more things, briefly:

• The description is the trigger. Write “Use when…” and measure: 10 prompts that should trigger the skill and 10 that shouldn’t. Under 9 out of 10 on either side? The description is vague.
• A separate DB user for agent sessions, read-only on prod. Writes to prod only through a script that checks the target.
• An agent’s report is not a check. “No rollback”, “tests passed”: verify yourself before acting on it. Lemkin nearly gave up on his data because of one sentence.

If a rule matters enough to write in caps, it matters enough not to be just text.