The Development Lifecycle Framework

Design a spec format that doubles as an agent prompt: preconditions, acceptance criteria, input/output examples, anti-examples. Machine-readable, not narrative prose.

Document what agents can/cannot modify, architectural patterns they must follow, file boundaries, and safety rails. The harness matters more than the prompt.

Every spec needs quantifiable pass/fail criteria. 'Improved performance' is not a criterion. '< 200ms p95 latency' is. Vague specs produce vague outputs.

Define what NOT to do for each major feature area. Anti-examples prevent agents from taking common wrong paths and reduce iteration cycles.

Store specs in version control next to the code they describe. Specs are living artifacts that evolve with the product, not static documents in a wiki.

Specs get reviewed before generation begins. A 30-minute spec review prevents 3 days of agent re-work. Include PM, engineering lead, and domain expert.

Establish three tiers: project-level context (architecture, conventions), feature-level context (domain rules, dependencies), and task-level context (specific requirements, examples).

Set up embeddings and vector storage for your codebase, docs, and domain knowledge. Agents should retrieve relevant context automatically, not rely on manual copy-paste.

Not every task needs the most expensive model. Define routing: fast models for boilerplate, frontier models for architecture decisions, specialized models for domain tasks.

Replace static architecture docs with living constraints that agents consume directly. Include: tech stack decisions, naming conventions, file structure rules, API patterns.

Context degrades over time. Set a cadence (weekly or per-sprint) to review, update, and prune stale context. Outdated context is worse than no context.

Define how to decompose features into parallel agent tasks. Each task should be independently executable with clear boundaries. Resolve merge conflicts as a human-in-the-loop step.

One human orchestrator manages multiple agent threads. Track what each agent is working on, what's blocked, and what needs human architectural decisions.

Each agent task gets explicit file/module boundaries. Overlapping scope between agents creates merge hell. Scope isolation is non-negotiable.

Establish token spend limits by task category. Bug fix: X tokens. New feature: Y tokens. Refactor: Z tokens. Track actual vs. budget to calibrate over time.

Agents generate code. Humans make architectural calls: when to abstract, when to duplicate, when to take on tech debt intentionally. Document these decisions.

You cannot improve what you cannot measure. Stand up automated eval (correctness, security, performance, style) before scaling agent-generated output.

Truth metrics measure real outcomes, not activity. Shipping velocity means nothing if regression rate climbs. Pair every speed metric with a quality counter-metric.

For every metric you optimize, track its counter-metric. Ship faster → track regression rate. Reduce cost → track quality score. This prevents optimizing into a corner.

Functional: does it work correctly and securely? Craft: is it elegant, maintainable, consistent with product quality bar? Both matter, but they're different review passes.

AI code has 2.74x more security vulnerabilities. Run SAST/DAST on every agent-generated PR. Flag patterns like hardcoded secrets, SQL injection, XSS, and auth bypasses.

AI can match 80% of quality. The last 20% - the craft that differentiates your product - requires human design review. Schedule these as a deliberate practice.

Measure what each AI feature costs per user action, not just aggregate token spend. A $0.03 action at 10K DAU is $300/day. Make cost visible to engineering and product.

Real-time visibility into token spend by feature, model, user tier, and action type. Shared across engineering, product, and finance. No surprises in the cloud bill.

Route requests to the cheapest model that meets quality requirements. GPT-4 for complex reasoning, GPT-3.5 for classification, local models for formatting. Save 60-80% on inference.

Your AI features have marginal cost. Price accordingly: credit-based, usage-based, or tier-gated. Don't eat inference cost on your lowest tier.

Model updates can silently change behavior. Pin specific model versions, test before upgrading, and maintain rollback capability. Treat model changes like dependency updates.

Heavy users can blow through your margin. Set per-customer or per-tier token budgets with graceful degradation when limits are reached. Rate-limit expensive operations.

After each sprint/cycle: What specs produced best agent output? Where did context fail? What harness constraints need updating? This is your compounding mechanism.

Track cycle velocity improvements over time. If each cycle isn't getting faster or higher-quality, your compounding loop is broken. This is the single most important AI-native metric.

When a spec produces excellent agent output, templatize it. Over time, your spec library becomes your competitive advantage - new features start from proven patterns, not blank pages.

Cognitive debt (Karpathy): the hidden cost of poorly managed AI interactions. Track: time spent re-prompting, context assembly time, review cycles per feature. Reduce systematically.

Production incidents and quality issues feed back into harness constraints. Every bug prevented by a constraint is a cycle saved in the future.

Context degrades. Stale embeddings, outdated conventions, deprecated patterns - all poison agent output. Schedule quarterly context audits like you schedule dependency updates.