Curate AI-Powered Monitoring & Maintenance
AI-powered monitoring, intelligent debugging, continuous documentation, and iterative improvement. Maintenance isn't an afterthought - it's an ongoing partnership.
What is the Curate Phase?
The Curate phase ensures your software remains healthy after launch. This is where AI helps with the ongoing work of monitoring, debugging, and improving systems over time.
Traditional maintenance is often reactive - you wait for something to break, then scramble to fix it. AI enables proactive maintenance by analyzing logs, identifying patterns, and suggesting improvements before issues become critical. Monitoring agents can now triage an alert, correlate it across logs and traces, and draft a fix as a reviewable pull request - turning 2am pages into a diff waiting for your approval.
In the Curate phase, AI helps you:
- Analyze logs and identify anomalies automatically
- Perform root cause analysis when incidents occur
- Suggest performance optimizations based on metrics
- Keep documentation current as code evolves
- Identify and prioritize technical debt
Quick Reference
SRE / Support Partner
Fixes, optimizations, docs
Claude, Datadog, Sentry, Grafana
On-call engineer and service owner
Findings filed as backlog items and rules committed
Inputs and Outputs
Curate keeps the release healthy and turns what happens to it into the input for the next cycle. It is the only phase whose main output is learning rather than a deliverable.
What it receives
- The running release from Launch, with its Release Plan, flags, and rollback path.
- Production signals: logs, traces, metrics, error reports, alerts, and user feedback.
- Incidents and escaped defects, with the pull requests and agents involved identified.
- The repo instruction file, runbooks, and documentation as they stand today.
What it hands off
- Dashboards and alerts that report against the baseline, not just raw counts.
- Runbook entries and updated documentation for anything that surprised the on-call engineer.
- Post-incident reviews for every incident that involved an agent or AI-authored code.
- New rules or skills in the repo instruction file, plus new backlog items that re-enter Analyze.
The output is what changes next time
A phase that only watches dashboards has not curated anything. Each incident, escaped defect, or surprising log pattern must end as a rule, a skill, a runbook entry, or a backlog item, so the next cycle starts from a better position than this one did.
Key Activities
Log Analysis
AI analyzes log files to identify patterns, anomalies, and potential issues before they become critical problems.
Root Cause Analysis
When incidents occur, AI helps trace through logs, code, and system state to identify root causes quickly.
Performance Optimization
AI suggests performance improvements based on profiling data, metrics, and code analysis.
Living Documentation
AI keeps documentation in sync with code changes, generating updates to READMEs, API docs, and runbooks.
Closing the Loop
Curate is the phase that makes the next cycle better than this one. The compounding asset is not the model, which every team can rent. It is the rules, skills, and context that your own incidents teach, kept in the repo. The mechanics are on The Learn Loop and Evals.
From defect to rule
Every escaped defect and every incident becomes a new rule in the review agent's library, versioned in the repo. Measure whether that defect class stops recurring, and feed the cases that keep escaping back into the rule.
From repeated task to skill
A task done the same way more than a couple of times becomes a skill, a versioned and testable procedure. An eval confirms the skill still produces the right output after a model or tool upgrade.
Keep the instruction file alive
The repo instruction file is a living spec. Review it on a schedule, remove rules the code no longer follows, and add the lessons from each incident. Left alone it drifts into the failure described under Instruction File Rot on Twelve Ways AI Adoption Fails, and agents keep obeying stale instructions.
Monitor model behavior when the product uses AI
Teams that ship AI-powered features have a second thing to watch: how the model behaves in production, including drift and quality over time. That is a different lifecycle from the software one. See AIDLC vs the ML Lifecycle.
Example Workflow
Alert Fires and the Agent Triages
A monitoring agent correlates the alert across logs and traces and drafts a summary of what changed and what it affects. It has read-only access to production data.
A Human Confirms and Mitigates
The on-call engineer checks the summary against the evidence, then mitigates: turn the flag off, roll back, or scale. The first priority is restoring service, not finding the cause.
Find the Root Cause
The agent helps trace logs, code, and recent merges to a cause. The human accepts or rejects the explanation.
Fix Through the Normal Pipeline
The fix is a pull request that goes through Develop and Launch like any other. An emergency does not remove review, it only changes how fast the tier-appropriate reviewer is paged.
Run the Post-Incident Review
Walk through the incident checklist for AI-related failures: which agent, which model version, what review missed, and what a human verified.
Turn Findings into Rules and Skills
Add a review rule, a skill, or an instruction-file change for each finding. Add the failure case to the eval fixtures so the rule is tested.
Update Runbooks and Docs
Have the agent draft updates to the runbook, README, and API docs from the diff and the incident. A human checks them against what actually happened.
Back to Analyze
File the remaining findings as backlog items with evidence attached. They re-enter the Analyze phase as the start of the next loop.
Who Owns It
The on-call engineer owns the incident while it is live, and the service owner owns what the team learns from it. The engineering manager owns the cadence of the learn loop, so it does not slip when the team is busy. The agent can compile and draft. It does not decide severity.
What the AI drafts
- Alert triage: correlating logs, traces, metrics, and recent merges.
- A draft post-incident review and a timeline built from the evidence.
- Proposed fixes as pull requests, and proposed rules and skills from the findings.
- Documentation and runbook updates generated from code changes.
What humans decide
- Severity, mitigation, and when to roll back.
- Whether the explanation of the root cause is the real one.
- Which findings become rules, which become backlog items, and which are accepted risk.
- Merging the fix. A monitoring agent proposes a diff and a person approves it.
Entry and Exit Criteria
Entry criteria
Start the phase when all of these are true.
- The release has gone through Launch and has a named service owner.
- Dashboards and alerts exist and report against a baseline.
- Monitoring agents have read-only access and their own identity.
Exit criteria
Close each cycle of the phase when all of these are true. Curate does not end, but each pass does.
- Every incident in the period has a post-incident review.
- Each finding became a rule, a skill, a runbook entry, or a backlog item.
- The repo instruction file was reviewed for rot and the new rules were committed.
- Evals were re-run after any model or tool change.
- Backlog items carry their evidence and have been sent to Analyze.
Best Practices
Turn every incident into a rule
If an incident does not change a rule, a skill, or a runbook, it will happen again. Version the rule in the repo and test it against the failure case.
Give monitoring agents read-only access
Let them read logs, traces, and metrics. Writes go through a pull request that a human approves.
Review the instruction file on a schedule
Put the review on the calendar, and treat the file like code: reviewed, owned, and versioned. Stale rules are worse than none.
Do not let an agent hot-fix production on its own
An unreviewed fix under pressure is how a small incident becomes a large one. The fix goes through a pull request, tiered like any other.
Anti-Patterns
The named patterns are catalogued on Twelve Ways AI Adoption Fails. Curate is where several of them are either caught or confirmed.
Instruction File Rot
The repo instruction file was written once and never revisited. It now contradicts the codebase, and the agent follows it anyway.
Model Upgrade Without Evals
A review agent or skill changes behavior with the model underneath it, and the defect class it used to catch begins to escape unnoticed.
Reporting Adoption as the Outcome
The dashboard shows how many engineers use AI. A leader asks what changed because of it, and nobody can say.
What to Measure
Curate owns the dashboard and the reading of it. Pick from Measuring Impact and always read against the baseline.
Change failure rate
The share of deploys that need a hotfix, rollback, or incident response. The clearest signal that quality is holding as volume grows.
Bug escape rate
Defects reaching production per release. Track whether each new rule from the learn loop reduces the repeat rate for its defect class.
Cycle time, ticket to production
Fixes should move through the same pipeline as features. If incident fixes are the slow tail, the pipeline is the problem.
Run the quarterly paradox test here. It is the check that execution speed became delivery and quality.
Go Deeper
Curate prompts
Prompts for log analysis, root cause, performance, and living documentation.
Read moreIncident checklist
A checklist for post-incident reviews where an agent or AI-authored code was involved.
Read moreAIDLC vs the ML Lifecycle
Monitoring model behavior in production for AI-powered features.
Read moreA worked example
One feature carried through all five phases, ending as new backlog items.
Read moreThe Cycle Continues
Curate isn't the end - it's a return to the beginning. Insights from monitoring inform the next Analyze phase. Bugs discovered lead to new Develop cycles. AIDLC is a continuous loop of improvement.
Back to Analyze PhaseFrequently Asked Questions
The Curate phase focuses on ongoing maintenance and improvement. AI helps with monitoring, debugging, documentation updates, and identifying opportunities for enhancement.
AI can analyze error logs, suggest root causes, propose fixes, and help track down issues across complex codebases. It remembers your project context and past issues to provide relevant suggestions.
A product that calls a model in production has behavior that can drift even when no code changed, so Curate must monitor output quality and model behavior as well as errors and latency. The AIDLC vs the ML Lifecycle page covers that model-side lifecycle. The software side of Curate still applies: alerts, runbooks, post-incident reviews, and new rules fed back into the repo.