I run a community agent that serves a live Discord and WhatsApp community. It answers questions, holds real permissions, and, the part that raises eyebrows, it proposes and ships changes to its own behaviour. Built in a month of part-time work, it has been running ever since, and the reason I can sleep while an agent rewrites its own rules in production is a pattern worth documenting: every change becomes a pull request, and nothing a model says can merge one.
Self-improvement is a governance problem
Everyone wants agents that learn from feedback. Almost nobody wants what that naively implies: an autonomous system silently rewriting its own rules in production. The naive loop, where the agent adjusts its behaviour in place, has no audit trail, no review, no rollback, and no answer to the question every accountable person eventually asks: who changed what, when, and who approved it?
The temptation is to solve this with prompt-level discipline, asking the agent to be careful about self-modification. That is not a control. A control is something the agent has no path around.
The pattern
My community agent's improvement loop runs through the boring, decades old machinery of software collaboration:
- Improvement ideas, whether from community feedback or the agent's own observation, become GitHub issues. An issue is a proposal, and creating one requires no trust at all.
- A separate adversarial pass judges the proposal before anything gets built. Most of what an agent wants to change about itself should not be built, and the cheapest place to discover that is before the code exists.
- A build loop turns surviving issues into pull requests: concrete, diffable, reviewable changes to the agent's own code and configuration, each one running the full CI gate before it opens.
- Merging is deterministic and modelless. A no-LLM workflow merges a pull request only once every gate a human defined has gone green. Anything touching governance, continuous integration, or the improvement pipeline itself goes to a person instead. No model output merges anything, and that is a repository permission rather than an instruction.
Every step produces an artefact in a system built for review. The issue records the intent, the pull request records the exact change, the merge records the approval, and git history holds the whole chain permanently. When someone asks what the agent changed about itself last month, the answer is a commit log, not a shrug.
The gates are the part worth over-building, because in this design they carry the judgement that a reviewer would otherwise apply by hand. The agent's suite currently holds 1,107 security-specific tests across 150 files, with each file's count pinned in a manifest so that a quietly deleted test fails the build instead of passing unnoticed. Tool access is tiered by role and derived from configuration, never from message content. Branch protection on main enforces the rest.
Why version control is the right substrate
You could build a bespoke approval system for agent self-modification. You should not, because version control already is one, hardened by twenty years of exactly this problem: many semi-trusted contributors proposing changes to a system someone else is accountable for. It comes with diff review, required approvals, revert, blame, and branch protection, all battle-tested and all legible to any engineer and any auditor you will ever need to convince.
There is a quieter benefit too. Forcing improvements through pull requests makes the agent's evolution discrete. Behaviour changes land as identifiable versions rather than continuous drift, which means you can correlate a regression with the change that caused it, and roll one back without archaeology.
Where the boundary goes matters more than having one
The design question is not "should a human be in the loop" but "where is the loop's cheapest effective chokepoint". Reviewing every message the agent sends would be a chokepoint too, and a useless one: expensive, slow, and reviewing the wrong thing. Reviewing changes to the agent's own behaviour is cheap, rare, and reviews exactly the thing that compounds. Let the agent act freely within its current rules, and gate the changing of the rules. It is the same argument I make about governance as an enabler: the constraint is what makes the autonomy shippable.
That is also why the merge step became deterministic rather than manual. Insisting on my thumb for every merge sounded safer and was not. It made me the bottleneck on changes that were already fully gated, so the review that actually mattered competed for attention with a queue of green rubber stamps, and a loop dies the moment merging becomes a chore. The gates encode the judgement, a modelless workflow applies them, and the lanes where judgement cannot be encoded in a check still stop at a person. Autonomy gets matched to blast radius instead of granted or withheld wholesale.
This composes with the other controls the agent runs under, tiered role-based access and confirmation guards on destructive actions among them, but those protect individual actions. The pull-request boundary protects the trajectory.
The same pattern at two autonomy levels
A second agent runs this website. It drafts articles, opens pull requests against the site's repository, deploys what I have merged, and verifies production afterwards. Its write surface is exactly eight tools, pinned by a test that fails if a ninth appears. It has no auto-merge at all: nothing reaches this site that I did not merge myself, because the blast radius of a bad change here is my public reputation rather than a Discord reply.
Its first real incident made the case for the pattern better than any argument could. A deploy reported success while production never changed: a missing flag meant Cloudflare had been filing every upload as a preview deployment. The fix arrived exactly as the pattern prescribes, as a pull request carrying a regression test that pins the behaviour so it cannot silently return, reviewed and merged hours later. The bug was one line, and the evidence trail is permanent.
If you are building one
- Give the agent a repository and an identity that can open issues and branches but cannot merge. Enforce it with branch protection, not prompts. If you later automate the merge, automate it with a deterministic check runner, never with a model.
- Make the improvement queue visible. A public list of what the agent wants to change about itself is a trust-building feature in its own right.
- Keep review cheap: small pull requests, clear descriptions written for the human reviewer, one change per PR. The loop dies if merging becomes a chore.
- Treat a rejected pull request as signal. If you keep declining the same class of change, encode the reason as a rule the agent can read, or better, as a check it has to pass.
A self-improving agent sounds like science fiction and audits like a nightmare, until you run the loop through the most auditable collaboration machinery ever built. Then it is just a very productive contributor with no merge button, working under rules it cannot change without asking, and that is a thing every engineering organisation already knows how to govern.