Why Agent Personas Started Breaking Down
When something that worked well stops working well, the question is whether it's your setup or the environment. I spent two months trying to distinguish between those before accepting the answer: it was both, and neither was fully fixable from my side.
The persona system had been delivering real value. The degradation was real too. Understanding exactly what was breaking helped clarify what I actually needed — which turned out to be a different architecture, not a better prompt.
The Three Failure Modes
Model drift. Copilot Chat updated its underlying model without notice. A system prompt that produced consistent code reviewer behavior under the previous version started producing inconsistent behavior under the new one. The persona's instructions were the same. The model's interpretation of those instructions had changed.
This is not a bug — it's an expected property of any system built on top of a model you don't control. The model provider's goal is to improve the model; your goal is to rely on it for a specific behavioral contract. Those goals are not aligned. The provider will ship improvements that change behavior in ways that break your prompts, and you have no advance notice and no rollback.
Scope creep under pressure. A model with a persona that limits its scope will stay in scope under normal conditions. Under prompting pressure — questions that push against the boundary, follow-up questions that assume the model will cross the boundary — it will drift. The scope constraint is advisory, not enforced. A model that is told "you do not write implementation code" will write implementation code if asked directly enough times.
This isn't a prompt engineering problem you can solve by writing stronger constraints. It's a fundamental property of how language models handle conflicting signals in a context window. If the conversation history contains enough examples of the model crossing a scope boundary, the next crossing becomes more likely. The persona is one signal among many, and it doesn't always win.
Context window contamination. Long sessions with a persona-driven model accumulated responses that conflicted with the persona's initial instructions. A code reviewer that started a session applying one set of standards would, thirty messages later, be applying a slightly different set because the intervening conversation had introduced variations. The persona provided a starting point, not a persistent constraint.
What the Failures Had in Common
All three failure modes share a root cause: personas rely on the model to self-enforce behavioral constraints. The persona instructs the model; the model decides whether to follow the instruction. There is no enforcement mechanism external to the model. When the model's interpretation of the instructions changes — because the underlying model changed, because the prompting context created pressure to deviate, because accumulated context diluted the original signal — you have no recourse except to restart the session.
Compare this to a software system with enforced constraints. If a function is restricted to a defined interface, the enforcement happens at the call site — you can't violate the interface accidentally. A persona prompt is more like a style guide than a type system. It describes intended behavior without enforcing it.
The Architectural Implication
The reliability I wanted from the persona system — consistent behavior, maintained scope, predictable output format — required enforcement that lived outside the model. Not a better prompt. A wrapper that controlled what the model could see, what it could return, and what happened when its output didn't match the expected pattern.
That's an orchestration problem. Not an inference problem, not a prompt engineering problem. The model is one component in a system; the system needs to control what the model does, verify what the model returns, and route between models or retry when the output fails the expected contract.
Copilot Chat didn't give me that control. It was a chat interface with a system prompt, not an orchestration layer. To get the reliability I wanted, I needed to build the orchestration myself — or find a framework that provided it.
The Search That Started
By the end of month five on persona work, I was evaluating alternatives. Not replacing Copilot — it was still doing its job on autocomplete. But the conversational AI layer needed something more robust than a chat interface with a system prompt.
What I was looking for: a framework that could enforce output contracts, manage multi-step task sequences, route between models based on task characteristics, and maintain session state in a way I controlled. That combination of requirements narrowed the field considerably. The options worth evaluating were a short list.
I spent the next several months doing that evaluation — testing models, frameworks, and approaches against the real workloads I needed them to handle. The results were instructive, occasionally frustrating, and ultimately pointed at a path forward that looked nothing like where I had started. As always, I'm here to help if you want to compare notes on how you've handled the scope enforcement problem.