“Why do you keep sending me hostels? I’ve never asked for a hostel in my life.”
That came from Mara, one of the travelers using Atlas, a trip-planning agent I built so I could instrument it and break it on purpose. The travelers are synthetic and nobody’s vacation was ruined. The bug underneath is real code, every trace in this post comes from a real run, and what follows is the sequence I actually went through.
It took about twenty minutes, and what I was looking for was not in any trace.
Agents that remember their users fail in a way stateless agents cannot. A stateless agent gets a request wrong and the next request starts clean. An agent with memory gets a fact wrong once and then applies it correctly, forever, to everything that follows. By the time anyone complains, the conversation where the damage happened is long over and every conversation since has looked fine.
That is what had happened to Mara. Weeks earlier, Atlas had written down that her hotel budget was sixty euros a night, less than half the number she had actually given it, and it had been following that faithfully ever since. The number belonged to her colleague Dan.
The cause is one line of logic. After every conversation, an extraction step reads back what was said and pulls out any durable facts worth remembering, things like a budget, a seat preference, or a trip in progress. It files each fact under whoever is speaking. When Mara asked Atlas to plan a trip for Dan, the extraction step filed Dan’s sixty-euro nightly cap under Mara’s own profile.
Every conversation with Mara after that read as normal. Atlas was correctly following her saved budget, and the number in it belonged to someone else. There is no single trace where this looks like a bug. The fault lives in a memory record, written weeks before anyone complained.
Finding the conversation
One complaint is an anecdote, and it arrived with no session id attached. All I had was a word she used and the account it came from. That turned out to be enough, because search ranks by relevance rather than exact match and the user filter does the rest.
Four sessions, filtered to her account.
I opened the top one and the wrong number was sitting in Atlas’s own reply, sixty euros a night, cited as her saved preference. It really was her saved preference. She had just never set it.
Atlas quoting its own note back to her, Dan’s name included.
The ledger holds the evidence the traces do not
Every memory operation an agent performs arrives as a span, so Latitude keeps each record’s full history the way git keeps commits. Opening her budget record showed me the damage as a diff. The line that came out was her own, mid-range, up to 150 euros a night. The line that went in belongs to Dan. Underneath the diff the panel lists seven reads of that record after the write, which is seven conversations Atlas planned against the wrong number.
The write itself, July 24. The span link leads back to the conversation behind it.
Following that link landed me in the conversation that caused it.
The conversation the link points at. Nothing in it looks wrong.
This is what ordinary tracing misses. A trace tells you what happened inside one request. A memory ledger tells you what the agent believed going into it, and when that belief changed. For a bug that ran two weeks before anyone complained, the second question is the one that finds it.
One user or a pattern
Fixing this for Mara was a support ticket. Working out whether it was happening to everyone else was a different job, and that one needed a number.
One thumbs-down on the conversation, with a sentence describing what went wrong, is enough to start counting.
A thumbs-down and one sentence.
That annotation becomes a tracked signal, and the same misattribution has now shown up thirty-eight times, across thirty-seven sessions and sixteen different travelers, each carrying somebody else’s constraints in their profile.
Thirty-eight occurrences, sixteen travelers, four percent of sessions.
The regression test
Crossing that threshold dispatched a coding agent, with the failing conversations and the memory history attached. It reached the conclusion a developer reading the ledger would reach: the extraction step attributes every fact to whoever is speaking, with no way to check who the fact is about.
The signal link sits at the top of the description.
The fix is one paragraph added to the extraction prompt, telling the model to attribute a stated fact to the person it is about, even when somebody else brought it up.
The more useful artifact is the test. The conversations that failed become the test set, which is the best kind of test set going, because none of it is invented. Three cases: Mara plans for Dan, Mara books for a group, and Mara changes her own budget. The first two must leave her profile untouched. The third must still write, which is what stops a lazy fix that disables memory writes from passing.
The same suite, before and after.
The second failure is the one I would have missed by hand. Planning for a group did not write Dan’s number into her profile. It rewrote her own budget note to mention the trip she was booking for other people, keeping her number and corrupting the record anyway.
The failing runs are also not deterministic. Running the suite twice on the unfixed branch gave one failure and then two, because whether the extraction step grabs the wrong subject depends on how the conversation went. That is exactly why the signal matters more than a single reproduction. One run tells you it can happen, thirty-eight occurrences tell you how often.
What to take from this
If your agent writes anything durable about its users, three things are worth having before you need them.
Record every memory operation as a span, so the write that poisoned a profile has a timestamp and a conversation attached to it. Make that history searchable, because complaints arrive as sentences rather than identifiers. And when you find one bad case, count the others before deciding what it is worth, since one annoyed user and sixteen are different problems.
Docs for the memory ledger and signals are at docs.latitude.so.