Vol.076 — TOO MANY FILES RECORDED! My AI staff read everything I gave them and still got it wrong

Date: 2026-10-16 | Newsletter · Multi to Single Agent Journey #3 of 10
Issue #2 was about the memory that was never written. This time, it is about the opposite failure, and as I said I worked with 6 AI staffs. The one I worked most hours naturally built up more files and rules. This is very obvious. However, when I went through each AI staff, what I found out was shocking! The agent (AI staff) who read the most rules was the one that got the most wrong.
My first instinct was to give it more to read
When an AI staff member forgets, the natural reaction is to write more down and make it read more at the start of every session. I did exactly that, for months, across six staff. By September one of them had a state file — the file that is supposed to say what is in progress right now — of 107,723 bytes: 686 lines under 33 headings. That file had been designed with one constraint, which I had set myself: the staff member must be able to read it back to me in thirty seconds, so that I know it knows where we are. At 107 kilobytes it could not. And as I said in #1, two of the six had files that were nearly empty. So I had one staff member drowning in its own notes and two with nothing, under the same rulebook.
I want to be honest about how this felt, because I think it is the feeling that sends most people to the wrong fix. It did not feel like a structure problem. It felt like the staff were careless — one hoarding, two lazy — and the fix that suggests itself is to scold, add a rule, and make them read the rule.
Not one problem. Six.
On 18 September the agent and I took the file structure apart on paper, with the metrics log as the worked example. The symptoms were familiar — things read at the wrong moment, stale facts, cross-project work falling between staff — but there turned out to be six separate causes, and any one of them would have produced the symptoms alone.
The first: files were keyed by owner, not by subject. Each staff member's rule was “my five files, in full, every session”. That works while one person owns one body of work. It fails the moment a subject crosses staff, because the subject has no file — only the staff do. The DCC website rebuild was never anyone's file.
The second: the separation between staff was never real. In the tool I use, skills and connectors are set at the account level, so all six staff had always seen the same twenty-eight skills and every connector. What was separated was the instruction file and the folder — the memory, not the capability.
The third: the memory file had no admission test that could be applied at the moment of writing. Its stated test was “information the user might ask again”. That is a question about the future, unanswerable at the time of writing, so in practice either everything got written or nothing did, depending on the mood of the session. The replacement is a test that can be applied on the spot: the trigger is backward-looking — I answered something, or corrected something unprompted — and the filter is my own three-month test: will this still be true in three months. Trigger first, filter second.
The fourth: the memory file mixed four kinds of content with four different lifetimes — durable facts, conventions, role boundaries and dated decisions — under one append-only rule, and had no way out. Every other file had an exit: a version number, a past/ folder, an ABANDONED mark. The memory file only grew.
The fifth: the state file's purpose had never been stated, so it drifted into narrative. Its real job is anti-compaction — holding exactly the specifics that a long session destroys: file names, figures, exact wording. Once that is named, the thirty-second read-back stops being an arbitrary limit and becomes a test of whether the file is still doing its job. A narrative fails the test; a list of specifics passes it.
The sixth, and the one that explains the other five: no file declared who reads it or when. So files were read at the wrong moment, or not at all, and nobody could say which was the error.
Smaller won. I did not expect that
With the causes named, there were four structures on the table.
Keep the six staff and fix their files. Rejected: it fixes the symptom and leaves work keyed by owner. One agent with one large instruction file, read in full every session. Rejected on cost and on attention — when everything is read every time, nothing is read carefully. Lean on the tool's built-in memory. Rejected: not portable, and able to contradict a written instruction, which is the same reasoning that had switched memory generation off on the 17th. And the one chosen: one agent, a small core read in full every session, plus lane files loaded only when that kind of work actually starts.
The core is twelve to fifteen kilobytes. It holds who the agent is, the words we use, a map of the live subjects, and the house rules — cited by section number, never restated. Everything else loads on a trigger: a subject file when I name the subject, a lane file when the work begins, the archive only when the subject file turns out to be thin. Reading every lane at the start would put about 230 kilobytes of rules into context before any work began. The small core is what keeps the start of a session short.
I had expected the big file to win. More context, fewer mistakes — that is the intuition. The argument that beat it was not about cost, it was about attention: a file that is read every time is read the way you read a terms-of-service page.
One admission. In the new agent's first week the core was over its own budget: 41.6 kilobytes, because the house rules file alone was 23.8. It was recorded as over budget rather than ignored, and it is still being trimmed. A budget nobody measures is a wish.
“I never use this brain data.” So stop building it
The next day, 19 September, the discussion turned to retrieval — how the agent finds the right fact at the right time — and it was heading toward a system: tags, an index, a scheme for finding anything. I stopped it. My words were that this whole discussion should be about improving the quality of detecting user intention. Two things from experience made me say it. I really never use these brain data before. And: don't make it too complicated, as it will become an unused database.
Three rules came out of that hour, and all three run against the instinct to be thorough. The agent does not judge what is worth recording; selection happens when something is retrieved, not when it is written. The system does not rely on auto-generated titles as its backbone — they are fine for finding things and unfit to build on. And the index is written in the words I would actually type when searching — “web renewal” — not in project vocabulary.
Underneath all three is one idea I have come to hold strongly: a memory system is judged by whether it is read, not by how complete it is. The most complete file I had was the 107-kilobyte one, and it was the least read.
“Memory should start with How to Retrieve!”
The rule that survived everything is a single line: every file declares its reader and its read trigger. Who opens this, and when.
A file that cannot answer those two questions is not memory; it is a pile. And once the reader and the trigger are written at the top, most of the other causes resolve themselves. The admission test becomes “will the reader need this at the trigger”. The eviction rule becomes “when the trigger no longer fires”. The size limit becomes whatever that reader can take in at that moment. Even the owner-versus-subject question answers itself, because the reader of a project file is whoever is working on the project.
Simply, all memory files recorded must be “easily-retrievable” and “easily-identifiable”.
“This way, I can save Token Use Volume as well!”
A large file read in full every time suits a narrow, single job where the whole context is the job: one client, one product, one format. A small core with lanes suits anyone whose agent does several kinds of work in a week, and it costs the discipline of routing — the agent must say which lane it is in before it reads one. Built-in memory suits the person who will never leave the tool and never needs to ask why. If your staff keeps forgetting and your answer is to give it more to read, you are treating the wrong cause: it will read all of it, slowly, and still get it wrong.
