3 years ago, a question you could not answer had a price attached to it.
You searched the internal wiki and found a page last edited in 2021 by someone who had left. You asked the two people most likely to know and got a shrug and a maybe. You then had a choice: drop it, or spend a day working it out. If you worked it out, you almost always wrote it down, not because the company had a documentation culture, but because you had just paid for that answer and had no intention of paying twice.
That cycle, repeated a few thousand times a year by a few hundred people, is how an institution accumulates knowledge. It is not elegant. It is driven almost entirely by irritation.
Today, the same question takes 4 seconds and produces a confident, well-structured, largely correct answer. Nothing is searched, nobody is asked, nothing is written down, and — crucially — nobody learns that the organisation had never worked it out in the first place.
The answer was excellent. The transaction that used to make the organisation smarter did not occur.
This is the shape of the thing I want to describe: not a failure, not a loss, but a quiet cessation of accumulation, taking place inside a system that has never felt better informed in its life.
What happens to a person
Start at the individual scale, where it has actually been measured.
In June 2025, a team at the MIT Media Lab, led by Nataliya Kosmyna, published a study with the deliberately provocative title Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task. 54 participants wrote essays across 4 months under 3 conditions: with an LLM, with a search engine, or with nothing but their own head. The researchers ran EEG throughout and tested recall afterwards.
The essays produced by the LLM group were fine. That matters, because this is not a story about bad output.
The recall was not fine. More than 80% of the LLM users could not quote a single sentence from an essay they had finished minutes earlier.
Asked to reproduce their own text after a delay, the LLM group managed around 17%; the unassisted group managed around 46%.
The EEG showed weaker connectivity across the networks associated with the effortful integration of new material. And the participants reported something subtler and more revealing: a diminished sense of authorship. They did not quite feel the essays were theirs.
They were right. In the only sense that matters for memory, the essays were not theirs. Memory is not a recording device that captures whatever passes in front of it. Encoding is a cost. It is paid in effort: in retrieval attempts, in failed sentences, in the specific discomfort of holding a half-formed idea while trying to make it a whole one. Remove the effort and you do not get the memory for free. You get the artefact without the memory.
The study is a preprint and has not yet been peer reviewed, and it deserves the usual caution.
But the finding it points at is not exotic. It is one of the most robust results in cognitive psychology, arriving in new clothes: we remember what we generate far better than what we are given.
Now stop thinking about one person and one essay. Think about 10,000 people, 3 years, and every decision, diagnosis, judgement and explanation those people produce.
What an organisation is actually made of
Institutions have always had a memory problem, and it has always been worse than anyone admits, because most of what an organisation knows has never been written anywhere.
In 1996, the anthropologist Julian Orr published Talking About Machines, an ethnography of Xerox photocopier technicians that has quietly influenced a generation of thinking about work. Orr rode along with technicians for months, and his central finding was awkward for everyone who had commissioned it: the knowledge that made these people good at the job was essentially absent from the service manuals they had been given. It lived in stories, told at breakfast, in car parks, over the phone. This machine in this office with this humidity. That fault which presents as a paper jam and is always a worn sensor. The customer who will describe the symptom wrongly, in a specific and predictable way.
“The circulation of stories among the technicians is the principal means by which they stay informed of the developing subtleties of machine behaviour.”
— Julian E. Orr, Talking About Machines: An Ethnography of a Modern Job, 1996
Xerox’s response has become a small legend in knowledge management: they gave the technicians two-way radios, so the stories could keep circulating without everyone having to be in the same room at the same time. The intervention was not to write the knowledge down. It was to protect the channel through which it moved.
The expensive version of the same lesson belongs to NASA. The F-1 engine that lifted Saturn V off the pad was, by any measure, one of the most thoroughly documented machines ever built; Rocketdyne ran a dedicated production knowledge retention programme that produced 20 volumes covering injector rings, valves, assembly, checkout, thermal insulation, cabling. Everything a reasonable person would think you needed.
When engineers returned to the F-1 decades later, the 20 volumes were not enough. Each engine had been effectively hand-built, with its own undocumented quirks. Welding and fabrication techniques lived in the hands of specific people, and those people had retired or died. The drawings had been produced under crushing deadline pressure without computer aids, and a drawing made that way records the shape of the thing but not the reasoning, the failed alternative, or the trick that made it work.
The blueprints survived. The knowledge did not. And nobody at NASA was careless: they had done everything the textbook says to do.
Both stories converge on the same uncomfortable point. The valuable part of institutional knowledge was never the record. It was the person who could tell you why the record was wrong.
The absence that did all the work
Here is the mechanism I think we have broken, and it is worth being precise about it, because it is easy to mistake for nostalgia.
Organisations do not learn because they are curious. They learn because they periodically run into the wall of their own ignorance, and the collision hurts enough to produce a response.
The collision had a very specific form: you asked, and nothing came back. That silence was not an inconvenience. It was a measurement. It told you, with total reliability, that this was something the organisation had never worked out — as distinct from something it had worked out and filed somewhere you had not looked. Those are entirely different states, and the old system distinguished between them for free.
Almost everything good followed from that distinction. It told you where the frontier was. It told you which questions were worth someone’s afternoon. And it created the only reliable motive for documentation ever observed in the wild, which is not diligence but resentment: I am never doing this again.
AI-mediated knowledge access removes the silence. Every question now returns a fluent, immediate, structurally impeccable answer, and the answer looks exactly the same whether it was retrieved from something the organisation genuinely knows or reconstructed on the spot from the general shape of the world.
This is not speculation. Anna Gausen, Bhaskar Mitra and Siân Lindley, writing at ACM FAccT in 2024, examined how AI-mediated enterprise knowledge access systems reshape organisational memory.
Among their findings: these systems increase the visibility of knowledge that already exists while simultaneously diminishing members’ organic awareness of knowledge gaps. Workers stop developing an accurate internal map of what the organisation does not know, because they no longer encounter its edges.
An organisation that cannot detect its own ignorance does not become ignorant dramatically. It simply stops filling anything in, and the gaps persist indefinitely, invisibly, being papered over on demand, 4 seconds at a time.
Four ways an organisation leaks
The failure is not one thing. It is four distinct leaks, and they compound, because each one hides the next.
The Vanished Gap. Knowledge that never gets created, because the absence that would have prompted its creation is no longer perceptible. Nobody decides not to investigate; the question simply never presents itself as open. This is the deepest leak and the only one that is genuinely invisible, because its symptom is the non-occurrence of an event. You cannot audit for work that was never triggered. The tell, when it eventually surfaces, is a company that has been operating for 3 years on an assumption nobody ever tested, defended by a confident answer that has been given 400 times.
The Ephemeral Answer. Knowledge that is created but has nowhere to land. The reasoning happens inside a conversation with a model, genuinely good reasoning, often better than what would have happened otherwise…and then the tab closes. It is not written down, because writing it down feels redundant when the thing that produced it is always available. But the model does not remember your organisation. It remembers nothing about the decision, the constraint, or the four options you rejected. The next person asks the same question and gets a different plausible answer, and neither of them will ever know they disagreed. Institutional memory is being replaced by a per-session hallucination of institutional memory.
The Hollow Fluency. Knowledge that is produced and delivered by a person who has not encoded it. This is the MIT finding at organisational scale. The output is real and often correct; the person cannot answer a follow-up question, cannot spot when the situation has changed enough to invalidate the answer, and cannot teach it to anyone. A workforce in this state looks extremely capable in every artefact it produces and extremely thin in every unscripted conversation. It is also, quietly, unable to train its own successors, because you cannot transmit what you never held.
The Severed Inheritance. Knowledge that used to move between people and now does not, because the channel has been rerouted. Orr’s whole point was that the stories were the transmission mechanism. When a junior colleague asks a senior one a question, the answer is the smallest part of what transfers; they also acquire the reasoning, the hesitation, the aside about the time this went wrong in 2019, and, not trivially, a relationship that makes the next question possible. Asking a model returns the answer with none of the freight. It is more efficient in precisely the way that a wire transfer is more efficient than a conversation about money.
The four compound in a specific order. The Severed Inheritance dries up the stories. The Hollow Fluency means nobody notices, because everyone still sounds competent. The Ephemeral Answer means nothing new is being written to replace what is being lost. And the Vanished Gap means no alarm ever sounds, because the organisation has lost the instrument that used to detect absence.
Where the leak becomes visible
If all of this were purely invisible, it would be hard to argue about. But there is one place the water shows on the ceiling, and it is worth looking at closely.
In July 2025, METR ran a randomised controlled trial with 16 experienced open-source developers, covering 246 real tasks in mature repositories on which the developers had an average of 5 years of prior experience. The developers expected AI tools to speed them up by 24%. Afterwards, they reported having been sped up by about 20%. Measured, they were 19% slower.
The perception gap, nearly 40 points, is the part that gets quoted. The mechanism is the part that matters here. These were extraordinarily high-context environments. The developers carried, in their heads, years of accumulated knowledge about why this module is structured strangely, which tests lie, what the maintainer will reject in review. The model had none of that and no way to get it. So the developers spent their time explaining their own tacit knowledge to a system that could not retain it, then reviewing output that did not reflect it.
The productivity finding is interesting. The structural finding is more important: the slowdown was largest exactly where human accumulated knowledge was deepest. Which tells you something uncomfortable about the incentive gradient.
In low-context work, where nobody has accumulated anything, the tools are transformative. In high-context work, they are a drag, and the way to make them stop being a drag is to lower the context. To standardise, flatten, document less idiosyncratically, and gradually stop building the deep specific knowledge that made the environment hard for the model in the first place.
Nobody will announce that decision. It will arrive as a series of sensible efficiency choices, each of which is defensible, and at the end of it the organisation will be perfectly legible to its tools and will contain nothing that took 15 years to learn.
How to hold water
The instinct at this point is to reach for documentation policy, and the instinct is wrong, or at least badly insufficient. NASA had 20 volumes. Documentation captures what was decided. The leak is in why, and in who could tell you the record is out of date.
If you lead an organisation: restore the ability to see absence. This is the single highest-leverage intervention available, and it is mostly a matter of instrumentation and honesty. Any internal AI system that answers from your own material should be able to tell the difference between “this is grounded in something we wrote and here it is” and “this is a reasonable reconstruction,” and it should say so, visibly, every time. That distinction is the modern equivalent of the empty wiki search, and if your system does not expose it, you have bought a machine whose primary function is to conceal the state of your own knowledge from you. Then measure the right thing. Every organisation now measures output, and output has never been less informative. Ask instead: what did we learn last quarter, where is it written, and who wrote it? If nobody can answer without opening a chat history, you do not have institutional memory.
If you manage a team: protect the channel, not the archive. Xerox’s answer to Orr was radios, and it was correct. The equivalent today is deliberately preserving the conditions in which people tell each other things: the review conversation that could have been a comment, the pairing session, the fifteen minutes at the end of an incident where somebody says what actually happened rather than what the report will say. These look like inefficiency and they are the transmission mechanism. And make the reasoning the deliverable. A decision record that says what was chosen is worth a fraction of one that says what was rejected and why, because the rejected options are the part a model will confidently reinvent for you next year.
If you are doing the work: understand that recall has become a choice, and that it now has to be made deliberately, because the default is no longer to remember. Pick two or three domains where you intend to be genuinely deep, and refuse the shortcut there: do it the slow way, generate the answer before you check it, be willing to be slower for a year. Be fast everywhere else, without guilt. And write down not what you concluded, but what you tried that failed. That is the part nobody else has, the part no model can reconstruct, and the part that will still be worth something when the answer to every well-posed question is free.
If you are early in your career: you are entering an environment that will let you produce a competent answer to almost any question, and remember almost none of it. That is an extraordinary offer and it is a trap, because in 10 years your value will be almost entirely the small set of things you can speak about with nothing open in front of you. Also: keep asking senior people questions you could ask a model. Not for the answer. For the 20 minutes around the answer, which is where the actual transfer happens and which is the thing quietly disappearing from professional life.
The uncomfortable truth
The prevailing story is that AI makes organisations smarter. Every measurement available supports it: questions answered, time to answer, coverage, satisfaction. On any dashboard you could build, an organisation with good AI-mediated knowledge access is dramatically better informed than the same organisation was three years ago.
That story is true and it is describing the wrong quantity. It is measuring access, not accumulation. Retrieval, not memory. And the two feel identical from inside, which is precisely why this can run for years without anyone raising it.
The distinction is not academic. A person with excellent access and no memory can answer any question and cannot notice that a question has stopped making sense. An organisation in the same state can serve every request and cannot tell that the ground beneath one of its oldest assumptions has moved, because the mechanism that used to report ground movement was the failure to find an answer, and it no longer fails.
Ruth Schwartz Cowan’s washing machines did not reduce housework because the standard rose. Jevons’ efficient engines did not reduce coal because demand rose. This one is stranger than either, because nothing rose. Something simply stopped: the small, irritating, unglamorous loop by which a company converts a moment of not-knowing into something it permanently knows.
Nobody switched it off. It just stopped being triggered, because the condition that triggered it — silence — no longer occurs anywhere in the building.
So ask your organisation what it learnt last year. Not what it produced, shipped, answered or automated. What it learnt, where that learning is written, and whose name is on it.
If the honest answer is that it knows a great deal more than it did and has learned nothing at all, that is not a paradox. That is the leak, and it has been running the whole time, at exactly the speed of your fastest answer.