A society of LLM agents will run on a laptop — but not blindly

A large society of LLM agents may turn out to be a job for a laptop rather than a cluster. But the main risk here is not a shortage of compute: if you simplify an agent's perception the wrong way, a cheap simulation will quickly compute the wrong thing.

The authors propose not copying each agent's "thinking" wholesale. Instead they take between a few hundred and a few thousand inexpensive LLM queries, build a compact behaviour model from them, and then run that model — for any number of agents N.

The idea is especially useful for research where what matters is not an individual participant's line but the behaviour of the whole system: phase transitions, stable macroeconomic regularities and scaling as the number of agents grows. In that case expensive per-agent detail may be superfluous.

The authors' key instrument is an "interaction order × memory" classification. It ties how much information an agent sees, and how long it keeps it, to the expected error of the simplified model. That is, the question "can the LLM be replaced?" is meant to be settled before the simulation runs, rather than checked blindly after the pretty graphs arrive.

The method was tested on a faithful implementation of the LLM macroeconomy EconAgent and seven other named simulations. Agent decisions were reproduced from genuine LLM answers, predominantly DeepSeek; the queries themselves cost a few dollars. The predicted error trends matched the results across every cell of the experiment. Two predictions did not hold directly, but the theory accounted for those deviations quantitatively as well — with no extra fitting parameters.

The limitations are substantial too. The source does not promise that any LLM agent turns into a simple formula without loss of meaning. The result depends on what the agent perceives, what memory it retains and how far the system's response saturates; strong saturation is precisely what caused the two discrepancies. Besides, this concerns simulations and statistical regularities, not a full replacement of the model in tasks where individual decisions and rare events matter.

The practical lesson for a simulation developer is simple: first measure the structure of the agents' perception and memory on cheap queries, and only then choose the scale of the run. Could you trust conclusions about the behaviour of a large group to a model that is not itself a full LLM agent? Source: cs.AI updates on arXiv.org