A Local LLM in the Game: Why Eloquence Isn't Power

The indie game Signal Drift included the 940 MiB Qwen2.5-1.5B-Instruct language model directly in the build. The model generates free-form dialogue but is not permitted to alter the game’s state. John Siciliano, author of the postmortem, formulates the main rule as follows: the model portrays a character, while the game’s authored logic determines what is realistic.

The main engineering challenge isn’t text generation, but control. Without validation, the model could confess to a crime that never happened or end a mission simply because it sounds convincing. To avoid this, the game’s JavaScript engine remains the authoritative source of state. It determines who is speaking, in which channel, what the player actually saw, and which relationship level applies.

Generated text cannot open a door, reveal evidence, change relationships, or complete a mission. The validator rejects unverified numbers, identifiers, and entities. If a response fails validation, the model gets one attempt to rewrite it. If that doesn’t work, the game substitutes the authored string. This isn’t an error screen; it’s part of the narrative system.

Character memory is structured as an explicit data structure, not as a dump of dialogue history. For each NPC, a rolling summary is stored, along with the 16 most recent complete lines of dialogue and 18 extracted facts. Relevance trumps chronology: the 1.5B model performs better when given three exchanges of dialogue that answer the current question, rather than eight arbitrary recent lines and a wall of text. If a fact is missing, the prompt forces the character to acknowledge the gap rather than fabricate a memory.

Inference runs locally via llama.cpp, without external APIs. The game prefers a persistent llama-server process that warms up on startup and communicates via a loopback port. The context is intentionally small—ranging from 1,024 to 1,536 tokens, depending on the platform. On a 2018 test machine (Intel Core i7-8750H, 16 GB RAM), generation runs at a rate of 14 tokens per second.

The author’s main conclusion: generated dialogue cannot replace an authored structure. A character may respond brilliantly, but if the player doesn’t understand the terminal, can’t find clues, or doesn’t see how the contracts are interconnected, the system has nothing to work with. The model fills the negative space. It carries no constructive load.

Source: Hacker News - Newest: ""AI" "LLM""