A Toolkit for AI Development: How to Build a Working System Around an LLM

Everyone’s talking about models: GPT, Claude, Codex, Gemini, Opus. But in real-world applications, it’s not just the most powerful model that wins—it’s the well-designed system built around it. This system is called an agent harness.

In the past, using AI in development was simple: you’d open a chat, type a prompt, get a snippet of code, paste it into your project, test it—and repeat this ten times until the model produced something suitable. The problem isn’t just the model; it’s that there’s often no working infrastructure around it: no project context, no rules, and no integration with the repository, documentation, server, or change history.

What Is a Harness

A harness is the software infrastructure surrounding an LLM: the orchestration loop, tools, memory, context management, state, error handling, constraints, and validations. To put it figuratively: the model is the engine, and the harness is the transmission, steering wheel, brakes, dashboard, and seat belts. Without a proper system in place, even a powerful engine won’t get you very far.

Twelve Components of a Production Harness

• Orchestration Cycle — the agent operates in iterations: it receives a task, processes it, performs an action, receives a result, and adjusts accordingly.
• Tools — access to files, the terminal, the browser, GitHub, APIs, and the database.
• Memory — allows the agent to avoid starting from scratch every time.
• Context Management — selecting what the model should see right now and what shouldn’t be included in the prompt.
• Prompt Construction — assembling system rules, tasks, history, tools, and project context.
• Output parsing — understanding what the model returned: a final answer, a tool invocation, an error, or a command.
• State management — tracking where the agent is in the task, what it has already done, and what remains.
• Error handling — the ability to recover from the first failure rather than giving up.
• Security — restrictions on dangerous actions: file deletion, migrations, deployments, and data access.
• Checks — tests, linters, builds, diff reviews, and visual reviews.
• Subagents — separation of roles: one investigates, another writes code, and a third verifies.
• Work Continuity — the ability to continue a task across sessions using Git, progress files, and change history.

Key takeaway: AI development of the future isn’t just about “buying access to a new model,” but about building a working system around it.

The author’s workflow setup

• Orca — a specialized IDE for developing and orchestrating code-based AI agents (Claude Code, Codex, OpenCode, Gemini, etc.) with isolated Git worktrees, separate terminals, a built-in browser, and a mobile app for remote management.
• GitHub — repositories, branches, pull requests, change history, and deployment.
• VPS — a stable environment with internet access where the project is stored and the agent runs.
• Pi Agent — the main harness through which the model is run.

Source: All Articles / Artificial Intelligence / Habr