Noisegate: The agent isn't trustworthy, and that's okay
Sh0ny
31 июля 20262 min read
In short
The open-source Noisegate gateway places the trust boundary not in the LLM, but in the code beneath it. Differential privacy protects data even from adversarial agents—including attacks directed at the CI itself.
Noisegate’s main architectural premise is simple: an LLM agent is an untrusted input source. You shouldn’t trust the model, you shouldn’t rely on its alignment, and you shouldn’t check to see if it has “leaked” data. Privacy is enforced in code that the agent does not control—further down the stack, at the level of the query engine and privacy budget. The model merely proposes a query. Everything else is handled by the gateway, which would behave the same way even if the query were entered manually by a human.
This is the correct approach. In production, you don’t trust user input—you validate it, sanitize it, and restrict it. It’s the same here, except the input comes from an AI agent via the MCP. The gateway compiles the query, executes it within the tracked privacy budget, and returns a deliberately noisier response with a confidence interval. If the query is too narrow—for example, if it attempts to single out one patient out of 20—the gateway rejects it at the trust boundary. If the agent has exhausted the budget, it receives a rejection rather than a “toned-down” response.
The project isn’t limited to mere statements. Three classic privacy attacks—differencing, membership inference, and singling out via re-identification—are built directly into the repository and run in CI. Each is demonstrated: it succeeds with privacy disabled and is defeated with privacy enabled. This means that the protection cannot quietly fail—any regression is caught in the pipeline.
The noise mechanism was written from scratch and verified against OpenDP—the industry reference implementation. All 35 noise scale checks match to within 1e-9. Budget accounting uses hybrid zCDP composition: 308 requests for the same budget, compared to 268 with advanced composition and 100 with naive sum-of-ε. The guarantee is (ε, δ)-DP, not pure ε-DP. To be fair: pure ε-DP would be stronger, but more expensive.
Stack: Python, DuckDB, FastAPI, Streamlit, MCP SDK, Docker, 250+ tests in CI. Apache 2.0 license. Works as an MCP server for Claude Desktop.
For practitioners connecting agents to sensitive data, this is a working pattern: not “trust the model,” but “the model is not on the critical path.” The question that remains is how well this pattern scales with real production datasets, where the differential privacy budget is depleted faster than one would like.