In Text-to-SQL, you need to save the context before executing the query

An agent that writes SQL has an unfortunate quirk: it determines on its own which observations it will receive next. If the query is too broad, the agent first consumes the database’s context and resources, and useful evidence may never make it into the working window. Compressing the result afterward does not mean retrieving the missed rows or undoing work that has already been done.

This is precisely the foundation of BAP-SQL. The authors propose controlling not only the model’s response but also the formation of observations: assessing the query’s risk, rewriting the SQL if necessary, and passing strict constraints to an independent runtime shield.

This is a significant shift in architecture. Budgeting occurs before the data reaches the agent, rather than after receiving a massive result set. For Text-to-SQL, this means that query planning becomes part of the agent’s policy, rather than a secondary optimization of the prompt.

On the baseline scenario derived from BIRD, BAP-SQL achieved a 3.4–3.6 percentage point improvement over a comparable SFT and used 4.5–5.0% fewer tokens. The results were tested on general-purpose 4B models and specialized FINER-SQL 4B and 7B models.

However, this is not a story of universal improvement. The effect weakens as the model’s capabilities and available budget increase, and at the most permissive setting, the advantage reverses. Furthermore, BAP-SQL does not reduce the amount of work performed by the database: the savings primarily apply to the agent’s observations and tokens.

The practical conclusion is simple: if an agent works with tools, it’s not just the final context that needs to be limited. Both the query format and the volume of data that is permitted to be delivered to the model at all must be restricted in advance. Otherwise, “optimization” after calling the tool will merely be a matter of cleaning up the aftermath.

Source: cs.AI updates on arXiv.org