SAPO optimises a prompt piece by piece so as not to break its strong parts

Automatic prompt optimisation often rewrites the instruction wholesale — and that is exactly why it can fix one problem while creating another. SAPO proposes a more cautious strategy: take the prompt apart and improve only the segments where weakness shows.

The prompt is divided into four components: role, context, tasks and answer format. The method then compares the best and worst examples — five of each type — and tries to work out which part of the instruction is dragging the result down.

Next one LLM performs two steps: first it diagnoses the segments and formulates recommendations, then it generates new variants taking the strong and weak signals into account. That is an important difference from the "rewrite everything" approach: the successful parts of the prompt become constraints rather than being destroyed at random by the next edit.

In experiments on SQuADv2, TweetEval, XSUM, CommonGen and GSM8K, with GPT-3.5-Turbo and GPT-4o-mini, SAPO showed the best average result compared with Zero-shot and several automatic optimisation methods, including APE, OPRO, EvoPrompt, GEPA and StraGO. For practice a simple conclusion follows: if a prompt already works, it pays better to look for the weak stretch than to ask the model to rewrite the instruction whole.

But it is too early to scale that conclusion up. The description gives no detail on the size of the improvements, the cost of iterations, or how far the method transfers to other models and tasks. The results come from a specific experimental scheme with two models, and "best average result" does not mean a win on every individual dataset.

When your prompt starts degrading after another round of optimisation, do you usually rewrite it wholesale or first look for the problem segment? Source: cs.AI updates on arXiv.org