Prompt Exploration with Prompt Regression

We thank you for your evaluation of our work and overall positive sentiment. We will work to address the edits you have suggested provided our paper is accepted.

My concern is whether this approach is task-specific.

PEPR allows prompt elements to depend on the input (footnote 1), thus the same prompt library can be used across different tasks of related nature, e.g., in Figure 5 we show such a prompt library used for both CAMEL Biology and Physics datasets. This brings in a certain degree of generality, but we note that prompt engineering is typically a task-specific problem.

Are multiple datasets mixed for training?

We don’t mix HateCheck and CAMEL, but we do mix categories within them (like Biology and Physics for CAMEL) when fitting prompt regression (prompt selection is per category).

Overconfidence in LLMs

We model relative differences in log-probability, so overconfidence might not be an issue if some prompt makes the LLM less overconfident.

Results in Figure 2 validate the assumption that "the elimination or addition of an element from the prompt library does not affect interactions between the other prompts."

Figure 2 validates that a linear model (corresponding to independence of irrelevant alternatives) holds because of high correlation and low MAE. If interactions were to have a major impact, a linear model would not do well.

The proposed method didn't compare with other related works, comparing only with its baselines.

We report 75th percentile and maximum performance as a proxy for budget-constrained searches seen in related work, but we can also report results of the method proposed by Shi et al. (2024) in a revision. Note that prior works are typically limited to around 100 prompt variants and they need to evaluate each at least a few times. Our method models the compositional structure of prompts and we applied it to over 1000 prompt variations (a prompt library with 10 elements) without evaluating each variant even once.

In some cases, even random prompt combinations can outperform both versions of PEPR. Is it possible that certain tasks are insensitive to prompts?

Random prompt combinations is a fairly strong baseline if the prompt library is good. In such cases both “random” and PEPR always outperform the base model and 75% percentile in most cases, suggesting that prompts always have an effect. Our method is able to filter out unhelpful prompt elements, while prompt regression also facilitates interpretability.

Thank you for your review. We appreciate your positive points and the chance to respond to your questions and concerns.

Motivation for prompt library search problem

System prompt selection is often a process of adding phrases to correct undesired behavior. Libraries of such phrases can grow big, thus requiring a method to identify an optimal combination algorithmically.

Generalizability of the PEPR prompts

PEPR allows prompt elements to depend on the input (footnote 1), thus the same prompt library can be used across different tasks of related nature, e.g., in Figure 5 we show such a prompt library used for both CAMEL Biology and Physics datasets. In comparison to prior prompt-engineering approaches which are often task-specific, this feature increases the generalizability of PEPR.

Scale of data used for parameter optimization

In Sections 3.2 and 4.2 we specify that we use little data for optimization (as many as 5 samples in several cases).

In Figure 2, regression performance deteriorates as the number of prompt elements increases, indicated by larger MAE and smaller r-values. Coupling between prompt elements.

While the results for predicting the effects of 2 prompt elements are the best across experiments, results for 4 elements are often better than for 3, indicating no clear trend w.r.t. the number of prompt elements. Moreover, our prompt regression yields $r \approx 0.9$ in most cases, indicating that a linear model fits data well, i.e. there is no significant effect due to coupling between the prompt elements.

"minimally-prompted model" and baselines

It is an LLM with basic instructions relative to the task (e.g., “You are a hate speech detector”; see Appendix B.1). We use it to see the effects of our prompt library relative to those of instruction tuning. Furthermore, we approximate methods involving constrained brute-force searches (e.g., Prasad et al. (2023)) by reporting 75th percentile and maximum possible metrics from all element combinations, but we can also report results with the method of Shi et al. (2024) in a revision.

Comparison to random baseline

The random baseline is quite strong when the prompt library is good, as can be seen by comparing results to 75% percentile and max performance. As noted in Section 5, our method can filter out unhelpful prompt elements and is more interpretable than trying prompts at random. We also reiterate our regression model is an interesting standalone contribution.

We thank you for your comments on our work and your very positive review. Please see our responses below:

Can you shed further light on what you mean by "solving (3.3) does not require knowledge of the reference (correct) y for the inputs (e.g., when there is a finite set of meaningful generations (such as classification or multiple-choice QA), one can simply plug every possible y for every input xi when fitting the regression coefficients)". In a quick reading it might sound like the PEPR method only supports classification usecases but that is patently not the case when considering the rest of the paper.

We mean that of a possible set of outputs, we do not need to know which one is correct in order to perform regression. This is because our regression approach models log-probabilities (and corresponding differences) of outputs themselves rather than assigning probabilities of correctness. In contrast, our proposed selection method does require knowledge of which outputs are correct in order to derive a performant prompt relative to correct outputs (see Table 1). Note that in using terms like “correct” and “possible set of outputs”, we do not mean to imply that our methods are only suited for classification (as you have deduced) but rather that even a given set of open-ended responses has associated qualities of correctness and desirability that are needed for selection and not for regression.

Would it be possible to quantify the level of interaction between prompt elements when their effects on language model outputs are not independent and make use of this information in the prompt selection step?

This is an interesting idea. Though we do not explore such an approach, it may be possible to detect dependence between prompt elements whenever our regression model is repeatedly or on average very inaccurate (relative to ground-truth log-probabilities and differences) for a given prompt element combination. In doing so, perhaps coefficients could be tweaked for specific combinations of elements such that selection is more accurate. More complicated models like linear regressions with interaction terms and factorization machines could account for pairwise interactions and increase effectiveness (perhaps at the risk of reducing interpretability). At any rate, this line of reasoning could be a fruitful future direction of work.

Thank you for your feedback and your highly positive opinion of our work. Please see our responses to your comments below:

There is a lack of comparison with existing prompt engineering methods. The existing comparison with a random baseline is not enough to show the advantage of the proposed PEPR method.

While we concede that we primarily compare results of PEPR with a random baseline, we also report 75th percentile performance metrics (relative to results from all possible prompt element combinations) in Table 2 and similar tables in our Appendix. In doing so, we approximate the results of brute-force searches with a fixed computation budget, which in turn are explored in other parts of the prompt engineering literature (e.g., GrIPS by Prasad et al. (2023)). We can also opt to include the performance results of the bandit algorithm introduced by Shi et al. (2024) in a revision of our paper.

The performance of this method highly relies on the quality of the prompt library.

Other related prompt engineering work (like that of Shi et al. (2024)) also assumes a prompt library or similar is used as a starting point for iterating on prompts. Therefore, though our method is dependent on the prompt library’s quality, this is also a constraint of other prompt engineering methods. Furthermore, other works consider relatively few prompt candidates (up to 100) whereas we are able to consider more (around 1000 due to possible combinations of prompt elements) because we account for the compositional structure of prompts.