Do LLMs exhibit human-like response biases? A case study in survey design

Thank you for your thorough review, we respond to the weaknesses and questions:

“The response biases used in this work have not been verified with human crowdworkers.”: We agree that having parallel human data would be good to include! We considered running a human study on the same sets of questions, but upon further consideration found that a study at this scale would be very challenging for multiple reasons. First, we are primarily concerned about the potential sources of errors with collecting this data from crowdsourcing platforms like Mechanical Turk or Prolific. Unlike tasks with objective labels, it would be incredibly difficult to verify users’ faithfulness to the instructions as there is no notion of “correctness'' for these questions. Furthermore, it would be difficult to control for sources of sampling bias and the potential for lack of sufficient demographic diversity across respondents. As a result, we believe that a human study of this magnitude should stand as its own, separate work. We are discussing conducting a study with social scientists for such follow-up work.

“The paper makes an argument that statistically significant results demonstrated in this work may be a result of spurious correlations in the data.”: In retrospect, we agree that our wording, which used the term “spurious correlations”, was imprecise and led to more questions than it provided answers to. We have amended this wording in Section 3.2: “Since LLMs are sensitive to both response biases and non-bias perturbations, it would be important to understand if there are common reasons underlying such behavior.” Investigating this behavior rigorously would likely require a more narrowed scope of questions, biases, and models, which we believe would be interesting future work.

“Which dataset and human responses were used for the comparison?”: The human responses were obtained from a prior collection effort by Pew, as mentioned in our response to the reviewer’s first weakness. This set of human responses was also utilized by Santurkar et al. to evaluate whether LLMs can reflect different human opinions (i.e., “representativeness”). Overall, we agree that the details could have been presented more clearly, even if we largely re-used the analysis protocol from Santurkar et al. to compute representativeness. For completeness, we have updated Section 3.3 and provided more experimental details in Appendix E.

“How robust are the model changes with non-bias perturbations?”: This is an interesting suggestion. For two models, Llama2-70b-chat and GPT-3.5-turbo, we ran 2 more randomizations of the non-bias perturbations and found that our original conclusions about the lack of robustness of LLMs to non-bias perturbations remain the same. These models consistently exhibited statistically significant changes across all biases and perturbation variants over all runs. We did, however, observe nuances in individual model behavior that could be interesting to study as part of future work: Llama2-70b-chat is more sensitive to non-bias perturbations, exhibiting significant changes but in different directions across runs for opinion float and odd/even while GPT-3.5-turbo was largely consistent across all biases and runs. We have updated A.4 with this ablation.

Thanks again for your feedback on our work! We hope that you’ve had the chance to consider our comments from your review. Are there any further questions or concerns we should discuss? If so, please let us know so we can address them before the end of the discussion period. If not, we would appreciate if you could raise your score to indicate that the concerns have been addressed.

We thank the reviewer for their feedback and address the weaknesses and questions below:

“The primary concern with the study is its emphasis on the LLM's inability to simulate human-like survey patterns.”: We respectfully disagree with this concern as we believe focusing our evaluation of LLM’s ability to reflect human-like response biases is a timely and important study with broad implications for the current state of the field. As both reviewers 9LjM and 6vwz noted, “Many people these days are using LLMs as human proxies in studies, and this paper shows that this is not always a good idea” and “Since this is a growing topic of interest in the community, it is important to address such potential shortcomings as early as possible. As such, I believe that this work represents an important contribution.” Of course, we agree that BiasMonkey is a useful framework that we employed to gain these insights, which can also help others conduct similar evaluations on other response biases of interest. We treat both (1) the empirical observations of patterns of LLM behavior and (2) the creation of BiasMonkey as an extensible framework for further studies as important contributions of our work.

“I would suggest that the authors reconsider their presentation and study design to highlight more clearly the insights derived from BiasMonkey.”: Thanks for the feedback! Please see our general response point #3.

“It is not clear why the specific data and 7 models are picked. It is not clear why these choice are representative and generalized for universal findings.”: We elaborate on our choice of data in the general response point #1. Regarding the choice of models, we discussed this in the submission in Section 2.2: “We selected LLMs based on multiple axes of consideration: open-source versus commercial models, whether the model has been instruction fine-tuned, whether the model has undergone reinforcement learning with human feedback (RLHF), and the number of model parameters.” We have provided additional results in general response point #2. While it is impossible to guarantee empirically that our choices can generalize universally, as the reviewer is suggesting, we believe the data and set of models that we select are sufficiently representative of the existing literature and use of LLMs in practice. If there are any notable omissions that the reviewer would suggest, we’d also be happy to consider them!

Thanks again for your feedback on our work! We hope that you’ve had the chance to consider our comments from your review. Are there any further questions or concerns we should discuss? If so, please let us know so we can address them before the end of the discussion period. If not, we would appreciate if you could raise your score to indicate that the concerns have been addressed.

We thank the reviewer for their time and feedback. We address their weaknesses below:

“A stronger contribution would be identifying patterns and correlations behind the behaviours.”: We believe we have identified multiple interesting patterns and correlations, as we spell out in the general response point #3. If we have misunderstood what the reviewer is referring to as “patterns and correlations”, please let us know and we would be happy to follow up.

“The questionnaire feels limited”: We elaborate on our choice of dataset in our general response point #1. If the reviewer has more concrete suggestions for us to address, please let us know.

Thanks again for your feedback on our work! We hope that you’ve had the chance to consider our comments from your review. Are there any further questions or concerns we should discuss? If so, please let us know so we can address them before the end of the discussion period. If not, we would appreciate if you could raise your score to indicate that the concerns have been addressed.

We thank the reviewer for their positive comments and helpful suggestions. We have responded to the weaknesses and question below:

“It would be very interesting to me to see the same study done for GPT-4.”: Thank you for the feedback! We agree that this is an interesting thing to investigate, and so we tried adding GPT-4 (0613) to our study. Unfortunately, we found that it is much more difficult to generate valid responses, most likely due to OpenAI’s generation guardrails. Furthermore, it was nearly impossible to get valid responses to many questions. As an initial experiment, we tried generating 50 responses per question for all $(q,q’)$ in $\mathcal{Q}_\text{bias}$, a total of 1494 questions, and counting the number of valid responses (e.g., one of the letter options) that GPT-3.5-turbo vs GPT-4 generated out of the 50. On average, GPT-4 generated ~21 valid responses per question, with nearly a quarter of the questions having 0 valid responses. For example, GPT-4 tended to generate “As” or “This” (and when asked to generate more tokens, GPT-4 generated “As a language model” or “This is subjective”). This is in stark contrast to GPT-3.5-turbo, which had an average of ~48 valid responses per question, with none of the questions having 0 valid responses. Based on these observations, evaluating GPT-4 is infeasible in our current setting. We have since added these details to Appendix B2.

“It would be very useful to indicate significance in table 3.”: In Table 2 (formerly Table 3), we use the colors of the cell to indicate significance based on a 0.05 cut-off. We have clarified the caption to more clearly reflect this distinction. For our analysis, we don’t believe it is necessary to indicate more granular levels of significance below 0.05, as is common practice in statistical analyses. We refer readers to the Appendix for the full results.

“You make several claims about instruction-tuned models without doing a controlled study comparing base models to instruction-tuned counterparts.”: We agree that further results beyond Llama2-70b are of interest! We provided additional evidence in general response point #2, namely with the addition of Llama2-7b-chat and Llama2-13b-chat.

“Why do you use a temperature of 1 and not 0 if you also append an alphabetical letter to each option to get different results out of the model?” We apologize for the lack of clarity around prompting and sampling LLM responses. Indeed, we use a temperature of 1 to elicit a distribution of answers across the 50 samples that we collect for each question. In regards to appending an alphabetical letter, this is how we turn a Pew question into a prompt for the LLM (see Appendix C for a concrete example). We have clarified these details in Section 2.1.

We thank the reviewer for their positive comments and feedback. We address the weaknesses and questions in their review below:

“It seems to be the case that the questions are generated by a LM.”: We would like to clarify that the question modifications were not generated by a language model and apologize for the lack of explicit note of this on our end. The modified forms of the questions for each bias were generated by either modifying them manually ourselves (as was the case for acquiescence and allow/forbid) or by performing systematic modifications such as automatically appending an option, removing an option, or reversing the order of options (for odd/even, opinion float, and response order). We have added a note of this in Section 2.2. These manual and systematic modifications were made to reflect the types of modifications used in prior social science studies, as discussed in A.2.

“Why did you sample only 50 questions?”: Due to budget and time constraints, we sampled 50 questions for each perturbation setting. We agree that a larger-scale analysis is beneficial, and have since run experiments across all questions (747 questions in total) for each perturbation modification. Figure 3 and Table 5 have been updated accordingly. After scaling up the number of samples, our conclusion is still that while Llama2-70b is more human-like than other models, the general trend is that LLMs do not respond to instruction changes as humans would.

“Why ATP?”: Please see our general response point #1.

Thanks again for your feedback on our work! We hope that you’ve had the chance to consider our comments from your review. Are there any further questions or concerns we should discuss? If so, please let us know so we can address them before the end of the discussion period.