An Incomplete Loop: Instruction Inference, Instruction Following, and In-Context Learning in Language Models

We’re very glad to hear that you enjoyed the paper, and thank you for your detailed suggestions! We have incorporated your suggestions into the current draft, and please see below for responses to your comments:

R1. Table 3 is opaque. What were the criteria for deciding whether a cell got a checkmark, partial check, or X? Were these qualitative judgments, or based on statistical significance tests?

This was qualitative, based on the actual accuracy for “Deductive/abductive/hypothesis proposal works”, and the checkmark/partial check, or X was based on whether respectively ground truth rules and instruction inference improved performance for the first two columns, and whether the hypotheses were actually mostly correct for the third column. The purpose of Table 3 was mainly to summarize the takeaways for readers, rather than to provide detailed results. We will clarify the meanings of the symbols in the caption for the camera-ready version.

R2. The Kalamang translation results are weak, and don’t seem to provide especially strong evidence for either RQ. That said, this is a very difficult task, and I appreciate its inclusion; I therefore don’t think this is a significant weakness.

Thank you for appreciating the inclusion of Kalamang, we are aware that the results are rather mixed in terms of not following the trends in the functions and colours domains, but we thought it was important to include a real-world task which was more difficult, especially since similar papers have also been tested on more synthetic domains. It is possible that with more work in the low-resource translation domain, instruction inference may work better and similar trends may be seen in this domain as well, but we leave this to future work.

Typos/suggestions

Thank you for your detailed reading in making these suggestions and finding all these typos! We have corrected all typos in our draft. In addition, we will also address the suggestions in the camera-ready draft.

Thank you for your comments, we’re glad that you see potential for future research! Please see responses below:

As described above, the experimental results are meaningful but no generalized knowledge is obtained from these results. What we can know is that the effectiveness of different prompting styles is mixed, and this is kind of expectable.

We agree with the need to further investigate in more domains and examine the broader applicability of the findings. However, technically a lot of results related to prompting can be summarized as “the effectiveness of different prompting styles is mixed”. For instance, chain-of-thought reasoning can be summarized this way, since CoT sometimes degrades performance as well. However, knowing this would still (1) provide an easy and practical way for people to get more performance out of their models (2) reveal future avenues for investigating the inner workings of LMs. In our case, instruction inference can be used relatively easily to guide LMs, at least on simple tasks. Furthermore, the dissociation between hypothesis proposal and in-context reasoning is interesting, and may merit further investigation. For instance, perhaps by training to make proposed hypotheses amenable with LM outputs without the ground truth label, we can increase reliability of models.

More broadly, we would like to point out that many highly impactful papers identify problems without proposing a solution in the same paper (such as the original papers on adversarial examples in NLP/vision, which are arguably still not fixed). However, knowing that this problem exists can guide the community in best practices for using and developing models.

The main focus of the paper is on three styles of prompting, i.e., instruction following, few-shot learning, and instruction inference, rather than generic logical inference like deductive, inductive, and abductive reasoning. The paper describes these three methods as deductive, inductive, and abductive learning, while they are just a specific instance [...]

Thank you for bringing this up, we plan to clarify the terminology more in the final version. We can clarify more in Section 2 that we are not making claims about these capabilities in general, but rather on common instantiations of these types of reasoning in LMs (hence the use of “learning” rather than “reasoning” in the title).

Thank you for your review, we're glad you appreciate our contributions and writing! See our responses below:

The paper primarily focuses on specific simple tasks, linear function The scope of tasks examined in the study—linear function learning, artificial language translation, and Kalamang translation—is somewhat narrow, limiting the broader applicability of the findings. Future studies could benefit from incorporating a wider array of tasks to strengthen the arguments presented.

Thanks for the comment! We agree that there are many potential future applications that are not explored here. As this was a first study exploring the relationship between these types of reasoning in language models, we thought it was important to use clear-cut domains such as the linear functions and colours domains, in which correctness of rules is unambiguous, easing verification. We attempted to show the differences between simple domains and a real-life task by incorporating the Kalamang translation results, in which instruction inference does not provide a large increase in performance, because of the breakdown in grammar rule proposal. We will explore more diverse domains as well as more avenues for improving instruction inference in future work.

There is room for deeper exploration regarding the limitations of the current methodologies. Discussing potential new domains for future research and outlining prospective directions would provide valuable insights.

This is a good point! We can certainly add more discussion of potential research directions and applications to the conclusion as part of future work. For instance, although the set of tasks we examined is more related to translation, one could imagine that similar methods could be used to help LMs reason about any domain with underlying rules or principles. For instance, LMs following a similar strategy could be useful to aid researchers in proposing scientific hypotheses to test given a body of existing literature.

Thanks for your comments! See below for response:

While the paper concludes that the ability to generate hypotheses is unrelated to learning from few-shot examples, it fails to suggest future directions or solutions to address this gap. This omission may limit the paper's impact on advancing the field's understanding of integrating these capabilities.

We definitely agree with the need to further investigate the results in more domains and examine the broader applicability of the findings. However, we also emphasize that this is the first paper to examine the relationship between these different avenues of reasoning, and thus we emphasized two controlled domains, linear functions and simple translation, in order to obtain more clear-cut and verifiable results. This is akin to the papers examining the expressivity of in-context learning starting with examining linear functions, rather than complex and varied domains, in which results may be unclear for a variety of reasons related to the domain – we did include a real-life task of Kalamang translation, in which we did not see benefits to instruction inference, unlike the first two domains, likely due to a breakdown in hypothesis proposal. Since we have now examined simple domains and have clear results for these simple domains, this makes it easier for future work to build on these results and examine more varied domains while keeping in mind the pieces of the reasoning loop as individual components.

One potential idea is self-consistency training, which could be a form of instruction tuning. For instance, we could use few-shot examples with underlying rules which can be expressed through natural language or code, and have the LM generate both an answer and explanatory rule separately. We can then have a combined loss, a generative loss which is the usual cross-entropy on the answer tokens (with multiple x_is tested for a single rule), and a consistency loss which penalizes the loss incurred by applying the generated rule to the answers generated by the model. We can certainly add more discussion of potential research directions and applications to the conclusion as part of future work.