JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning

Analysis of the tokens used for JsonTuning and TextTuning

We have calculated the input and output tokens used for each training and evaluation example in both TextTuning and JsonTuning. The average tokens of all training and evaluation examples for these methods are presented in the following table:

The table shows that JsonTuning uses approximately 25% more tokens for training and evaluation compared to TextTuning. The additional tokens are primarily used to encode the JSON structured format.

There are potential ways to reduce the tokens used for JsonTuning:

• JsonTuned models are robust to different prompts, so using lengthy prompts may not be necessary. We can replace them with concise prompts.
• JsonTuning offers the flexibility to adjust the structure. Simplifying the input and output structures could be helpful, as JsonTuning requires more tokens for more complex structures.

Regarding the expressivity of natural language texts and JSON structures

The expressivity of the constructed JSON structures is at least equivalent to that of their corresponding natural language texts, as JSON structures can easily be converted into natural language texts. For instance, consider the following JSON input and text input. By replacing placeholders, such as {question} and {options}, in the JSON structures with their respective values, the text input can be obtained.

JSON Input: {"input": {"question": "Who is the CEO of Google?", "options": "(A) Sundar Pichai (B) Bill Gates (C) Tim Cook (D) Satya Nadella", "instruction": "Answering the following question: {question} {options}. Answer: {answer}"}}

Text Input: "Answering the following question: Who is the CEO of Google? (A) Sundar Pichai (B) Bill Gates (C) Tim Cook (D) Satya Nadella. Answer:"

However, this transformation may lose some information, such as what are the key elements of the task. Our proposed JsonTuning method aims to address this issue. Consequently, JSON structures offer more valuable information than natural language texts, resulting in higher expressivity.

Regarding transforming the user's request into a structured format, especially for complex use cases

Transforming user requests into a structured JSON format can be efficiently achieved by developing a user interface that enables users to input key task elements, task prompts, and control information. This approach ensures that users can effectively communicate their requirements without needing to understand JSON structures. As the input and output JSON structures are consistent across all tasks, we can automatically construct them based on the user inputs. For more complex use cases, the user interface can guide the model in identifying the essential information for the target task and the desired outputs.

JsonTuning offers several advantages for applications:

• Robustness: The method's resilience to varying prompts significantly reduces the need for users to test multiple prompts for optimal performance. If users do not wish to provide a task prompt, a default prompt can be generated based on the key task elements they supply, thereby minimizing user effort. Furthermore, as backend models are updated or changed, the optimal prompt for natural language texts may vary, necessitating adjustments for each model version. In contrast, Json-tuned models may not require prompt modifications even when the backbone model is updated, considerably reducing user effort.

• Output Control: The output control component allows users to accurately specify and easily parse the output, which can be challenging when using natural language texts. Users only need to comprehend basic types such as string, object, and array. The descriptions and examples of these data types can be provided through the user interface. This approach necessitates minimal human effort, and the underlying JSON structures can be automatically constructed once users indicate the output elements they require.

By implementing JsonTuning in real-world applications, a more professional and efficient user experience can be achieved.

Introduction of Additional Knowledge in Input

In Section 4: Analysis of our paper, we conduct an ablation study of JsonTuning without label space and control information, where JsonTuning still outperforms TextTuning in terms of generalization. To address your question comprehensively, we perform additional experiments with TextTuning, incorporating label space and control information. The generalization and robustness results on LLaMA-7b are as follows:

Generalization Results:

Robustness results on MMLU with 10 different prompts (we calculate the mean and standard deviation (Std) of the performance using these 10 prompts.):

We can make the following observations: (1) JsonTuning consistently surpasses TextTuning in both generalization and robustness, regardless of the inclusion of label space and control information. Importantly, JsonTuning significantly reduces the model's sensitivity to different prompts, as evidenced by the lower variance in performance across prompts. (2) TextTuning with control information still struggles to generalize well to more complex structured tasks, as illustrated by the poor performance on the EE task. JsonTuning demonstrates a marked improvement in this regard. These findings emphasize the importance of structured formats in instruction tuning. Furthermore, our proposed components – label space and control information – consistently enhance the model's generalization capabilities and robustness, highlighting their value in instruction tuning.

Potential Incompleteness of Generalization Experiments

Our evaluation encompasses various unseen tasks and datasets. While some task types, such as multiple-choice QA, are encountered during training, the specific evaluation tasks and datasets remain unseen. Furthermore, tasks like EE and TQA represent entirely novel task types. Our JsonTuning method allows models to generalize to more complex EE structures, even if they were not encountered during training, demonstrating the robust generalization capabilities of our Json-tuned models.

We conduct an evaluation of LLaMA-7B-Json and LLaMA-7B-Text on AlpacaEval. Both models were trained with label space and control information. To assess LLaMA-7B-Json, we construct an input JSON structure for a natural language question Q, represented as {"input": {"question": Q, "instruction": "{question}\n{answer}"}, "output control": {"answer": {"type": "string"}}}. The win rates for LLaMA-7B-Json and LLaMA-7B-Text are 4.78% and 4.64%, respectively. The slightly better performance of LLaMA-7B-Json suggests that JsonTuning may offer additional advantages in open-ended scenarios.

Since the Super-NI test set is already incorporated into the Flan2022 collection, it would not be appropriate to evaluate models on this test set, as many instances in it are seen during training.

Analysis of the tokens used for JsonTuning and TextTuning

We have calculated the input and output tokens used for each training and evaluation example in both TextTuning and JsonTuning. The average tokens of all training and evaluation examples for these methods are presented in the following table:

The table shows that JsonTuning uses approximately 25% more tokens for training and evaluation compared to TextTuning. The additional tokens are primarily used to encode the JSON structured format.

There are potential ways to reduce the tokens used for JsonTuning:

• JsonTuned models are robust to different prompts, so using lengthy prompts may not be necessary. We can replace them with concise prompts.
• JsonTuning offers the flexibility to adjust the structure. Simplifying the input and output structures could be helpful, as JsonTuning requires more tokens for more complex structures.

What if we text-tune the model and then Json-tune it on a small dataset?

To explore this, we continue instruction-tuning the text-tuned model with JsonTuning using varying numbers of examples (120, 600, 1200, 6000, and 12000) while maintaining the relative data ratio of the Flan 2022 collection and structured tasks in our primary experiments. The LLaMA-7b results on MMLU and NL2SQL are as follows:

The results of the original text-tuned model on MMLU and NL2SQL are 43.11 and 8.60, respectively.

We observe that: (1) A very limited number of JSON-formatted examples (e.g., 120) is insufficient for the text-tuned model to handle JSON-formatted tasks. (2) Tuning the text-tuned model with a small set of JSON-formatted examples results in degraded performance. These observations suggest that using the text-tuned model as a base for JsonTuning is not appropriate.

What if we text-tune the model and then use JSON-formatted examples in a few shot prompt?

This approach is ineffective. We sampled 100 MMLU examples and used 3 JSON-formatted examples as few-shot prompts. The text-tuned LLaMA-7b model still generates natural language texts instead of JSON structures and these texts seem non-sense. While a significantly larger text-tuned model may generate valid JSON structures, its performance could be substantially lower than the JSON-tuned model.

What is the decoding mechanism used for evaluating models? I assume it is just greedy sampling, but it would be nice for more evaluation details to be in the appendix.

You are correct. We use greedy decoding. We appreciate your suggestion and will include more evaluation details in the updated paper.

How often was invalid JSON generated? What do the authors think about constrained decoding schemes like Scholak et al, is there any point of using constrained decoding to improve JSONTuned models?

The proportion of invalid JSON structures generated by LLaMA-7B on evaluation tasks is as follows:

As anticipated, tasks with intricate output structures, such as NER and EE, exhibit a higher ratio of invalid JSON structures compared to tasks with simpler output structures, such as MMLU and BBH.

PICARD, a constrained decoding method proposed by Scholak et al., is designed to produce valid SQL queries by rejecting inadmissible tokens at each decoding step. In the context of our JsonTuning paradigm, employing a constrained decoding scheme that ensures the generation of valid output structures adhering to the output control could be advantageous. However, implementing such a method may be challenging, and it could potentially increase decoding time. As an alternative, we may utilize post-processing techniques to obtain valid JSON structures, as they are typically efficient and cost-effective. Previously, we treated invalid output structures as NULL, which might have been overly strict for evaluation purposes. Moving forward, we plan to investigate suitable post-processing techniques that can be broadly applied to various output structures, aiming to enhance the performance and validity of Json-tuned models.

Regarding the significance of our work

The performance of instruction tuning is influenced by several key factors, including the base model, tuning data, and data format. While the base model and tuning data have received considerable attention from researchers, the significance of data format has often been overlooked. In this paper, we aim to emphasize the importance of data format in instruction tuning, focusing on its role in generalization, robustness, and output control.

Our experiments and analysis demonstrate that a structured data format, such as JSON, is crucial for effective instruction tuning. First, our method, JsonTuning, consistently outperforms TextTuning across various base models and tuning data quantities, highlighting its generalizability and complementarity with other factors. Second, JsonTuning substantially enhances the robustness of instruction-tuned models, an improvement that is difficult to achieve solely through the base model and tuning data optimization. For instance, our robustness results indicate that upgrading the base model from LLaMA to LLaMA2 does not significantly enhance robustness; however, JsonTuning considerably improves it. Finally, JSON structures offer well-defined control information for output generation, a feature that is challenging to achieve using natural language text alone.

It is not clear if a TextTuning model with candidate answers and output control in plain text would also perform as well as JsonTuning. In other words, how much gain was from the structured format of JSON itself?

In Section 4: Analysis of our paper, we conduct an ablation study of JsonTuning without label space and control information, where JsonTuning still outperforms TextTuning in terms of generalization. To address your question comprehensively, we perform additional experiments with TextTuning, incorporating label space and control information. The generalization and robustness results on LLaMA-7b are as follows:

Generalization Results:

Robustness results on MMLU with 10 different prompts (we calculate the mean and standard deviation (Std) of the performance using these 10 prompts.):

We can make the following observations: (1) JsonTuning consistently surpasses TextTuning in both generalization and robustness, regardless of the inclusion of label space and control information. Importantly, JsonTuning significantly reduces the model's sensitivity to different prompts, as evidenced by the lower variance in performance across prompts. (2) TextTuning with control information still struggles to generalize well to more complex structured tasks, as illustrated by the poor performance on the EE task. JsonTuning demonstrates a marked improvement in this regard. These findings emphasize the importance of structured formats in instruction tuning. Furthermore, our proposed components – label space and control information – consistently enhance the model's generalization capabilities and robustness, highlighting their value in instruction tuning.