We greatly appreciate your feedback.

Fit for ICML

Thank you for raising this point. We believe that this paper is a strong fit for the Application-Driven Machine Learning track of ICML, defined as papers that “introduce novel methods, datasets, tasks, and/or metrics according to the needs of a real-world use case”. We have made contributions along all of these dimensions. We have combined LLMs and optimization to address the sustainable menu design task (outperforming an expert human chef and several other baselines), introduced two datasets not previously studied by the ML community, and introduced four novel tasks and associated evaluation metrics.

We believe that publication of our work in ICML would be mutually beneficial for the ML community, serving to raise awareness of this important testbed for ML algorithms, and the sustainable food community. Climate Change AI was founded in 2019 by AI researchers to encourage more research at the intersection of AI and sustainability, and has held 12 workshops at NeurIPS, ICML, and ICLR over the past 6 years, suggesting that the intersection of AI/ML and climate change mitigation is indeed of interest to the ML community. Additionally, in recent years, sustainability organizations such as the Good Food Institute, Food Systems Innovation, New Harvest, and the Bezos Earth Fund have called for research at the intersection of AI/ML and sustainable food, particularly novel sustainable protein sources. Food systems are responsible for one third of human-caused greenhouse-gas emissions, and thus represent an important part of AI/ML for climate change mitigation efforts. Despite this, no prior work on sustainable food, to our knowledge, has been published in an AI/ML conference. Our work is a step in this direction. We plan to publish followup work in domain journals to also ensure impact in the sustainable food community.

ML contribution

We would like to clarify our ML contribution. For the recipe preference prediction, sensory profile prediction, and experimental design tasks, yes, we apply LLMs and analyze their performance, without contributing new methods. However, for the menu design task, we find that LLMs on their own do not adequately balance multiple constraints, and present a framework for combining LLMs with optimization techniques, which we find reduces emissions by 79% while maintaining patron satisfaction. We also provide an initial theoretical analysis of our framework (Proposition 6.1), showing that the error of our approach can be bounded as a function of the number of items selected and the maximum item-level prediction error of the LLM. While we currently only apply this framework to sustainable menu design, we believe it can also inspire future work. Besides the sustainable menu design task we study, examples of applications we are motivated by include health coaching (“Generate a diet and exercise plan for achieving my goals while meeting constraints on ingredients, cost, preparation time, and current injuries”), curriculum design (“Generate a curriculum that will maximize student engagement while meeting constraints on topics covered and class time”), and travel planning (“Generate a travel plan that meets my preferences and covers the following locations”). In these applications, both background knowledge (of nutrition, fitness, education, travel, human preferences, etc.) and optimization (to meet constraints) are necessary, and we show how to combine the background knowledge of LLMs with optimization tools.

Our framework contributes to two strands of prior work in the ML and optimization communities. The first is the literature on LLMs for optimization modeling [1,2,3], which focuses on reducing barriers to the use of specialized optimization software via allowing users to specify optimization problems in natural language. Our work builds upon this literature, which assumes that the optimization problems are precisely specified. In our framework, LLMs are used to provide background knowledge on topics such as human preferences, to convert an imprecisely specified optimization problem (as is the case with many real-world planning, scheduling, decision making, etc. problems, such as the three mentioned above) to a precise optimization problem. The second is the predict-then-optimize framework [4,5], as pointed out by reviewer XTjM. Here, our contribution is to add a generation step, in which an LLM generates the elements of the ground set (e.g. recipes, exercises, travel destinations) from which a solution is produced, as well as to apply LLMs for the prediction step.

[1] AhmadiTeshnizi, A., et al. “OptiMUS.” ICML 2024.

[2] Jiang, C., et al. “LLMOPT.” ICLR 2025.

[3] Huang, C., et al. "ORLM." arXiv 2024.

[4] Elmachtoub, A. et al. "Smart `Predict, then Optimize.’” Management Science 68.1 (2022).

[5] Bertsimas, D. et al. "From Predictive to Prescriptive Analytics." Management Science 66.3 (2020).

Thank you for your review!

We greatly appreciate your feedback.

Statistical analysis

Thank you for raising this point. We note that we could have analyzed this data in alternative ways, given both the small sample size ($n=60$ ratings across 30 products) and clustering in the data (namely, pairs of products that were evaluated by the same food scientist, or where their associated experimental design was generated by the same food scientist). We ran additional statistical tests and found that the statistical significance of our findings are robust to all analysis variations we tested, which includes methods that do not rely on the central limit theorem. We will include this in the revised paper. The table of $p$-values across dimensions and methods is below. For all dimensions the mean value was higher (better) for o1-preview than the human food scientists.

Additionally, we would like to clarify our sample size for this task. Each data point was a rating of a human or LLM experimental design to improve a product. 30 products total were evaluated, for each of the two groups (human and LLM), yielding $n=60$ ratings total. The total number of human evaluators was 20. As stated in Appendix D.1, in Phase 1 (generation phase), 15 food scientists were recruited to generate experimental designs for 2 products each, yielding 30 products with associated experimental designs. o1-preview was also prompted to generate experimental designs for each of the 30 products. Then, in Phase 2 (evaluation phase), another 15 food scientists (with overlap from Phase 1, but ensuring that no one evaluated their own designs) evaluated both human and LLM designs for 2 products each, on the four dimensions of accuracy, complementarity, specificity, and percent time saved. Then, for each of the four dimensions, a $t$-test was performed on the human vs. LLM scores, though as we show above the result is robust to alternative tests. We will make this more clear in the final version.

Evaluation criteria and satisfaction metric

Regarding the comment “Something similar happens with the satisfaction metric, which relies on a proxy obtained by the LLMs, which the paper points out as biased towards omnivore options”, we clarify that for the menu design task, the satisfaction metric is based on responses of actual human participants, in response to the question, “How satisfied are you with your set of choices?” Please let us know if we misunderstood your comment; we will be happy to respond further.

Across tasks, we use a combination of automated metrics and those that rely on human judgment. In the preference prediction tasks, we use automated metrics (accuracy relative to ground truth sensory panel or recipe rating data). In the menu design task, the emissions computation is automated.

We agree that the human-reliant evaluation in the experimental design task could introduce bias, but we recruited 20 distinct expert human raters to minimize systematic bias.

Definitions of accuracy

We appreciate this point and will make the distinction clear in our revision.

Theoretical claim and typo

You are correct that the upper bound was overestimated, and should be $2Kϵ$ - thank you! We will fix this as well as the typo in the updated version.

Purpose of theoretical bound

The theoretical bound provides the insight that the maximum error of our approach can be bounded based on the number of items selected (rather than e.g. the size of the ground set) and the maximum item-level error of the LLM. We agree that future work could assess how downstream performance varies depending on the quality of the preference prediction method, and compare empirical results with the theoretical bound.

Supplementary material

We apologize for the insufficient documentation. Though we are not allowed to modify our submission at this stage, we commit to releasing a public GitHub repository that is well documented and provides full instructions for reproducing our results, other than information we cannot release due to IRB restrictions or the policy of the data provider (for the sensory panel data).

Thank you for your thoughtful review!

We greatly appreciate your feedback.

Baselines

We have added two baselines (expert chef, and the beef to chicken substitution you suggest), both of which we outperform, and three ablations (remove preferences component, remove diversity component, and remove both) for the menu design task. The chef was given the same set of instructions as the LLMs, but with more time: one hour. (Our o1-preview+IQP procedure takes ~3.5 minutes to run). We found that the chef did not always meet the ingredient availability constraint. Additionally, while the chef generated creative recipes, they lacked a strategy for reducing emissions, choosing to preserve several high-emissions dishes.

Below are our new results, compared with the original menu and our proposed method o1-preview+IQP. SD is in parentheses.

Novelty of empirical findings

We would like to clarify that we do have several novel findings, most notably that 1) LLMs can outperform expert food scientists in the generation of experimental designs for improving sustainable protein formulations and 2) our LLM+IQP algorithm outperforms an expert human chef and several baselines in the design of sustainable menus. Additionally, we characterize performance of LLMs on preference prediction tasks, finding that they can be useful for coarse-grained prediction but that further work is needed for fine-grained prediction. This is in addition to our novel task formulations.

Related work

Thank you so much for pointing us to the “predict-then-optimize” literature. We will cite it in the updated version. The ML contribution of our paper can be viewed as extending the predict-then-optimize framework by incorporating a component where the elements of the ground set (e.g. recipes) are generated, and applying LLMs to both the generation and prediction steps. Incorporating measures of LLM uncertainty and conducting additional theoretical analysis of this generate-predict-then-optimize framework is an ongoing area of work.

Here we also address Vdt3’s references. Regarding [1] and [2] from their review, our framework differs in that the LLM is used to provide background knowledge on topics such as human preferences, to convert an imprecisely specified optimization problem (as is the case with many real-world planning, scheduling, decision making, etc. problems) to a precise optimization problem. Besides the sustainable menu design task we study, examples of applications include health coaching (“Generate a diet and exercise plan for achieving my goals while meeting constraints on ingredients and current injuries”), curriculum design (“Generate a curriculum that will maximize student engagement while meeting constraints on topics covered and class time”), and travel planning (“Generate a travel plan that meets my preferences and covers the following locations”). These applications cannot be readily addressed by the work in [1] and [2], which assume that the provided optimization problem is fully specified. Moreover, our work adds to the set of applications in this literature. Regarding Vdt3’s reference [3], we appreciate the pointer to this comprehensive survey. Our work differs from this literature due to our focus on leveraging background knowledge of LLMs to expand the class of reasoning problems that can easily be addressed with optimization tools.

Our contributions

The Application-Driven ML reviewer instructions state, “Originality need not mean wholly novel methods. It may mean a novel combination of existing methods to solve the task at hand, a novel dataset, or a new way of framing tasks or evaluating performance so as to match the needs of the user.” We have made contributions along all of these dimensions. We have combined LLMs and optimization to address the sustainable menu design task (outperforming an expert human chef and several other baselines), introduced two datasets not previously studied by the ML community, and introduced four novel tasks and associated evaluation metrics.

We agree that we do not have significant theoretical contributions. However, we do not believe that this is required for the Application-Driven ML track, which notes that the form of the contribution may differ from papers in the main track.

Thanks again!

We greatly appreciate your feedback.

Theoretical claims

A proof is in Appendix A. YVD5 noted that the bound can be improved, which we will incorporate into the final version.

Related work

Thank you, we will cite these works. Please see our response to XTjM, where we discuss these papers; we did not have space to include it here.

Results mainly for food systems

Please see our response to VYkF explaining why we believe our paper is a strong fit for the Application-Driven ML track of ICML.

Dataset size and complexity of taste preferences

The Food.com dataset contains 522,517 recipes and 1,401,982 reviews. The recipes contain at least 50, and on average 121.34 reviews. The mean SD of ratings across the recipes is 1.21 (on a 5 pt scale), suggesting diversity of ratings. The NECTAR dataset consists of 47 products, with at least 100 human sensory evaluations per product along 21 dimensions, with 1150 distinct human taste testers. The subjects were restricted to American omnivores, and we will add to the Limitations section that our results may not generalize outside of this population. However, the sample was designed to be representative of American omnivores as a whole, e.g. was diverse along dimensions of age, gender, and race. Please see the NECTAR 2024 report for exact statistics (we will include this in the final version). The mean SD of the ratings across products for the dimensions we study in our sensory profile prediction task ranged from 0.89 (Greasiness) to 1.84 (Overall Satisfaction) - both on a 7 pt scale - suggesting diversity of ratings.

Finally, we note that the NECTAR dataset is expanding, and will reach 500 products and 50,000 sensory evaluations by 2026, further increasing the potential impact of introducing this dataset to the ML community. It has already expanded by 126 products (each with at least 100 sensory evaluations) since the submission of this paper. We plan to re-run our experiments on this expanded dataset.

Hypothetical setting

Please see our response to 7evN.

Optimal $\lambda$, and sensitivity analysis

We placed the maximum weight on diversity (for our input data $\lambda=100$ achieves the same result as $\lambda=\infty$) to ensure a diverse set of options for online participants, who may have allergies or other constraints. Assuming a high quality ground set, we think this is a reasonable default choice in general. We studied the sensitivity of the optimal menu to the value of $\lambda$. When decreasing $\lambda$ with a step size of 1, the generated menus are identical until $\lambda=3$, at which point the optimal menu changes by 2 recipes (out of 36), suggesting robustness to the exact choice of $\lambda$. Future work could use the LLM-as-judge framework to optimize the value of $\lambda$ without running human subjects experiments.

Ratcliff/Obershelp

This algorithm computes similarity between $S_1$ and $S_2$ as $\frac{2K_m}{|S_1| + |S_2|}$. $K_m$ is the number of matching characters, defined as the length of the longest common substring (LCS) plus recursively the number of matching characters on both sides of the LCS. More details can be found in the difflib documentation. We used this algorithm for recipe similarity because it is a simple and widely used algorithm that allows for partial matches. As a sensitivity analysis, we tested the Levenshtein distance, another commonly used method for text similarity, and found that the optimal menu changed by 4 recipes (out of 36), suggesting that that performance boost, if any, would be limited. We will include this and other similarity metrics (e.g. those based on semantic similarity) in the final version.

Style standardization

We used o1-preview to standardize the style. Our prompt is in Appendix D.1, Figure 5. As in [1], we did not assess how standardization affected the outcomes; in general, responses remained similar after the standardization step, with a few exceptions, e.g. where the humans referred to their personal experiences.

Trade-off between computational complexity and solution quality

Our problem sizes are relatively small, with a ground set of size 56. We were thus able to solve our problem instances optimally in less than a second. Essentially, there was no tradeoff between computational complexity and solution quality for our use case.

Prompt engineering

For style standardization, we adapted a prompt from prior work [1]. For the other tasks, the ML and culinary/food science members of our team collaborated to iteratively design the prompt. We aimed to test LLMs’ potential off the shelf, which is why we didn't do extensive prompt engineering, as in [2].

[1] Si, C. et al. "Can LLMs Generate Novel Research Ideas?" ICLR 2025.

[2] Bulian, J., et al. “Assessing LLMs on Climate Information.” ICML 2025.

Thank you for your review!

We greatly appreciate your feedback.

Ablations and analysis

Our submission included two ablations - removing the IQP component entirely (just prompting o1-preview directly to revise the menu) and removing the descriptions. We add three other ablations: removing the estimated preferences, removing the diversity term ($\lambda=0$), and removing both. Satisfaction declines ($p=0.01$) when both are removed. We also add two baselines: expert chef and replacing beef with chicken. Our method improves upon both. Please see our response to XTjM for our results table, which we did not have space to replicate in this response.

Ground-truth validation for menu-based emissions reductions

Our setting, an online experiment, is commonly used in the sustainable food literature [1,2,3,4] since it mimics popular online food delivery platforms. We acknowledge that some prior work tried to align incentives via delivering the meals to 1 in 30 participants [3] or providing food vouchers to 1 in 20 participants [4]. Given the complexities of replicating this setup for LLM generated recipes, we leave this to future work. Regardless, it is not guaranteed that conclusions based on online environments will generalize to real-world food environments. We will note this in the Limitations section.

[1] Attwood, S. et al. "Menu engineering to encourage sustainable food choices when dining out: An online trial of priced-based decoys." Appetite 149 (2020).

[2] Weijers, RJ, et al. "Nudging towards sustainable dining: Exploring menu nudges to promote vegetarian meal choices in restaurants." Appetite 198 (2024).

[3] Lohmann, PM., et al. "Choice architecture promotes sustainable choices in online food-delivery apps." PNAS Nexus 3.10 (2024).

[4] Banerjee, S, et al. "Sustainable dietary choices improved by reflection before a nudge in an online experiment." Nature Sustainability 6.12 (2023).

LLM generated recipes are not tested for real taste, only inferred preferences

Yes, in the menu design task, the generated recipes are not actually prepared and tasted by participants. This would involve a number of logistical and ethical challenges and is beyond the scope of the current study. We will note this in the Limitations section. We do use actual tasting data in our sensory profile prediction and experimental design tasks, and actual ratings in our recipe preference prediction task.

Sensory profile prediction task lacks stronger benchmarks

Our baselines in the submitted version were constructed on the basis of the food science literature and the available nutritional information, e.g. for overall satisfaction and purchase intent, the average of normalized fat and sodium content is used. We agree that more sophisticated baselines could be tested.

To address this, we have obtained an expert food scientist baseline for the Overall Satisfaction dimension of the sensory profile prediction task, which we think is the most relevant baseline for food science practitioners. The expert food scientist spent 2 hours on the task (approximately 1.5 minutes per pair). Statistically significant results (after Bonferroni correction) are in bold. We find that across all pairs for the Overall Satisfaction dimension, the expert food scientist does not achieve a statistically significant improvement over a random baseline, underscoring the difficulty of the task. The expert food scientist’s accuracy is similar to that of o1-preview, and both underperform our baseline based on the nutritional information and food science literature. We find that for coarse-grained prediction (quartile 4 of the ground truth preference gap), the expert food scientist again does not achieve a statistically significant improvement over a random baseline. However, both o1-preview and the nutritional baseline do outperform a random baseline (81% and 86% accuracy respectively), supporting the potential of automated methods, particularly for coarse-grained comparisons, which could save time in testing and development. We will include this result in the final version.

Thank you for your support!