Personalized Decision Modeling: Utility Optimization or Textualized-Symbolic Reasoning

We thank you for your time and effort in reviewing our paper! We find your suggestions very helpful and we hereby address your questions:

W1: Complexity, inference time, and token / cost footprint

We sincerely thank the reviewer for highlighting this point. Here we performed an additional experiment that tracks the number of the token consumed and estimated cost, which is reported below.

All runs use GPT-4o-mini (128 k ctx.; $0.15 / M input tok.,$0.60 / M output tok.).
Costs below list input → output fees. Pricing source: OpenAI docs (July 2025).

Stage 1 Symbolic search

Stage 2 TextGrad refinement

Take-aways

• Stage 1 dominates both tokens and time; Stage 2 is negligible (< 7 k tokens, < 6 min).
• Even the heaviest configuration (30 + 5 iters) costs < $0.35/run, very affordable.
• Token growth matches our theoretical $\mathcal{O}\bigl((K T + N T′)\bigr)$ prediction; adding more threads lets us parallel-slice Stage 1 to reduce wall-time.
• Since LLM outputs are not that stable nad can occasionally drift from instructions (e.g., formatting glitches), we should consider the budget for automatic retries; the table above is just for reference, actual costs fluctuate and are model-dependent.

W2: Value range of hyperparameters

Thank you for pointing out that several run-time hyper-parameters were not explicitly reported. We have added the following table for clarity:

W3: LLM-based Decision-Making Models

Thank you for drawing our attention to the recent work that applies LLMs directly to decision making under uncertainty! We have carefully read the papers you mentioned (e.g., DeLLMa [1], STRUX [2], and the follow-up studies on supply-chain optimization, personalized medicine, and autonomous driving [3–4]) and found them extremely insightful. We will incorporate these contributions into the final version, expanding the “LLM-based Decision-Making Models” subsection to include those works and discuss the complementary design choices each line of work makes. We greatly appreciate the pointer and will keep a close eye on the continued progress of these research groups.

W4: Qualitative Analysis for Formulas in Main Text

Thanks for your suggestions to enhance the qualitative analysis of the formulas presented in the main text! We have discussed about some representative examples in the rebuttal section of Reviewer HnWo.

Q1: Dataset, Subset Selection Concern

We thank the reviewer for raising this insightful question. We only down-sampled via stratified sampling; every feature and label stays, so the joint distribution matches the originals.

Q2: Scaling Behavior

Stage 1 – Group-level symbolic discovery
The scalability of Stage 1 is shown in Table2 of our paper: as we extend the symbolic-search loop from 0 → 30 iterations, performance gains steadily increase. But it will reach a threshold where further iterations will not yield significant improvements.

Stage 2 – Individual-level semantic adaptation

Experimental setup. We selected 5 individuals from each dataset and ran Stage 2 for 1, 5, 10, and 15 iterations. The results are summarized in the tables below. All scaling runs use gpt-4o-mini as base model via OpenAI API (July 2025 release) for Stage 2. For each experiment T (Stage 1) is set to 30.

Swissmetro (5 individuals)

Vaccine (5 individuals)

Key observations

1. Fast saturation. For Swissmetro, metrics stabilise after 5 iterations, with only a slight bump at 15. Vaccine peaks at 5–15 iterations, with no monotonic gains thereafter.
2. Modest Stage‑2 gain. The largest improvement over the 1‑iteration baseline is +0.10 accuracy, confirming that Stage 1 contributes the bulk of predictive power.
3. Practical default. Capping Stage 2 at 5 iterations captures > 95 % of the attainable benefit while keeping extra cost (< 7 k tokens) negligible.

We thank you for your time and effort in reviewing our paper! We appreciated for noting that “integrating symbolic utility modeling with LLM-powered semantic adaptation for personalized decision-making is novel and will inspire future work.” Your recognition of ATHENA’s ability to deliver accurate, interpretable, individual-level utility models is highly encouraging.

We are also gratified that the other reviewers independently highlighted the same strengths. Reviewer F7TL praised the “clear motivation, technically sound two-stage design” as well as our visualizations and writing clarity. Reviewer HnWo called the approach “very interesting and original,” commending the sound methodology and thorough ablations. Reviewer zWuZ valued the detailed formalism, the importance of combining LLMs with symbolic methods, and the originality of our neural-symbolic framework.

Together, the reviewers’ comments affirm that ATHENA makes a timely and impactful contribution to human-centric, interpretable personalized decision modeling.

Regarding your concerns, we added below experiments:

• End-to-end models: BERT, TabNet, MLP.
• Additional LLMs: Qwen3-32b, DeepSeek-R1-Distill-Qwen-32B, GPT-4o.

Results: Please see the table below. All new runs use a randomly selected subset (size of 100 individuals) from the dataset described in §4.1.

Note. Qwen3‑32B TextGrad baseline experiments are still running; we will publish the final numbers and update the table within the next 1–2 days.

We evaluate ATHENA against two state-of-the-art open-source models (Qwen3-32b and DeepSeek-R1-Distill-Qwen-32B) and three leading commercial offerings (GPT-4o-mini, GPT-4o, and Gemini-2.0-Flash). Across both tasks, ATHENA achieves superior performance—recording the highest accuracy, F1 score, and AUC of all baselines. To further broaden our comparison, we also include three additional benchmarks: BERT, TabNet, and a multilayer perceptron (MLP).

Why GPT-4o-mini & Gemini-Flash were the original choices

1. Practical adaptation cost: Individual-level TextGrad Adaptation requires hundreds of forward passes per person. Inferencing large open-source models like Qwen3 or DeepSeek-r1 locally or stronger models like GPT-4o via API is prohibitively expensive. GPT-4o-mini and Gemini-Flash are smaller, more efficient models that still provide strong reasoning capabilities.
2. Representative reasoning strength: GPT-4o-mini is explicitly tuned for chain-of-thought tasks, while Gemini-Flash is optimized for low-latency inference. Together they bracket a typical speed-versus-reasoning trade-off faced in practice.

We thank you for your time and effort in reviewing our paper! We find your suggestions very helpful and we address all your questions/comments as follows:

W1/Q3: End-to-End Baseline Comparison & More Powerful Base Models

Thank you for the suggestions of adding end-to-end baselines and more powerful base models. We've been paying close attention to new cutting-edge models and have supplemented our experiments accordingly.

• End-to-end models: BERT, TabNet, 3-layer-MLP.
• Additional LLMs: Qwen3-32b, DeepSeek-R1-Distill-Qwen-32B, GPT-4o.

Please see the updated results in the table published in rebuttal section of Reviewer FBCn

Q2: Explainability of ATHENA utilities

We agree that interpretability is crucial and confirm that ATHENA yields fully interpretable, end-to-end utility functions. Below is two representative segments from the Swissmetro and Vaccine dataset.

• Representative Example — Swissmetro Dataset

Feature: Between 39 and 54 years old, identify as female, and have an income between 50 and 100.

• Key take-aways for domain experts

These insights translate raw coefficients into specific levers for service design and policy -- exactly the actionable value the reviewer is seeking. Taken together, these conclusions are highly valuable for practitioners and align closely with the extensive body of prior research on travel behavior and mode choice.

[1] Shires, Jeremy D., and Gerard C. De Jong. "An international meta-analysis of values of travel time savings." Evaluation and program planning 32, no. 4 (2009): 315-325.

[2] Abrantes, Pedro AL, and Mark R. Wardman. "Meta-analysis of UK values of travel time: An update." Transportation Research Part A: Policy and Practice 45, no. 1 (2011): 1-17.

[3] Chang, Yu-Chun. "Factors affecting airport access mode choice for elderly air passengers." Transportation research part E: logistics and transportation review 57 (2013): 105-112.

[4] Weis, Claude, Kay W. Axhausen, Robert Schlich, and René Zbinden. "Models of mode choice and mobility tool ownership beyond 2008 fuel prices." Transportation Research Record 2157, no. 1 (2010): 86-94.

• Representative Example — Vaccine Dataset

Feature: Age 18–38, income above county median.

• Key take-aways for domain experts

These findings convert raw symbolic coefficients into tailored interventions: age-targeted messaging, trust-building coalitions, and occupation-based booster incentives.

[5] Green, Manfred S. "Rational and irrational vaccine hesitancy." Israel Journal of Health Policy Research 12, no. 1 (2023): 11.

[6] Noh, Yunha, Ju Hwan Kim, Dongwon Yoon, Young June Choe, Seung-Ah Choe, Jaehun Jung, Sang-Won Lee, and Ju-Young Shin. "Predictors of COVID-19 booster vaccine hesitancy among fully vaccinated adults in Korea: a nationwide cross-sectional survey." Epidemiology and Health 44 (2022): e2022061.

[7] Biswas, Nirbachita, Toheeb Mustapha, Jagdish Khubchandani, and James H. Price. "The nature and extent of COVID-19 vaccination hesitancy in healthcare workers." Journal of community health 46, no. 6 (2021): 1244-1251.

[8] Trent, Mallory, et al. "Trust in government, intention to vaccinate and COVID-19 vaccine hesitancy: A comparative survey of five large cities in the United States, United Kingdom, and Australia." Vaccine 40.17 (2022): 2498-2505.

W2: Ablation on the LLM component

Thank you for the suggestion regarding an ablation of the LLM component. We appreciate the opportunity to clarify the role and necessity of the LLM in our framework. 1) Our LLM-based utility search relies on two distinct libraries: a Symbolic Library, which contains standard algebraic operators, and a Concept Library, which includes semantically rich, high-level terms derived from domain-specific text. The Concept Library is only accessible through the LLM. 2) Ablating the LLM would not simply remove a component of the model; it would eliminate access to the Concept Library entirely. This makes a clean ablation infeasible: any such comparison would conflate the removal of the LLM with the removal of high-level semantic concepts, thereby blurring the effect under study. 3) Traditional symbolic regression methods (e.g., GP, SRBench) rely solely on the Symbolic Library and therefore cannot propose or reason over high-level, text-derived concepts. As a result, their outputs often lack the semantic richness and real-world relevance that LLM-enhanced models provide. This is not an apples-to-apples comparison, and such a comparison would overlook the core novelty of our approach.

In short, the LLM is not just a computational enhancement; it is essential to the inclusion of semantically meaningful, multimodal domain knowledge in the search process. We hope this clarifies the role of the LLM and why a traditional ablation would not offer a fair ablation. That said, we are open to alternative suggestions for more targeted comparisons, should the reviewer have specific ideas in mind.

Q1: Initialization in the Stage-2-only Ablation

Paper setting. For the “Individual-Level Semantic Adaptation only” ablation we seeded TextGrad with one symbolic template drawn uniformly at random. No best-of-N filtering was applied.

• New control experiment: best / median / worst of all formulas after 30 iterations

We sampled 10 individuals from Swissmetro datasets. We selected the best, median, and worst symbolic utility formula after 30 iterations of Stage 1. We then ran TextGrad on these formulas as initial seeds.

Key Observation After TextGrad refinement, median-quality seeds converge to within ≈ 3 pp Acc and ≈ 0.6 pp F1 of the best seed, while even the worst seed retains ≈ 73 % of the best accuracy. Hence Stage 2’s boost comes from gradient adaptation, not from simply picking the “best of N” start.

We thank you for your time and effort in reviewing our paper! We find your suggestions very helpful and we hereby address your questions:

W1/W2 & L2/L3 – Predefined Demographic Grouping & Robustness

To clarify, all baseline models received the same demographic features and groups. Our choice to use these groups (common demo-based group) was a deliberate decision motivated by:

• Theoretical Grounding: It aligns with established practices in decision choice modeling researches that use demographics to explain behavioral heterogeneity [1-3].

• Interpretability: It yields human-readable symbolic rules that translate directly into levers. E.g., for women aged 39-54 earning 50–100 k CHF, the rules reveal that shorter travel-time, premium comfort, and baggage assistance most sway their choice -- actionable insight that guides operators toward faster services, targeted upsells, and better luggage support for this segment.

• Robustness: Automatically discovering behaviorally consistent latent clusters is a challenging task that typically requires large datasets. Given the scale of our experimental data, relying on established demographic priors provides a more robust initial partition. It mitigates the risk of overfitting and learning spurious patterns. Also, we don't want to increase complexity or introduce additional hyperparameters to make the model more difficult to use.

Crucially, our ablation study shows that removing the group-level symbolic discovery stage causes a severe performance drop. This result underscores that a well-structured, group-informed prior is not a limitation but a necessary component for the model's success, preventing the unguided adaptation from converging to poor local optima.

While we agree automated group discovery is a valuable direction for future research, our work demonstrates the marked effectiveness of this principled, two-stage approach.

[1] Tymula, Agnieszka, et al. "Adolescents’ risk-taking behavior is driven by tolerance to ambiguity." Proceedings of the National Academy of Sciences 109.42 (2012): 17135-17140.

[2] De Bruijn, Ernst-Jan, and Gerrit Antonides. "Poverty and economic decision making: a review of scarcity theory." Theory and Decision 92, no. 1 (2022): 5-37.

[3] Croson, Rachel, and Uri Gneezy. "Gender differences in preferences." Journal of Economic literature 47, no. 2 (2009): 448-474.

W3: Component novelty

The contribution should be evaluated on the end-to-end paradigm and its conceptual novelty, not on isolated components. As noted in Section 1, population-level models often overlook each person’s unique cognitive calculus -- the belief, the unique constraints and the context-specific trade-offs that drive real decisions. ATHENA captures this calculus through a novel two-stage pipeline: symbolic regression extracts compact, group-level utility laws, and TextGrad then personalizes them to each individual.

• Symbolic regression for insight: Generates interpretable, domain-aware formulas that reveal feature relations far beyond merely black-box predictors.
• Informed semantic optimization: Uses those formulas as data-driven priors, mitigating TextGrad’s sensitivity to initialization and yielding faster, more reliable convergence to personalized models.

W4: Qualitative Analysis of symbolic representations

We agree that deeper qualitative analysis and actionable takeaways from our group-level utility formulas would strengthen the manuscript. Below, we provide a detailed qualitative analysis of the example Swissmetro symbolic utilities. For a parallel example on the Vaccine dataset, see the “Q2: Explainability of ATHENA utilities” section in Reviewer HnWo’s rebuttal. These examples demonstrate that our group-level utilities are both interpretable and directly actionable for domain experts.

• Representative Example — Swissmetro Dataset

Feature: Between 39 and 54 years old, identify as female, and have an income between 50 and 100.

• Key take-aways for domain experts

These insights translate raw coefficients into specific levers for service design and policy -- exactly the actionable value the reviewer is seeking. Taken together, these conclusions are highly valuable for practitioners and align closely with the extensive body of prior research on travel behavior and mode choice.

[1] Shires, Jeremy D., and Gerard C. De Jong. "An international meta-analysis of values of travel time savings." Evaluation and program planning 32, no. 4 (2009): 315-325.

[2] Abrantes, Pedro AL, and Mark R. Wardman. "Meta-analysis of UK values of travel time: An update." Transportation Research Part A: Policy and Practice 45, no. 1 (2011): 1-17.

[3] Chang, Yu-Chun. "Factors affecting airport access mode choice for elderly air passengers." Transportation research part E: logistics and transportation review 57 (2013): 105-112.

[4] Weis, Claude, Kay W. Axhausen, Robert Schlich, and René Zbinden. "Models of mode choice and mobility tool ownership beyond 2008 fuel prices." Transportation Research Record 2157, no. 1 (2010): 86-94.

Q1: Hyper-parameter clarification

Yes. In ATHENA both $T$ and $T’$ are user-set hyper-parameters:

• $T$ – maximum iterations of Stage 1: Group-level Symbolic Utility Discovery.
• $T’$ – maximum iterations of Stage 2: Individual-level Semantic Adaptation.

Figure 3 tracks the symbolic-discovery loop, so its x-axis is $T$ (30 iterations in our runs).

W5 & L1: Scalability and parameter $T$ and $T'$

1. Theoretical analysis

With $T$, $T'$ fixed, the algorithm is linear, not constant:

$$t_{\text{total}} = t_{\text{Stage1}} + t_{\text{Stage2}} \sim \underbrace{\lvert\mathcal{G}\rvert\cdot K\cdot T}\_{\text{symbolic utility search}} + \underbrace{N\cdot T'}\_{\text{textual gradient refinement}}$$

Each term is multiplied by the average LLM latency $\tau$ (roughly linear in token count):

$$\mathcal{O}\bigl((\lvert\mathcal{G}\rvert K T + N T')\,\tau_{\text{tok}}\bigr)\;.$$

Both stages can be parallelized:

• Stage 1: symbolic searches for each demographic group run independently.
• Stage 2: TextGrad refinements for each individual can be batched or dispatched to separate workers.

2. Empirical evidence

Using gpt-4o-mini, we measured wall-clock time and token usage on the Swissmetro subset:

Stage 2 – Individual adaptation

Stage 1 – Group-level discovery

Here we can see runtime and tokens grow linearly with iterations: Stage 2: 48 s → 316 s (T′ 1 → 5); Stage 1: 31 min → 66 min (T 5 → 30)—so a full Swissmetro run (T = 30, T′ = 5) finishes in < 70 min and ≈ 0.5 M tokens on one worker, with near-perfect parallel speed-up.

S1\S2\S3\S4: Paper Writing

Thank you for the helpful suggestions. We have revised all relevant instances in the manuscript. The updates are summarized below.

S5: Cross Entropy metric

Cross-Entropy is important because the ATHENA models decision-making as a probabilistic choice, consistent with the Random Utility Maximization theory. A lower CE value shows the model assigns higher probabilities to the choices individuals actually make.

While F1/AUC measures classification performance, CE provides a distinct and complementary evaluation of the model's confidence calibration. With all those metrics, we can better understand the model's performance and robustness.