Counterfactual Reasoning for Steerable Pluralistic Value Alignment of Large Language Models

Response to reviewer efFE:

Thank you very much for your insightful comments. We provide responses to your concerns about the causality usage (Weaknesses) and additional questions (Q) as follows.

W1: The use of causality in the framework feels forced.

The adoption of causality (a structured causal model) in our counterfactual reasoning framework is not forced but theoretically well-motivated by the core requirements of pluralistic value alignment. The effectiveness is also empirically validated by the ablation study in Sec 5.4.

For the task of pluralistic value alignment, we need to model how different value profiles lead to different LLM responses. As stated in Line 29-31, value studies in social science and psychology (e.g., Schwartz’s Value Theory) claim that multiple value dimensions with priorities serve as the guidance of human behaviors. This inspires us to address value alignment by modeling the inherent causality between complex values and LLMs behaviors. Therefore, we introduce the structural causal model (SCM) to capture the causal dependency between values and behaviors, as well as the interdependency among value dimension themselves.

Furthermore, Challenge 2(Value Steerability) in Line 45-48 highlights that different value profiles may have minor differences, such as the change in the priority of two value dimensions. We introduce counterfactual reasoning to simulate such shifts, however, without an explicit SCM, only LLMs prompts struggle to capture such nuanced adjustments.

In Sec 5.4, we conduct an ablation study to validate the significance of SCM. Specifically: i) removing the SCM while retaining counterfactual reasoning leads to a significant performance drop; ii) removing Counterfactual but keep the SCM for direct behavior prediction leads less performance drop. The results demonstrate that SCM is essential to capture the causality between values and behviors, but also enabling more effective counterfactual reasoning.

W2: Acquiring an accurate causal model is quite difficult.

In this paper, we do not build a value-to-behavior causal models from scratch using extensive annotated (value profiles, behaviors) pairs due to two main challenges:1) Large-scale datasets with comprehensive annotated values and behaviors are rare and costly to obtain; 2) Traditional approaches for building an SCM often struggle with high-dimensional, unstructured natural language inputs[7][8][9].

Instead, inspired by that LLMs have already embedded rich knowledge about values and LLMs have been regarded as a promising tool for causal inference tasks with natural language data, such as such as causal discovery and counterfactual reasoning[2][3][4][5][6], we adopt an LLM to build the causal model to encode the structural relationship among multi-dimensional values and how they jointly determine the final model responses, i.e., V $\rightarrow$ R (as introduced in Sec 3.2).

We acknowledge that this is an approximated causal model for value-to-behaviors, however, this already capture rich knowledge about the relationship between values and behaviors. The effectiveness has been validated by both quantitative experiments (in Sec 5.2, Sec 5.3) and qualitative experiments (Sec 6). We consider learning more faithful causal model as an important future work.

W3：How SCM is utilized in the framework?(with several sub-items in the following)

W3-1:In the abduction step, the endogenous variables are inferred but not the exogenous noises.

Regarding abduction: Given the structure of our framework and the black-box nature of LLMs, inferring exogenous noise variables strictly is not feasible. We instead use counterfactual reasoning to avoid the influence of exogenous variables. More specifically, we focus on the transitions of endogenous variables (from value to intermediate concept to response) and use LLM inference as a proxy for capturing the underlying causal relationships. Concerning the “value attributor” and priorities: for each response, we estimate the attribution for all values, not just a subset, and compare the estimated values with the target values to ensure alignment with the model. The value extraction process itself also utilizes LLMs, following current NLP and causal reasoning work [7][8], where LLMs are employed to identify relevant latent concepts and perform causal reasoning, leveraging their inherent causal attribution abilities [4].

W3-2: Details about the second intervention step. After the last abduction step, the value attributor extract priorities for all predefined values, obtaining the estimated value $v’$. Then, we intervene the priority score of $v’$ towards the target value $v$, i.e. do (V=v) as described in Line 157.

Comparing the estimated $v’$ with $v$ just aims to calculate the differences, and when there is no difference $|v’-v| < \theta$, there is no need to conduct following steps.

W3-3: Variables of the causal model in the counterfactual reasoning step

As introduced in Sec 3.2 (Line 130-131), our causal model aims to encode the relationship between values $V$ and the model response $R$, thus we have $X=[V,R]$. As the textual responses could usually consider some side information that may not be correlated with the values, such as the rephrase of descption, we introduce value concepts, which are behavioral indicators of values beyond redundant and noisy text as a more essential proxy of response (as introduced in Line 182-183). Thus, in practical implementation, we replace the variable $R$ in the causal model with concepts $C$, which is also used in the counterfactual reasoning step. In Line 208, we highlight that we would generate the final textual responses based on the value concepts.

W3-4: A relational graph and covariance matrix are sufficient to capture a causal model.

We would like to clarify that the relational graph and covariance matrix in our method is not intended to capture the causal model, as described in Line 201-206, they are used to capture the complex dependencies between value dimensions (addressing Challenge 1 - Value Complexity). In the response to W2, we explained that the value-to-behavior causal model is built or captured by an LLM with rich knowledge about values.

More specifically, the relational graph captures the original correlation among value dimensions, i.e. congruent, opposite or irrelevant; while the covariance matrix captures their priority scores.

Q1: Concern about why answers are not generated directly

This design—introducing explicit intermediate concepts or states for counterfactual reasoning—is now common in NLP (see, e.g., [relevant literature]). We introduce concepts/intermediate states because direct value-to-text mapping lacks interpretability and flexibility. Having an explicit intermediate stage allows for better control, counterfactual testing, and more nuanced understanding of model behavior. As shown in the ablation results in Table 4, removing this module leads to a significant drop in performance. The causal decomposition (value → concept → response) reflects the underlying logic of value-driven generation, and aligns with the way many recent controllable generation methods are designed.

Q2: Definition of "Strong" LLMs

By “strong” LLMs, we mean models that score higher on standard benchmarks of text understanding and instruction following (e.g., MMLU, BIG-Bench, GSM8K). There is no absolute threshold, but in practice, we select models with proven high performance in such evaluations. This is also common practice in the literature (e.g., OpenAI GPT-4, Google Gemini; see, e.g., [10,11]).

References:
[1] Pearl, J. (2009). Causality: Models, Reasoning, and Inference. Cambridge University Press.

[2] Jing Ma et al. (2024). Causal inference with large language models: A survey. arXiv preprint arXiv:2409.09822.

[3] Emre Kiciman et al. (2023). Causal reasoning and large language models: Opening a new frontier for c ausality. Transactions on Machine Learning Research.

[4] Zhijing Jin et al. (2023). Cladder: Assessing causal reasoning in language models. Advances in Neural Information Processing Systems, 36:31038–31065.

[5] Nick Pawlowski et al. (2023). Answering causal questions with augmented LLMs.

[6] Yuzhe Zhang et al. (2024). Causal graph discovery with retrieval-augmented generation based large language models. arXiv preprint arXiv:2402.15301.

[7] Amrita Bhattacharjee et al. (2023). Towards LLM-guided causal explainability for black-box text classifiers. arXiv preprint arXiv:2309.13340.

[8] Amrita Bhattacharjee et al. (2024). Zero-shot LLM-guided counterfactual generation for text. arXiv e-prints, arXiv:2405.

[9] Amir Feder et al. (2023). Data augmentations for improved (large) language model generalization. Advances in Neural Information Processing Systems, 36:70638–70653.

[10] OpenAI (2023). GPT-4 Technical Report.

[11] Gemini Team, Google et al. (2023). Gemini: A Family of Highly Capable Multimodal Models. arXiv preprint arXiv:2312.11805.

Thank you for the rebuttal. However, the response does not address my core concern about the usage of causality. In fact, with the provided clarification, it reinforces my concerns. I want to clarify that I am not questioning the significance of using SCM for this task, but rather whether the proposed method actually achieves appropriate and technically sound causal reasoning (as it claimed).

First, I don't see a properly defined SCM in the context of this task. The authors provide the original formal definition of SCM but fail to explain how it applies to the specific task at hand. Even though the actual structural equations might be difficult to estimate precisely, a proper definition is crucial. It seems to me there are several key endogenous variables: multiple values, concepts, and the final response. The authors do not provide a clear description of how the structural equations between these variables will be modeled, which leads to confusion. For instance, the authors claim that "We would like to clarify that the relational graph and covariance matrix in our method is not intended to capture the causal model, as described in Line 201-206, they are used to capture the complex dependencies between value dimensions" However, values and their relationships are inherently part of the SCM under consideration—in fact, they might be one of the most important components.

I feel like the proposed method simply uses a large black-box LLM with some constraints (the provided covariance matrix and relational graph) to perform some kind of "counterfactual reasoning." However, I don't believe this constitutes counterfactual querying as defined in the context of SCM (see Ch. 7 in [1]). In the rebuttal, the authors claim "We instead use counterfactual reasoning to avoid the influence of exogenous variables." However, inferring exogenous noise given observed evidence is a key component of proper counterfactual reasoning in SCM frameworks.

I think that the empirical benefits might stem from finer-grained control enabled by the constraints of multiple values and their predefined correlations. However, the current misconceptions regarding SCM and counterfactual reasoning represent a significant technical flaw in my view and make it hard to analyze why the proposed method might work or not.

Some other questions

1. (from the rebuttal) “This design—introducing explicit intermediate concepts or states for counterfactual reasoning—is now common in NLP (see, e.g., [relevant literature]).” I am not sure what the authors are referring to as [relevant literature]
2. In the "value intervention" step, is it necessary to provide all values? In other words, if we only specify "self-direction: 5," would the model be able to infer the remaining values automatically?

[1] Pearl, J. (2009). Causality: Models, Reasoning, and Inference. Cambridge University Press.

3. Clarification for other questions.

• Regarding [relevant literature]: Sorry for this typo. The relevant literature correponds to the recent papers [1][2] that explicitly introduce intermediate concepts or states for counterfactual reasoning in NLP area.

• For the "value intervention" step: For each specific question, we do not consider all values but only the value dimensions directly involved in the question (as described in Line 240-241, we consider up to 5 most related dimension to define the value profile $v$). For example, a scneario of "taking care of my parents" is related to Benevolence but irrelevant to Power. The model would only infer the related values but ignore others. When the given question involve the value "self-direction", we can specify "self-direction: 5" in the target, otherwise, we can only intervene other related dimensions.

We hope we have addressed your concerns

We hope our responses above could address your concerns and we're more than willing to respond to any further questions.

We would sincerely appreciate it if you could read our responses, and kindly reconsider the assessment of our work.

References

[1] A. Bhattacharjee et al. (2023). Towards LLM-guided causal explainability for black-box text classifiers. arXiv:2309.13340.

[2] A. Bhattacharjee et al. (2024). Zero-shot LLM-guided counterfactual generation for text. arXiv:2405.04793.

[3] A. Feder et al. (2023). Data augmentations for improved (large) language model generalisation. Advances in Neural Information Processing Systems 36, 70638–70653.

[4] Emre Kiciman et al. (2023). Causal reasoning and large language models: Opening a new frontier for causality. Transactions on Machine Learning Research.

[5] Zhijing Jin et al. (2023). Cladder: Assessing causal reasoning in language models. Advances in Neural Information Processing Systems, 36:31038–31065.

[6] Jing Ma et al. (2024). Causal inference with large language models: A survey. arXiv preprint arXiv:2409.09822. [7] Longxuan Yu et al. (2024). Causaleval: Towards better causal reasoning in language models. arXiv preprint arXiv:2410.16676.

[8] Alexander Marx and Jilles Vreeken. (2017) Telling cause from effect using mdl-based local and global regression. In 2017 IEEE International Conference on Data Mining (ICDM), pp. 307–316. IEEE.

[9] Natasa Tagasovska et al. (2020). Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 9311–9323. PMLR, 13–18 Jul 2020.

[10] Kun Zhang and Aapo Hyvarinen. (2012). On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv:1205.2599.

[11] Pengzhou Wu and Kenji Fukumizu. (2020). Causal mosaic: Cause-effect inference via nonlinear ICA and ensemble method. In International Conference on Artificial Intelligence and Statistics, pp. 1157–1167. PMLR.

[12] Kristy Choi et al. (2022). Lmpriors: Pre-trained language models as taskspecific priors. arXiv preprint arXiv:2210.12530.

Sec 4.2 and Sec 4.3 in our submission detail how we leverage an LLM to implement the above formalized causal inference process respectively. Thus, our method is not just a large black-box LLM with some constraints, but a solid simulation of the counterfactual querying process as defined in the context of SCM. And extensive experiments verify the effectiveness of this simulated SCM counterfactual process.

We will add the formal definition of SCM and the corresponding implementation steps into our revision to enhance the technical soundness of causal reasoning in our framework.

2. Justification for using LLMs to implement causal inference

We approximate the above SCM inference process using an LLM, allowing the model to implicitly handle both structural relationships and exogenous factors. We are not the first to conduct causal inference with LLMs, which has become a popular strategy especially for natural language tasks with complex causality that are hard for traditional causal models, and achieved even better performance than classical methods. We have reviewed most studies in Sec 2.2, such as [4–6]. And we further present some empirical evidence in the following.

This table shows the performance of LLMs on the causal discovery task, evaluated on the Tübingen cause-effect pairs dataset [4]. Notably, models such as GPT-4 achieve accuracy on par with or better than traditional algorithms.

Beyond causal discovery, LLMs also demonstrate competitive performance across a range of causal reasoning tasks. This table summarizes results from [7], covering three major task categories: Causal Discovery (COPA, NPDS, e-CARE, Corr2Cause), Causal Inference (CLADDER), and Additional Causal Tasks (CRASS, MoCa, Tram).

These results confirm that LLMs are increasingly capable of addressing not only standard causal discovery but also a wide range of more complex causal inference and reasoning challenges.

Thanks for your constructive feedback. We address each of your main concerns as follows.

1. Formal definition of SCM in our framework

In our framework, an SCM $(X, \mathcal{F}, \epsilon)$ is built to encode the structural relationship among value dimensions and the final model responses to given questions. Thus, we mainly consider the questions $q$, the value dimensions $v = (v_1, v_2, ...)$, the value concepts $c_v = (c_v^1, c_v^2, ...)$ which are behavioral indicators of values beyond redundant and noisy text (as introduced in Line 182), and the final response $r$ as the endogenous variables. Other variables ($\epsilon_1, \epsilon_2$) that influences the response generation process as exogenous variables, as detailed in the Table.

All functions $\mathcal{F}$ to capture the relationships among variables are as follows. We first describe the procedure for obtaining the concept variable.

$

c_v = \mathcal{F}_c(P_a(c_v), \epsilon_1) = \mathcal{F}_c(v, G_v, \Sigma _{v}, q, \epsilon_1),

$

where $G_v$ is the relational graph capturing the correlations among values, i.e., congruent, opposite or irrelevant; $\Sigma_{v}$ is the covariance matrix capturing the relative importance among values, such as self-direction: 5 > conformity: 3. This functions is introduced in Line 206 (Eq.(2)).

We then present the modeling steps for obtaining the response variable.

$

r_v = \mathcal{F}_r(P_a(r_v), \epsilon_2) = \mathcal{F}_r(c_v, q, \epsilon_2).

$

As introduced in Line 208-210, we prompt an LLM to aggregate the concepts to generate the final response.

Since traditional causal reasoning methods mainly focus on tabular data and struggle with causal inference of natural language tasks, we follow recent studies[1][2][3] about causal inference with LLMs to implement the above two functions using a powerful LLM. Specifically, the three-step counterfactual reasoning step is completed as follows:

Dear Reviewer efFE,

Thank you again for your valuable comments and suggestions. We have posted further responses to address your concerns.

As the discussion period is nearing its end with less than two days remaining, we sincerely appreciate it if you could take some time to reply with further feedback on whether our responses have addressed all your concerns. If there are any other comments, we are more than willing to address them.

Best regards,

The authors

Response to Reviewer fCkw:

W1: Question (on value objectives and country alignment):

To clarify, in Section 5.3 (lines 300–302), responses are aligned according to four different countries/groups, not four value objectives. In all cases, the value objectives are defined as combinations of the ten Schwartz basic value dimensions.

• The "ten" refers to the ten basic value dimensions defined by Schwartz, which form the full set of value objectives used throughout our evaluation. In all cases, no Schwartz value dimension was ever discarded or omitted.
• The "four" refers to four alignment targets: two clusters/groups (Group 1 and Group 5, derived from our own user data for their clear differences), and two real-world country profiles (the UK and India).

None of the Schwartz value dimensions were discarded at any point in our evaluation. The four groups or countries used for alignment consist of two clusters identified from our own data (Group 1 and Group 5, chosen for their clear differences) and two real-world country profiles (the UK and India). The specific mappings for these four cases, along with their definitions, are provided in Appendix B.2.2 (lines 107–127), and the rationale for selecting these representative combinations is visualized in Appendix Fig. 1.

In summary, our evaluation always considers all ten Schwartz dimensions. The grouping is based on country-specific value profiles, not by omitting any objectives.

W2, Q3: Question (on interpretability analysis, Table 5): The reviewer finds the interpretability analysis in Section 6 and Table 5 unclear and questions the semantic meaning of certain frequent words.

Table 5 presents the most frequent words associated with each value and score level. Words marked in red indicate a strong correlation with the score polarity: for example, words frequently appearing in responses rated 5 (high score) tend to be associated with positive or supportive attitudes, while words appearing in responses rated 1 (low score) reflect negative or rejecting attitudes (e.g., "not").

Further examples can be found in Appendix C.5 (lines 271–274) and Appendix Table 17.

We will revise the Table and the revised text will explicitly explain how word polarity and context relate to each value and score, and clarify the interpretation of Table 5, including how to read the highlighted words.

W3: Question (on baseline clarity, Section 5.1.2):

Next, we will clarify baseline identification in Section 5.1.2:

• All baseline models discussed and highlighted (in bold) in Section 5.1.2 were actually used in our experiments, and their results are explicitly reported in Table 2 and Table 3.
• For fine-tuning-based baselines, experiments were conducted only on open-source models, as shown in Table 3. For closed-source models, only prompt-based baselines were tested, as shown in Table 2.

To improve clarity, we will revise the writing in Section 5.1.2 to explicitly state which models are used as baselines.

For full transparency, detailed descriptions of each baseline are provided in Appendix B.4 (lines 148–199).

Q1: Question (on human evaluation and value objectives)

Response:
Next, we discuss the question regarding our human evaluation and the selection of value objectives. In our study, we consider a total of 15 value objectives: five representative groups obtained by clustering real user data, five major European countries, and five major countries globally. For the human evaluation, we selected two representative groups (Group 1 and Group 5) and two countries (the UK and India) because their value profiles are more representative and diverse within our dataset.

For dataset alignment (e.g., with specific country profiles), we use established mappings as described in Appendix B.2.2 (lines 107–127). The full selection process and all value mappings are documented in the appendix.

Q2: Question (on Table 4, possible typo):

Response:
Sorry for the trouble caused by the typo. You are correct—there is a typo in Table 4. We will correct this in the revised version. We have also double-checked the other entries and can confirm that there are no further errors.

Response to Reviewer b5WM:

Thank you for your thorough and constructive feedback. Below we address each of your concerns directly:

W1: Clarify the paper’s focus on counterfactual reasoning.

While much of the paper discusses value abduction, counterfactual reasoning is also given equal attention in our framework. In fact, abduction is a prerequisite for meaningful counterfactual intervention: without accurate value attribution, downstream interventions would be arbitrary.

Furthermore, both the main experimental results and the human evaluation focus on measuring the alignment between our counterfactual generation answers and target values, rather than value abduction.

W2, Q5: Figure 4(b/c/d) explanation

• Subplot 4b focuses on the alignment distance between the original LLM response and the target values. We group the cases by this distance, computed as sum|v_original - v_target|, and calculate the MAE loss for each group. This shows that the greater the initial misalignment, the harder it is for the model to achieve value alignment—demonstrated by increasing MAE with distance.
  At the same time, our method achieves better alignment performance than the strongest baseline (Plan-and-Solve). Moreover, as the distance increases, the alignment loss of our method does not increase significantly, indicating better robustness.

• Subplots 4c and 4d illustrate the difficulty of aligning the LLM’s original value on a given dimension to the target aligned value. Each cell in the heatmap (e.g., coordinate 5,1) represents the MAE loss when transforming responses with an original value of 1 to align with a target value of 5. This allows us to quantify how challenging it is to shift model outputs across different value levels. Subplot 4c shows the results for the strongest baseline, while 4d presents the results for our method.
  The results show that it is particularly challenging to change responses with a very high original value to a very low value (and vice versa). Nevertheless, our method achieves strong performance across all value transitions.

W3: Concern about the impact of question-irrelevant values.

We agree this is an important point. We conduct an experiment where we randomly set unrelated values as "important" and evaluated the impact on output MAE. The results are summarized below:

Output MAE before and after randomly setting unrelated values as "important" (100 cases).

The negligible change in MAE demonstrates that interventions on non-relevant concepts/values do not significantly affect the model's outputs, further supporting the robustness of the causal structure learned by the model.

W4: Concern about the concept layer in interpretability results.

We conduct an ablation study to remove key concepts from the model and measure the impact on alignment performance. The results are summarized below:

For 100 randomly sampled questions, we report the average alignment score with the "Achievement" value objective for the US value profile, before and after ablation (removal of Achievement-related concepts):

Table : Average alignment scores for the "Achievement" value objective (US value profile) before and after ablation of Achievement-related concepts (100 random questions).

The observed drop in alignment after ablation quantitatively demonstrates the model’s reliance on key concepts for achieving value-specific alignment, further supporting the interpretability of the model.

W5: Concern about maintaining the priority between values in qualitative case studies results.

To facilitate understanding, we provide additional qualitative examples that illustrate how model outputs change when value priorities are reversed (not simply shifted in magnitude). For the 15 alignment objectives, which include 5 groups and 10 countries, we randomly selected 100 questions from each and evaluated the model's performance on two dimensions (Achievement vs Security).

Overall Statistics (Achievement vs Security, All Groups & Countries):

The table below summarizes the overall statistics for the value pair Achievement and Security across all groups and countries:

In summary, 87.4% of the model outputs successfully maintained the expected value priority relationship between Achievement and Security after intervention, demonstrating that our approach reliably preserves the intended value order in the majority of cases.

Q1: Concern about the relevance and valence of values.

Thank you for pointing out the Kaleido paper. We acknowledge that our current rating combines both relevance and valence, and we will discuss the implications of this in the revision. In our approach, value relevance is determined at the question level (see example in Table 7 in the Appendix): For each question, the set of relevant values is limited—we first extract the possible values applicable to the scenario, and then the model is instructed to output only for these values. We will clarify this distinction in the revised text.

Q2: Sensitivity to number of key concepts

We have added a hyperparameter analysis to test sensitivity to the number of intermediate concepts.

Sensitivity Analysis: Number of Key Concepts. The table below summarizes the impact of varying the number of key concepts on the model performance, measured by MAE and Correlation:

Table: Sensitivity analysis of the model performance with different numbers of key concepts, measured by MAE and Correlation.

As the number of key concepts increases, the model performance stabilizes. This is because the number of values involved in each question is fixed, so when the number of concepts is too low, it becomes difficult to generate a good answer. However, when the number of concepts is sufficiently large, performance improves and stabilizes.

Q3: Sensitivity to value intervention threshold

The value intervention threshold is set as a hyperparameter (currently 0, i.e., any change triggers intervention, making it very fine-grained). We will include an ablation showing performance under different thresholds and discuss the rationale and impact.

Formally, for each answer, we compute the sum of value differences across all dimensions where the value in the natural answer differs from the value in the target answer:

where $\mathcal{I} = \{i \mid v^\text{nat}_i \neq v^\text{target}_i \}$, and $v^\text{nat}_i$ and $v^\text{target}_i$ denote the value of the $i$-th dimension in the natural answer and target values, respectively.

Model performance (MAE and Correlation) under different value intervention thresholds.

Q4: Concern about alignment targets for DailyDilemma

There is indeed a challenge here: the DailyDilemma dataset involves 50 dimensions of values, but each character portrayal actually focuses on a few distinct dimensions. Therefore, there is inevitably a difference in values between characters. While we currently adopt scenario-specific alignment targets, this method may make it more difficult to quantify consistency in alignment across different scenarios.

Of course, based on your suggestion, we would be happy to conduct an experiment on clustering the value profiles in the dataset and using them as shared alignment targets to see if it improves this issue.

Limitations: scalability, LLM bias

We acknowledge that the limitations section could be further elaborated. We will expand discussion of:

• Scalability: As the number of values increases, accuracy drops (see Fig. 4a); this will be elaborated as a key limitation for real-world use. However, we note that the ten Schwartz dimensions already form a comprehensive and widely-accepted value system. According to our statistics, most scenarios (i.e., for each question) typically involve only a small subset of these values, rather than all ten at once. This practical observation mitigates, to some extent, the scalability issue in realistic applications.

• LLM-based evaluator bias: We will acknowledge this risk and its implications for evaluation fairness. To address this, we have also conducted human-annotated evaluations and survey-based assessments. In addition, we compared the results of LLM-based automatic evaluation with human judgment to ensure consistency between automated and human assessments.

Thank you again for your feedback. We hope the planned experiments and clarifications will fully address your concerns.

I have read the author's rebuttal, and thank them for their effort in addressing my feedback and questions. Their responses help me with better understanding the proposed approach and corresponding results. As a result, I will change my rating to a 5.

Regarding W5, my intention was to understand whether it was possible to "reverse" a model's output based on swapped value priorities, although the authors seem to show the model is quite robust and preserves relative value priorities across a set of questions. This may be due to the learned relationships in the SCM.

Response to reviewer e3SB:

W1: Regarding the concern on the number of experimental backbones

We acknowledge the reviewer’s view that having only two backbone models may be insufficient, and using more backbone models would make the paper more convincing. In fact, our current experiments already include two open-source models (Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct) and three closed-source models (gpt-4.1-mini, o3-mini, Deepseek-R1) as backbones, covering a diverse range of architectures and model capacities. The results for the closed-source models (gpt-4.1-mini and Deepseek-R1) are reported in Tab.2 of the main text. Results for o3-mini can be found in Appendix C.1.1 (lines 227–229, Tab.5). The two open-source models are presented in Tab.3 and Appendix C.1.2 (lines 235–242, Appendix Tab.12).

Based on the reviewer's suggestion, we conducted experiments and obtained results from GPT-4o-mini. On this backbone, our model's performance is also significantly improved.

Touché23-ValueEval

DailyDilemma

W2: Regarding writing clarity about the method framework

The following paragraph provides clarifications regarding the writing. We will also revise and enhance readability further based on these points.

• Algorithm: The algorithm is presented below in pseudocode with proper mathematical notation and conventions, following best practices from the literature. Due to the limitations of the Markdown environment, the table is somewhat simplified. The full version will use LaTeX in the final paper.

• Line 123: Here, "priority" refers to the internal ranking among values. For example, if the alignment target places more emphasis on security than on achievement (e.g., $v_{\text{security}}=5$, $v_{\text{achievement}}=3$), then the response should reflect this priority order, with security being prioritized over achievement in the generated answer.

(i) Mathematically, let $s = [s_1, s_2, ..., s_n]$ denote the target value scores, and $s' = [s_1', s_2', ..., s_n']$ the model output. Our objective is twofold:

• Minimize the score differences: $\mathrm{MAE} = \frac{1}{n} \sum_{i=1}^n |s_i - s_i'|$,
• Preserve priority: For all $i, j$, if $s_i > s_j$ then $s_i' > s_j'$.

(ii) These objectives directly correspond to the evaluation metrics used in our experiments: MAE quantifies the absolute difference between targets and outputs, while Correlation assesses how well the relative priorities among values are preserved (e.g., via Spearman correlation).

• Line 127: We appreciate the reviewer’s point. However, as clearly stated in lines 130–131 of our paper, we treat both values and answers as endogenous variables and explicitly model the value-to-answer relationship. By specifying the value→answer mapping and performing interventions, we can conduct further inference (value→?) after perturbation, which helps avoid the influence of exogenous variables. This approach addresses the concern raised and clarifies that the value-to-answer modeling is already handled appropriately in our methodology.

• Lines 130–131: Thank you for raising this point. To clarify:

(i) Description of the initial SCM graph:
As described in our paper, the initial causal graph in our Structural Causal Model (SCM) is constructed primarily based on domain knowledge and prior literature. Specifically, we define the main relationships—such as value→concept and concept→response—as the backbone structure for downstream causal inference. In our framework, we focus primarily on modeling the relationships among endogenous variables, as counterfactual interventions allow us to mitigate the influence of exogenous factors (e.g., linguistic style, tone, etc.). This approach ensures that our analysis targets the underlying causal mechanisms rather than superficial artifacts in the data.

(ii) Rationale for not explicitly displaying the full graph:
In line with existing work, both node-level and graph-level causal reasoning have been effectively addressed by leveraging the inherent knowledge within large language models (LLMs). Therefore, it is common practice not to explicitly enumerate the entire causal graph. Instead, recent studies typically specify only the key edges relevant to the target tasks, relying on the LLM’s internal representations to capture other dependencies as needed.

(iii) Relevant literature on LLMs + SCM:
This modeling approach is consistent with recent work that integrates LLMs with causal modeling or SCM frameworks. For further reference, please see:

[1] Jing Ma et al. Causal inference with large language model: A survey. arXiv preprint arXiv:2409.09822, 2024.

[2] Emre Kiciman et al. Causal reasoning and large language models: Opening a new frontier for causality. Transactions on Machine Learning Research, 2023.

[3] Zhijing Jin et al. Cladder: Assessing causal reasoning in language models. Advances in Neural Information Processing Systems, 36:31038–31065, 2023.

[4] Yuzhe Zhang et al. Causal graph discovery with retrieval-augmented generation based large language models. arXiv preprint arXiv:2402.15301, 2024.

[5] Amrita Bhattacharjee et al. Towards LLM-guided causal explainability for black-box text classifiers. arXiv preprint arXiv:2309.13340, 2023.

[6] Amrita Bhattacharjee et al. Zero-shot LLM-guided counterfactual generation for text. arXiv e-prints, pages arXiv–2405, 2024.

[7] Amir Feder et al. Data augmentations for improved (large) language model generalization. Advances in Neural Information Processing Systems, 36:70638–70653, 2023.

• Line 157: We acknowledge the typo and will correct do(V=v) as required.

• Lines 189–191: To clarify, we use our own annotated data and the "LLM as judge" approach to refine the evaluation metrics and enhance the accuracy of our model's performance assessment. The purpose of this approach is to iteratively improve the effectiveness of LLM as an automatic evaluator, allowing us to better assess model outputs.

While detailed information about this process is provided in Appendix A.2 (lines 23–35), we want to emphasize that this is already included in the original version of our paper. The reference to the appendix is meant to provide further supporting details, but we will explicitly explain this process in the revised manuscript to ensure the information is clear and accessible without requiring the reviewer to revisit the appendix.

• Line 218: For Touché23-ValueEval, the dataset consists of statements expressing a point of view, such as "One should support homeschooling." We convert these statements into opinion-seeking questions (e.g., "Should we support homeschooling?") in order to elicit value-involving responses from the LLM using the GPT-4o-mini API.

W3: Concern about the reliability of experiments

There is a misunderstanding of our task definition. Our focus is on correction and improving accuracy, rather than simply optimizing a single automated metric. Moreover, in addition to automatic evaluations, we also conducted a survey-based evaluation using PVQ (Appendix C2.1, lines 243–253, Tab. 12), so our findings are not solely dependent on the automated evaluator.

Furthermore, we provide additional human evaluation results in the main text (Section 5.3, lines 299–306, Fig. 3), as well as in the appendix (Appendix C.3, lines 254–261, Appendix Fig. 2). Therefore, the concern about reporting results with special designation is unwarranted; our evaluation approach is transparent and multi-faceted.

I'd like to thank the authors for their clarification and additional experiments! My concerns about backbones and experimental results are addressed. I'll raise my score to 5, and I hope the clarification of writing problems can be added to the paper. Thanks!