Emergence of Hierarchical Emotion Representations in Large Language Models

Thank you for recognizing the originality, quality, and impact of our paper. We also appreciate your suggestions on how to improve this paper, which we address below.

Regarding Weaknesses

1. Limited data for emotion tree construction: The study utilized 5,000 prompts to test 135 emotion types, resulting in an average of only 37.03 prompts per emotion type. This relatively small sample size per emotion suggests the need for expanded data collection to construct a more detailed and robust emotion tree.

We agree that expanding the dataset could enhance the robustness and granularity of the constructed emotion tree. While our current dataset provides a proof-of-concept, we acknowledge that a larger sample size per emotion type would help refine the hierarchical structure and capture nuances more effectively. As part of future work, we plan to scale up the number of prompts to provide greater coverage for each emotion type. Additionally, we hope to explore methods for leveraging external datasets or employing data augmentation techniques to further strengthen the emotion tree construction.

2. Dataset limitations: The study exclusively relies on GPT-4 generated datasets. It would benefit from incorporating data from real-world scenarios (such as EDOS[1], EmpatheticDialogues[2], and GoEmotions[3]) for experimental validation.

Thank you for your insightful suggestion. We conducted an additional experiment using the GoEmotions dataset for you. The results have been added to Figure 10 and are discussed in the paragraph titled "Expanding to realistic datasets" in Section 4.1. This experiment further supports our findings by highlighting biases in emotion recognition observed in both humans and LLMs, thereby strengthening our conclusions. We greatly appreciate your feedback!

3. Format error: (3.1) Citation formats require standardization (inconsistencies noted in lines 34, 48, 249, and 250); (3.2) Possible typo: "eutral" appears on line 362 (should this be "neutral"?)

Thank you for pointing these out. We have fixed the citation formats and typos in the updated draft.

Response to questions

1. Methodology Consideration: Did the research employ various prompt types when constructing the emotion tree? Given that LLMs are typically sensitive to prompting, different prompt structures might elicit varying responses. This could potentially affect the emotional relationships identified in the study - for example, the strong connection observed between fear and shock in Llama3.1_8b might be weakened or altered with different prompt formulations. Therefore, I suggest conducting an ablation study on prompt sensitivity to quantify how different prompts affect the emotional hierarchy.

Due to resource constraints, we are not able to rerun the hierarchy experiments before the end of the rebuttal period. However, this is a great suggestion, and we will add an ablation study to the final version of the paper.

2. Data Representation Query: Considering that all the data used in the study was generated by GPT-4o, to what extent might this deviate from authentic human emotional expressions and patterns? I recommend that the authors compare GPT-4o generated data with existing human-annotated emotion datasets to quantify any differences. Additionally, human experts could evaluate the differences between GPT-4o generated data and real-world data.

Thank you for your thoughtful suggestion! In response to your comment, we have:

• Conducted an experiment using the human-annotated emotion dataset, GoEmotions [3], to quantify differences between human data and LLM-generated data (Figure 10).
• Performed a user study with 60 participants to evaluate differences in emotion recognition between humans and LLMs (Figure 9, 20 and 22).
• Added new experiments with a stronger focus on psychological perspectives, connecting our findings to psychology literature (Figures 11 and 12).

We've added a new Section 4.1 to provide a detailed discussion of these results. Interestingly, LLM shows strong overall emotion recognition abilities but can also display human-like behavior when adopting specific personas, as highlighted in our discussion. Additionally, we've clarified how our experiments are grounded in psychology research by improving the introduction (Section 1) and refining the opening of Section 4. We appreciate your thoughtful feedback and are happy to address any additional suggestions you may have!

Thank you for your response; you have addressed my concerns.

Overall, I believe this paper has value and can help future work clarify the emotional mechanisms within LLMs. The research content is quite comprehensive and convincing.

However, I think the paper has some minor flaws. The most critical part is the construction of the hierarchical emotion tree. This section's construction strategy relies solely on statistical metrics and lacks significant innovation, despite the interesting motivation.

In summary, I believe the accurate rating should be 6.5. However, the system does not offer this option, so I have not adjusted the score.

2. Character Background Analysis

Chapter 4 provides quantitative analysis from multiple perspectives, but could you offer specific examples of how different character background settings lead to different model emotion predictions? This would help provide more substantial insights.

Thank you for your valuable suggestion. We have significantly revised Figure 7 and introduced new figures, Figure 8, 9, 20, 21, and 22, together with new experimental results for you. These chord diagrams highlight key emotion prediction patterns unique to each character background.

Figure 8 and 21 show additional results for intersectional underrepresented groups, highlighting unique emotion prediction patterns across various detailed character background settings. Figure 9, 20 and 22 compare emotion recognition patterns between LLMs and human responses from our experiments, visualizing how these patterns vary across different character background settings both for LLMs and human.

Additionally, in response to your feedback, we have developed a JavaScript-based interactive project website, included as supplementary material (see the general response for access to the demo).

3. Advancement of Previous Work

Could you clarify what new insights your experiments provide to advance previous work in perceiving, predicting, and potentially influencing human emotions? Some aspects have been discussed individually in previous studies.

Thank you for raising this important point. As mentioned in our response to Weakness 3: Paper Structure and Contributions, we have substantially revised the Introduction (Section 1) and Related Work (Section 2) to better highlight the context and novelty of our study. Additionally, we have incorporated new experimental results in Sections 4 and 5 to further strengthen and showcase the contributions of our work.

3. Paper Structure and Contributions

The contributions of the paper are somewhat scattered, covering three different aspects, but the discussions on these points are inadequate.

Thank you for this important observation about the paper's structure. We have made substantial revisions together with new experimental results (Section 4 and 5) to create a more cohesive narrative that follows a clear logical progression through our three main contributions.

Our investigation began with a fundamental question: Do LLMs have a human-like understanding of emotions? In Section 3, using a novel tree-construction algorithm inspired by established psychological concepts, we demonstrated that LLMs do exhibit human-like representation of emotional hierarchies. This finding naturally led to two follow-up questions: Can LLMs accurately perceive human emotions adapting various personas (Section 4), and how might this capability affect their ability to influence emotions through dialogues (Section 5)?

Following your feedback, we've now strengthened the connections between these sections. The detailed persona analysis newly introduced in Section 4's emotion classification study now extends into Section 5. To reflect your feedback, we now examine

• How LLMs' intrinsic understanding of emotions, captured through the geometric measures of the emotion trees, affects emotion recognition accuracy. Our new analysis, presented in Figures 23 and 24, demonstrates that the structure of the emotion tree significantly influences recognition performance, highlighting the relationship between hierarchical understanding and accuracy. This analysis tightly links Sections 3 and 4.
• How biases in emotion recognition affect persuasion outcomes in sales scenarios. Specifically, we conducted new experiments using personas with 2x2x2 combinations of education level (high/low), race (Black/White), and gender (man/woman). Our findings in Figure 25 reveal that combinations of underrepresented attributes correlate with both lower emotion predictions and decreased ability to manipulate, consistent with the biases observed in Section 4, thus tightly weaving those sections (Section 4 and 5).

While I understand that involving human subjects may incur additional costs, drawing conclusions solely from LLM dialogues in isolated scenarios lacks persuasiveness.

Thank you for your thoughtful feedback. We understand the need for more comprehensive coverage on influencing human emotions. To address this, we'll expand emotion dynamics experiments (Section 5) by including dialogue tasks with human participants to validate our conclusions.

Response to Questions

1. Underlying Principles and Conclusions

What is the underlying principle behind Chapter 3? Your algorithm extracts more nuanced and hierarchical emotional information, but can you elaborate on what further conclusions can be drawn from this? If I understand correctly, does the model's ability to use more emotion-related vocabulary lead to greater hierarchical richness?

The principle and goal of Chapter 3 is to extract the "structure of emotion representation" in LLMs, going beyond traditional benchmarking metrics like classification accuracy. Our hierarchical analysis provides several key insights:

First, the extracted tree structure reveals two important dimensions:

• The breadth of emotional understanding (represented by the number of nodes)
• The depth of emotional comprehension (shown through hierarchical relationships)

The number of nodes correlates with the LLM's vocabulary size of emotions, while tree depth indicates how sophisticated the model is in grouping related emotions. Thanks to your comments, we included this discussion in Section 3.2.

This is clearly demonstrated in Figure 3, where larger LLM models show increased complexity in both dimensions. For instance, the Llama-405B model demonstrates particularly nuanced understanding through its five-layer hierarchical groupings. It correctly places embarrassment and humiliation as subcategories of shame, while clustering thrill and enthusiasm under excitement – arrangements that align well with human psychological understanding.

Our methodology also offers novel insights into emotional understanding across different demographic groups and psychological conditions. As shown in Figure 12, by adapting personas (such as individuals with ASD), we can analyze how emotional representation structures vary across different populations. While traditional psychology has explored these relationships through careful methods involving human studies, our approach offers a unique, data-driven technique for rapidly extracting these structures. This represents a novel approach to research methodology in machine psychology.

Thank you for your detailed comments. We are delighted that you found our hierarchical emotion extraction method effective and our paper clear. We are happy to address your comments below.

Regarding Weaknesses

1. Model Limitations and Emotion Perception

After reading the introduction, I expected Chapter 3 to discuss the model's ability and limitations in perceiving emotions, especially focusing on the circumstances under which the model fails. However, the paper mainly discusses how larger models outperform smaller ones in understanding emotions, which is rather obvious and does not provide sufficient novel insight.

Inspired by your comment, "discuss the model's ability and limitations in perceiving emotions, especially focusing on the circumstances under which the model fails.", we now have included a new set of "human experiments" to see where and how the models' prediction differs in emotion state prediction. We find that LLM struggles to recognize the emotion of "surprise," but demonstrates a greater overall ability in emotion perception compared to humans (Figure 9a). Interestingly, LLMs can also "fail" like humans when adopting certain personas (Figure 9b and 9c).

However, the paper mainly discusses how larger models outperform smaller ones in understanding emotions, which is rather obvious and does not provide sufficient novel insight.

Thank you for your thoughtful feedback. One of our key contributions is introducing a new problem: evaluating LLMs' intrinsic understanding of emotion recognition. To address this, we developed a novel algorithm that goes beyond standard benchmark metrics. While many existing approaches tackle similar tasks, most of them overlook the hierarchical relationships among emotions, a well-established concept in psychology. Hierarchical clustering algorithms assume hierarchical relationships between clusters, but not among emotions themselves. Our work is the first to explore LLMs' intrinsic understanding of relationships among emotions purely through their internal representations. Our tree-construction approach is deeply inspired by the concept of emotion differentiation and "emotion wheel" (Figure 1), pioneered and widely used in cognitive psychology. Our algorithm can capture the hierarchical relationships between pairs of emotions and reconstruct the overall structure of the emotional hierarchy. By applying our proposed algorithm, we uncovered a nontrivial phenomenon: LLMs' intrinsic understanding of emotions emerges as the model size increases. To clarify these points further, we have updated the related work section and Section 3.2 to ground our methodology and insights more clearly.

2. Synthetic Data Testing and Validation

Chapter 4 employs synthetic data for testing but lacks sufficient quality validation. Including human and LLM prediction accuracy in a figure, such as Figure 6, would be beneficial, even if only for a subset.

Thank you for your suggestion to include human experiments! We conducted new user experiments and generated a figure analogous to Figure 6. These new results are presented in Figure 9, and the corresponding discussions are included in the paragraph titled "User Study: Comparing emotion recognition in humans and LLMs." in Section 4.1. In summary, while LLMs have superior emotion recognition ability than human overall, it has human-like bias in emotion recognition accuracy and misclassification patterns when adopting specific personas. Our user study involved 60 participants due to time constraints; however, we plan to scale this up to a larger sample size for the camera-ready version to ensure robustness and generalizability of our findings.

Dear Reviewer Lh9i,

Thank you for your insightful and constructive feedback that has helped us greatly improve our paper. As we approach the end of the extended discussion period in two days, we would like to summarize the substantial updates we've made to specifically address your key concerns:

1. Model limitations and emotion perception (Chapter 3):

   • Demonstrated specific failure cases, particularly with recognizing "surprise"
   • Added detailed analysis of human-like bias patterns with personas (new Fig. 9b,c)
   • Enhanced validation through integration with the human labeled GoEmotions dataset (new Fig. 10)

2. Regarding your questions about synthetic data validation:

   • Conducted new user study with 60 participants to validate emotion recognition patterns
   • Added comparative visualization between human and LLM prediction accuracy (new Fig. 9)
   • Expanded demographic analysis showing biases across intersectional groups (new Fig. 6b-g)
   • Developed interactive visualization tool for detailed result exploration

3. Paper structure and contributions:

   • Strengthened connections between sections with new analyses:
     • Demonstrated correlation between understanding of hierarchies of emotions and recognition accuracy (new Fig. 24)
     • Added detailed persona analysis showing how biases affect persuasion outcomes (new Fig. 25)
   • Added validation across different model sizes to demonstrate robustness of our findings
   • Enhanced psychological grounding with emotion wheel analysis (new Fig. 1)

Given these substantial updates and new experimental results, we would greatly appreciate your feedback on whether these changes adequately address your concerns. If you have any remaining questions or need clarification on any point, we are ready to respond promptly.

Thank you for your thoughtful feedback. We’re glad you appreciated our method for studying hierarchical emotion representations in LLMs, measuring bias and persuasion abilities, and our practical insights for applying AI in emotionally sensitive environments. We address your suggestions below.

Regarding Weaknesses

1. The first two conclusions are quite obvious and lack in-depth exploration of their underlying causes. For example, what is the relationship between the breadth and depth of model emotional stratification and model parameters and pre-training corpora?

We share the reviewer’s interest in how particular factors such as model size and pre-training data influence emotion representations. Indeed, one of our goals in Section 3 was to explore how different size LLMs represent emotions. As noted by the reviewer, our analysis reveals two critical dimensions of hierarchical emotion representations that vary across model size: breadth (related to node count) and depth (measured through hierarchical relationships). More specifically we find that while both dimensions generally increase with model size, emotional breadth appears to grow with parameter count, while depth shows diminishing returns beyond certain model sizes.

This relationship is demonstrated in Figure 3, where we compare models with 8B, 70B, and 405B parameters. While the 8B model shows only basic emotion groupings, the 70B model demonstrates increased sophistication with four layers. The 405B model shows the most nuanced understanding with a five-layer structure, correctly placing embarrassment and humiliation as subcategories of shame, while grouping thrill and enthusiasm under excitement—arrangements that align well with human psychological understanding.

While we did not explore the impact of pre-training data on emotion representations in this work, we see this as a promising avenue for future work. In particular, it may be that better understanding which data leads to which emotion representations may help us better fine-tune models to improve the complexity of their emotional understanding, or to mitigate harmful biases.

The discussion on ethics and biases is somewhat coarse in terms of categorization by region, ethnicity, cultural background, and other living conditions.

Thank you for your thoughtful suggestion. Following your comments as well as similar points raised by other reviewers, we have added 20+ new experiments to our paper, including:

• Detailed demographics: Groups categorized by income (high/low), race (Black/White), and gender (man/woman), creating 2x2x2 combinations.
• Broader test roles: Expanded categories such as race (e.g., White, Black, Hispanic, Asian), religion (e.g., Christian, Muslim, Buddhist, Hindu), and psychological conditions (e.g., ASD, depression, anxiety). Including psychological conditions allows our findings to connect with existing psychological research.
• Granular roles: A breakdown of education levels (e.g., postgraduate, college graduate, some college, high school, less than high school).

These new findings are included in Figure 6, 7, 8, 20, 21 and 22 with detailed discussions in Section 4. Additionally, we plan to analyze data by country to explore cultural dimensions and refine demographics for deeper intersectionality insights. We would be very happy to incorporate any further suggestions you may have.

3. There is a lack of discussion on how to leverage LLMs' emotional prediction capabilities to optimize downstream dialogue tasks.

Thank you for your suggestion. The emotion tree provides distance measures to quantify the depth of LLMs' understanding of the relationships between emotions. By leveraging such distance measures within the emotion tree as a reward for the model, we expect it to achieve better performance in downstream tasks such as persuasion and negotiation. We have added this discussion to Section 3.2.

Weakness 4: Figure Readability

The font in Figure 2 is too small to see easily.

We have increased the font size in Figure 2 (now Figure 3) to improve readability. Additionally, in response to your feedback, we have developed a JavaScript-based interactive project website, included as supplementary material (see the general response for access to the demo). This platform allows users to dynamically adjust scaling, enhancing both accessibility and interactivity.

Question 4

Besides emotion, I guess your method can visualize the structure of other entities. Can you extend this part more to enlarge the generalization of your method?

Thank you for your insightful suggestion! Inspired by your comment, we conducted an additional experiment in another domain: scent. We identified a reasonable hierarchical relationship between aroma words, aligning with human-annotated aroma wheel. The results are now included in Figure 19, demonstrating the generality of our method.

Thank you for the detailed comments and for appreciating the novelty and timeliness of our work. We appreciate your suggestions, which we address below.

Weakness 1: Emotion Extraction Technique

Emotion Extraction Technique Concern: The method for extracting hierarchical structures based on next-word probabilities lacks rigorous justification. There is no comparison with alternative methods or validation.

Question 1

Can you provide a more detailed justification for using next-word probabilities to extract hierarchical emotion structures?

Thank you for this question about our methodological approach. We have added a detailed explanation of our probabilistic framework in Appendix A. Our prompting format ("[Emotion scenario.] As a [person from a specific demographic group], I think the emotion involved in this situation is") allows us to interpret next-word probabilities as approximations of the model's estimated likelihood for each emotion given the scenario. Building on this interpretation, Appendix A provides a formal probabilistic foundation for our hierarchical structures. The elements in our matching matrix represent the joint probabilities of emotion pairs co-occurring across scenarios. Furthermore, our criteria for including edges in the hierarchy are based on conditional probabilities between emotions—specifically, how strongly the presence of one emotion predicts another. This probabilistic framework provides theoretical grounding for our extraction of hierarchical emotion structures from language model predictions.

Weakness 2: Threshold Selection

Threshold Selection: The paper sets a threshold (0 < t < 1) for determining parent-child relationships but does not explain how this threshold is chosen or its impact on the results.

Question 2

How did you determine the appropriate threshold value (0 < t < 1) for establishing parent-child relationships between emotions? Was this threshold empirically validated?

Thank you for your valuable feedback. We sweeped threshold and selected the threshold to ensure the hierarchy trees have a reasonable number of edges and nodes, with the largest tree's number of nodes approximately equal to the number of emotions considered (135). We have now included plots illustrating the threshold sweep (see Figure 15 in Appendix D). We can see that the increasing trend with model size remains robust across different threshold selections.

Weakness 3: Quantitative Metrics

Quantitative Metrics: Although the visual representations of emotion hierarchies are compelling, incorporating additional quantitative metrics or comparisons with human-annotated emotion hierarchies could provide stronger validation of the proposed method.

Question 3

Besides visual representations, can you use some quantitative metrics to validate the integrity and accuracy of the extracted hierarchical emotion structures?

Thank you for your insightful feedback. Based on your suggestions, we included a quantitative comparison between the extracted hierarchical emotion structures (Figure 3) and the human-annotated emotion hierarchies (i.e., the emotion wheel shown in Figure 1) in Figure 16. We observed significant correlations (correlation > 0.5, p < 0.001) between the node distances in the hierarchical tree and their corresponding distances on the emotion wheel across the LLMs, supporting the accuracy of the LLM-derived emotion structures.

We also want to emphasize that traditional emotion wheels have been largely intuition-driven, as their empirical construction through human experiments faces significant challenges. These include the need for a large number of participants, the overwhelming difficulty for humans to evaluate relationships among all pairs of 135 emotion classes, and the influence of subjective biases. By leveraging the vast corpora accessible to LLMs, we demonstrate that emotion wheels can now be constructed in a fully data-driven manner. The reconstructed hierarchical trees provide a more granular and nuanced representation of emotional relationships compared to the simpler two-layer structure of traditional emotion wheels. Our work, originally inspired by psychology and cognitive science, might now have the potential to offer new insights back to these fields.

Dear Reviewer qTjt,

We sincerely thank you for your thoughtful feedback that highlighted critical methodological aspects requiring stronger validation. As the discussion period ends in two days, we would like to summarize how we've addressed your key concerns:

1. Justification of emotion extraction method:

   • Added formal probabilistic framework and mathematical foundations in Appendix A
   • Added validation of our method by comparing results with human-annotated emotion hierarchies (new Fig. 16 and 17)
   • Demonstrated strong correlation (>0.5, p<0.001) between our extracted structures and established psychological frameworks
   • Conducted new user study with 60 human participants to validate emotion recognition patterns (new Fig. 9)

2. Threshold selection and validation:

   • Added comprehensive threshold sweep analysis (new Fig. 15)
   • Demonstrated robustness of our findings across different threshold values
   • Added quantitative criteria for threshold selection, targeting approximately 135 nodes to match the number of emotions considered

3. Quantitative metrics and method generalization:

   • Added correlation analysis between hierarchical tree distances and emotion wheel distances (new Fig. 16)
   • Expanded methodology to a new domain by successfully applying our tree-construction algorithm to wine aroma hierarchies (new Fig. 19)
   • Validated results through comparison with the established Davis Wine Aroma Wheel, demonstrating the generalizability of our approach

Given these substantial updates strengthening the methodological foundations of our work, we would greatly appreciate your feedback on whether these changes adequately address your concerns. If you have any remaining questions or need clarification, we are ready to respond promptly.