From Tokens to Lattices: Emergent Lattice Structures in Language Models

Thank you for your valuable comments. Below, we provide our responses.

Weakness 1: The study relies on carefully curated datasets that may not capture the complexity or variability of real-world language use. Expanding the dataset to include a broader range of contexts and ambiguities would enhance the framework’s robustness and applicability.

Answer: We agree it is important to choose datasets that capture the complexity and variability of real-world language usage. Actually, we used datasets from different domains (e.g., Disease-symptom is a medical domain dataset) and vary in terms of density, single/multi-token, etc, which reflect the real-world language usage. Our datasets are the largest datasets for formal concept analysis.

Weakness 2: While concept lattices offer a structured hierarchy, they are currently limited to modelling simple object-attribute relationships. Incorporating more nuanced or multi-relational knowledge types would broaden the framework’s applicability.

Answer: Our framework can be easily extended to multi-relational data. We just have to design suitable patterns for different relations and extract the relationships with corresponding patterns. However, constructing lattices from a multi-relational formal context is time-consuming, as the time complexity is exponential due to the combinatorial nature of the subset search across multiple dimensions.

Weakness 3: Emphasizing intuitive explanations would make the work more accessible and engaging.

We understand that the writing of our paper is more mathematically rigorous than other papers that also work on language model probing. This is because our contribution includes also theoretical analysis. But we agree some parts of it can be improved with some intuitive explanations. To echo this, we will revise the descriptions of the method to include more intuitive explanations.

Question 1: Can this framework be applied to autoregressive language models (e.g., GPT series)?

Answer: Yes. To do so, we only have to put either the attribute or object tokens at the end of the context. This is because autoregressive models can only predict the next-token probability.

Question 2: How are concept patterns selected for this framework? Are they designed empirically by humans, and to what extent could different patterns impact the results?

Answer: Yes, they are designed by humans, but in a way such that the patterns reflect how the objects and attributes are used.

Question 3: How might this framework perform on larger, more complex datasets with more ambiguous or polysemous relationships?

Answer: According to our Lemma 2, the performance depends on how well the patterns capture the real-world usage of the objects and attributes, but does not depend on the size of the datasets. If there are more ambiguous or polysemous relationships between objects and attributes or if the pattern does not capture enough information of the relationships, then the probability of their corresponding relationships will also become more uncertain, e.g., 0.5.

Thank you very much for raising your score. We will definitely include more intuitive explanations in our final version.

Thank you for your valuable comments. Below, we provide our responses.

Weakness 1: The baseline models only consider naive BERT embeddings, omitting a range of established probing techniques known for their effectiveness, such as prompt-based probing in natural language inference (NLI) formats. Incorporating these methods would provide a stronger basis for claiming that this FCA-based framework is more effective in extracting knowledge from MLMs.

Answer: We have added two more baselines: one naive baseline with a template generated by the definitional hypothesis denoted as BertLattice (definitional), and another baseline with natural language inference (NLI) formats denoted as BertLattice (NLI) . The results indeed show that the patterns are very important to the results, and the patterns should reflect how the objects and attributes are used in real-world.

The corresponding templates and results are seen in the following tables:

Weakness 2: The evaluation is limited to "single-token" named entities, which may restrict dataset coverage. While the [MASK] token only predicts one token at a time, several methods exist to approximate multi-token predictions.

Answer: We want to clarify our datasets do contain multi-token objects or attributes. We only limit that either attributes or objects are single-token. For exampl, in the animal-behavior datasets, most of behaviors are multi-token. We also agree that it is possible to approximate multi-token predictions, but this goes beyond our target of this paper and we leave it as our future work.

Question 1: Have you considered the issue of potential ill-formed entity or relation names fitting into the template, which might introduce bias into evaluation?

Answer: According to our Lemma 2, the performance depends on how well the patterns capture the relationships between the objects and attributes. So if there are ill-formed entity or relation names that are ambiguous or polysemous, it is also hard to find a good template/pattern that uniquely captures the ill-formed entity or relation names, so the uncertainty of their corresponding relationships will also be high. This is due to the statistical nature of the language model training.

Thank you for your comments. Below, we provide our responses.

Weakness 1: Definition 5 is vogue, it is more like an intuition rather than definition

Answer: In our revision, we have split the definition 5 (triadic formal context and triadic formal concept) into two separate definitions and make them more clear and formal.

Weakness 2: In L (line) 263, it is said that data generation (i.e., Abstraction 1) is an inherently noisy process. However, I do not see anything noisy in Abstraction 1, everything is nice and clean. What is meant by noisy here?

Answer: You are right. Abstraction 1 is indeed a clean and perfect process. What we meant by “noisy” is that real-world data generation is not always as perfect as Abstraction 1 and there are some other noises influencing the generation. Hence, our reconstruction (Def. 9) is only approximately possible. We paraphrased the description to avoid confusion.

Weakness 3: In L 253, Animal-behaviour dataset is presented. But where does the data come from? Is it just the authors’ beliefs and nothing more?

Answer: Animal-behaviour was constructed by humans with external knowledge (e.g., wikipedia). We construct it by both humans (for very easy cases such as dogs can fly?) and verify them by checking their Wikipedia description.

Weakness 4: In L 374, it is said “We formalize the problem as a recommendation system problem”. But this is not clear at all without further details, and so the essence of the first experiment is not clear for me.

Answer: Recommendation system is a ranking problem, i.e., given a user, ranking all possible items. Our problem is essentially also a ranking problem, i.e., given an object, rank all possible attributes or vice versa. We paraphrase this sentence to make it more clear.

Weakness 5: In the introduction and the conclusion, it is emphasised that the suggested method does not depend on predefined human concepts and ontologies. However, it is misleading: it clearly depends on humans, who have to provide patterns B.

Answer: What we meant is that our method can discover concepts that are not defined by humans and the evaluation does not depend on human ontologies. Our algorithm does depend on a human-designed pattern, but a pattern is just a part of the algorithm so it is just like a hyperparameter users give to the algorithm.

Minor issues: We fixed all minor issues in the writing. Thanks for the very detailed suggestions.

Dear reviewer t7YG,

Thank you again for your thoughtful review and constructive feedback. We wanted to kindly remind you to review our rebuttal and share any further thoughts or updates. We provided straightforward answers to your questions and have addressed your comments about the writings in our revised version.

If these have addressed your concerns, we kindly ask you to increase your score. Also, please let us know if there is any additional clarification or further information we can provide. This is very important to us as we are eager to ensure that all points have been adequately addressed to your satisfaction.

Best regards

Authors

Thank you for your valuable comments. Below, we provide our responses.

Weakness 1: There is a lack of general motivation on why it is important to uncover the lattice of the concepts embedded in an MLM and how it will be used. If the goal is XAI, the authors should highlight it.

Answer: Our goal is indeed XAI. In particular, we aim to understand the mechanisms of language models in learning conceptual knowledge. We have highlighted this point in our revision.

Weakness 2: The scalability of the framework to multi-relation is unclear and using one single relation seems quite limiting. While it is mentioned by the authors as a limitation, a discussion about how limiting the lattice to one relation can miss patterns would be good. The authors could discuss the challenges that might arise when tackling multi-relations. This might include the complexity of the lattice itself.

Answer: To handle multi-relational data, we just need to design separate patterns for different relations and apply multi-relational FCA. Everything is done then. The challenges would be to design good-quality patterns for different relations as they may have confounding factors that influence each other and the increase of the computational complexity . We add this discussion in the conclusion section.

Weakness 3: The experimental section is limited in the number and choice of the datasets.

Answer: Unfortunately, there are no existing datasets suitable for evaluating our framework. We have created three datasets with human curation. They vary in terms of domains, single/multi-tokens, density, etc.

Question 1: How can the lattice of concepts be leveraged in NLP tasks?

Answer: The lattice of concepts from FCA is a structured semantic hierarchy from unstructured text or corpora. All NLP tasks that require a structured representation of knowledge (e.g., ontologies, taxonomies) such as question answering and named entity recognition, entity typing can benefit from the lattice of concepts.

Question 2: Does using only one relation mean that FCA does not allow capturing the full extent of the conceptual understanding the MLM is capable of?

Answer: No, FCA can be generalised to multi-relational data, e.g., Pizza can be classified based on both its toppings and sauces. However, such generalization requires more computation and it is less useful in practice. We left this extension as our future work.

Question 2: How are the uncovered concepts labelled?

Answer: In FCA, concepts are unlabelled (that is why do not assume human-defined concepts). Labels, if required, can be named using all the attributes it has. For example, the set of objects that can fly and hunt can be named “swim-huntable”.

Thanks for addressing my questions. I think your work is promising and therefore, I raised my score.

Thank you for your comments. It seems that the reviewer misunderstands the significant aspects of the paper. Below, we provide our responses.

Question 1: directly generating a concept lattice or generating a formal context

Answer: We generate formal context first and then derive the concept lattice with FCA. FCA does require a formal context as input.

Question 2: Computational complexity of generating the context and then the lattice?

Answer: The time complexity of context generation is linear with respect to the number of objects or attributes (e.g., O(|M|) or O(G) depending on which one is masked to predict). The time complexity of lattice construction, in the worst case, is indeed exponential in the size of the input formal context, as there is as most 2^{min(|G|, |M|)} number of formal concepts. Although FCA suffers from computation complexity issues, this does not hurt our contribution. Our contribution is NOT to obtain SoTA results on concept lattice construction, but rather to understand and interpret the mechanism of MLMs (i.e., how do MLMs learn conceptual knowledge). Our major finding is that MLMs learn conceptual knowledge similarly to FCA, and we verify our findings by FCA. Of course, we fully agree that FCA is computationally expensive but that does not contradict our findings.

Question 3: The contributions/research questions of the paper should be explicitly mentioned in the introduction.

Answer: To echo your suggestion, we have highlighted our contributions/research questions more explicitly in our revision with colors.

Question 4: Why not using Large Language Models？why not prompt Large Language Models which contain much more information?

Answer: We agree that larger models may obtain better results. However, our goal is NOT to seek SoTA results on lattice construction but rather to understand and interpret the behaviors or mechanisms of masked language models. So at this point, we choose to use BERT because it is the most popular masked language models.

Question 5: The broader applicability of these lattices is a bit unclear, where would these lattices be used? If the objective is to classify the terms into the classes then this can be done without passing through the concept lattice generation phase.

Answer: First, the lattice of concepts from FCA is a structured semantic hierarchy from unstructured text or corpora. All NLP tasks that require a structured representation of knowledge (e.g., ontologies, taxonomies) such as question answering and named entity recognition, entity typing can benefit from the lattice of concepts.

Second, our objective is NOT to obtain a good lattice, but rather to understand, interpret, and verify the behaviors or mechanisms of masked language models. In particular, we investigate how MLMs learn conceptual knowledge. FCA is just a mathematical tool used to verify our findings.

Dear reviewer RPj7,

The deadline of the author-rebuttal session is approaching. We sincerely appreciate if you can take a look at our response and reconsider your score. We believe our response has effectively addressed your concerns. If you have any other concerns, please do not hesitate to contact us.