Understanding Inter-Session Intentions via Complex Logical Reasoning

Thank you for providing such constructive feedback. We have put in significant effort to revise the paper based on your comments and suggestions, and all the rewritten parts are highlighted in blue.

Re W1:

Regarding the task formulation, we have taken your advice and extended the problem definition of CQA to include sessions as ordered hypergraphs of varying arity, namely LS-CQA. We also want to emphasize that the order of items in the session is crucial for recommending relevant items, and top-performing models such as SQE, LSGT, and session encoders all take this into account.

Furthermore, we want to highlight the novelty of our problem formulation, as we are the first to bridge the CQA problem with session-based recommendations. CQA methods and knowledge graph reasoning are essential for session understanding, and our approach is unique in that it addresses user anonymity, which is a common feature of session-based recommendation tasks. By incorporating logical graph reasoning methods, we believe our approach is not only novel but also useful for improving session-based recommendation systems.

Re W2 Encoder + Experiments:

We have taken your advice and demonstrated that transformer-based methods have comparable performance despite using different linearization strategies. However, we want to highlight that the linearization strategy employed in LSGT outperforms others in queries with negations.

Furthermore, we have conducted additional experiments to investigate the compositional generalization of different linearization strategies. Our results show that the linearization strategy used in LSGT exhibits better compositional generalization than other existing strategies. Below are the detailed performance results:

Re W3 Theory:

We have read and analyzed the reference papers [1,2], and we developed a new expressive claim based on the relational expressiveness of LSGT. We proved that the LSGT is at least as powerful as 1-RWL, namely the expressiveness of R-GCN and CompGCN. We include the proof in the appendix and mark it in blue.

[1] Barcelo et al. Weisfeiler and Leman Go Relational. LOG 2022. [2] Huang et al. A theory of link prediction via relational Weisfeiler-Leman. NeurIPS 2023.

We really appreciate your constructive feedback. If you have any other concerns please do not hesitate to tell us.

Re W1

As part of our technical contribution, LSGT proposes a new strategy of using tokens to represent sessions and logical structures. This approach demonstrates its novelty and advantage by achieving an average improvement of 2.32 on complex structures including negations, such as 2ina, 2ins, 3in, inp, and pin, over three datasets compared to a model that also uses a Transformer and backbone (SQE).

Furthermore, we conducted additional experiments on four previously unseen query types (3iA, 3ip, 3inA, and 3inp) and showed that our method achieves an average improvement of 2.44 in MRR over three datasets compared to a previous method that also uses a Transformer encoder (SQE+Transformer). The details of this experiment are provided in Re W2.

Re W2

Following your suggestions, we included two ablation studies on the hypergraph and logical structures. In the first study, we removed the tokens representing the logical structures. In the second study, we removed the order information in the hypergraph by removing the positional encoding features of item tokens in each session.

Here are the results:

Removing the logical structure information drastically dropped the model's performance, especially for queries involving negations and multi-hop reasoning, such as ip, pi, inp, and pin. Without the logical structure, the model could only use co-occurrence information like "bag of sessions" and "bag of items" to rank candidate answers. While this information may be useful for simple structured queries, it is not very useful for complex structured queries.

Similarly, removing the order information within each session also drastically dropped the overall performance. This demonstrates two things: First, the item orders in each session are critical in this task. Second, the LSGT model is effectively able to utilize the order information for this task.

In addition, we included a new evaluation on the compositional generalization of LSGT. The newly included query types are 3iA, 3ip, 3inA, and 3inp. We selected them because they are all complex types involving three anchors, cover both EPFO queries and queries involving negations, and have both 1-hop and 2-hop relational projections in the reasoning process. These query types were not trained during the training process but were evaluated in a zero-shot manner.

Here are their performances against the baselines:

Thank you for your review, we would like to address your concerns individually.

Re W1

Thank you so much, we have fixed the typos you mentioned and marked them in blue in the revised paper. We have also conducted a thorough check on the grammatical issues and typos and also fixed them at the same time.

Re W2

We have made some updates to our work. Specifically, we added the notation of "u" and "n" as in previous work to distinguish between “2iS” and “2uS”. Additionally, we included query structures for the four out-of-distribution query types in Figure 5, which addresses your concerns on compositional generalizability. You can find more details in Re W4.

Re W3

We've made some updates to our paper. Specifically, we added a detailed description for Figure 3 to better explain our problem setting and relevant query encoding methods. We marked the newly added explanations in blue for your convenience.

Re W4

We have followed your suggestions and added new experiment results on compositional generalization. The newly included query types are 3iA, 3ip, 3inA, and 3inp. We selected these query types because they are complex, involve three anchors, cover both EPFO queries and queries involving negations, and have both 1-hop and 2-hop relational projections in the reasoning process. These query types were not trained during the training process but were evaluated in a zero-shot manner.

We compared their performance against the baselines and found that our proposed method showed stronger compositional generalization on these unseen query types. It achieved MRR improvement ranging from 1.28 to 3.22 on three datasets.

Re Q1 We have added the evaluation of compositional generalization, as in previous work, and included the results in the original paper. Initially, we did not include compositional generalization because we believed it was not very important in the real setting of session reasoning. We thought that if we wanted to improve performance on some unseen query types, a better approach would be to sample more data with that query type and conduct supervised training on it.

However, we now realize the importance of compositional generalization in the general problem of complex query answering. Therefore, we further included the experiment in Re W4 to demonstrate the effectiveness of our method on compositional generalization.

Re Q2

We selected and derived fourteen query types from the original fourteen types of queries in [1,2]. The 2iS and 2iA are both derived from 2i queries. They involve 1 session 1 item and 2 sessions as their anchors, respectively.

Regarding pni, we found it difficult to interpret its practical meaning in a real problem setting. It can be interpreted as "Predict the attribute value of a given product that is not desired by a given session." However, we believe that predicting the attribute value for a single item is not meaningful in session understanding. Therefore, we did not include this type in our training and evaluation so that we could focus on other important query types.