We sincerely thank the reviewer for their detailed feedback and valuable suggestions. Below, we address each concern and question raised. We believe the revisions and clarifications provided significantly strengthen the paper.

Response to Weaknesses

1. Circuit Size We agree with the reviewer that the circuit size plays a crucial role in the model's performance. To address this, we have conducted additional experiments with larger quantum circuits by using Dense Encoding: and IQP Encoding. The details are in Section 4.1, Appendix C and G. We acknowledge that the circuits in Figure 5 primarily introduce local entanglement. We are exploring better entangling circuits as a future direction, as enhanced feature maps and circuit designs could further improve performance.

2. Motivations We appreciate the reviewer’s comment on insufficient discussion of the motivations behind replacing the dot product with entanglement. To address this, we have elaborated on our rationale in the revised introduction:

3. Efficiency The concern regarding the computational cost of entanglement measurements is valid. We have addressed this in the updated manuscript:

   • Classical Shadows: Entanglement entropy can be efficiently approximated using classical shadows.
   • Parallelization: Calculations for query-key pairs can be parallelized, mitigating computational burden.
   • Novelty: This work introduces entanglement entropy to classical ML, laying groundwork for future optimization. A complexity analysis for different entanglement measures is included in Section 4.3.

4. Missing Details We have made the following updates:

   • Query and Key States: Clarified that these are row vectors of query and key matrices.
   • Encoding Techniques: Descriptions of encoding methods (super dense, dense, and IQP) are now in Section 4.1, with corresponding circuit diagrams in Appendix C.
   • Complexities: A comparison of computational complexities is included in Section 4.3.

5. Model Performance We agree that larger quantum circuits and better hyperparameter tuning could improve performance. To address this:

   • Larger Circuits: Expanding to 12 qubits (dense encoding) improved performance, as reported in Table 3 of the revised manuscript.
   • Classical vs. Quantum Models: While classical models perform better with larger datasets, our focus remains on small-data quantum-inspired mechanisms. We observe that larger systems and better encoding lead to overfitting in quantum attention, a limitation we highlight as a future direction.

6. Citation Errors We apologize for the citation errors and have corrected them in the revised manuscript.

Response to Questions

• Concrete Relations Between Entanglement and Model Performance:
  Our approach allows the model to learn optimal entanglement via parameterized CRX gates. However, we do not assert a direct relationship between stronger entanglement and better performance. The model’s adaptability to task-specific correlations determines its effectiveness.

• Performance with Larger Circuits:
  Preliminary results using larger systems (12 qubits, dense encoding) suggest improved performance, which we report in Table 3 and Appendix G of the revised paper. We aim to investigate scalability with larger quantum systems in future work.

• Comparison with Existing Works:
  We appreciate the reviewer highlighting related works:

  • Ren-Xin Zhao et al.: Their work introduces quantum kernel-based attention but is limited to MNIST/Fashion-MNIST with a small dataset. Our method shows scalability limitations when implemented on full datasets (all 10 classes), with accuracy dropping to ~10%.
  • Cherrat and Kerenidis (Quantum 2024): They focus on orthogonal layers in quantum transformers and classical preprocessing, which contrasts with our integration of quantum entanglement in the attention mechanism directly.
  • Khatri (arXiv:2406.04305): Their fully quantum transformer represents a significant advancement but is fundamentally different from our hybrid quantum-classical approach focused on classical datasets.

Resource Analysis

The revised manuscript now includes:

• Time Complexity: A note on the complexity of various entanglement measures.
• Qubit Requirements: Discussed for each encoding technique.

Number of Trainable Parameters

We confirm that our model has 469 trainable parameters. A breakdown is included in the updated manuscript.

Closing Remarks

We are grateful for the reviewer’s thoughtful feedback, which has led to meaningful improvements in the paper. We hope these clarifications address all concerns. If there are further questions, we would be happy to elaborate.
We kindly request the reviewer to reconsider their evaluation based on the revisions provided. Thank you for your time and effort in reviewing our work.

General Response

We sincerely thank the reviewer for their detailed feedback and constructive suggestions, which have significantly helped us improve the paper. Below, we address each point raised, along with changes made to the paper or clarifications to strengthen our contributions.

Response to Weakness 1: Noise in Quantum Systems

We appreciate the reviewer's observation about noise in quantum systems. We initially noted this as future work but have since conducted noisy simulations on the RP dataset, where quantum attention showed the most significant performance gain. Specifically, we applied 1-qubit and 2-qubit depolarization noise models, along with thermal relaxation noise, for 12 qubits under dense encoding, with 2 and 4 attention layers. Remarkably, the noisy quantum attention still outperformed the classical attention mechanism. These results have been added in Appendix F of the revised manuscript, along with a discussion of noise models.

Response to Weakness 2: Lack of Theoretical Analysis

We acknowledge the need for a deeper theoretical analysis of quantum entanglement in the attention mechanism. Our work was empirical, showing that entanglement entropy can effectively model correlations in quantum-inspired attention. We agree that a rigorous theoretical understanding of its benefits for small-data generalization would be valuable but note that this is beyond the scope of our current study. We have added this point to the discussion as a future direction.

Response to Weakness 3: Parameter Comparisons

We appreciate the suggestion to clarify the parameter and computational trade-offs. The primary difference between our quantum and classical models lies in the attention layer, where the dot product similarity is replaced by entanglement entropy via a Parameterized Quantum Circuit (PQC). The PQC introduces additional parameters—specifically half the number of qubits used. We’ve added this response in the revised manuscript.

Response to Weakness 4: Comparison with MLPs

Thank you for recommending additional baselines such as MLPs. We tested an MLP with 3 hidden layers and 1 output layer on the RP dataset, where each hidden layer used weight matrices of size (12×4). This resulted in ~7,000 trainable parameters, significantly more than the attention-based models. The MLP still did not generalize well on the RP dataset and showed lower test accuracy compared to both classical and quantum attention mechanisms. These results are included in Appendix E for a more comprehensive comparison of the quantum model’s performance.

Response to Weakness 5: Missing References

We have added the references suggested by the reviewer and compared them to our approach:

• Shi et al. (2023): Their quantum state mappings for dot product similarity are evaluated on simpler datasets. Our method outperforms theirs in test accuracy.
• Shi et al. (2022): They propose a Quantum Self-Attention Network but focus on binary classification with MNIST. Our approach generalizes to more complex tasks and uses entanglement entropy.
• Di Sipio et al. (2022): They develop a quantum transformer using PQCs but lack empirical evaluation. Our work fills this gap by demonstrating performance improvements empirically.

These comparisons, along with the added references, strengthen our contribution’s contextualization.

Questions Addressed

• Noise Performance: Noisy simulations showing model robustness have been added (Weakness 1).
• Theoretical Foundation: Acknowledged as a future direction (Weakness 2).
• MLP Comparison: Conducted and included in the Appendix (Weakness 4).
• Relevant References: Added and discussed (Weakness 5).

Closing Remarks

We are grateful for the reviewer’s constructive feedback, which has greatly improved the paper. We hope the additional experiments, comparisons, and references address the concerns raised. If there are any remaining questions or suggestions, we would be delighted to incorporate them.

We kindly ask the reviewer to consider these clarifications and updates when re-evaluating the paper and its contribution. Thank you for your time and thoughtful review.

General Response

We sincerely thank the reviewer for their detailed feedback and constructive suggestions. We appreciate the opportunity to clarify and improve our paper. Below, we address each concern raised and describe the corresponding changes made to the manuscript.

Response to Concerns

1. Dataset Size and Transformer Suitability

We respectfully disagree with the notion that Transformers are exclusively suited for large-scale datasets. While Transformers are often employed in large-scale scenarios, numerous studies use simplified architectures or small datasets to analyze specific Transformer components. Our study follows this line, focusing on the applicability of quantum entanglement attention.

• Performance on RP Dataset: Our quantum attention model consistently outperforms classical attention-based Transformers, MLPs, and other quantum models on the RP dataset.
• Scaling with Larger Systems: We observed improved performance when scaling the quantum circuit size, indicating that enhancements to the quantum architecture can further benefit the approach.
• Novel Contribution: This work is the first to leverage entanglement entropy in attention mechanisms, demonstrating its potential to enhance performance even on small datasets. While our current experiments focus on smaller data, the methodology can be extended to larger datasets in future work.

We have updated the manuscript to emphasize these points and provide additional context.

2. Visualizing and Analyzing Attention

We appreciate the reviewer’s suggestion to visualize and analyze the attention mechanism. In response:

• Attention Heatmaps: We have added visualizations of the quantum and classical attention matrices in the Appendix. These heatmaps show that the quantum attention mechanism does not treat all tokens equally, validating its functionality and effectiveness.
• CLS Token: The classification in our model relies on the learned CLS token, which effectively aggregates information through attention. This further distinguishes our quantum attention mechanism from MLP behavior.

These results are included in the revised manuscript, with attention heatmaps added to Appendix D and E.

3. Dataset Descriptions

To address the lack of clarity regarding the datasets, we have updated the manuscript to include descriptions of RP and MC datasets and the methodology involved in tokenization. These updates enhance the reproducibility and transparency of our study.

Closing Remarks

We are grateful for the reviewer’s constructive feedback, which has led to significant improvements in the manuscript. The updates address concerns about quantum system size, architecture details, and visualization. If further clarification is needed, we would be happy to provide it.

We kindly ask the reviewer to consider these revisions and their impact on the paper’s quality and clarity when re-evaluating the manuscript. Thank you for your time and effort.

General Response

We sincerely thank the reviewer for their detailed comments and valuable suggestions. We are pleased that the reviewer engaged deeply with our work and raised critical points that help clarify and improve our presentation. Below, we address each concern raised and provide the corresponding updates or explanations.

Clarity on Self-Attention and Entanglement Measures

We appreciate the feedback on the lack of mathematical clarity in Section 4. Classical attention is modeled using softmax on similarities between key and query vectors, computed via the dot product. In our approach, we replace this similarity computation with entanglement measurements derived from quantum states.

To address this concern, we have significantly expanded Section 4 in the revised manuscript. Specifically, we:

• Provided explicit mathematical definitions for each entanglement measure used to compute attention.
• Explained the embedding techniques in detail, including how the Quantum Feature Map (QFM) and Parameterized Quantum Circuit (PQC) operate on the quantum states.
• Added circuit diagrams for both the QFM and PQC to visually illustrate these operations.
• Included complexity analyses for the various entanglement measurement techniques.

These revisions aim to make the operations and the computation of attention more transparent to the reader.

Parameterization and Generation of PQCs

The reviewer noted the lack of explanation for generating the PQC in Figure 5. We clarify that the PQCs in our work are heuristically designed, comprising Controlled-RX (CRX) gates between the key and query states. The model performance and complexity depend on the choice of PQC. A CRX gates-based PQC was chosen to ensure it entangles the query and key state.

Study of Model Performance with Varying Model Sizes

We acknowledge the reviewer’s concern about studying model performance with varying model sizes. Due to computational resource limitations, we were unable to test models with more than two attention layers. However, to investigate the effect of a larger quantum system, we conducted additional experiments using a 12-qubit system. The results indicate improved performance with the increased number of qubits, suggesting that larger quantum systems could further enhance the model’s effectiveness. We have included a table in the revised manuscript (Section 5) summarizing these results using dense angle encoding.

Qualitative Analysis of Attention Maps

The reviewer’s suggestion to include qualitative analysis of attention maps is highly appreciated. To address this, we analyzed the attention heatmaps produced by our model and observed distinct patterns across different classes. Specifically, we plotted the average attention per class to demonstrate how quantum and classical attention mechanisms differ in their behavior. These visualizations and corresponding discussions have been added to Appendix D of the revised manuscript to provide a better understanding of the model’s attention dynamics.

QSANN’s Test Accuracy and CLS Token Usage

We appreciate the reviewer’s insightful questions regarding Table 1. The observed 100% test accuracy on the MC dataset is due to the simplicity of the dataset. When using the QSANN model without the CLS token, the model aggregates information across all tokens for classification. This approach diminishes the role of the attention layer, resulting in behavior more akin to nonlinear layers. By appending a CLS token and restricting classification to this token, as is standard practice in Transformer models, the performance of QSANN dropped significantly to 56%. This suggests that the QSANN’s performance is not primarily attributable to its attention mechanism, underscoring the value of our proposed approach.

Comparison with QSANN and Other Models

We focused on comparing our method with QSANN due to the lack of accessible implementations for many existing works. While other models, such as QKSAN, were implemented, they underperformed (e.g., performing no better than random guessing on tasks beyond binary classification). However, following the suggestion of Reviewer 3, we compared our model with Shi et al. (2023), which also utilizes the RP dataset, and found that our approach outperforms their reported results. We have included this comparison in the revised manuscript (Section 5.1).

Closing Remarks

We thank the reviewer for their constructive feedback, which has allowed us to significantly enhance the clarity, depth, and presentation of our work. We hope these updates adequately address the concerns raised. We respectfully ask the reviewer to reconsider their score in light of the revisions made and the additional results provided. Please let us know if there are any further questions or areas for improvement.