Semi-supervised Knowledge Transfer Across Multi-omic Single-cell Data

We sincerely appreciate the time you've taken to review our paper and for your insightful comments. Your positive feedback is highly encouraging for us! We'd like to address your concerns in the following response.

Q1. In the Introduction, the authors claim that "a small fraction of scRNA-seq data with cell types annotated" aligns more closely with practical scenarios. Do you have any evidence to support the claim? You may provide an example to show that scRNA-seq data are not processed and annotated as a whole, but only a small portion is annotated.

A1. Thanks for your comment. The label scarcity in scRNA-seq data is practical as supported in the previous works [1,2,3]. They have developed semi-supervised approaches to address the label scarcity in the scRNA-seq data. We will include the citations in our revised version.

Q2. The uniform distribution assumption in the optimal transport process could be wrong, as most scRNA-seq data are unbalanced across different cell types.

A2. Thanks for your comment. In real-world scenarios, we often have no prior knowledge about the distribution of specific cell types and adopting the uniform distribution assumption is a popular choice without prior information. In future work, we aim to extend our methods to scenarios with prior distribution information for imbalanced scenarios. We will include the discussion of potential future works in our revised version.

Q3. In Eq. 8, the negative cell types are removed with a significant difference from the cell type with the highest probability. However, does a threshold of 1e-3 represent significant differences?

A3. Thanks for your question. We have included a sensitivity analysis of the threshold $\mu$ by varying it in {0.0005,0.001,0.0015,0.002}. The following results indicate that a threshold of 1e-3 brings good performance with significant differences. In particular, after the softmax operation, the sum of the confidences for all cell types equals 1 with more than 10 cell types. In this context, the confidence scores are quite small and a difference of 1e-3 indicates a sufficient difference.

Q4. What does $S_i$ in Eq. 10 mean?

A4. Thanks for your question. $S_i$ is a typo and it should be $Y_i$ in Eq. 9. We will correct it in the revised version.

Q5. What is the motivation of mixing scRNA-seq data to form scATAC-seq data? Is such an operation biologically reasonable?

A5. Thanks for your question. We do not generate scATAC-seq data from scRNA-seq data. Instead, we generate virtual scRNA-seq data using the Mixup technique and then reduce the semantic gap between scATAC-seq data and this virtual scRNA-seq data in the embedding space. From a biological perspective, scATAC-seq data is characterized by extreme sparsity. This results in input matrices that are significantly heterogeneous compared to scRNA-seq data. To address the challenges in data integration, our motivation is to reduce the distance between scATAC-seq data and the mixed (virtual) scRNA-seq data in the embedding space.

Q6. Typo: In line 253 cBridge -> scBridge.

A6. Thanks for pointing out the typo. We will correct it in the revised version.

Q7. Currently the author split the MouseAtlas dataset into several subsets with different combinations for evaluation. I recommend the author provide the experimental results on the full dataset.

A7. Thanks for your suggestion. We have included the experimental results on the full dataset. The results below indicate that our approach still outperforms other methods. We will include this in our revised version.

Reference

[1] Dong et al., Semi-Supervised Deep Learning for Cell Type Identification From Single-Cell Transcriptomic Data, IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022
[2] Kimmel et al., scNym: Semi-supervised adversarial neural networks for single cell classification, bioRxiv, 2020
[3] We et al., CALLR: a semi-supervised cell-type annotation method for single-cell RNA sequencing data, Bioinformatics, 2021

Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.

We are truly grateful for the time you have taken to review our paper and your insightful review. Here we address your concerns in the following.

Q1. I'm not entirely convinced about the prevalence of the problem setting described by the author, where only a small subset of the scRNA-seq data is annotated.

A1. Thanks for your comment. The label scarcity in scRNA-seq data is practical as supported in the previous works [1,2,3]. They have developed semi-supervised approaches to address the label scarcity in the scRNA-seq data. We will include the citations in our revised version.

Q2. It may occur in scenarios such as large experiments conducted in batches, where only certain batches are annotated. It raises questions about the experimental setup. Are annotations removed randomly or based on specific conditions or batches? I wonder how does the availability of annotated subsets (randomly or by batch) affect the overall study results.

A2. Thanks for your question. The experiments in the manuscript are randomly removed. We have added an experiment to compare the performance of removing annotations by batches. Specifically, we collect different batches of MouseAtlas dataset and remove some of the batches. From the following results, we can observe that our approach still outperforms other methods, which verifies the robustness of our approach in different scenarios. We will include this in our revised version.

Q3. I wonder how the methodology comparision is conducted. As the authors argued in the paper, the SOTA methods in the experimental studies does not deal with scenarios where the scRNA-seq is only partially annotated. Therefore, in the comparison experiment, such as in the "low label ratio" setup, is it that only the 1% of the scRNA-seq is used for cross-modality matching? There are popular existing methods such as Seurat and Harmony that provided a potential two-step approach: first learn the full annotation on scRNA-seq, and then transfer to scATAC-seq, which might be a more fair comparison.

A3. Thanks for your suggestion. To solve your concern, we have included Seurat [4] and Harmony [5] for performance comparison. For these two methods, we first learn learn the full annotation on scRNA-seq data, and then transfer the knowledge to scATAC-seq data. From the following results on CITE-ASAP and snRNA_10X_v3_A-ATAC, we can find that our method surpasses other methods consistently. We will include this in our revised version.

Reference

[1] Dong et al., Semi-Supervised Deep Learning for Cell Type Identification From Single-Cell Transcriptomic Data, IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022

[2] Kimmel et al., scNym: Semi-supervised adversarial neural networks for single cell classification, bioRxiv, 2020

[3] We et al., CALLR: a semi-supervised cell-type annotation method for single-cell RNA sequencing data, Bioinformatics, 2021

[4] Stuart et al., Comprehensive integration of single-cell data, Cell, 2019

[5] Korsunsky et al., Fast, sensitive and accurate integration of single-cell data with Harmony, Nature Methods, 2019

In light of these responses, we hope we have addressed your concerns, and hope you will consider raising your score. If there are any additional notable points of concern that we have not yet addressed, please do not hesitate to share them, and we will promptly attend to those points.

We greatly appreciate your time in reviewing our paper and your insightful comments. Your positive feedback is incredibly encouraging for us! We'd like to address your concerns in the following response.

Q1. The compared methods, especially the DA method, are relatively old. Comparison with newer methods (e.g., [1], [2], etc.) helps to understand the performance of the methods.

A1. Thanks for your suggestion. We have included the recommended baselines, i.e., SLA [1] and COT [2] on different datasets for performance comparison. The results are shown below, which demonstrate the superiority of our method.

Q2. Repeated use of symbols may cause confusion and misunderstanding among readers. Eq. (1), (3), (16), and Theory 3.2 all contain $\lambda$. As far as I understand, the meanings of these symbols may be different.

A2. Thanks for pointing out this typo. The term in Eq. 1 and Eq. 3 should be $\sigma$, and in Theory 3.2 should be $\beta$. We will correct all typos in the revised version.

Q3. The method includes too many empirical hyperparameters, and the ‘crucial’ parameters selected by parameter analysis seem not comprehensive enough. A more comprehensive description of each $\lambda$ and threshold $\tau$ will help to understand the method and promote future work.

A3. Thanks for your suggestion. We have added the parameter sensitivity analysis by varying $\lambda$ (Eq. 1) and $\tau$ (Eq. 6). From the results below, we can observe that the performance is not sensitive to the choice of $\lambda$ when we change $\lambda$ from 5 to 20, so we empirically set its value to 10. We also vary the value of $\tau$ with the range of {0.8,0.85,0.9,0.95}. The results below indicate that the method is not sensitive to $\tau$ in the interval [0.8,0.95]. Therefore, we set $\tau$ to 0.9 as the default.

Q4. The quality of images seems to be low, affecting comprehension. To be specific, the method process is not clearly and comprehensively shown in Figure 1. The text in Figure 2 is too small.

A4. Thanks for your comment. We have revised the figures according to your suggestions and uploaded the pdf file in the global response section.

Q5. The proposed method seems to be a patchwork of methods in domain adaptation, cross-modal alignment, etc. Maybe its technical innovation is questionable. A more targeted and detailed description is recommended.

A5. Thanks for your comment. We would like to point out that the innovation of our approach against existing methods is threefold:

• Underexplored Pratical Problem. We focus on an underexplored yet practical problem of knowledge transfer across multi-omic single-cell data under label scarcity, while previous works focus on utilizing fully labeled scRNA-seq data.
• A Holistic Framework. To both challenges of label scarcity and cell heterogeneity, we propose a holistic framework, which consists of OT-based dataset expansion and divide-and-conquer strategy for multi-omics semantic learning, followed by cross-omic multi-sample Mixup for semantics integration. All these designs are totally new for single-cell data integration.
• Theortical Analysis. We provide a comprehensive theoretical analysis to support our designs, which makes our framework more solid.
• Superior Performance. Our method achieves superior performance on benchmark datasets compared with state-of-the-art methods. In particular, the performance increasement on snRNA_10X_v3_A-ATAC is up to 96.4% compared to the best baseline.

Reference

[1] Yu et al., Semi-Supervised Domain Adaptation with Source Label Adaptation, CVPR 2023
[2] Liu et al., COT: Unsupervised Domain Adaptation with Clustering and Optimal Transport, CVPR 2023

Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.

Thanks for your feedback! We are pleased to know that you keep the positive rating and your support. We will properly include all the rebuttal contents in the revised version, following your valuable suggestions.

We are truly grateful for the time you have taken to review our paper, and your insightful comments and support. Your positive feedback is incredibly encouraging for us! In the following response, we would like to address your concerns and provide additional clarification.

Q1. The exclusion of OT in scATAC-seq data due to cellular heterogeneity needs practical examples or quantitative analysis to substantiate its significance.

A1. Thanks for your suggestion. We have added a model variant w/ OT for scATAC-seq to support our point. The compared results on CITE-ASAP dataset are shown below. We can find that our full model outperforms w/ OT for scATAC-seq, which validates the OT strategy may not be the optimal choice given the significant cellular heterogeneity inherent in scATAC-seq data.

Q2. The initial prediction's dependency on the divide-and-conquer strategy could lead to misclassification. Discussion on handling wrongly divided samples or providing statistics is needed.

A2. Thanks for your suggestion. We have added the comparison of the pseudo-labeling accuracy between unlabeled scRNA-seq data and source-like scATAC-seq data. The results on the snRNA_SMARTer-snmC dataset are shown below. From the results, we can observe that the accuracy of source-like scATAC-seq data is close to that of unlabeled scRNA-seq data, verifying the effectiveness of our division.

We have also added a model variant w/o divide-and-conquer on snRNA_10X_v3_A-ATAC dataset for performance comparison. As evidenced by the results in the table below, the incorporation of the divide-and-conquer strategy yields positive gains.

Q3. Exploring if DANCE can be conducted in the opposite direction (scATAC-seq with OT and scRNA-seq with divide-and-conquer) would be beneficial.

A3. Thanks for your suggestion. We have included an experiment, which transfers cell type knowledge from scATAC-seq data to scRNA-seq data on the CITE-ASAP dataset. From the results below, we can validate the superiority of our method in the opposite direction.

Q4. A discussion on the complexity of OT and divide-and-conquer in terms of memory and time compared to existing studies is required.

A4. Thanks for your suggestion. We have included the comparison of memory and time as follows and we can observe that our method has a competitive computation cost. In particular, the performance of scNCL is much worse than ours (the performance increasement of ours is over 105.9%), while our memory cost only increases a little with even less training time.

Q5. Including a discussion on the use of scCLIP [1] in scenarios with scarce labels would enhance the paper.

A5. Thanks for your suggestion. We will include the discussion in the revised version as follows.

"scCLIP [1] introduces a contrastive learning approach to integrate multi-omic single-cell data. It aligns the representations of pairwise multi-omic single-cell data without the usage of cell type labels. In contrast, we focus on the cell type knowledge transfer task in label scarcity conditions."

Reference

[1] Xiong et al.,scCLIP: Multi-modal Single-cell Contrastive Learning Integration Pre-training, NeurIPS Workshop 2023

Thanks again for appreciating our work and for your constructive suggestions. Please let us know if you have further questions.