Biology Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models

Weakness6: Comparison with Task-Specific Models

TABLE r1：Comparison with SOTA in DNA Tasks

TABLE r2: Comparison with SOTA in RNA Tasks

TABLE r3: Comparison with SOTA in Protein Tasks

TABLE r4: Comparison with SOTA in Multi-Molecule Tasks

TABLE r5: Comparison Between DNABERT2 and Our Model Trained on GUE Benchmark

As shown in TABLE r1-4, ChatMultiOmics currently exhibits a significant performance gap compared to specialized models. We attribute this to the following reasons:

• Specialized models are often fine-tuned with task-specific heads, enabling them to excel at individual tasks. In contrast, training a single model to handle multiple tasks simultaneously introduces significant complexity due to the need for broader generalization. Despite these challenges, our approach successfully integrates 21 tasks into a single unified model.

• Encoder-only models, particularly those based on the BERT architecture, demonstrate exceptional capabilities in biological sequence understanding tasks, resulting in superior performance on specialized tasks. Notably, current SOTA models[r2, r3] predominantly rely on encoder-only architectures, which contribute to their optimal performance compared with decoder-only model like HyenaDNA[r4].

• The amount of pre-training data available for ChatMultiOmics is substantially smaller than that for models like DNABERT2. To achieve comparable training data sizes for large language models, such as the notably larger Llama-3.1-8B, significantly greater computational resources are required. We believe that further extensive pre-training of ChatMultiOmics could effectively enhance its performance. To support this claim, we conducted experiments on the GUE benchmark, comparing the performance of encoder-only models to the Llama3.1-8B model without instruction tuning. The specific settings are: 1. Training was performed using the GUE dataset without any instruction tuning; 2. The model utilized the same pre-trained checkpoint as ChatMultiOmics; 3. Share the same training setting such as learning rate weight ChatMultiOmics. The results are shown in TABLE r5.

• LoRA [r5] shows suboptimal performance in tasks involving biological sequence understanding. While full fine-tuning could potentially yield better results, it requires significantly longer computational time due to increased communication demands compared to LoRA.

Based on the above factors, we acknowledge the considerable performance gap between ChatMultiOmics and specialized models. However, we believe that with better training paradigms and additional computational resources, ChatMultiOmics could achieve results comparable to those of specialized models in the future.

That said, the primary focus of this work is not on achieving SOTA performance but on the construction of the Biology-Instructions dataset. We hope that future research will leverage our dataset to develop more powerful biological large language models.

Weakness1: Multi-omics Interactions

Thank you for pointing out this concern. In this work, our primary focus is on tasks involving multi-molecular interactions, encompassing both intra-modal and cross-modal tasks. Our benchmark already includes tasks that explore interactions between different molecular modalities, enhancing its diversity:

RNA-Protein Interaction Prediction: Investigating interactions between ncRNAs and proteins can deepen our understanding of the biological mechanisms involving ncRNAs. siRNA Efficiency Prediction: Examining the silencing efficiency of siRNA-DNA interactions supports the development of more effective gene therapy drugs. In the future, we plan to further expand the range of multimodal tasks, enabling models like ChatMultiOmics to gain a more comprehensive understanding of molecular and cellular biology.

Weakness2: Dataset Balance

We conducted experiments on this question. For further information, please refer to Global response 3.Data imbalance in Biology-Instructions.

Weakness3: Effect of Stage 3 training

We appreciate your valuable suggestions regarding the data from the third stage. As you have pointed out, training in the third stage has indeed had a negative impact on some regression tasks. We speculate that this may be due to the following reasons:

Complexity of Large Model Construction: The data in the third stage is generated by the large language model. For the model, interpreting regression tasks may be more challenging than classification tasks. Despite providing relevant information for each task in the hints, the model may still struggle with specific analyses. This often results in the model mentioning generalized information like, "For further validation, one might reference databases such as ENCODE," rather than conducting targeted analysis. We conduct multiple rounds of quality control to filter these data out; however, the data for regression tasks are still not fully desirable.

Inherent Challenges of Regression Tasks: Training a large model to handle regression tasks is inherently challenging. Regression tasks require the model to make accurate numerical predictions, and the additional textual information may negatively impact the training process, exacerbating this difficulty. We will continue to review and optimize the data construction and training process in the third stage to mitigate any adverse effects on regression tasks and provide more effective model support for these tasks.

Weakness4: Compare to Other Bio-LLM Dataset

We present the first large-scale multi-omics instruction fine-tuning dataset constructed around the central dogma of molecular biology. In terms of task design, our dataset is unique, encompassing tasks related to DNA, RNA, and proteins, as well as tasks involving multi-molecular interactions. In contrast, Mol-Instruction [r1] is an excellent work focused on molecular biology, with its tasks primarily revolving around molecule-related tasks, protein-related tasks, and molecule-related textual data. This results in a substantial difference in task coverage between our dataset and Mol-Instruction.

Regarding dataset construction, the approach used in Mol-Instruction and similar works is a well-established and proven methodology. Therefore, we have drawn inspiration from their construction methods while designing our task set to suit the multi-omics domain. Methodologically, such adaptation is both reasonable and necessary, as it ensures the reliability of the dataset construction process and provides a strong foundation for our research goals. However, our contribution lies in extending this framework to encompass multi-omics tasks centered on the central dogma, thereby addressing a gap in existing datasets. Additionally, we note that we are the first to integrate reasoning data in Bio-LLM datasets.

We believe that the Biology-Instructions dataset establishes a novel benchmark for research in genomics, transcriptomics, proteomics, and their interactions. This not only complements the scope of Mol-Instruction but also paves the way for the further development of biological large language models.

Weakness5: Data quality control

We conducted quality control of the dataset in two main stages:

Quality Control for Stage 2 Dataset

At this stage, we focused primarily on the quality control of question-answer templates. As mentioned in the main text, we ensured diversity in the grammar and style of the data templates and constructed a sufficient number of question-answer template pairs. Additionally, to further ensure the quality of the templates, we required task template authors to perform cross-checking to ensure that all templates met the predefined requirements and achieved high-quality standards.

Quality Control for Stage 3 Dataset

For specific information , please refer to Global response 2.Data Quality Control for Stage 3 Reasoning Data

[r1] Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models. ICLR 2024.

[r2] DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes. ICLR 2024.

[r3] The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics. bioRxiv 2023.

[r4] HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution. NeuralIPS 2023.

[r5] LoRA: Low-Rank Adaptation of Large Language Models. ICLR 2022.

Weakness 1: Focus Solely on Predictive Tasks

We appreciate this valuable feedback and would like to take the opportunity to clarify the intent and scope of our work. Our study intends to focus on evaluating tasks on sequence understanding through, specifically assessing how instruction-tuned language models, designed for general-purpose tasks, can perform on biological sequence-based prediction tasks.

While generative tasks, such as sequence design, are indeed important and promising, they are beyond the scope of this study. Our primary goal is to integrate multiple omics data and provide a comprehensive benchmark from a biological perspective, rather than aiming for versatility across both predictive and generative task types. Incorporating generative experiments would require distinct evaluation methodologies, which are not addressed in this work.

We recognize the value of generative tasks and view them as an important direction for future research. We plan to explore generative applications in subsequent work based on insights gained from this benchmarking effort. To ensure clarity, we have revised the paper to more explicitly define the scope of this study in the Abstract, Section 1 Introduction, Section 3.2.1 Tasks and included generative tasks as a future direction in Section 6 Discussion, highlighted in blue.

Weakness 2: Lack of Structural Data Integration

We appreciate this advice and agree on the importance of molecular structural data in functional prediction tasks. However, incorporating structural data to develop a method that excels in function prediction is not our primary intention for this work.

The core research question we aim to answer is whether instruction-tuned language models, which are proficient in understanding human language, can also demonstrate strong performance in biological sequence understanding to address biologically critical tasks. The motivation guiding us to select our tasks and dataset stems from the intrinsic similarity between biological sequence data and human language—both being discrete and sequential in nature. Incorporating structural data, which is continuous in nature, represents a distinct research direction that falls beyond the scope of this work. Our contribution lies in establishing a benchmark and dataset that covers multiple omics in biology, serving as a foundation for the research community to advance multi-omics studies.

We acknowledge the value of structural data and its potential to complement sequence-based approaches. As such, we have included the integration of structural data as a future direction in Section 6 Discussion to explore in subsequent studies, highlighted in blue.

Thanks for your feedback. We selected 200k data points from the Biology-Instructions dataset to balance its volume with PubMedQA and conducted experiments using models pre-trained in the first phase. The models were fine-tuned separately on Biology-Instruction-200k and on a combination of Biology-Instruction-200k and PubMedQA. The results indicate that while the inclusion of PubMedQA had a generally negative impact on performance, it showed notable improvements in specific tasks, such as rna_protein_interaction (MCC: 4.44 → 7.17) and thermostability prediction (spearman: 13.69 → 23.27).

1. Positive Impact:

The inclusion of PubMedQA boosted performance in certain tasks, such as RNA-protein interaction and thermostability prediction. This suggests that PubMedQA may provide complementary knowledge or context for tasks that benefit from enriched textual information, potentially related to functional annotations or interaction studies.

2. Negative Impact:

For several other tasks, such as enhancer activity prediction and Isoform prediction, performance decreased when PubMedQA data was included. This might be due to task-specific conflicts or noise introduced by the PubMedQA dataset, which was not curated specifically for these genomic tasks. The textual nature of PubMedQA might have diluted the task-specific signal learned from Biology-Instruction data.

3. Possible Reasons for Mixed Results:

• Task Relevance: PubMedQA primarily focuses on textual biomedical Q&A, which may not align closely with structured biological tasks like regression or classification on sequence-based features.
• Data Volume Balance: The relatively small size of Biology-Instruction-200k compared to the scale of pre-training data may have made the model more susceptible to overfitting or being biased by PubMedQA's domain.
• Domain Compatibility: Some tasks likely require more structured biological data (e.g., sequence-to-function predictions), whereas PubMedQA introduces noisy textual knowledge that could overshadow task-specific patterns.

I will keep my score since no response.

Question1: Typo in Figure 7

Thank you for identifying this. We confirm that this was indeed a typo in the figure and we have updated it in the manuscript Figure 7.

Question2: Detailed Results of Figure 6

We appreciate your suggestion to use a table for better clarity. Due to space constraints, we were unable to include detailed tables in the main text. However, we have provided comprehensive tables (Table 4,5,6 and 7) in the original manuscript Appendix, detailing the results of all methods across all tasks. In the revised version, we have included clearer references to these tables in the main text to guide readers in green.

Question3: Insights

Thank you for your feedback. While we agree that some of the findings may appear intuitive, we believe they are crucial in addressing the unique challenges of applying large language models (LLMs) to biological sequence tasks.

Biological language modeling presents specific challenges due to the unique nature of biological sequences, which differ significantly from natural language. For instance, biological sequences often require specialized tokens and representations to capture their structural and functional information. Although this may seem intuitive, our work is the first to systematically validate this observation through extensive experiments.

We tested a wide range of existing LLMs and consistently found that their performance on biological sequence tasks was inadequate without specialized pre-training and instruction tuning. This suggests that the issue is not isolated to individual models but is a broader limitation across current approaches. Moreover, through these findings, we identified key problems and proposed potential solutions, such as tailored pre-training strategies, instruction datasets, and reasoning-based enhancements, to bridge this gap. These findings provide a foundational understanding for future work.

Question4: Mixed Score Formula

Thanks for the suggestion. We have added detailed calculation in Appendix A.2.1 in green for clarity.

The Mixed Score is a custom metric adopted from data source [r1] designed to balance regression error and classification accuracy, integrating three components: Mean Absolute Error (MAE), Range-based MAE (Range-MAE), and F1 Score. This metric offers a comprehensive evaluation of model performance by considering both overall accuracy and precision within a critical value range.

Components

1. Mean Absolute Error (MAE): Measures the average magnitude of prediction errors across all samples, defined as:

   $$\text{MAE} = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|,$$

   where $n$ is the total number of samples, $y_i$ is the ground truth, and $\hat{y}_i$ is the predicted value. The range of MAE is $[0, 100]$.

2. Range-based MAE (Range-MAE): Evaluates the MAE within a specific range of interest, focusing on regions where high accuracy is critical. For the siRNA task, the critical range is defined as $[0, 30]$. The formula is:

   $$\text{Range-MAE} = \frac{1}{m} \sum_{j=1}^{m} |y_j - \hat{y}_j|,$$

   where $m$ is the number of samples within the range, and $y_j, \hat{y}_j$ are the ground truth and predicted values within this range. Range-MAE is bounded within $[0, 100]$.

3. F1 Score: Combines precision and recall into a harmonic mean for evaluating predictions within the designated range:

   $$\text{F1} = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}.$$

Mixed Score Formula

The final Mixed Score integrates the above components to balance global and range-specific performance:

where the first term emphasizes overall regression performance (via MAE), and the second term focuses on classification precision and recall within the specified range (via F1 and Range-MAE).

This metric ensures a well-rounded evaluation of model capabilities, rewarding models that perform consistently across global and critical sub-ranges.

[r1] SAIS. sirna data. http://competition.sais.com.cn/competitionDetail/ 532230/format, 2020. Accessed: 2024-05-26.

Thank you for your feedback. Our primary focus is on addressing the critical need for instruction tuning datasets specifically designed for biological sequence understanding tasks, which are essential for effectively integrating LLMs into multi-omics research. We sincerely thank you for your insightful feedback. Below are our detailed responses to your concerns:

Weakness1: Training Sample Size

The 3 million samples refer specifically to the instruction-tuning dataset, which is constrained by the availability of high-quality source data and is therefore both precious and rare. Prior to instruction tuning, we utilized a significantly larger dataset for pre-training to ensure the model learns essential biological knowledge.

Notably, many studies on LLMs in domains such as molecular biology [r1,r2] and mathematics [r3,r4], and general LLM [r5,r6,r7] have demonstrated that instruction tuning on datasets of this scale is highly effective. For example: Mol-Instructions has 2,043,587 instruction-tuning samples across biomolecular tasks, InstructGPT has only 112,801 instruction-tuning samples focused on text-based tasks, and LLaVA[r8] has 158,000 instruction-tuning samples. Compared to these works, our Biology-Instructions dataset, with over 3 million samples, represents one of the largest instruction-tuning datasets in this domain.

Our experiments further confirm that the scale and diversity of Biology-Instructions enhance the model’s performance in biological sequence understanding tasks, aligning with findings from the above studies in other domains.

[r1] Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models. ICLR 2024.

[r2] InstructMol: Multi-Modal Integration for Building a Versatile and Reliable Molecular Assistant in Drug Discovery. arXiv, 2023.

[r3] MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning. arXiv, 2023.

[r4] MathScale: Scaling Instruction Tuning for Mathematical Reasoning. arXiv, 2024.

[r5] Training language models to follow instructions with human feedback. NeurIPS 2022. (InstructGPT)

[r6] Scaling Instruction-Tuned Language Models. JMLR 2024. (Flan-T5)

[r7] Crosslingual Generalization through Multitask Finetuning on a Few Languages. ACL 2023. (BLOOMZ)

[r8] Visual Instruction Tuning. NeurIPS 2023. (LLaVA)

Weakness2: Alignment of Human and Biological Language

Thank you for highlighting the importance of aligning human and biological language. We agree that this is both a critical and fascinating direction for research. In fact, our study addresses this challenge, as aligning these two domains is essential for using large language models (LLMs) in multi-omics research.

Our experimental results indicate that without pre-training, LLMs struggle to effectively align human and biological language. This gap motivated us to design a three-stage training pipeline to systematically address the alignment challenge. Through this pipeline, our model demonstrates substantial improvements in aligning human and biological language.

Thank you for your time and effort in reviewing our work and for providing valuable feedback. We have been actively working to address your comments, conducting additional experiments, and refining our responses to ensure they are comprehensive and satisfactory. Thank you again for your constructive review and understanding.

Dear reviewer, we sincerely thank you for your insightful feedback. Below are our detailed responses to your concerns:

Weakness1: Specialized Pre-training Feasibility

We emphasize the unique characteristics of biological sequence tasks. Our experiments on baseline large language models demonstrate that existing open and closed-source models lack pre-training on biological sequences, which is distinctly different from tasks like AI in chemistry [r1]. Although ChatNT [r2] presents a training approach that does not require pre-training, it effectively connects a large language model with an encoder that has already undergone extensive pre-training. This integration is achieved through a percevier-resampler module. However, since there is no well-trained multi-omics encoder model available, we opted not to adopt this method.

Therefore, we highlight the importance of both pre-training and instruction fine-tuning, as they complement each other in achieving optimal training outcomes. We point out that without robust pre-training, even training a model on a large-scale biological sequence instruction fine-tuning dataset would fail to significantly enhance its performance in biological sequence understanding tasks. Conversely, without a large-scale instruction fine-tuning dataset, pre-training loses its value. Given the availability of sufficient resources for biological sequence pre-training data, we propose the Biology-Instructions dataset to bridge the gap in the lack of instruction fine-tuning datasets for biological sequence tasks. Additionally, we validate that incorporating reasoning processes into biological sequence understanding is beneficial for performance and can provide a better interactive experience.

[r1] ChemLLM: A Chemical Large Language Model. arXiv, 2024.

[r2] ChatNT: A Multimodal Conversational Agent for DNA, RNA and Protein Tasks. bioRxiv, 2024.

Weakness2: Dataset Imbalance

We appreciate your observation regarding dataset imbalance. We conduct experiment on this question, for further information please refer to Global response 3.Data imbalance in Biology-Instructions.

Weakness3: Data Availability Statement

Thank you for pointing this out. We confirm that the Biology-Instructions dataset is publicly available. It can now be accessed through an anonymous data link https://anonymous.4open.science/r/Biology-Instructions-FD66/ , also in the revised version of the paper 3.1 Overview of Biology-Instructions in green.

• The amount of pre-training data available for ChatMultiOmics is substantially smaller than that for models like DNABERT2. To achieve comparable training data sizes for large language models, such as the notably larger Llama-3.1-8B, significantly greater computational resources are required. We believe that further extensive pre-training of ChatMultiOmics could effectively enhance its performance. To support this claim, we conducted experiments on the GUE benchmark, comparing the performance of encoder-only models to the Llama3.1-8B model without instruction tuning. The specific settings are: 1. Training was performed using the GUE dataset without any instruction tuning; 2. The model utilized the same pre-trained checkpoint as ChatMultiOmics; 3. Share the same training setting such as learning rate weight ChatMultiOmics.
• LoRA [r4] shows suboptimal performance in tasks involving biological sequence understanding. While full fine-tuning could potentially yield better results, it requires significantly longer computational time due to increased communication demands compared to LoRA.

Based on the above factors, we acknowledge the considerable performance gap between ChatMultiOmics and specialized models. However, we believe that with better training paradigms and additional computational resources, ChatMultiOmics could achieve results comparable to those of specialized models in the future.

That said, the primary focus of this work is not on achieving SOTA performance but on the construction of the Biology-Instructions dataset. We have made a significant step forward in building an instruction fine-tuning dataset for biological sequences, and we hope that future research will leverage our dataset to develop more powerful biological large language models.

[r1] DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes. ICLR 2024.

[r2] The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics. bioRxiv 2023.

[r3] HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution. NeuralIPS 2023.

[r4] LoRA: Low-Rank Adaptation of Large Language Models. ICLR 2022.

Question2: Interpretability of Predictions

We acknowledge that interpretability in biological sequence understanding tasks presents significant challenges. However, we have taken deliberate steps to enhance interpretability through the following three approaches:

• Reasoning Process in Dataset Construction To improve interpretability, we designed the dataset construction process to include reasoning explicitly. Specifically, we required the large language model to generate end-to-end responses that follow a structured reasoning framework. Each response was designed to include:

1. Detailed analysis of the biological sequence – A thorough examination of the sequence characteristics.
2. Task-related knowledge presentation – An explanation of relevant theoretical or domain-specific knowledge.
3. Integrated predictions – Predictions derived from the intersection of sequence analysis and related knowledge.
4. Literature references – References to relevant literature to substantiate the knowledge or reasoning provided. This structured approach ensures that the dataset itself inherently contributes to the interpretability of the tasks, as the reasoning process is embedded within it.

• Quality Control of Reasoning Path For specific information about quality control, please refer to Global response 2.Data Quality Control for Stage 3 Reasoning Data

• Task-Specific Interpretability Analysis We conducted an interpretability analysis on the promoter detection task, a task that can be manually analyzed. The results are as follows:

Thank you very much for your insightful comments. Below are our responses:

Weakness1: Evaluation on Practical Applications

We fully agree that wet lab experiments provide the most robust evaluation.

• On the one hand, we have included several tasks in our benchmark that are highly relevant to real-world applications. For instance, tasks like Mean Ribosome Loading Prediction and Solubility Prediction can facilitate vaccine design by improving protein yield predictions [r4, r5]. Additionally, tasks such as siRNA Efficiency Prediction contribute to advancing siRNA-based gene silencing therapies [r6], while Antibody-Antigen Neutralization supports the development of antibodies with enhanced binding affinity for therapeutic purposes [r7].
• On the other hand, other similar benchmark papers [r1,r2,r3]published at ICLR or NeurIPS also do not include wet-lab experiments, we argue that the high cost and resource demands of wet-lab experiments are beyond the scope of this benchmark work.

[r1] Peer: a comprehensive and multi-task benchmark for protein sequence understanding. NeurIPS 2022.

[r2] BEACON: Benchmark for Comprehensive RNA Tasks and Language Models. NeurIPS 2024.

[r3] Dnabert-2: Efficient foundation model and benchmark for multi-species genome. ICLR 2024.

[r4] High-throughput 5′ UTR engineering for enhanced protein production in non-viral gene therapies. Nature Communications 2021.

[r5] Algorithm for optimized mRNA design improves stability and immunogenicity. Nature 2023.

[r6] On the art of identifying effective and specific siRNAs. Nature Methods 2006.

[r7] AIntibody: an experimentally validated in silico antibody discovery design challenge. Nature Biotechnology 2024.

Weakness2: Cross-modality Interaction Tasks

Thank you for pointing out this concern. In this work, our primary focus is on tasks involving multi-molecular interactions, encompassing both intra-modal and cross-modal tasks. Our benchmark already includes tasks that explore interactions between different molecular modalities, enhancing the benchmark's diversity:

• RNA-Protein Interaction Prediction: Investigating interactions between ncRNAs and proteins can deepen our understanding of the biological mechanisms involving ncRNAs.
• siRNA Efficiency Prediction: Examining the silencing efficiency of siRNA-DNA interactions supports the development of more effective gene therapy drugs.

In the future, we plan to expand the range of multimodal tasks further, enabling models like ChatMultiOmics to gain a more comprehensive understanding of molecular and cellular biology.

Weakness3: Potential for Data Leakage

Thank you for your suggestion to discuss the potential for data leakage.

• The majority of the data in our tasks do not pose a risk of data leakage based on the central dogma. For DNA tasks, most sequences come from non-coding regions (e.g., promoters, enhancers, and transcription factor binding sites), which cannot be aligned to transcribed RNA or translated proteins. Similarly, for RNA tasks, most do not involve codon fragments, eliminating alignment with proteins.
• From another perspective, even if a small fraction of the data does involve alignment based on the central dogma, none of the DNA, RNA, or protein tasks share the same prediction target, meaning the labels are different. This ensures that no data leakage occurs.
• Furthermore, we did not explicitly integrate the central dogma into our model training. Some recent works [r1, r2] have begun aligning the central dogma in model training to enhance multi-modal understanding, which could be a promising direction for future expansion.

[r1] LucaOne: Generalized Biological Foundation Model with Unified Nucleic Acid and Protein Language. bioRxiv 2024.

[r2] CD-GPT: Biological Foundation Model at Full-molecular Level. bioRxiv 2024.

Weakness4: Limited Error Analysis

Thank you for pointing out the lack of error case analysis in our work. Since the performance metrics primarily involve pure classification and regression tasks without any reasoning path, it is inherently challenging to conduct specific error localization and performance evaluation for the final model results. However, to address this and improve data quality, we conducted a thorough quality check and error analysis on the training data in the third stage. For further information, please refer to our response to Question2: Interpretability of Predictions below.

4.An Example of Self-Check

Origin Data

To assess the level of ribosome loading associated with the provided RNA sequence, we first need to analyze its structural features. The RNA sequence given is:

TAAGTTAAGCGCCTGACACCATCTAGCCGGAATTGGACTACATCTCACTGACATGTGTCTC

Analysis of the Sequence

1. Ribosome Binding Sites: The effectiveness of ribosome loading can often be influenced by the presence of specific motifs such as the Shine-Dalgarno sequence in prokaryotes or the 5' cap and poly-A tail in eukaryotes. Although the exact sequencing around the start codon is essential for identification, this RNA may contain regions that facilitate ribosomal attachment.

2. Secondary Structure: mRNA can form secondary structures that might inhibit ribosome loading. For this specific sequence, a computational assessment would typically involve predicting stem-loop formations or other structures that could impede ribosomal access.

3. Codon Usage Bias: The codon sequence can affect loading efficiency. Codons that correspond to abundant tRNAs are generally associated with higher translation efficiency and therefore better ribosome loading.

Ribosome Loading Level

Without direct laboratory data or additional computational tools to model the interaction dynamics, we hypothesize that the sequence exhibits a ribosome loading level of 0.85.

This numeric value suggests a high level of ribosome occupancy, where 0.00 indicates no loading and 1.00 indicates maximal ribosome binding. A loading level of 0.85 is indicative of conditions favorable for translation initiation, possibly resulting from optimal sequence features and minimal secondary structure hindering ribosomal access.

Conclusion

The level of ribosome loading associated with the provided sequence is 0.85. This indicates a high capacity for ribosomal engagement, likely due to favorable structural attributes and codon composition. For a more definitive assessment, further experimental validation or computational modeling would be useful, including techniques such as ribosome profiling or RNA secondary structure prediction.

Self-Check Result

{"result": 0, "reason": "The answer lacks detailed analysis of the specific sequence and does not provide references or citations to support claims."}

3.Data Imbalance in Biology-Instructions

We appreciate your observation regarding dataset imbalance. Our dataset proportions are constructed to reflect the distribution of source data, simulating real-world scenarios. To address potential challenges of overfitting to overrepresented tasks, we have implemented strategies during training to balance sample representation and enhance the model's performance on less-represented tasks.

In the revised version of the paper, we have included additional experimental results and a detailed discussion on the impact of data imbalance and our solutions in Appendix C in orange. We conducted experiments comparing models trained on a balanced dataset Ours(stage 1+balanced stage 2) with those trained on our imbalanced dataset Ours(stage 1+imbalanced stage 2). The final training results are shown as TABLE r1-r4 below.

TABLE r1：Compare with Balanced Dataset in DNA Tasks

TABLE r2：Compare with Balanced Dataset in RNA Tasks

TABLE r3：Compare with Balanced Dataset in Protein Tasks

TABLE r4：Compare with Balanced Dataset in Multi-Molecule Tasks

Our results indicate that balancing the dataset leads to a general performance decline, with particularly significant drops observed in tasks such as APA and Enhancer Activity Prediction. We believe that balanced datasets may distort the natural distribution of real-world biological data and reduce overall data size to match the smallest task, which contains only a few thousand samples, limiting the model's ability to fully utilize available data.