TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondence

We thank the reviewer for their feedback and highlighting the innovative model design and SOTA performance of our TRIDENT model. We address the valuable questions and comments raised by the reviewer point-by-point:

1. Sensitivity analysis on hyperparameter settings: Following the reviewer’s suggestion, we performed sensitivity analysis on key hyperparameters: learning rate, contrastive loss temperature, and the momentum parameter β (used for weighting global and local alignment losses). We conducted this analysis on three MoleculeNet datasets and one TDC dataset (Caco-2). The results show that within a reasonable range of hyperparameters (learning rate 1e−4 to 1e−5, β between 0.5–0.9, and temperature 0.07–0.03), TRIDENT consistently achieves similar performance. This indicates that TRIDENT is robust to hyperparameter choices and does not require tuning for each individual dataset.

2. Additional recent or multimodal baselines: In response to the reviewer’s comment (and also reviewer g9iG), we have extended our comparisons to include recent state-of-the-art baselines, namely Uni-Mol (3D molecular modeling) and multimodal large-scale models such as BioT5, BioT5+, and MolXPT. (The suggested comparison with MolFM is already reported in Table 1 of the paper.) We have now evaluated these new models on MoleculeNet regression tasks and seven TDC datasets, where TRIDENT consistently outperforms these strong baselines. These additional results and corresponding citations to these baselines will be incorporated into the camera-ready version of the paper. For completeness, we also report the results below.

MoleculeNet regression tasks:

7 TDC datasets:

We thank the reviewer again for their questions! Given that we have addressed all your questions, we request you to kindly consider raising your score!

We thank the reviewer for the follow-up comment.

Comparison against Uni-Mol baseline on TDC datasets: Due to the limited rebuttal period, we initially prioritized experiments that addressed multiple reviewer comments simultaneously, which led us to focus on Uni-Mol primarily for the MoleculeNet benchmarks. Following the reviewer’s suggestion, we have now run Uni-Mol on the TDC benchmarks and summarize the results below:

These results show consistent improvements across all seven TDC benchmarks, with gains as high as 22.64% (Caco-2) and 13.05% (Carcinogens).

We also agree that MoleculeNet has known limitations, which is why we conducted comprehensive experiments on seven TDC benchmarks additionally. We hope this addresses the reviewer’s questions and concerns fully. If so, we would kindly appreciate reconsideration of the overall score!

We thank the reviewer for their positive feedback and highlighting the value of our hierarchical taxonomic annotations for the field as well as for our substructure-subtext alignment approach which obtains SOTA performance on molecular property prediction benchmarks. We address the valuable questions and comments raised by the reviewer grouping similar questions together:

1. Results on MoleculeNet regression tasks and discussion/comparison with 3D models: In addition to the MoleculeNet classification tasks reported in the main text, we have now also evaluated TRIDENT on three MoleculeNet regression tasks, and we report the results below (metric is RMSE). For a more comprehensive comparison, we extend the baseline set beyond those in the main text to include: (a) 2D topological graph models (GCN, GIN, GINE, SR-GINE) and 3D molecular graph models (Uni-Mol). These additional results demonstrate TRIDENT’s strong performance over existing and recent baselines (including recent 2D and 3D graph models ). We will cite and include the discussion of these models in the camera-ready version along with the new results on MoleculeNet regression tasks shown here.

2. Comparison with BioT5, BioT5+, MolXPT: We also compared TRIDENT against models BioT5, BioT5+, and MolXPT as suggested by the reviewer. We reported the comparison in the table above for three MoleculeNet regression tasks. We are also including the results for 7 TDC datasets in the table below. TRIDENT outperforms the baselines here as well. We will also cite BioT5, BioT5+, and MolXPT in the camera-ready version and will include these results.

3. Ablations (a) with 380k SMILES-text original dataset, (b) using further fine-tuning with larger SMILES-text dataset starting with pre-trained TRIDENT weights: We appreciate the reviewer’s suggestion to explore this. For this ablation, we pre-trained TRIDENT on the full 380k SMILES–Text pairs without HTA and evaluated it on three MoleculeNet regression tasks and one TDC dataset (Caco-2). Training solely on the full 380k SMILES–Text pairs degrades performance due to known dataset issues (artificially templated text, limited semantic richness (too short descriptions), and duplicates) [1]. For the second ablation, we used the model weights from triplet-based trained TRIDENT model and fine-tuned it with remaining (380k-47,269) SMILES-text pairs. The performance improved slightly but still lagged behind TRIDENT trained solely on curated triplets as fine-tuning on potentially low quality data is detrimental for learning. This confirms the value of our data curation contribution and the inclusion of hierarchical taxonomic annotations (HTA) for representation learning.

We believe we have addressed all your questions and request the reviewer if they may consider raising their score!

[1]: Ross, Jerret, et al. "Large-scale chemical language representations capture molecular structure and properties." Nature Machine Intelligence 4.12 (2022): 1256-1264.

Dear reviewer,

We greatly appreciate the time and effort you’ve put into reviewing our submission. As the discussion period is nearing conclusion, we kindly request your feedback regarding our detailed responses to your valuable comments. We believe we have effectively addressed all questions. Please let us know if you have any further questions. We’d be happy to clarify. Otherwise, if your concerns have been addressed, we’d greatly appreciate it if you could consider raising the score. Thank you!

We thank the reviewer for appreciating the novelty and highlighting the state-of-the-art performance of our proposed model on molecular property prediction tasks. We also thank the reviewer for raising valuable questions. We address all questions raised by the reviewer point-by-point.

1. Why chose only 3 TDC datasets/more TDC datasets/other downstream tasks on larger datasets: We actually evaluated TRIDENT on five TDC datasets: three in the main text (as noted by the reviewer) and two additional ones in Appendix Table 8. The appendix includes results on larger datasets (AMES with 7,255 drugs and hERG with 648 drugs), where TRIDENT also outperforms all baselines. Additionally, for the rebuttal, we conducted experiments on two other larger datasets (CYP P450 2C19 with 12,665 drugs and Caco-2 with 906 drugs ), bringing the total to seven TDC datasets. For completeness, we report all results below, which further confirm TRIDENT’s consistent performance across diverse TDC benchmarks.

In case the reviewer referred to large molecules such as proteins by larger datasets, we emphasize that since the pre-training is done on small molecules data curated from PubChem, we have benchmarked the model on many tasks from commonly used small molecule property prediction benchmarks (MoleculeNet and TDC) relevant for small molecule drug-discovery applications. Extending TRIDENT to larger molecules (such as proteins) is an interesting future work!

2. Ablation and discussion on local alignment hyperparameters: The local alignment module has only two main hyperparameters:(a) the relative weighting of local vs. global alignment terms, (b) the aggregation (pooling) strategy for consolidating sub-text and sub-structure representations.

a. In the main text (Table 4), we already presented ablation results for different weighting strategies for local vs. global alignment terms, showing that momentum-based dynamic weighting performs best.

b. Following the reviewer’s suggestion, we additionally ablated the aggregation strategy, comparing multiple pooling operators. Note that the “weighted mean pooling” variant refers to assigning weights based on the frequency of functional groups present in the molecule. The results (reported below: three MoleculeNet regression tasks and one TDC dataset (Caco-2)) indicate that TRIDENT’s performance is similar irrespective of the choice of aggregation function.

We will include these results in the Appendix of camera-ready version. Given that we have addressed all your questions, we request if the reviewer may kindly consider raising their score!

We thank the reviewer for their positive feedback and appreciating the novelty of our method. We address the valuable comments and questions raised by the reviewer point-by-point and will include them at appropriate places in camera-ready version.

1. Training time and memory usage: We use pre-trained frozen encoders for SMILES, HTA, and functional descriptions, with only lightweight trainable projection layers (2.5 M parameters vs. 223 M in MoMu). This parameter-efficient design significantly reduces computational requirements for training and deployment. Under the same experimental settings, MoMu processes 32.5 samples per second while our method processes 16.4 samples per second on a single NVIDIA H100 GPU, with the difference primarily attributed to processing additional modalities (functional groups and HTA information). The total GPU memory usage is ~45 GB versus ~35 GB for MoMu, with the modest increase due to loading multiple frozen encoders and handling additional data modalities, but still easily affordable by most researchers.

2. Details on prompting strategy and validation process to synthesize HTA texts with GPT-4o:

Prompting strategy: The exact prompt template used is shown in Appendix A, Figure 4. This template explicitly instructs the model to summarize taxonomic annotations from PubChem without introducing additional information. All HTA summaries were generated using the same fixed prompt and model parameters (temperature=0.2, frequency_penalty = 0.0).

Filtering and validation: Because GPT-4o was summarizing well-defined, scientifically validated PubChem Taxonomic Annotations, the risk of introducing extraneous information was minimized. Additionally, we conducted a validation step where domain experts independently reviewed a random sample of 1,000 HTA summaries, comparing them against the corresponding PubChem annotations. No factual hallucinations or taxonomic inconsistencies were observed in this review.

3. Error analysis (failure cases): We thank the reviewer for highlighting this important point. While TRIDENT demonstrates strong overall performance, we note a performance decline on specific downstream tasks. For example, in the SIDER dataset, it achieves a ROC-AUC of 0.5904 on the Product issues task, which is notably lower than the 0.7383 ROC-AUC on the Blood and Lymphatic System Disorders task. A key factor is class imbalance in the former dataset, for which TRIDENT does not currently employ explicit rebalancing mechanisms. Addressing such imbalance-aware training strategies (e.g., loss reweighting or data augmentation) is a promising direction for future work to combine with TRIDENT.

4. Analysis of the embedding space: We thank the reviewer for this suggestion. To analyze the embedding space, we calculated the average Euclidean distance between positive pairs across all modalities during pretraining. The distances between positive pairs across all modality combinations consistently decrease throughout training: SMILES-Text distances reduce from 0.7930 (Epoch 1) to 0.4805 (Epoch 40), SMILES-HTA distances decrease from 0.8269 to 0.4708, and Text-HTA distances drop from 0.9511 to 0.8698, with the overall average distance declining from 0.8570 to 0.6070. This systematic reduction in intra-sample cross-modal distances indicates that TRIDENT successfully learns to align semantically related representations across different modalities. We will include this analysis in camera-ready version. Given the rebuttal policy, we cannot include figures in this response but we will also include tSNE projections of embeddings and crossmodal attention heatmaps in the camera-ready version to further illustrate interpretability.

5. Performance on non-biomedical or non-taxonomic datasets: We thank the reviewer for this valuable suggestion. Our work is primarily focused on molecular property prediction for drug discovery applications. Exploring non-biomedical datasets is an interesting future direction but is beyond the current scope. Notably, taxonomy information is used only during pre-training; all downstream molecular property prediction tasks rely solely on the SMILES encoder and projector, demonstrating that TRIDENT performs effectively even when taxonomic information is unavailable at inference time.

6. Architecture of encoders and projectors: We thank the reviewer for the suggestion. Our model architecture and training configuration are described in Appendix B.2 and Algorithm 2 (Appendix page 20). Briefly, we use pre-trained MoLFormer for SMILES and MolT5/SciBERT for text (including HTA descriptions), both of which are frozen during training. We train only lightweight modality-specific projection heads (three linear layers with GELU, LayerNorm, and dropout) to map encoder outputs to a shared 512-dimensional space. For HTA and Text tokenization, we adopt the default SciBERT WordPiece vocabulary (scivocab) tokenizer, ensuring consistency with its pre-training. When using MolT5 as the HTA and Text encoder, we employ its pre-trained T5Tokenizer. This design minimizes computational overhead while retaining rich semantic representations.

Thank you for the explanations and for taking the time to answer. My concerns are well addressed.