Can Transformers Smell Like Humans?

We thank the reviewer for the comments and for the time spent reviewing our paper We now address the questions (Q), weaknesses (W), and limitations (L) raised by the reviewer:

• (W2, Q2, L1) Fine-tuning results: We thank the reviewer for the suggestion. We present the results of fine-tuning the complete MoLFormer model in Figure III of the general rebuttal document and will add these results to the paper. The change compared to MoLFormer is not significant in many cases. This raises interesting questions regarding the feasibility of fine-tuning pre-trained large-scale chemical models for human olfaction (and the method to do so).
• (W1, L2) Model Architecture: We agree that the development of novel network architectures for this task is interesting but, as it stands, we believe our work responds to a different, but also very relevant question. We explore whether representations extracted from pre-trained large-scale models on chemical data are aligned with human perception. Our finding suggests that pre-trained models are highly aligned with human perception of odorants without being explicitly trained on it, even when compared to supervised learning methods explicitly trained on human olfactory perceptual data.
• (Q1, Q3) Models employed in our work: In our work, we consider three types of models to generate odorant representations: (i) the MoLFormer, a self-supervised, transformer-based, large-scale pre-trained model of chemical data, which is not trained on human assessments of olfactory stimuli; (ii) the Open-POM, a supervised graph neural network model that is trained explicitly using human assessments of olfactory stimuli; (iii) the DAM model, a feature-engineered representation model that was specifically proposed to be used in prediction tasks of the human olfactory experience. To the best of our knowledge, Open-POM is the SOTA deep-learning model trained for prediction tasks of the human olfactory experience, and the DAM model is the SOTA feature-engineered model for prediction tasks of the human olfactory experience.

We thank the reviewer for the comments and for the time spent reviewing our paper. We now address the questions (Q) and weaknesses (W) raised by the reviewer:

• (W1.1) Dimensionality Reduction: In this work, we use PCA to reduce the dimensionality of the representations for MoLFormer and Open-POM. You are correct that DAM does not require PCA. We have clarified this in the updated version of the document.
• (W1.2) DAM on Figure 3: We thank the reviewer for the comment. We have added the t-SNE visualization of the DAM model (Figure II. c) in the general rebuttal document. We also added such visualizations using UMAP and PCA in the updated version of the paper. (We didn’t include them in the general rebuttal document due to the lack of space.)
• (W2) Noise ceilings: We thank the reviewer for the suggestion to use noise ceilings in Section 4.2 and Section 4.3. We have computed the noise ceilings for the "Sagar" and "Keller" datasets (as these are the only ones that have multiple evaluators for each odorant, and those are publicly available.), as shown in the following tables. The results show that we have noise ceilings of $0.28\pm{0.1}$ for the Keller dataset and of $0.7\pm{0.05}$ for the Sagar dataset. The results show that the data of the Sagar dataset is less noisy, and there is still room for the models to increase the alignment. However, the Keller dataset alignment results are relatively close to the noise ceiling value. We will also add the results divided by noise ceiling value in the appendix of the paper.

Sagar dataset:

Keller dataset:

• (W3) Introduction references: We do agree that DiCarlo & Cox, 2007 and Olshausen & Fields, 2004, are more suitable for the purpose of this sentence and, as such, we have followed the reviewer’s suggestion and replaced them in the updated version of the paper.

• (W4) RSM analysis: We thank the reviewer for the analysis suggestions. We provided the RSM figure only for the Ravia dataset and for MoLFormer, OpenPom and Human Perception in Figure I of the general rebuttal document, and we will add that for both the Ravia and Snitz datasets and also for the DAM model in the updated version of the paper (we don't include Snitz dataset and DAM model in the pdf due to the lack of space). We also added a table showing the correspondence of labels on axes with the real CIDs in the paper appendix.

• (W5) SMILES/Input for the different models: The SMILES representation encodes the three-dimensional structure of a chemical (e.g., a molecule) into a short ASCII string. It does so by first performing a depth-first tree search over the chemical graph, adding the elements, bonds, and rings, and subsequently removing hydrogen atoms and breaking cycles (e.g., water is represented by “O” in the SMILES notation). We use SMILES representations of odorants as input to the MoLFormer. For example, toluene is written as a string like "Cc1ccccc1". For the Open-POM, each odorant is represented as a graph, where each atom is represented as a node feature vector (e.g., valence, hydrogen count, atomic number…), and each bond is represented as a edge feature vector (e.g., degree, aromaticity, whether it is a ring or not). Finally, for the DAM model, we use a set of predefined physio-chemical features, which we extract using the AlvaDesc software. We have clarified these representations in the updated version of the document.

• (Q1) Correspondences of Inputs: We thank the reviewer for raising this point. In our work, “smell like a human” corresponds to whether representations of odorant chemical structures extracted from learning-based models are aligned with human olfactory perception (line 37). The underlying assumption to this question is that the "stimuli" provided to human participants are similar to the ones provided to the models using different types of representations. We agree that there are limitations that are not captured by our representation, such as the concentration and intensity of different molecules in an odorant, and we will try to discuss this better in the updated paper. But in our study, input to the olfactory receptors are the molecules of a specific odorant, and we used the representations of the exact same molecules as the input of the model.

We thank the reviewer for the comments and for the time spent reviewing our paper. We address the questions (Q), weaknesses (W), and comments (C) raised by the reviewer as follows:

• (W1) Description of GS-LF dataset: GS-LF dataset is a multi-label binary dataset, where each data point (the “odorant”) has a set of labels (the “descriptors”) evaluated by a single expert evaluator (the “judge”). As such, no odorant has ratings from multiple judges, but the single judge was asked to assign labels to each odorant chosen among 138 descriptors. We clarified this point in the updated version of the paper.

• (W2 and Q1) Dimensionality reduction method and variance captured: In this work, we use PCA to reduce the dimensionality of the representations. We report the amount of variance explained by the first 20 PCs in the following table. The results show that our 20 PCs capture the majority of the variance of the data.

• (W3) Descriptors in the similarity dataset (line 123): You are correct. Line 123 is not accurate. Similarity datasets do not include descriptors, and in this case, $y_i \in [a, b]^{n \times 1}$. We have corrected this line in the updated version of the paper.
• (W4) References in the introduction: We thank the reviewer for this comment. As pointed out as well by reviewer UQYS, we have replaced Damasio references by the work of DiCarlo & Cox, (2007) and Olshausen & Fields, (2004). We agree that reference [4] is out of place: in line 34 it should read “(...) impressive performance in various tasks such as control [4], image [13], (...)”. We have also added the reference to the work of Vaswani et al. (2017) as the original reference for the transformer. Moreover, we agree that [13] uses a supervised objective to train the transformer, and we also added the reference to He et al. (2022) on MAEs to the discussion on self-supervised image transformers.
• (C1, Q3) UMAP and T-SNE visualizations: We add the T-SNE figures to the attached rebuttal PDF. We also ran experiments with UMAP and will add it to the appendix of the paper (we don't include in the attached PDF due to lack of space). Overall, we found these figures to be harder to interpret than the PCA figure, which was already suggested in [20].
• (C2) Train-test split: In all evaluations in our work, we use a 80-20% train-test split, with 30 different randomly-seeded partitions. Moreover, we used nested cross-validation to estimate the hyper-parameters of the linear models. We have added this information to the beginning of Section 4 of the updated version of the paper.
• (C3, Q2) Physio-chemical descriptors and Figure 6: We extract the physio-chemical descriptors using the AlvaDesc software. These physio-chemical descriptors are those that are used as input features of the DAM model proposed by Snitz et al. (2013). As such, in Figure 6, we don’t evaluate the prediction capabilities of the DAM model, as the targets would be the same as its input features. We have clarified this point in Section 4.4 of the updated version of the paper.
• (C4) Motivation of section 4.4 (line 284): We have clarified the motivation for the additional analysis in the updated version of the paper.
• (C5) Alignment with low-level physio-chemical descriptors: We thank the reviewer for the insightful comment on the results regarding the alignment of the model with physio-chemical descriptions, which we fully agree with. The reviewer pointed out correctly that in vision models, low-level features are extracted in the early layers of the model, and the same behavior can be observed here. We added this insightful interpretation in our final paper.