Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models

Motivation for studying intermediate layers:

I don't see the point of analysing the intermediate layers of a foundation model for domain specific scientific reasoning. It makes sense in trying to undestand what foundation models do exactly but that does not seem to be the goal here. I am not sure I fully understand the main ambition of the paper. Can you elaborate on the significance of the obtained features?

We would like to refer the reviewer to the section Positioning of the paper in the response addressing all reviewers and may ask them whether this clarified their concern. We specifically chose intermediate layers over the final layer because we aim to extract features from the point in the residual stream where the model retains the most biological information. For MAEs, we observe that the peak information is located in an intermediate layer. Additionally, using intermediate layers is a common practice in mechanistic interpretability research on large language models (LLMs) as noted in previous studies ([1,2]).

Scientific significance:

Overall the contribution seems novel and non-trivial, however, its future scientific significance appears to be vague to me at this point.

We would like to refer the reviewer to the section Scientific relevance in the common response to all reviewers.

Questions:

More computationally efficient variants of ICFL:

A termination condition based on the norm would seem more sensible to me.

We agree that future research in this direction would be interesting! Since compute is not a limiting factor in comparison with the required compute of training the foundation models, we did not explore more computationally efficient methods yet. However, when scaling DL compute certainly becomes a valid concern and techniques that adaptively choose the number of steps could be a promising solution.

More computationally efficient variants of ICFL:

Can you elaborate on the significance of the obtained features?

We thank the reviewer for raising this important question. We would like to refer the reviewer to Section Scientific relevance in the common response addressing all reviewers.

Features extracted directly from the microscopy image data:

What features do you get when you apply ICFL directly on cellular microscopy images?

May we ask the reviewer to specify their question? Is the reviewer suggesting applying ICLF to the flattened image containing RGB values? If this is the case, we do not expect any meaningful features as ICFL only extracts linear directions. Thus, ICFL can only be used in combination with a large scale (transformer) model, in strong contrast to disentangled representation learning methods mentioned in the related works.

Features extracted from the foundation model:

Can you give more details on the foundation model (e.g. input, output)?

The foundation model is a gigantic masked auto-encoder (MAE-G) based on a vision transformer (ViT) architecture that takes multi-channel images as input into the encoder and returns the reconstructed version of the image of an identical shape as an output of the decoder. In the process of the image encoding, token-wise embeddings are produced at masking ratio of 0 at inference time.

What do you think your model can learn from a foundation model, something related to its input or its output, or both?

By extracting features from the intermediate representations of the MAE, we hope to learn abstract features capturing biologically relevant information about the underlying input data. We would like to refer the reviewer also to the section Positioning of the paper in the common response to all reviewers.

[1] Gao, Leo, et al. "Scaling and evaluating sparse autoencoders." arXiv preprint arXiv:2406.04093 (2024).

[2] Bricken, Trenton, et al. "Towards monosemanticity: Decomposing language models with dictionary learning." Transformer Circuits Thread 2 (2023).

Technical novelty:

With respect to machine learning methodology, the technical novelty of the paper is somewhat limited. The overall problem and approach follows TopK, except rather than optimizing the specific TopK formulation, an orthogonal matching pursuit approach is used instead. ICFL was shown to be superior to TopK on some quantitative measures, but these measures are proxies and not an end goal in themselves. Qualitative results were only given for ICFL. Was any qualitative analysis done on the TopK learned features? Was it the case that those features were clearly sub-optimal to ICFL from a qualitative perspective? Or were they also able to capture any concepts like the morphologies shown in Figure 1?

We would like to refer the reviewer to the section Novelty of the ICFL algorithm in the common response to all reviewers.

Questions:

Number of interpretable features:

The qualitative analysis gives a discussion and intuition for a small number of the learned features. Was it the case that many of the learned features had this kind of interpretability as human-understandable concepts? Or was this only true for the small subset shown?

The reviewer raises an important question. While our qualitative analysis suggests that many of the features capture at least some biologically meaningful information, expanding this analysis poses challenges. We extend our qualitative analysis in the section Interpretability Analysis in the common response to all reviewers. However, we would like to stress that, in its current form, we cannot provide a definitive answer to the reviewer's question, as addressing it would require significant effort. In particular, it is non-trivial for a scientist to determine, by examining examples, whether a randomly chosen feature captures potentially complex biological patterns or simply overfits to batch effects. Developing protocols that involve domain experts to assess the quality of SAEs and to facilitate the discovery of features capturing novel, biologically relevant patterns represents a very important and promising interdisciplinary direction for future research.

Application in biological research:

More generally, how do you see the results of this method being used in biological research? Is the desired goal that by looking through the learned features, a researcher might find a single concept that captures something complex and unexpected, warranting further study?

We thank the reviewer for raising this important question. Yes, this is one of the potential applications. We would like to refer the reviewer to section Scientific relevance in the common response to all reviewers.

Alternative approaches:

For instance, from the perspective of biological research and understanding, why not instead take a CRL approach that directly tries to learn interpretable features/representation? Or if certain tasks are of key importance, why not focus on learning concept vectors with respect to that task/class-labeled data. It would be interesting to compare the proposed method with class-based concepts using something like Ghorbani et al Automatic Concept-based Explanations, for instance using the specific tasks already being considered in the paper.

We thank the reviewer for raising this important question. We mention alternative approaches in the related work and will extend this section in a revised version of the paper. To answer the reviewers question, we would like to mention the following two aspects:

Supervised vs unsupervised features: Supervised features are expected to be more predictive of the labels and help researchers better understand which patterns separate classes. However, multiple factors limit supervised feature learning. First, off-target effects of the CRISPR guides are significant [Lazar et al 2024, “Proximity bias”], and so there is a very real risk that the features that are most predictive of a given target, are those that depend on cells experiencing off target effects rather than the phenotype of interest. In this context, we point the reviewer to recent studies that demonstrate the superiority of biological representations learned through self-supervised methods over supervised methods, particularly when evaluating the recall of known biological genetic interactions.

Large scale foundation models: In this paper we extract features from self-supervised vision models that can be efficiently scaled and trained on large scale datasets. In contrast, many works on disentangled representation learning usually learn an auto-encoder from scratch, drastically limiting their applicability on large scale datasets.

Available labels:

B) The interpretability scores presented in the quantitative evaluation are based only on a few known labels. Are those labels all that can be interpreted in the models?

This is indeed an important question. The 5 types of labels that we use in this paper are natural labels of greater significance in the community. In general, one wants to avoid relying too much on labels that require domain expertise for training the probes as generating these labels is expensive. An alternative would be to rely on CellProfiler, which provides a very extensive list of hand-crafted features by domain experts. We use CellProfiler Features in Figure 3. However, limitation in the usage of CellProfiler features is that these features capture visual patterns and not more abstract concepts. One hope of this line of research is that future efforts allow us to extract features that achieve higher selectivity scores for genetic perturbations than CellProfiler, and thus are individually more predictive of these perturbations.

There are two important points to note in addition to this analysis. First, doing this type of analysis requires the human annotators to precisely know and understand the detailed function of the gene which is knocked out, which means we are naturally restricted for analysis of only a few dozens of genes the function (and morphological consequences) of which we understand and can interpret. Second, observing the images with 20-50 cells each, studying each cell in detail and giving a human verdict about what's going on with that particular cell instance is extremely laborious and doesn't scale well to the repertoires of publicly available cell microscopy datasets, including the ones used in this study. We currently didn't possess the resources of several scientist annotators available, so had to restrict ourselves to a smaller yet more reliable analysis, which is now fully backed up by the quantitative aspect as well.

Entanglement is still present:

The results in Fig. 3 seem to indicate that there is still considerable entanglement between features.

We agree with the reviewer. However, to put this comment into perspective, we view it already as a big step that we are able to obtain features that have a selectivity score close to the one of CellProfile features and hope that future works will further improve the selectivity scores. We tried to be transparent about this limitation in the conclusions by writing:

That said, these sparse features are clearly incomplete:we see significant drops in their linear-probing performance on tasks that involve more subtle changes in morphology. It is not clear to what extent this is a limitation of our current dictionary learning techniques, the scale of our models, or whether these more subtle changes are simply not represented linearly in embedding space. Nonetheless, it is clear that the choice of dictionary learning algorithm matters to extract meaningful features.

Use of proprietary models:

Not reproducible research. As far as this reviewer can tell, the models and data to reproduce this research are not publicly available. The machine learning community strives for openness and reproducibility, exemplified by the open source, open review, and open access practices. It is a significant downside of this manuscript that the presented results cannot be independently verified.

We understand the reviewer's concerns regarding the use of proprietary models, especially given the community's emphasis on reproducibility and openness. However, we believe it's important to note that the field of mechanistic interpretability often relies heavily on proprietary models. Many prominent works in this area, for instance [1,2], use models and frameworks that are not fully open. In general, it is particularly common for large-scale foundational models to use proprietary non-public models. Limiting research in this space based on accessibility to proprietary tools may hinder overall progress and innovation in the field.

Nevertheless, we strongly believe in open science and would also like to contribute to this important community effort. To do so, we’ve made a deliberate effort to use publicly available datasets like RxRx1, RxRx3, and benchmark against the publicly available CellProfiler. Although this does not allow for the full reproducibility of our results directly, our approach allows for comparative benchmarking. By evaluating selectivity scores on RxRx1 and comparing them with CellProfiler features, we’ve created a novel and accessible benchmark that can be independently referenced and utilized. This compromise, we believe, aligns with the spirit of openness while also pushing forward impactful research.

Nevertheless, we would like to argue against the claim made by the reviewer that the transformations make it more difficult to deduce insights about the inner working mechanisms of the underlying model in comparison with the pre-processing applied in e.g., [1,2]:

Token aggregation: To comment first on token aggregation. Assuming that concepts are laid out by the model as linear directions, these linear directions persist when aggregating the tokens. However, the advantage of aggregating the tokens is that concepts that depend on capturing highly local information, such as whether patch i contains a cell or not, will shrink due to averaging effects. Thus, aggregating the tokens can be seen as a way to reduce the signal to noise ratio that is particularly troublesome in multi-cell image data where batch effects are substantial. Yet, DL should still be able to recover directions of concepts that appear across the entire image, such as exactly the ones we are interested in studying. We are curious to hear the response of the reviewer to our explanation.

PCA whitening: While we would agree with the reviewer that the assumptions behind DL, formalized in the superposition hypothesis, is strong, we disagree with the reviewer that applying our weakly supervised PCA transform necessarily results in features that reveal less information about what the model is actually doing compared to ones obtained from apply DL directly. Indeed, in this paper, we propose to minimize the L2 norm after a PCA transform, which can equivalently be understood by applying DL on the original features but minimizing a weighted L2-norm. If we understand correctly, the reviewer now claims that these features are less informative of what the model does compared to the ones obtained when minimizing the l2-norm. However, what justifies the l2-norm as the right norm to minimize? It is very plausible that some concepts are represented by linear directions with smaller magnitudes and others with directions with larger magnitudes. In this case, a data driven approach using weak supervision as we do in this paper is crucial to recover all features. While we do not want to claim that this is indeed the case, we want to challenge the statement by the reviewer that minimizing simply the L2-Norm yields features that are more descriptive of the features actually used by the model. We are curious to hear the thoughts of the reviewer to this response.

Monosemanticity:

The paper also claims that their approach finds directions that are monosemantic, but the results do not support such claims. The quantitative evaluation is focused on making sure that the learned sparse features capture as much signal as the original features. Monosemantic features would be able to classify these concepts with a single dimension, which is not what the evaluation presents.

The reviewer highlights an important point, namely that claiming monosemanticity is indeed a strong statement. However, we usually use monosemanticity in quotation marks and specify what we mean with monosemanticity, such as in Line 537:

In our experiments, we found that both ICFL and PCA significantly improve the selectivity or “monosemanticity” of extracted features, compared to TopK sparse autoencoders.

In particular, we do not claim that we find monosemantic features, but instead argue that ICFL together with PCA whitening improves the “monosemanticity” of the features, in the sense that it improves the selectivity score which is indeed a proxy for monosemanticity. Could it be that the reviewer views monosenacity as an attribute that either is present or not, while we treat monosemanticity as an attribute that can also occur only to some extent.

After consideration, we understand that our use of the term monosemanticity can be misleading, and we will adjust the writing accordingly. In particular, we will remove the term monosemanticity in the abstract, and thus only write: “improve the selectivity score of extracted features compared to TopK sparse autoencoders.”. Nevertheless, we want to clearly object to the reviewers statement that this paper “ also claims that their approach finds directions that are monosemantic”. We explicitly always say that our method improves the ``monosemanticy’’ of the features and specify that we mean the selectivity score of the features.

Nature of the main contributions of the paper: We thank the reviewer for raising their critical comments on this paper. We begin by addressing the following comment made by the reviewer.

There are several transformations applied to the tokens before dictionary learning, which obscures the claims made in the paper about interpreting the inner workings of the models. The fundamental question for interpretability is what is the model doing and how we can use that information to keep it under control? Discovering biases and understanding how they are encoded in the model is at the heart of these approaches, but in this paper, the model outputs are first aggregated and decorrelated with PCA whitening using labeled data to proceed with dictionary learning. This contradiction between the claims / motivation of the paper and the actual procedure followed for interpretation is confusing. This also suggests that the interpretation is performed at the level of one downstream task rather than at the level of model activations and inner workings.

We believe that the reviewer may have a diverging perception of what the main contributions of this paper are, and we would like to revise the manuscript to address the sources of the misunderstandings. As the reviewer correctly points out, the goal of mechanistic interpretability is to understand “what is the model doing and how we can use that information to keep it under control”. However, we would like to emphasize that this is not the main goal of this paper. Instead, we investigate with this paper how the tools from mechanistic interpretability can help us to better understand multi-cell image data. In particular, the goal of this paper is not to find hidden biases in the model, but instead, given the premise that the model lays out abstract concepts as linear directions, to extract features using DL capturing biologically meaningful patterns. We would like to refer the reviewer also to the section Positioning of the paper in the common response addressing all reviewers.

To address the following comments made by the reviewer:

This contradiction between the claims / motivation of the paper and the actual procedure followed for interpretation is confusing , which obscures the claims made in the paper about interpreting the inner workings of the models.

May we ask the reviewer to point us to the section in the paper where they got the impression that this paper makes these claims? We would like to address the writing accordingly. Based on the reviewer's comment, we believe that the use of the term ‘interpretable features’ in the caption of Appendix B could be misleading and we will rename this section to Examples of evident patterns captured by the features.

Additional transformations on top of features from MAE:

There are two reasons for this A) the approach presented in the paper depends on additional token transformations, which makes the reader believe that the interpretation is not really associated to what the model encodes. There are several transformations applied to the tokens before dictionary learning, which obscures the claims made in the paper about interpreting the inner workings of the models

We thank the reviewer for raising this important concern. We would like provide multiple arguments for why we do not believe that the transformations, i.e. PCA whitening and aggregation of the tokens, stand in contradiction with our analysis.

Finding linear concept directions capturing biologically meaningful information: The underlying assumption that we exploit in this paper is that the model represents abstract concepts as linear directions, which is also the starting point of recent works using DL in mechanistic interpretability. Since we only apply linear transformation before applying DL, the underlying assumption exploited by DL remains unchanged. Moreover, since the goal is primarily to find features capturing biologically striking patterns, we do not see any contradiction in applying these linear transformations, especially as we demonstrate that they improve the scores.

To clarify the comment about “without interventions”, early experiments in mechanistic interpretability attempted to understand if the original activations of the model are interpretable or not. Later on, sparse coding was introduced to identify feature combinations that were more “monosemantic”. This work makes the assumption that sparse codes are necessary, but the author’s argument could be strengthened if they show that the original feature space is not as interpretable as the codes they learn in this domain.

Finally, it's true that previous work have used and relied on proprietary models to conduct the evaluations, but even the papers that the authors cite have conducted experiments in open models (e.g. GPT2) for the community to reproduce and continue exploring such research directions. This is something that this paper does not have in its current form.

Thank you for offering clarifications about the random classifier and challenges of what is interpretable in this domain. Overall, I appreciate the responses and the work behind this submission. I have modified the rating from 3 to 5 in light of these. Please, update the manuscript with the suggested changes and clarifications.

Thank you for addressing my comments and for clarifying. I acknowledge that the common response to all reviewers was helpful, including the position of the paper and scientific relevance. In addition, I see value in the proposed algorithm for general purpose mechanistic interpretation.

To clarify my comment about contradiction between claims / motivation, I believe the paper reuses terminology from mechanistic interpretability in natural language processing which results in mismatched expectation and results. For instance, the fact that the authors use the word “token” for the superposition hypothesis and subsequent formulations, to later on find out that the paper is not doing a single token analysis, is confusing. The use of the “monosemanticity” term (more below) was also confusing. This is a pattern where the paper seems to reuse ideas from NLP without critically thinking how that matches the problem at hand. I think there is a unique opportunity to highlight the challenges in this domain, inspired by success in NLP, but without suggesting that the same thing is happening.

I appreciate the clarification about PCA being a linear transformation too and the detailed discussion about optimizing L2 vs a weighted L2. I agree with the authors that this is very unique and it is an important fundamental question to address. However, this is not clearly motivated in the paper beyond “they improve the scores”. This could be an example where differences in domain may result in rich opportunities to advance the field as mentioned above.

Writing style:

The authors could have simply dived in to why dictionary learning is desirable.

While we appreciate the reviewer's constructive feedback regarding the paper's writing, we would like to emphasize that our motivation for using deep learning stems directly from the superposition hypothesis. Furthermore, the field of causal representation learning addresses a similar problem to this paper but adopts a very different approach, and indeed some papers have already begun to make formal connections between the two areas [ Rajendran et al 2024, “Learning interpretable concepts: Unifying causal representation learning and foundation models”].

Contribution of individual experiments to the story line:

Also for explaining results, I sometimes found myself lost in why each of the experiment is important and how it ties to the central message of ICFL (e.g., Fig.2b, Fig.2C, Fig.3f, Fig.3g).

We would like to refer the reviewer to the paragraphs Positioning of the paper and Novelty of the Novelty of the ICFL algorithm in the comment addressing all reviewers.

Switching between different classification tasks in experiments:

Also the authors in the results section switch between different tasks (among the five tasks and experiments) without sufficiently explaining, and this made it hard to follow the paper.

We agree with the reviewer that switching between different tasks can interrupt the flow while reading the paper. We tried to always select the tasks that are most relevant for the specific experiment and we will add additional explanations to better guide the reader through the experimental section.

Technical novelty of the ICFL algorithm:

Technical novelty At the end of the day, ICFL is a variation of Orthogonal Matching Pursuit and also restricted to L0 recovery methods, which has been around since early 1990s.

We would like to refer the reviewer again to the sections Positioning of the paper and Novelty of the ICFL algorithm in the response to all reviewers. While the algorithm itself is indeed a variation of the orthogonal matching pursuit algorithm, the novelty lies in the usage of this algorithm to extract features from large scale foundation models, and especially in the context of biological image data.

Comparison with CellProfiler features:

The reason is that ICFL lags behind on almost all metrics behind the original embedding baseline and CellProfiler-derived baselines - As a reader/possible user of the algorithm, I find it less convincing to use ICFL, if it is always performing worse than simpler and older approaches.

We would like to put this comment into perspective as we believe that the results in Figure 3 are a very strong motivation for our approach. CellProfiler features have been developed by domain experts, and thus are building upon decades of biological research. They are (relatively) interpretable, but they are significantly outperformed by black-box deep learning approaches on recall of biological relationships [Kraus et al 2024, others]. ICFL extracts features in an unsupervised manner (with the only signal being used are labels for unperturbed control cells) from a self-supervised trained foundation model. Because these features are extracted in an unsupervised fashion, they can be used complementary to CellPaint features.

Questions:

In line 319, why use center crop, instead of the whole image?

The MAE requires us to divide the entire image into crops of specified size, divisible by the token size of 8x8 pixels. There is no particular reason why one could not use the tokens aggregated over all crops per image instead of the center crop only.

Benchmarking of ICFL:

Their method appears to very effectively achieve this goal (Table 1); however, this contribution would be more significant if the authors included an evaluation on additional datasets, such as those used in Gao et al.

We agree with the reviewer, but view such an analysis outside of the scope of this paper. We would like to refer the reviewer to the section Benchmark comparison of ICFL in the comment addressing all reviewers.

Confirmation bias of qualitative analysis:

By visualizing a heat map of the inner product of individual tokens with the learned feature, they conclude that specific cells within each image have escaped the perturbation, exhibiting normal phenotypes, and correspondingly are not highlighted in the heat maps. However, this qualitative analysis could inadvertently suffer from confirmation bias.

While we generally agree that there are serious risks of confirmation bias with any qualitative output of an interpretability method so it is good to be wary, we disagree with this characterization. The fact that some of the cells escaped the perturbation is evident from the images themselves, and the fact that the inner product is significantly smaller for these cells is evident from the heatmap. With the qualitative analysis we demonstrate that we can find features that capture highly non-random patterns, similar to the ‘’golden gate bridge’’ feature extensively discussed in [1].

Extensions of the qualitative analysis:

I suggest a more rigorous analysis, such as having a blinded expert identify cells that are not exhibiting an expected phenotype for a genetic perturbation. This would produce a cell-level annotation of outlier cells, amongst other cells that are exhibiting the intended phenotype. Based on this, it would be interesting to evaluate the correspondence of token-level heat maps with this annotation.

We thank the reviewer for sharing this advanced and intriguing approach for evaluating the features. We took a major step towards addressing your concern in section Interpretability analysis in the response addressing all reviewers. Furthermore, we strongly believe that developing robust processes for assessing the biological value of features extracted from foundation models along the lines of what the reviewer described, and especially ones that involve domain experts, is a very valuable and important contribution on its own.

The clarity of Figure 2 could be improved. Here are some suggestions:

We thank the reviewer for the suggestions and will implement them in a revised version of this paper.

Questions:

I was confused by the histograms in Figure 3 a-d. In particular, for some genetic relationships (a,c) the result of paired genetic perturbations was to reduce the mean cosine similarity, yielding a negative value. Is this expected for these specific gene pairs? It would be helpful to include additional discussion on the expected cosine-similarity for each pairs of perturbations.

We plot the correlation of tokens from specific genetic perturbations with selected feature directions, showing that the feature directions can separate the distribution from the distribution of all tokens. We would like to stress that the features learned by the ICFL algorithm can be both positive and negative, and the algorithm has no specific preference towards positive values (in strong contrast to TopK SAE). Indeed, one can simply replace the feature directions of the ICFL algorithm w_i with -w_i (Section 4) with the only impact that z_i becomes -z_i, however, without changing the reconstruction \hat x_i. May we ask the Reviewer whether this explanation answers their question?

Novelty of the ICFL algorithm

To extract features from our foundation model, we use TopK-SAE, which is a SOTA technique proposed in the mechanistic interpretability community. Furthermore, we find that our own algorithm, ICFL, which is indeed just a variant of the orthogonal matching pursuit yields substantially better features. Since we believe that this finding may be of independent interest for the mechanistic interpretability community, we mention it as a separate technical contribution. Different reviewers raised concerns about the novelty of the ICFL algorithm:

Technical novelty At the end of the day, ICFL is a variation of Orthogonal Matching Pursuit and also restricted to L0 recovery methods, which has been around since early 1990s. With respect to machine learning methodology, the technical novelty of the paper is somewhat limited. The overall problem and approach follows TopK, except rather than optimizing the specific TopK formulation, an orthogonal matching pursuit approach is used instead.

We would like to put these comments into perspective and stress that most well-known results in mechanistic interpretability use TopK-SAE [2] or L1-SAE [2] which were both already well known [4] algorithms. However, while these methods are simple to implement in deep learning frameworks, it is not obvious that they lead to good disentanglement results (especially in the context of images where there have been fewer successful applications), and our results show you can get significantly more disentangled transformer representations with matching pursuit. To ensure that this point is clear, we will add 2 lines to the revised version of the paper.

Benchmark comparison of ICFL

Reviewers SfZF and 2RjZ also raised concerns about the scope of the benchmark comparison against TopK-SAE:

Their method appears to very effectively achieve this goal (Table 1); however, this contribution would be more significant if the authors included an evaluation on additional datasets, such as those used in Gao et al. The authors restrict the analyses on MAE-pretrained ViTs, whereas if the authors also perform similar analyses on other types of SSL-pretrained ViTs for this domain, I think that would strengthen the argument that ICFL is robust. ICFL was shown to be superior to TopK on some quantitative measures, but these measures are proxies and not an end goal in themselves. Qualitative results were only given for ICFL. Was any qualitative analysis done on the TopK learned features?

In this paper, we favor an in-depth analysis of features extracted from one specific model over multiple models. This is common in the existing literature (see e.g., [2,3]) and has two reasons: the first one is the access to such models, the corresponding datasets and the necessary infrastructure to run these experiments. We highlight here that many works e.g., asdf use proprietary models. The second reason is that assessing the quality of the extracted features is a highly domain-specific task. In this paper we focus on multi-cell image data, and propose an evaluation strategy specifically targeted for this data modality.

To further comment on the metrics used in this paper to compare TopK-SAE with ICFL, we use the number of dead features (Table 1), the linear probing scores (Figure 2), the selectivity scores (Figure 2 and 3), and the reconstruction error (Figure 2 and 7). These scores are similar to the ones used in [2] to evaluate SAEs, with the addition of the selectivity score. Regarding a qualitative analysis comparing ICFL with TopK-SAE, we do not think that such an analysis is the right approach for comparing the two algorithms; confirmation bias is risky in any interpretability work, and this is especially true in cell imaging where the differences between images can be extremely subtle and hard to interpret, even for human experts. We would like to stress that the main purpose of the qualitative analysis in Section 7 is to illustrate striking patterns captured by the features and therefore motivate the potential of this line of research as a whole.

[1] Kraus, Oren, et al. "Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.

[2] Gao, Leo, et al. "Scaling and evaluating sparse autoencoders." arXiv preprint arXiv:2406.04093 (2024).

[3] Bricken, Trenton, et al. "Towards monosemanticity: Decomposing language models with dictionary learning." Transformer Circuits Thread 2 (2023).

[4] Makhzani, Alireza, and Brendan Frey. "K-sparse autoencoders." arXiv preprint arXiv:1312.5663 (2013).