Manipulating Feature Visualizations with Gradient Slingshots

We thank the reviewer for their thorough and constructive feedback. We are glad they found the Gradient Slingshots (GS) attack to be novel and difficult to detect, and appreciated the thoroughness and convincing nature of our experimental results. Below, we address the reviewer’s concerns point by point.

Weaknesses

W2: Lemma 3.1, used as the theoretical basis for the method, is a very mundane result that gradient descent (see Q1) on Euclidean distance will converge. The most novel part of the result is establishing that $\mathbb{M} = \mathbb{T_{\text{B, L}}}$, but the proof uses that as a condition instead of emphasizing that fact.

We clarify that since we control how the manipulation set $\mathbb{M}$ is sampled, we explicitly define it as $\mathbb{T_{\text{B, L}}}$; the non-trivial aspect lies in ensuring that each step $q^{(i)}$ during optimization remains in $\mathbb{T_{\text{B, L}}}$, which is guaranteed for gradient descent but not necessarily for other optimizers. This point is already noted in lines 149–151, where we explain that while Lemma 3.1 formally applies to gradient descent, our empirical results indicate that similar behavior extends to other Feature Visualization (FV) optimization strategies.

W3: Few details are given for the hyperparameter tuning process besides that it was computationally intensive. Was it done via grid search on a validation set? How many models needed to be trained during tuning?

We thank the reviewer for raising this concern. A detailed description of our hyperparameter tuning and selection procedure is provided in our response to Question 3 below, and we have incorporated this explanation into the revised paper.

Questions

Q1: I think Eq. 4 needs to be updated to use the negative square Euclidean distance instead of the positive distance. Currently the gradient with respect to Eq. 4 would result in times Eq. 3.

We thank the reviewer for pointing out this oversight. We have corrected the Eq. 4 accordingly. We have also updated Equation 10 to fix the same issue. We emphasize that this correction only affects the presentation in the manuscript; the code implementation already uses the correct formulation and remains unaffected.

Q2: How were the standard deviation values calculated? Multiple trained models? Bootstrapping?

As detailed in Appendix C.8, the reported standard deviations are computed over multiple runs of FV generation. Specifically, we generated 100 samples per model (30 for CLIP ViT-L/14 due to computational constraints), and computed similarity metrics over these samples. Since FV generation involves sampling the initialization point from a distribution, each run yields a slightly different result. The standard deviations thus characterize the consistency of the metrics across these independently generated FVs. We have incorporated this clarification into all relevant table captions to prevent any confusion.

Q3: As detailed in W3, how was hyperparameter tuning performed?

For the parameter $\alpha$, our tuning process follows a coarse-to-fine iterative approach. We begin with a broad sweep of values (e.g., $\alpha \in [0.01, 0.05, 0.5, 0.9]$) to identify a promising region. Once a candidate such as $\alpha = 0.5$ shows good results, we perform a finer-grained search around it (e.g., 0.4 and 0.6), and continue refining around the best-performing values, e.g., near 0.6.

Importantly, we perform tuning using only 1–3 epochs and on small subsets of large datasets (see Table 8 in the Appendix). For ViT experiments, for example, we use less than 1% of ImageNet, with a single epoch taking under 5 minutes. While we acknowledge that tuning of $\alpha$ is computationally intense, the process remains practical even with modest resources.

For the parameter $w$, we set it to 0.1 when the target layer contains 10 neurons, and to 0.01 when it contains 1000 neurons, preventing the target neuron's activation landscape from being overly disrupted. For $\gamma$, we use the heuristic $\nabla (\phi \circ \eta)(\mathbf{0}) = \gamma \mathbf{q^t}$ to preserve the gradient magnitude in the initialization zone.

We have incorporated this explanation into the paper in order to improve transparency around hyperparameter selection.

Please let the authors know whether their rebuttal has adequately addressed your concerns. If any issues remain, please communicate your specific, unresolved concerns as soon as possible to ensure timely discussion.

We thank the reviewer for their clear summary and thoughtful feedback. We are glad they found our paper well-written, our idea novel, and appreciated the breadth of model choices and ablations. Below, we address the reviewer’s concerns in detail.

Weaknesses

1. The paper only focuses on last layer features. It does not conduct any experiments on features in middle layers.

While Section 4 primarily focuses on output-layer features to enable direct quantitative evaluation (e.g., via AUROC against known ground-truth labels), our Gradient Slingshots (GS) method is not limited to the final layer. The general formulation in Section 3 applies to any differentiable feature function within the network.

Moreover, the submitted version of our paper already includes a case study involving the manipulation of a mid-layer feature (Section 5). Specifically, we target a neuron in the 22nd residual block of a CLIP ViT-L/14-based model, identified through a supervised discovery method. Both quantitative results (Table 3) and qualitative visualization (Figure 1) demonstrate that GS manipulation is effective when applied to middle layers, and preserves the feature performance.

2. Hyperparameter tuning of loss seems very hard as both large and small values lead to the method not working.

We thank the reviewer for raising this concern. For the parameter $\alpha$, our tuning process follows a coarse-to-fine iterative approach. We begin with a broad sweep of values (e.g., $\alpha \in [0.01, 0.05, 0.5, 0.9]$) to identify a promising region. Once a candidate such as $\alpha = 0.5$ shows good results, we perform a finer-grained search around it (e.g., 0.4 and 0.6), and continue refining around the best-performing values, e.g., near 0.6.

Importantly, we perform tuning using only 1–3 epochs and on small subsets of large training datasets (see Table 8 in the Appendix). For ViT experiments, for example, we use less than 1% of ImageNet, with a single epoch taking under 5 minutes. While we acknowledge that tuning of $\alpha$ is computationally intense, the process remains practical even with modest resources.

For the parameter $w$, we set it to 0.1 when the target layer contains 10 neurons, and to 0.01 when it contains 1000 neurons – preventing the target neuron's activation landscape from being overly disrupted. For $\gamma$, we use the heuristic $\nabla (\phi \circ \eta)(\mathbf{0}) = \gamma \mathbf{q^t}$ to preserve the gradient magnitude in the initialization zone.

We have incorporated this explanation into the paper in order to improve transparency around hyperparameter selection.

3. The paper claims that internal mechanisms of the model remain unchanged but because they didn’t investigate middle layers their argument is only based on accuracy.

As noted earlier, we explicitly evaluate manipulation on a feature located in an intermediate layer (Section 5, Case Study with CLIP ViT-L/14). In all our quantitative experiments, beyond overall accuracy, we measure the AUROC of the manipulated feature output with respect to its ground truth label. This targeted metric directly assesses whether the feature’s original performance is preserved. Our key argument is that the target feature is the component most susceptible to alteration by our manipulation. The fact that it maintains its original predictive behavior demonstrates that the model’s reliance on this feature remains intact, and that our method effectively “covers up” the FV without removing or degrading its internal role. We have made this argument more prominent and explicit in the paper.

Questions

Is there an experiment that shows the effect of model manipulation on other features Feature Visualization ? Do other features FV also look like the target image?

We understand the reviewer’s concern — it is important to demonstrate that GS does not inadvertently perturb FVs of non-manipulated features, which could make GS detectable. To address this, we evaluated FVs of the first 10 non-manipulated output neurons in a GS-attacked ResNet-50 (Sec. 4, $\alpha=0.64$). For each neuron, we sampled 100 FVs from the original and manipulated models and measured their similarity to a single original FV and the target image.

The results, presented in Table C below, show virtually no difference in the FVs of non-manipulated neurons between the two models across all metrics, confirming that GS leaves non-manipulated features unaffected. We have included these quantitative results, along with example FVs, to the paper’s Appendix.

Table C: Before-and-after-manipulation comparison of visualizations of non-manipulated features in the GS attack on the ResNet-50 model where the “wolf spider” output neuron was manipulated. Reported are the mean ± standard deviation of similarity between FVs and a single sampled original FV (CLIP, MSE, LPIPS), and the mean ± standard deviation of similarity between manipulated FV and the target image. Similarity statistics are computed over 100 independent FV runs. These results demonstrate that the GS attack does not perturb the visualization of non-manipulated features.

Please let the authors know whether their rebuttal has adequately addressed your concerns. If any issues remain, please communicate your specific, unresolved concerns as soon as possible to ensure timely discussion.

We thank the reviewer for their thoughtful feedback. We’re pleased they found our method novel and simple, our paper well-written, and appreciated that our attack is difficult to detect. Below, we address their concerns to clarify that our results are comprehensive and not selectively presented, provide additional quantitative evidence, discuss theoretical assumptions, and elaborate on broader applicability.

Weaknesses

W1: The evaluation is only conducted on one or a few neurons, so it might be subject to cherry-picking.

We thank the reviewer for raising concerns about cherry-picking. To address this, we further manipulated the first 10 output neurons in the ResNet-50 setting from Sec. 4 (target image: Dalmatian). Beyond standard similarity metrics, we added two semantic alignment metrics:

• Target Label Alignment ("Target Lbl."): CLIP cosine similarity between manipulated FV and the target label embeddings (e.g., “an image of a Dalmatian”).
• GT Label Alignment ("GT Lbl."): CLIP similarity between manipulated FV and the ground-truth feature label (e.g., “an image of a wolf spider”).

Results from Table A (only the first 5 neurons shown due to character limits) show that manipulated FVs are similar to the target image and align more with the target than ground truth label; classification accuracy and AUROC indicate the stability of model and feature performance. This supports that our method generalizes beyond isolated neurons.

Table A: Manipulation results for the Gradient Slingshots (GS) attack on the first 10 output neurons of a ResNet-50 (partial, 5/10 neurons).

W2: The quantitative metrics are not impressive. I am suspicious that GS works on only a few examples selected for the paper illustrations, [...] It is even more true for ViT (see Appendix, p. 32), for which the metrics barely change compared to the FV without GS.

We thank the reviewer for raising the concern. As noted in Sec. 4 and Appendix B, low-level metrics (MSE, SSIM, LPIPS) are more useful in settings with low-res datasets (e.g., MNIST, CIFAR), where manipulated FVs resemble targets pixel-wise. In high-res settings, these metrics fail to capture compositional or semantic alignment (see Fig. 3). CLIP similarity better reflects high-level alignment but may still miss compositional effects.

To address the selective representation concerns, we added multiple samples of manipulated FVs per evaluation. We also extended all quantitative evaluations using Target/GT Label Alignment metrics, capturing semantic similarity. Table B (extending Table 16) for ViT-L/32 shows that at some $\alpha$ values, FVs align more with the target image label (“sea lions”) than the ground truth (“broccoli”), supporting GS’s manipulation efficacy on transformer models.

Table B: Accuracy–manipulation trade-off for the GS attack on the ``broccoli'' output neuron in ViT-L/32 (partial Tab. 16).

W3: The assumption of l. 139 for Lemma 3.1 seems to make GS artificial. As far as I understand, it means that the initialization must be known by the manipulator and fall in. It seems very easy to circumvent for an auditor, for instance, by uniformly sampling several initializations in $Q$.

We thank the reviewer and agree the assumption in l.139 may appear strong. We provide further explanation for why this assumption is reasonable in our response to Q1.

The assumption of l. 139 for Lemma 3.1 […] should appear in the Lemma. The authors spend time on the proof of Lemma 3.1, which is intuitive, […].

We thank the reviewer for the suggestion. The assumption is now stated explicitly in Lemma 3.1, and the detailed proof has been moved to the Appendix. The freed space was used to clarify evaluation metrics and hyperparameter selection.

Questions

1. How can a manipulator make sure that the FV initialization will fall into and will be appropriately steered towards the wrong optimum?

We thank the reviewer for the question. Standard FV libraries (Lucid, Lucent, Horama) use fixed Gaussian noise initialization, making the process predictable. While auditors could randomize or broaden initialization, adversaries can expand the “slingshot zone” $\mathbb{B}$ accordingly. Alternative methods (e.g., seeding from real images) are rare since they often yield FVs resembling the seed (Hamblin et al., 2024).

• Hamblin et al. “Feature Accentuation: Revealing 'What' Features Respond to in Natural Images.”

2. How does GS affect the FV of other features? If the FV is pertubated for every neurons, it might seem suspicious to auditors.

We appreciate the reviewer’s concern and agree it’s important to show GS doesn’t alter FVs of non-manipulated features. To address this, we evaluated FVs of the first 10 non-manipulated output neurons in a GS-attacked ResNet-50 (Sec. 4, $\alpha=0.64$). For each neuron, we sampled 100 FVs from the original and manipulated models and measured their similarity to a single original FV and the target image.

Table C (first 5 neurons shown due to length limits) shows that non-manipulated neuron FVs remain nearly unchanged, confirming that GS preserves non-manipulated features. Qualitative and quantitative results are now included in the Appendix.

Table C: Before-and-after-manipulation comparison of visualizations of non-manipulated features in the GS attack on the ResNet-50 model (partial, 5/10 neurons).

3. In Sec 5, the manipulation and the FV seem to target the feature vector representing an assault rifle. However, FV and GS are only formulated for the maximization of a single neuron. How to make this fall under a formalism similar to Eq. 1, 2, 3, 4, 5?

We thank the reviewer for highlighting this point. Both the theory and implementation (Eqs. 1–5) define GS manipulation over a feature function $f$, not just individual neurons. As stated in l. 68–70, $f(\mathbf{x}):= \mathbf{v} \cdot g(\mathbf{x})$, where $g(\mathbf{x})$ is a vector of activations and $\mathbf{v}$ defines a feature direction.

4. The FV method from Olah et al. 2016 [1] notoriously fails on modern networks like ViT [2]. Could the authors run GS with modern FV methods like MACO [2]

We thank the reviewer for the suggestion. While we show that standard Fourier FV with an appropriate transformation robustness recipe (Appendix C.7) can produce interpretable results for modern architectures like CLIP ViT-L/14 (Fig. 1), we agree that MACO is a popular FV approach for transformers.

MACO, like Fourier FV (Olah et al., 2017), operates in the Fourier frequency domain, but with a restricted optimization space, making it compatible with GS. We applied MACO to the existing manipulated ViT-L/32 model (Sec. 4), generating FVs before and after manipulation and evaluating alignment to the target and ground truth labels. As shown in Table D, manipulated MACO FVs show stronger alignment with the target class (“sea lions”) than the ground-truth (“broccoli”), mirroring results with Fourier FVs. We have included both quantitative and qualitative MACO results in the revised manuscript.

Table D: Manipulation results of GS applied to MACO visualizations of the “broccoli” output neuron in ViT-L/32.

5. I am surprised that there is no factor in Eq. 10 before , or a factor in Eq. 9, that stems from derivatives of square expressions.

We thank the reviewer for noticing this oversight on our side. We have corrected the Eq. 4 and the Eq. 10 accordingly. This error is limited only to the paper formalisms and not reflected in our code implementation.

We thank the reviewer for their thoughtful and constructive feedback. We are glad that they found the core idea of our method theoretically sound, appreciated the visual presentation and figures, and found the supplementary material helpful. Below, we address the reviewer’s concerns in detail, providing additional experimental results, methodological details, and elaborating on the practical applicability of our method.

Weaknesses

1. Lack of Comparative Analysis: The paper fails to compare GS against any existing baselines. While it suggests that methods like Nanfack et al. might degrade model performance, no empirical evidence is provided to substantiate this claim. […]

We appreciate the reviewer’s concern about the lack of empirical comparison with prior work. While conceptual differences were discussed in Appendix A, we now include a direct qualitative and quantitative comparison with the only comparable fine-tuning-based method: Prox-Pulse (PP) [Nanfack et al., 2024]. This comparison uses a ResNet-50 setting from Sec. 4 (target image: Dalmatian; manipulated neuron: wolf spider). We follow PP’s reported hyperparameters and additionally vary its $\alpha$ to show the trade-off between model accuracy and manipulation result.

Alongside original similarity metrics, we introduce two new semantic alignment measures:

• Target Label Alignment ("Target Lbl."): CLIP cosine similarity between a FV and the target label embeddings (e.g., “an image of a Dalmatian”).
• GT Label Alignment ("GT Lbl."): CLIP similarity between a FV and the ground-truth feature label (e.g., “an image of a wolf spider”).

Results (Table A) show that Gradient Slingshots (GS) yield FVs more semantically aligned with the target image and less aligned with the ground truth label, while preserving model performance better than Prox-Pulse. GS outperforms Prox-Pulse on all similarity metrics except low-level metrics MSE and SSIM.

Table A: Comparison of FV manipulation results on the “wolf spider” output neuron in ResNet-50 between GS and Prox-Pulse (PP).

2. Unconvincing Similarity Metrics: The reported similarity metrics (e.g., MSE and SSIM) between the manipulated FV and the target image do not show statistically significant improvement. […]

We thank the reviewer for raising the concern. Low-level metrics (MSE, SSIM) are more useful in settings with low-res datasets (e.g., MNIST, CIFAR), where manipulated FVs resemble targets pixel-wise more closely (as noted in Section 4 and Appendix B). In high-res settings, these metrics fail to capture compositional or semantic alignment (see Fig. 3). CLIP similarity better reflects high-level alignment but may still disregard compositional structure.

To further support our evaluation results, we added multiple samples of manipulated FVs per evaluation. We also extended all quantitative evaluations using Target/GT Label Alignment metrics described above. These metrics explicitly quantify conceptual alignments that are invisible to traditional similarity scores. Some of the results using these metrics are shown in Tables A, B, C in this rebuttal.

3. Apparent Cherry-Picking of Results: The selection of specific alpha values in Table 1 (e.g., 0.64 and 0.62, which are very close to 0.6) while omitting results for 0.7 or 0.8 suggests a selective presentation of data. […]

The $\alpha$ values in Table 1 were chosen to show representative trade-off states between model performance and FV manipulation success. Within 0.64-0.9, metrics vary marginally, so we did not report these values.

To address the concern, we have now extended Tab. 1 with $\alpha=0.7$ and $0.8$ (see Table B), providing a full sweep from 0.1 to 0.9. These additional results confirm the trends described in Section 4.1. Additionally, in response to reviewer PQ6K, we provide quantitative evidence showing that the neuron targeted for manipulation itself was not cherry-picked by providing evaluation results for 10 more neurons in the ResNet-50 setting. We hope this clarifies our reporting and alleviates concerns about selective presentation.

Table B: Accuracy–manipulation trade-off for the GS attack on the “wolf spider” output neuron in ResNet-50 (partial Tab. 1).

4. Excessive Hyperparameter Tuning: The wide range and extreme precision of alpha values […] strongly indicate a reliance on extensive grid searching to find optimal parameters. […]

We thank the reviewer for raising this concern. A detailed description of our hyperparameter tuning and selection procedure is provided in our response to Question 2 below, and we have incorporated this explanation into the revised paper.

5. Unproven Efficacy on Transformers: Despite being a purported advantage over prior work, the paper does not convincingly demonstrate the methodology's effectiveness on transformer architectures. The extremely high alpha values used for ViT models (e.g., at least 0.9995) suggest that the method may simply be injecting noise, rather than performing a targeted attack.

We thank the reviewer for the observation. $\alpha$ parameter balances manipulation and preservation losses; high $\alpha$ does not indicate noise injection but rather that the manipulation loss remains small relative to activation changes during training.

Figure 1 and Section 5 demonstrate strong efficacy of GS on a ViT model (CLIP ViT-L/14). While quantitative metrics for ViT-L/32 (Appendix D) are less convincing than the qualitative results (Fig. 3), this highlights the limitations of conventional metrics in capturing similarity to the target. To address this, we extended the evaluation with the two new alignment metrics introduced above. Results in Table C show that for certain $\alpha$ values, manipulated FVs align better with the target label (“sea lions”) than the ground truth (“broccoli”), sometimes even surpassing the original FV’s alignment with its ground truth label.

Table C: Accuracy–manipulation trade-off for the GS attack on the ``broccoli'' output neuron in ViT-L/32 (partial Tab. 16).

Questions

1. Could the authors clarify the meaning of the standard deviations reported throughout the paper? […]

As explained in Appendix C.8, the standard deviations reflect variability across multiple FV generations—100 per experiment (30 for CLIP ViT-L/14 due to compute limits). Since FV generation involves sampling initializations, each run differs. The deviations thus capture metric consistency across independent runs. We’ve clarified this in all relevant table captions.

2. Is there a methodology for determining reasonable hyperparameter values without resorting to an exhaustive grid search? […]

Our tuning of the parameter $\alpha$ uses a coarse-to-fine iterative approach rather than an exhaustive grid search. We start with a broad sweep (e.g., $\alpha \in$ 0.01, 0.05, 0.5, 0.9 $$) to identify promising regions, then refine around top candidates (e.g., testing values near 0.5), iteratively narrowing down.

Importantly, tuning is done using only 1–3 epochs on small subsets of large datasets (see Table 8 in the Appendix). For ViT experiments, this involves under 1% of ImageNet and takes less than 5 minutes per epoch, making the process practical even on modest resources.

For parameter $w$, we set $w=0.1$ when the target layer has 10 neurons, and $w=0.01$ when it has 1000 neurons, to avoid excessive disruption of the activation landscape of the feature. For $\gamma$, we use the heuristic $\nabla (\phi \circ \eta)(\mathbf{0}) = \gamma \mathbf{q^t}$ to preserve the gradient magnitude in the initialization zone.

We have incorporated this explanation into the paper.

3. In real-world auditing practices, do auditors commonly rely on FV to assess models?

While FV is not yet standard in traditional audits, it has proven useful for detecting undesirable behaviors (Goh et al., 2021), uncovering backdoor attacks (Casper et al., 2023), interpreting time-series patterns (Schlegel et al., 2024), and understanding CNN filters in materials science (Ling et al., 2017; Zhong et al., 2022). These show FV’s value in the broader context of model interpretability and safety.