Topological Signatures of Adversaries in Multimodal Alignments

We thank the reviewer for their positive feedback.

In response to the reviewer's query about attacks on text modality, we conducted additional experiments. In particular, we studied the confusion prompts (also referred to as cross-class prompt injection attacks) [Refs. 1 and 2 below]. We tracked the changes in Total Persistence (TP) and Multi-scale Kernel (MK) losses as adversarial text was incrementally injected into the data batch (Adv. ratio) (similer to the experiments shown in the first two columns of Figure 1 for images) using three different CLIP configurations (RN50, ViT-B/32 and ViT-L/14) on the ImageNet dataset. These losses were computed between the two modalities (Equations (3) and (5) of the paper). Examples of typical benign and adversarial text are:

• Benign: "a photo of an apple" (Prediction: "apple")
  Adversarial: "a photo of an apple that resmbles an aquarium fish" (Prediction: "aquarium fish")

• Benign: "This is a castle" (Prediction: "castle")
  Adversarial: "This is a castle that mimics a baby" (Prediction: "baby")

(Here, the prediction "apple" means the CLIP will point the user to a set of images with label apple in the ImageNet dataset)

Interestingly, we obtain results consistent with our prior findings in images: Both losses are monotonically changed as more adversarial text are injected into the data batch:

We chose not to include these results in our initial submission for three main reasons. First, we believe the paper is already rich with the presented analyses. Second, a comprehensive examination of the signatures associated with textual attacks would require more thorough investigations due to the variety of attack methods and text-encoding modules, making it more appropriate for future dedicated research. Third, from a detection perspective, given that adversarial attacks significantly alter the original textual content, topological signatures might not be necessary for effective detection of malicious interventions. If the reviewer believes that including these additional results would strengthen the scientific contribution of the paper, we will gladly incorporate them into our final manuscript.

We hope the reviewer finds that the manuscript is thorough, well-rounded, and deserving of a higher evaluation than a weak accept.

---

References:

[1] Maus, Natalie, et al. "Black box adversarial prompting for foundation models." arXiv preprint arXiv:2302.04237 (2023).

[2] Xu, Yue, and Wenjie Wang. "LinkPrompt: Natural and universal adversarial attacks on prompt-based language models." In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024.

We thank the reviewer for the careful examination and thoughtful feedback. We address the following 3 reviewer's concerns:

Q1: Number of adversarial and the size of the hold-out data

We agree they are critical factors:

1. Number of adversarial samples:
   Detection performance depends more on the absolute number of adversarial samples ($|Y|$ in Eq. 6) than on their ratio. Increasing the hold-out dataset size ($|Z|$) (effectively decreases the ratio) improves detection by better modeling the logit space.

2. Hold-out dataset size:
   The size of the hold-out dataset should be sufficiently large to accurately represent the topological structure of the logits. Typically, this size is determined by the dimensionality of the logits—10 for CIFAR-10 and 1000 for ImageNet. We find $|Z| = 1000$ is sufficient for CIFAR-10 (100 points per label). For ImageNet, we chose $|Z| = 3000$ to ensure at least 3 samples per class. However, as demonstrated below, even $|Z| = 1000$ (1 image per class) can be sufficient.

We provide additional experiments using CLIP RN50 with FGSM and PGD to illustrate the above claims: https://docs.google.com/drawings/d/e/2PACX-1vTtwOJT6miiHXree45vV0plXMhUj3z5kwflHhQMSXZWT-kd1ssTaYofZ6OgYrmu9jL9waulVseAtQT9/pub?w=960&h=720

As reported in our paper, test power reaches near 1 at $|Y| = 100$ with $|Z| = 1000$. Furthermore, while increasing $Z$ helps for lower $|Y|$ (40–60), the gain diminishes as $|Y|$ grows.

Q2: Adaptive attacks

Since Reviewer M3Mh (Q1) raised a similar interests, we provided a detailed response in that thread. We summarize the experimental results of an attacker with full gradient knowledge in the following. The attacker enhances FGSM with topological gradients. Specifically, we integrate standard FGSM gradients ($\nabla_{\text{FGSM}}$) and $\nabla_{\text{TC}}$ as follows:

$x_{\text{adv}} = x + \epsilon \left[ (1 - \alpha)\ \text{sign}(\nabla_{\text{FGSM}}) - \alpha\ \text{sign}(\nabla_{\text{TC}}) \right]$

Results with $\epsilon = 4/255$ on CLIP ViT-B/16 in ImageNet:

The results indicate that possessing complete gradient information ($\nabla_{\text{TC}}$) allows attackers to circumvent our detection. Nevertheless, realizing the above attack is generally not feasible in practice due to:

• Unknown $Z$: Attackers can't reliably match the detector’s randomly chosen hold-out set.
• High Cost: Gradients from Eq. (6) are expensive to compute, especially for iterative attacks.
• Loss Specificity: Evasion for one detector (e.g., Total Persistence) may fail against others (e.g., Multi-scale Kernel).

Q3: Topological signatures of FGSM

We agree that FGSM exhibits distinctive behaviors compared to more complex attacks. We elaborate this discussion with the following assumption and observation concerning FGSM:

• Assumption (A1): FGSM, being a relatively simple, generates adversarial logits that less closely resemble the logits of the target class.

• Observation (O1): Although the Total Persistence (TP) measure isn't always strictly monotonic, FGSM typically exhibits an initial steady increase followed by a slight decrease (Figures 10, 11, 13, and most cases in 12).

We argue that (A1) explains (O1), which is different from the monotonic of more complex attacks. Formally, (O1) means the TP of a adversarial-only point cloud is smaller than TP of mix point cloud in FGSM, i.e., $TP(adv) < TP(mix)$. On the other hand, $TP(adv) > TP(mix)$ for more complex attacks. Our explanation is:

• Assuming we have $K$ labels, the original clean logits point cloud has $K$ clusters. In more complex attacks, the adversarial logits are also that $K$ clusters but have slightly different structures. As the adversarial ratio increases, the $K$-cluster clean point cloud gradually shifts to the $K$-cluster adversarial point cloud. Thus, the TP gradually changes monotonically from $TP(clean)$ to $TP(adv)$.
• However, due to (A1), we can think of the mixture point cloud of FGSM has $2K$ clusters ($K$ for clean and $K$ for adversarial). As the adversarial ratio increases, the $K$-cluster clean point cloud first becomes a $2K$-cluster point cloud (which can have significantly high TP), then becomes the $K$-cluster adversarial point cloud. This explains why $TP(adv) < TP(mix)$ for some intermediate mixtures. Nevertheless, we expect this distinctive behavior of FGSM can be better detected by higher-order homology.

---

We will include the above discussion in our final manuscript if the reviewer finds them beneficial.

We thank the reviewer for careful feedbacks. We address the concerns of the reviewer via the 3 following topics:

Q1: Adaptive attack against topological-based detection

Due to the broad landscape of adaptive attacks, we focus this discussion on gradient-based attacks against: an attacker requires the gradient with respect to the topological signatures utilized by our detector (Equation 6). We first discuss why obtaining these gradients is challenging:

1. Lack of knowledge of the hold-out $Z$:
   An attacker does not typically know the specific hold-out dataset $Z$ used by the detector. Each time the detector runs, it can select a different $Z$ detection. Thus, gradients used by the attacker using a different hold-out set can be significantly different.

2. Computational cost:
   Even with complete knowledge of $Z$, accurately computing the gradients from Equation (6) is computationally expensive for attacker—especially for iterative attacks. This complexity is from the construction of the Vietoris–Rips (VR) complex. While the detector computes this once, iterative attacks require repeated computations. Note that the attacker also need to back-propagate that gradients on the logit to the input for adversarial perturbations.

3. Loss-specific gradients:
   The attacker’s gradients must be specific to the loss used by the detector. Thus, gradients for a Total Persistence-based detector may not necessarily enable the attacker to bypass another detection, such as a Multi-scale Kernel detector.

Impact of complete knowledge:

Nevertheless, we agree that it is important to study an attacker having complete gradient. Due to the computational impracticality of iterative attacks (point 2), we simulate the attacker using FGSM. We enhance the standard FGSM gradient ($\nabla_{\text{FGSM}}$) with our topological loss gradient ($\nabla_{\text{TC}}$ back-propagate from Equation (6)):

$x_{\text{adv}} = x + \epsilon \left[ (1 - \alpha)\text{sign}(\nabla_{\text{FGSM}}) - \alpha\text{sign}(\nabla_{\text{TC}}) \right]$

Here, α is a parameter balancing the standard and topological gradients. Results with $\epsilon = 4/255$ on CLIP ViT-16/B in ImageNet are:

As observed, having complete gradient helps the attacker in bypassing our detection, though it comes with a reduction in attack effectiveness.

Recommendations for robust detection:

• Utilizing a larger hold-out dataset.
• Combining multiple detection methods simultaneously.

Q2: The usage of higher-dimensional topological features

During our research, we form a hypothesis explaining why higher-degree summary is less effective for adversarial detection. Specifically, we observe that critical distinctions between adversarial and non-adversarial configurations primarily arise from lower-degree homology, particularly degree-0, rather than higher-degree topological features such as loops (degree-1) or cavities (degree-2).

This limitation of degree-1 homology can be shown via our PCP modeling. For example, as shown by the 3 configurations on the right side of Figure 4, when the filtration radius approaches half the distance between vertices of a simplex, a prominent one-dimensional hole emerges across all cases. As this degree-1 feature appears uniformly, it provides limited discriminative power.

However, we also observe some unique situations where higher-order topological information helps. That is the situation with FGSM (detailed discussion is in Q3 of Reviewer 9vo5). We argue that, due to the simplicity of the attack, the resulting logits introduce new clusters (rather than being near existing clusters of the clean data). As the occurrence of those clusters can be better detected by higher-order summaries, we expect they can help detect that situation.

Q3: Complexity and real-time application

Compared to benchmark (Gao, 2021), our approach introduces additional complexity due to the integration of topological data analysis (TDA). However, our Maximum Mean Discrepancy (MMD) detection learning (Line 376) operates on logis rather than input data, significantly reducing complexity due to the lower dimensionality of logits. Specifically, in CIFAR10 (about 1,000 points), the additional computational cost from TDA involves 2 VR complex computations, taking around 0.5 seconds on our hardware. In comparison, the training time for MMD is approximately 10–20 seconds. At inference, the combined time for TDA computation and MMD is about 0.5 seconds per sample. For ImageNet (about 3,000 points due to larger hold-out requirements), the additional time introduced by TDA during training is about 10 seconds, which is still lower than MMD training.

We thank the reviewer for their positive feedback and insightful comments. We fully agree with the reviewer’s suggestions, including:

1. Explicitly mentioning that the Multi-scale Kernel (MK) loss is quadratic,
2. Replacing the term "homologies" (line 200) with "topological summary" to avoid confusion,
3. Addressing the identified typos and grammatical errors.

We now address the specific questions raised by the reviewer:

---

Q1: PCP analysis on Multi-scale kernel (MK) Loss

We performed a PCP analysis on the MK loss, following the same setting used for Total Persistence (TP) in Figure 5. The results can be found here: https://drive.google.com/file/d/1Es536XanO-4MTSgSzimci3NoidjvmzkC/view?usp=share_link

From the results, we observe that the MK loss landscape (analogous to the TP landscape in Figure 5) exhibits less monotonic behavior. While the MK loss generally decreases with increasing $\alpha_s$, it first increases and then decreases as the bias ratio $r$ increases. We hypothesize that this non-monotonicity may account for the observed flips in MK loss under certain experimental settings.

Although MK loss is less monotonic than TP, it may serve as a more effective topological summary for adversarial detection in settings where the spatial distribution of persistence points carries important information—something MK captures and TP does not.

Q2: Hypothesis for the flip in MK loss (Figure 3, Row 4)

As discussed in Q1, the MK loss can exhibit both increasing and decreasing trends with respect to the bias ratio $r$, in contrast to Total Persistence (TP), which typically decreases more consistently (see Figure 5).

This suggests that when clean logits shift toward adversarial logits—corresponding to decreasing $\alpha_s$ and decreasing $r$ in the PCP model—the MK loss may behave non-monotonically.

We hypothesize that the decrease (or flip) in MK loss observed in Figure 3 (row 4, for CLIP under AutoAttack) arises because the clean and adversarial logits fall within a region of the MK landscape where the loss decreases as the shift occurs from clean to adversarial. This implies that the behavior of MK loss is more sensitive to the geometric positioning of the logits within the manifold, which is inherently influenced by the dataset and the specific attack method.

Q3: Use of higher-dimensional topological features

Through our investigations, we hypothesize that higher-degree homology (e.g., loops or cavities) offers less utility in adversarial detection under the zero-shot setting.

We find that lower-degree homology, especially degree-0, captures the most discriminative structure between clean and adversarial configurations. For example, in our PCP analysis (Figure 4, right side), when the filtration radius reaches half the inter-point distance in a simplex, a degree-1 hole (loop) emerges uniformly across all three configurations. This uniformity reduces its discriminative power. Thus, we believe that focusing on degree-0 information is more effective in our examined tasks.

However, we also observe some unique situations where higher-order topological information helps. That is the situation with FGSM (detailed discussion is in Q3 of Reviewer 9vo5). We argue that, due to the simplicity of the attack, the resulting logits point clouds introduce new adversarial clusters (rather than being near existing clusters of the clean data). As the occurrence of those clusters can be better detected by higher-order summaries, we expect they can help detect that situation.

Q4: Attacking methods targeting the entire input

We agree this is a very interesting idea. Given recent advances in TDA, there are promising tools for constructing topological features from raw input data. However, at this time, we are not aware of a straightforward way to design such an attack.

Q5: Natural interpretation of TP Loss analogous to MK in MMD

At present, we do not have a natural interpretation of the Total Persistence (TP) loss similar to how MK loss aligns with Maximum Mean Discrepancy (MMD), but we suspect that such an interpretation actually does not exist. The reason is that the multi-scale kernel was originally created (Reininghaus, 2014) to solve exactly this problem of Wasserstein distance (and Total Persistence as a consequence) not embedding naturally into a Hilbert space.

We thank the reviewer for their positive feedback and careful examination of the manuscript. We agree with the reviewer’s comments regarding grammatical issues and typos, particularly the corrections at Line 104 and in Algorithm 1 (indeed, $\nabla_X$ should be corrected to $\nabla_Y$).

We now address the following questions of the reviewer:

Q1: Vectorization of persistence diagrams

Q2: Meaning and usage of higher-degree homology

Q3: Usage of Alpha complex

We appreciate these insightful questions, as they are natural considerations of our work. In particular, our research had considered them and we later found out that (perhaps counter-intuitively) these three suggested approaches were less effective for the adversarial detection task presented in our paper.

---

Q1 and Q2:

We form the following two hypotheses during our research, which explain why the current vectorization approach and the usage of higher-degree homology is less effective:

• Hypothesis H1 (Addressing Q1):
  The main difference between persistence diagrams (PDs) of clean and adversarially perturbed point clouds predominantly appear in small (often called noisy) features, those located near the diagonal with short lifespans, rather than in dominant, long-lived topological features. Under H1, current vectorization methods (commonly emphasize persistent, large-scale features) will fail to capture subtle yet crucial signatures induced by adversarial.

• Hypothesis H2 (Addressing Q2):
  These critical distinctions mainly arise at low-degree homology groups (especially degree 0), rather than higher-degree homology. Higher-degree features, such as loops (degree 1 homology) or cavities (degree 2 homology), tend not to offer substantial discriminative information for adversarial detection (except for some less common cases as FGSM, which we will discuss later).

Supporting evidence for H1:

Empirical evidence supporting H1 can be observed by examining the Total Persistence (TP) with different choices of order parameter $\alpha$ (as defined in Equations (2) and (3)). Specifically, a smaller order (e.g., $\alpha = 1$) emphasizes dominant components of the PD, whereas larger $\alpha$ more evenly balance contributions from both dominant and small components. Our results (first column of Figure 1 and the corresponding Section 5) show that using a larger $\alpha$ leads to a clearer monotonic change in TP, translating into significantly better detection performance. Nevertheless, we refrain from further increasing $\alpha$, as doing so might violate the stability property of Total Persistence, as discussed by Divol (2019).

Supporting evidence for H2:

Regarding H2, the highest-degree homology we tested is degree 1, which captures one-dimensional holes. Figure 4 employing PCP modeling adversarial logits can be used to illustrate the limitation of degree-1 summary compared to degree-0. As shown in the 3 plots on the right of Figure 4, when the filtration radius approaches haft the distance between vertices of the simplex, a dominant 1-dimensional hole appears across all configurations. Thus, this feature contains less discriminative power for differentiating those configurations. Our detection experiments further reinforce this claim, demonstrating that homology of degree 0 contributes more to the detection.

However, we also observe some unique situations where higher-order topological information helps. That is the situation with FGSM (detailed discussion is in Q3 of Reviewer 9vo5). We argue that, due to the simplicity of the attack, the resulting logits point clouds introduce new adversarial clusters (rather than being near existing clusters of the clean data). As the occurrence of those clusters can be better detected by higher-order summaries, we expect they can help detect that situation.

---

In conclusion, we find that the combination of Total Persistence using the Wasserstein distance, with an emphasis on small-scale features (appropriately chosen $\alpha$), and low-degree homology (specifically degree 0) constitutes a "sweet spot" for adversarial detection tasks.

---

Q3:

The Alpha complex is unsuitable for our problem because it does not scale well to high-dimensional data. Specifically, the dimension of the logits ($X$ and $Y$ discussed in Section 3.1) equals the number of labels predicted by the model, ranging from 10 for CIFAR10 to 1000 for ImageNet. Constructing the Alpha complex typically relies on Delaunay triangulation, which has a complexity of $O(n^{d/2})$, making it computationally infeasible for our scenario.

In contrast, constructing the Vietoris-Rips (VR) complex only requires an adjacency matrix, independent of the data dimension, which is crucial to scale up the experiment to ImageNet. This key difference is the primary reason we prefer the VR complex over the Alpha complex in our work.