VisualTrap: A Stealthy Backdoor Attack on GUI Agents via Visual Grounding Manipulation

W1: The authors don't report the exact number of poisoned data. Since SeeClick pertaining data is 1M, even 5% or 10% of them is non-trivial. This poses questions about whether the proposed attack is realistic.

Thank you for this important clarification request. We would like to clarify that we first sampled 10% of the SeeClick dataset for our experiments, which corresponds to 101,040 samples in total. Among these, approximately 65k are grounding data samples. Specifically, 10% equals 6,551 grounding samples, and 5% equals 3,276 grounding samples. We will include these exact figures clearly in the revised manuscript.

W2: The authors provide a limited explanation of the relatively poor Desktop ASR results in Table 3. If higher resolution is an important factor, I think it should be systematically studied in section 4.2?

Thank you for the valueable suggestion, In response, we conducted additional experiments to systematically study the impact of image resolution on attack performance. Specifically, we resized images size by various scale factors before adding the trigger, allowing us to observe how changes in resolution affect the attack's effectiveness. The results of these experiments are presented in the following figure:

https://anonymous.4open.science/api/repo/2i8k0kl2r1B38/file/scale_factor_study_results.pdf

Our findings indicate that significantly increasing the image resolution can indeed lead to a noticeable decline in attack performance, compared to simply reducing the trigger size. We believe this is because VLMs resize image inputs to fit within their maximum pixels. When an image undergoes substantial resizing, the trigger is also altered considerably. Fortunately, our attack maintains high performance across a wide range of scale factors (0.5-2), demonstrating its robustness under varying image resolutions. We will incorporate these new results and analyses into our revised version.

Thank you once again for your valuable feedback, which has helped us enhance the depth of our analysis

Thanks for the response.

I increased my score to 7.

Q1: Figure 4 seems to be not consistent with the description about LLM component in Line 363-365?

Thank you for pointing out this inconsistency. After reviewing our experimental result, It appears that the positions of "LLM Poison" and "Vision Poison" were inadvertently switched in Figure 4. We will revise the figure to correct these labels and ensure alignment with the description in Lines 363-365

Q2: What are the authors’ thoughts on other potential defense mechanisms? Also, could you elaborate on the input-side filtering methods mentioned in Line 367?

Thank you for your insightful question regarding defense mechanisms, which are indeed vital for the practical deployment of GUI agents. We appreciate the opportunity to elaborate on potential strategies, including input-side filtering methods mentioned in Line 367.

Regarding input-side filtering, we refer to preprocessing techniques applied to GUI screenshots before they are processed by the agent. Specifically, these techniques could include:

• Visual Anomaly Detection: This involves implementing statistical analyses to identify unusual pixel patterns or patches that deviate from standard GUI elements. For instance, techniques such as frequency domain analysis or local variance measurements could be used to detect Gaussian noise patches.
• Patch-based Analysis: This method involves segmenting input images into smaller patches and analyzing each for potential trigger signatures, akin to approaches utilized in computer vision security.

Beyond input-side filtering, we believe additional defense strategies merit exploration:

• Tracking Agent Actions for Suspicious Behavior: Developing mechanisms to monitor agent actions for anomalies, such as verifying the legitimacy of click locations, could enhance security.

In summary, our VisionTrap backdoor introduces two significant challenges that require further defense exploration:

• Identifying Trigger Patterns: While our paper uses Gaussian noise as a trigger, real-world attackers might employ specific shapes (e.g., "+", ".") or text embedded within images as triggers. These elements can appear normal within a UI, making it difficult to detect unusual pixel patterns and ascertain the cause of an attack.

• Detecting Attack Steps: In practical scenarios, GUI agents operate in multiple steps to achieve their objectives. An attacker might inject a trigger at any stage, leading to biased choices or dangerous operations. Detecting threats at each step is inefficient and costly, underscoring the need for new methods or mechanisms that can accurately and efficiently identify attack points.

We appreciate your feedback and look forward to further discussions on enhancing the security of GUI agents.

W1: The paper states that trigger placement is done at a random location, but it is unclear whether the gold location is excluded during this process. If not, this could affect the validity of the results. The statistics on any overlap between trigger and gold locations would be needed.

Thank you for your valuable feedback. We would like to clarify the trigger placement procedure in our experiments:

Downstream Tasks: Since our goal is to demonstrate real-world threats by injecting triggers only on interactable elements, we explicitly exclude the gold element locations from trigger injection. This ensures that the trigger does not overlap with the ground truth locations, preserving the validity of the evaluation.

Pretraining Poisoning: In this setting, trigger placement is randomized, and there is a small percentage of overlap between trigger and gold locations. Specifically, the overlap ratios are as follows:

We will include this detailed information in the revised version of the paper to improve clarity and address this concern.

W2: The experiments are limited to a single model family—Qwen2-VL 2B/7B as the LVLM backbone. To show the broader effectiveness of the proposed backdoor attack, more recent versions like Qwen2.5-VL [1] and other model families like LLaVA [2] are needed.

Thank you for the valueable suggestion, We have conducted following addtional experiments on Qwen2-VL-3B and LLaVA-NeXT-Mistral-7B:

The results for Qwen2.5-VL-3B are consistent with those observed in the Qwen2-VL series. Even with just 5% poisoned data, we achieve a high attack success rate of around 90%.

For LLaVA-NeXT, which did not undergo grounding-specific pretraining and has a relatively smaller vision tower compared to Qwen, Due to our resource constraint, training on approximately 65k grounding data for 1 epoch is insufficient for it to accurately determine the target position. Nevertheless, our method still achieves an average attack success rate of around 60%, demonstrating the effectiveness of VisionTrap.

Thanks for the response and conducting additional experiments. It addresses my concerns, and I have increased my score.

W1&Q1: Multi-Agent and Multi-Round Complexities

We appreciate the insightful question regarding the manifestation of our attack in real-world GUI systems involving multiple steps and multiple agents. we want to first clarify that our experiments already encompass multi-step task executions and a modular multi-agent architectures.

● Multi-Step Tasks: the environments used in our evaluations—AITW, Mind2Web, and OmniAct—each involve sequential decision-making processes spanning multiple steps(as detailed in appendix B.2)

●Multi-Agent Architectures: Our experiments on OmniAct also investigated a recent widely used modular multi-agent framework(SeeClick-V [1] [2]), where a planning agent interprets task instructions to generate descriptive text actions, and a grounding agent maps these descriptions to precise UI coordinates.

We believe our Attack Remains Effective in More Complex Multi-Step and Multi-Agent Contexts.

● Persistence Across Steps and Agents: Once the backdoor is implanted into the visual grounding component, it consistently influences every grounding operation throughout the entire multi-step task. This means that whenever the trigger appears at any stage, regardless of which agent or module is currently active, the system’s behavior is redirected toward the trigger location.

● Stealthiness in Multi-Step Tasks: In practical multi-step GUI tasks, agents perform sequences of operations to achieve high-level goals. An attacker can strategically inject the trigger at any step, causing subtle biases or incorrect actions that may propagate unnoticed through subsequent steps. This stealthy manipulation makes the attack difficult to detect and mitigate in real-world multi-round interactions

Our attack specifically targets the visual grounding capabilities of GUI agents powered by large vision-language models (LVLMs). Therefore, it provides an effective attack vector in any scenario where an LVLM is required to directly provide precise UI locations based on textual plans.

W2&Q2: Complex Real-World Task Generalization

We thank the reviewer for highlighting the importance of discussing our attack’s applicability to more complex, real-world scenarios. In fact, the effectiveness of our attack tends to increase as task complexity grows because:

Vulnerability at Higher Levels of Abstraction: Complex tasks often involve longer sequences of actions and interaction with a wider variety of interface elements. This expanded scope provides attackers with more opportunities to strategically embed triggers at critical points.

Consider a "book a business trip" task involving:

● Steps 1–3: Flight Booking — A trigger embedded in the airline selection interface could redirect choices toward a competitor, causing unfair market manipulation.

● Steps 4–6: Hotel Reservation — A trigger placed in the hotel selection could steer the user toward overpriced options, resulting in economic loss.

● Steps 7–9: Ground Transportation — A trigger in the pickup location input could cause misdirection to an incorrect address, potentially creating security risks.

Our attack operates at the visual grounding level,enabling it to compromise any or all of these steps regardless of the high-level task complexity. This capability provides attackers with a potent means to exert direct and covert control over user interactions across diverse, intricate workflows.

Q3: Real-world deployment of GUI agents

We recognize the value of evaluating our attack in real-world settings, such as mobile apps or desktop automation tools, to provide concrete evidence of its practical impact. While full real-world deployment requires extensive ethical approval, human label realworld tasks and controlled environments, our experiments on realistic datasets (Aitw for mobile, Mind2Web for web, OmniACT for desktop) provide strong evidence of practical threat potential.We are actively exploring opportunities to conduct such evaluations and will include these results in future work to further substantiate our findings.

Reference:

[1] Gou, Boyu, et al. Navigating the Digital World as Humans Do: ICLR2025.

[2] Wu, Zhiyong, et al. Os-Atlas: A Foundation Action Model for Generalist Gui Agents. ICLR2025.