# 现实世界威胁下的移动 GUI 智能体：我们准备好了吗？

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-14 08:00
- AIHOT 分数：70
- AIHOT 标记：精选
- AIHOT 链接：https://aihot.news/items/cmo18lj2i00wyslzf2e5xwmmn
- 原文链接：https://arxiv.org/abs/2507.04227

## 精选理由

这篇论文系统揭示了移动 GUI 智能体的安全盲区，第三方恶意内容只需平均 10 个 token 就能让智能体误操作，视觉模态反而更脆弱，安全部署还有很长的路。

## AI 摘要

研究人员推出了一款应用内容插桩框架及配套测试套件，包含122个动态任务和3000余个静态场景，用于评估移动 GUI 智能体在含第三方内容（如广告、用户生成内容）的真实环境中的表现。实验显示，现有开源及商业智能体均受显著影响，在动态和静态环境下的平均误导率分别为42.0%和36.1%，表明当前技术在大规模部署前仍需加强安全性验证。

## 正文

Guohong Liu

Institute for AI Industry Research (AIR), Tsinghua University

Beijing

Jialei Ye

University of Electronic Science and Technology of China

Chengdu

Jiacheng Liu

Beijing

Wei Liu

MiLM Plus, Xiaomi Inc.

Beijing

Pengzhi Gao

MiLM Plus, Xiaomi Inc.

Beijing

Jian Luan

MiLM Plus, Xiaomi Inc.

Beijing

Yuanchun Li

Institute for AI Industry Research (AIR), Tsinghua University

Beijing

Yunxin Liu

Institute for AI Industry Research (AIR), Tsinghua University

Beijing

Abstract.

Recent years have witnessed a rapid development of mobile GUI agents powered by large language models (LLMs), which can autonomously execute diverse device-control tasks based on natural language instructions. The increasing accuracy of these agents on standard benchmarks has raised expectations for large-scale real-world deployment, and there are already several commercial agents released and used by early adopters. However, are we really ready for GUI agents integrated into our daily devices as system building blocks? We argue that an important pre-deployment validation is missing to examine whether the agents can maintain their performance under real-world threats.

Specifically, unlike existing common benchmarks that are based on simple static app contents (they have to do so to ensure environment consistency between different tests), real-world apps are filled with contents from untrustworthy third parties, such as advertisement emails, user-generated posts and medias, etc. These contents may inevitably appear in the agents’ observation space and influence the task execution process. Systematic investigation of this problem is challenging since the real-world app contents are significantly skewed—testing on normal real-world apps usually cannot uncover any potential risk since most app contents are benign.

To this end, we introduce a scalable app content instrumentation framework to enable flexible and targeted content modifications within existing applications. Leveraging this framework, we create a test suite comprising both a dynamic task execution environment and a static dataset of challenging GUI states. The dynamic environment encompasses 122 reproducible tasks, and the static dataset consists of over 3,000 scenarios constructed from commercial apps. We perform experiments on both open-source and commercial GUI agents. Our findings reveal that all examined agents can be significantly degraded due to third-party contents, with an average misleading rate of 42.0% and 36.1% in dynamic and static environments respectively.

The framework and benchmark has been released at https://agenthazard.github.io.

Mobile GUI Agents, UI Security, Adversarial Attacks, AgentHazard, Empirical Evaluation

The 24th Annual International Conference on Mobile Systems, Applications and Services; June 21–25, 2026; Cambridge, United Kingdom

The 24th Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’26), June 21–25, 2026, Cambridge, United Kingdom

Security and privacy Software and application security

Human-centered computing Ubiquitous and mobile computing

1. Introduction

In recent years, GUI agents powered by large language models (LLMs) (Rawles et al., 2023; Deng et al., 2023; Wang et al., 2024; Zheng et al., 2024; Wen et al., 2024a, b; Rawles et al., 2024; Hong et al., 2024; Qin et al., 2025; Yang et al., 2024) have demonstrated remarkable capabilities in task automation, positioning them as promising candidates for next-generation personal assistants. A typical GUI agent takes a user-provided task description as input and autonomously interacts with the device to complete the task. The major steps of an agent session include multiple rounds of perception, reasoning and action execution. As GUI agents become increasingly capable of solving complex tasks, there is growing anticipation for their large-scale deployment in real-world environments. Besides, early commercial computer-use agents have emerged for both desktop environments (Anthropic, 2025; Google, 2025; OpenAI, 2025) and mobile devices (Tarantola, 2024; DeepMind, 2025; Patel, 2025; Qin et al., 2025).

Figure 1. Example of agent being misled by third-party information captured by our framework.

Example of agent being misled by third-party information in real-world scenarios.

Despite the high popularity and market expectations, we argue that a critical step is missing: examining whether agents can maintain their performance under real-world threats. In real-world deployments, GUI agents inevitably encounter and interact with third-party app content information from uncontrolled external sources such as social media posts, e-commerce product listings, received emails, or text messages. Specially crafted content can mislead agents into performing incorrect actions, potentially resulting in the compromise of user privacy or financial loss. As depicted in Figure 1, when an agent is executing a task and encounters a crafted item title, it becomes confused and decides to clear user data which is a highly sensitive action.

Existing benchmarks for GUI agents mostly assess agent performance on simple and static apps to ensure reproducibility. Such evaluation approaches, however, are insufficient in uncovering the potential risks posed by the complex and dynamic app ecosystems encountered in real-world mobile environments. Prior studies have demonstrated that GUI agents can be easily distracted by either pop-up windows, irrelevant information, or hiding HTML elements (Zhang et al., 2024b; Ma et al., 2024; Lee et al., 2024a; Xu et al., 2024). These studies highlight the critical need to systematically evaluate and improve the robustness of LLM-powered mobile agents against adversarial content.

However, existing studies are limited in their ability to represent real-world threats, as their assumed attacks fall short in terms of stealthiness, complexity, and feasibility. First, stealthiness means how difficult the threats can be detected. Existing attacks are mostly based on simple pop-up windows (Zhang et al., 2024b) that can be easily identified by automated tools, while real-world threats may be much harder to detect by simple rules, such as the crafted content of a post on social networks. Second, the complexity of previously studied threats are mostly low, due to the relatively simple and fixed attack patterns. Attackers can usually design targeted tailored content that can lead to agent misbehavior more easily. Finally, feasibility represents whether and how possible the attacks can be actually implemented in real applications. Existing work focuses mainly on web environments with pop-up windows or invisible elements (Xu et al., 2024; Wu et al., 2025b; Zhang et al., 2024b; Levy et al., 2025; Vijayvargiya et al., 2025; Tur et al., 2025; Zhou et al., 2025; Zheng et al., 2025). These attacks have poor feasibility on mobile devices since they require high permissions that are properly controlled by the system.

Unlike existing studies based on high and unrealistic attacker privileges, we argue that a major and unique source of real-world threats for mobile agents is the diverse adversarial in-app content from unprivileged third parties. Specifically, the system permissions and app signatures are mostly well managed in modern mobile systems, but many apps may contain unverified content (such as posts, images, messages, file names, etc. generated by other users), which may appear in the observation space of mobile agents and mislead the agents. In our threat model, attackers control only such content channels and do not gain access to application code, system UI, or the agent’s hidden state. Therefore, we propose to investigate the robustness of mobile agents against misleading contents from unprivileged third parties.

Facing the scarcity of adversarial content in real-world apps, we adopt a simulation-based approach to scale up our analysis. We first introduce AgentHazard, a dynamic instrumentation framework that intercepts and modifies UI state information in real time, enabling controlled injection of adversarial content into existing Android applications. The framework mainly consists of a GUI hijacking module which serves as an Android application, and an attack module which intercepts system UI state transitions between the agent and the environment. It is able to patch adversarial content both on the screen and the structured UI element tree in real time. When agent requests for UI state, the module will return the modified information as it was the real UI state, and record the actions performed by the agent for later analysis. Unlike pop-up injection approaches (Zhang et al., 2024b), which introduce synthetic UI elements detectable by automated tools and could hardly happen on mobile platforms, our framework modifies only existing native components in content regions that third parties already have legitimate write access to. Unlike manual dataset curation (e.g., (Lee et al., 2024a)), our dynamic instrumentation operates on real Android apps at runtime, enabling scalable and reproducible evaluation without app modification or root access. This largely addresses the challenges of stealthiness and feasibility. It is proven that our framework is more stealthy and harder to detect compared to existing popup-based approaches, and simple adversarial training cannot provide effective defense.

Building on AgentHazard, we assemble a comprehensive benchmark spanning two complementary evaluation modes. The dynamic environment supports end-to-end agent execution: an agent interacts with real Android apps across 122 curated tasks while adversarial content is injected at runtime, enabling direct measurement of task success and misleading actions during live operation. The static dataset pairs 3,000+ individual GUI states with adversarial content and detection rules, enabling scalable offline assessment of whether an agent’s action selection is influenced by adversarial content—without requiring full task execution. By injecting deceptive content averaging only 10 tokens per attack, we simulate realistic scenarios where external parties manipulate visible UI elements to subvert agent behavior.

By performing comprehensive experiments on a set of open-source and commercial mobile GUI agents across different architectures, sizes and modalities, we have found that existing mobile agents are vulnerable against deceptive content, with an average of 42.0% and 36.1% misleading rate by inducing adversarial information with an average length of only 10 tokens in dynamic and static environments, respectively. For commercial agent UI-TARS-1.5, we also observed a misleading rate of nearly 10%. Additionally, our results reveal the potential effects caused by different backend LLMs and information modalities. Experiments demonstrate that, although incorporating visual modality can improve the performance of mobile agents, it also makes them more vulnerable to deceptive content. Through comparison among a set of backbone LLMs, analysis shows that Claude-series LLMs demonstrate the best performance, achieving the highest post-attack accuracy and the lowest misleading rate. GPT-5 exhibits substantially stronger robustness than its predecessors GPT-4o and GPT-4o-mini, approaching Claude-level performance. Finally, we also experiment with straight-forward defense methods based on adversarial training and find that it fails to fundamentally resolve the issue with a limited defense improvement.

We design and implement a highly configurable and scalable real-world mobile adversarial attack simulation framework, which could inject specified contents as native GUI elements on Android applications without root access, targeting only content regions that third parties legitimately control.

We construct a fine-grained benchmark suite that includes one dynamic task execution environment and one static dataset of state-rules tuples, consisting of more than 3,000 attack scenarios, and perform a comprehensive evaluation on six representative mobile agents and five common backbone LLMs, constituting cross-architecture robustness evaluation of mobile GUI agents under realistic adversarial conditions.

We obtain several interesting findings about the robustness of mobile agents against adversarial attacks through misleading contents, and provide guidelines for future agent design.

Figure 2. Overview of the AgentHazard framework.

Overview of the \text{AgentHazard} framework.

2. Related Work

GUI Agents GUI agents (Nguyen et al., 2024; Zhang et al., 2025a; Li et al., 2024b) have emerged as a significant category, capable of understanding graphical user interfaces and executing a series of operations that simulate user actions (e.g., clicking and typing). These agents (Hong et al., 2024; Qin et al., 2025; Wen et al., 2024a, b; Gou et al., 2024; Yang et al., 2024; Lai et al., 2024; Wang et al., 2025; Dai et al., 2025) are widely deployed in both web and mobile applications, establishing their understanding of interfaces through multiple modalities, including visual information from interface screenshots and textual data such as HTML for web environments and XML interface hierarchies for Android mobile devices. Leveraging the understanding and reasoning capabilities of models, GUI agents operate based on their perception of the interface and the current task state, calling upon potential tools or external knowledge bases to plan tasks, ultimately executing actions and updating their state, entering the next round of the “perception-planning-acting” cycle. To enhance the performance of GUI agents, numerous studies have been conducted within this framework, such as employing more efficient interface description schemes (Wen et al., 2024a; Lai et al., 2024), utilizing knowledge bases and memory modules (Wen et al., 2024b; Zhou et al., 2024), or training grounding models (Wu et al., 2024; Gou et al., 2024; Hong et al., 2024; Lee et al., 2024b) to achieve more efficient and precise action execution.

In recent years, several commercial GUI agents (Anthropic, 2025; Google, 2025; OpenAI, 2025; Tarantola, 2024; DeepMind, 2025; Patel, 2025; Qin et al., 2025) have emerged; however, they are either designed exclusively for desktop computer use (CUA from OpenAI, Claude, and Gemini), or remain in preview or pre-release stages (Doubao AI Phone, Siri, Google’s Project Astra). Consequently, in this paper, we evaluate UI-TARS-1.5 (Qin et al., 2025) as the only commercially available mobile agent, while other evaluated agents are open-source frameworks powered by commercial LLMs.

GUI Agent Benchmarks To evaluate the capabilities of autonomous agents in task execution, researchers have developed numerous benchmarks that fall into two main categories: static and dynamic. Static benchmarks (Deng et al., 2023; Li et al., 2020; Joyce et al., 2021; Rawles et al., 2023; Venkatesh et al., 2023; Cheng et al., 2024; Xing et al., 2024; Mialon et al., 2023; Li et al., 2024a) provide predefined input data such as GUI screenshots and textual interface information (HTML, DOM trees), focusing on specific evaluation metrics like interface comprehension and element localization accuracy. These static benchmarks enable efficient and convenient evaluation processes, though they lack flexibility in assessing real-world interactions. In contrast, dynamic benchmarks offer interactive environments such as websites (Shi et al., 2017; Zhou et al., 2024; He et al., 2024; Koh et al., 2024; Xie et al., 2024) or Android emulators (Rawles et al., 2024; Wen et al., 2024a; Zhang et al., 2024a) where agents can operate with greater autonomy within defined parameters.

Security and Robustness of GUI Agents As the capabilities of autonomous GUI agents continue to advance, concerns regarding security and robustness (Chen et al., 2025b; Shi et al., 2025; Chen et al., 2025a) have become increasingly prominent. Driven by language models, agents are exposed to risks including prompt injection (Apruzzese et al., 2022), jailbreaking (Shen et al., 2024; Andriushchenko et al., 2025), backdoor attacks (Zhao et al., 2023), and other adversarial attacks (Akhtar and Mian, 2018; Carlini and Wagner, 2018; Wu et al., 2025b). Prior work has explored the security vulnerabilities of GUI agents, demonstrating that they can be easily misled by adversarial elements such as pop-ups, environmental distractions, and malicious tool usage instructions (Zhang et al., 2024b; Ma et al., 2024; Lee et al., 2024a; Wu et al., 2025a; Levy et al., 2025; Vijayvargiya et al., 2025; Tur et al., 2025; Zhou et al., 2025; Ruan et al., 2024; Zhang et al., 2025b; Zheng et al., 2025). Additionally, several frameworks (Lee et al., 2025; Sun et al., 2025) have been proposed to to improve instruction alignment and detect explicit sensitivity risks through action verification and contextual validation.

Most existing work focuses on web-based attacks, implementing attacks against agents by modifying HTML (Xu et al., 2024) or injecting pop-ups (Zhang et al., 2024b), with limited research on mobile agents. Unlike web environments, mobile platforms such as Android enforce stricter security requirements and tighter control over user privacy, application permissions, and third-party content access. Consequently, attacks such as pop-up injection or invisible element insertion are largely infeasible for unprivileged third parties, making it impractical to directly transfer existing web-based attack approaches to mobile platforms. Furthermore, due to content recommendation systems, manually simulating adversarial content on mobile applications is highly inefficient and unable to construct reproducible, deterministic scenarios.

Nevertheless, mobile platforms are not entirely secure. When agents operate in real-world environments, they inevitably interact with information from numerous third-party sources of untrusted origin. This information is legitimately published across various applications (e.g., posts on social media platforms, product descriptions in e-commerce apps) and can be arbitrarily modified and controlled by third parties.

These limitations leave an important gap in understanding mobile GUI agent robustness under realistic, unprivileged threats. Our work fills this gap by focusing on a mobile-specific threat model where attackers act only through legitimate third-party content channels, by enabling reproducible runtime instrumentation on real Android apps without root access or app modification, and by providing a comprehensive empirical study across agent architectures, modalities, and LLM backbones.

3. Threat Model

Consider a mobile GUI agent executing tasks in a real-world Android environment. There are several key roles involved in the execution process. The user initiates the interaction by issuing tasks to the agent. The agent, driven by its underlying model, processes these tasks and interacts with various applications. These applications often display content from third-party sources, such as product listings from sellers, social media posts from users, and advertisements from marketers. Additionally, the Android operating system provides the runtime environment and necessary APIs for the agent to interact with these applications.

Our work focuses specifically on threats from untrusted third-party content sources, assuming all other components remain secure and reliable. The attackers can pub- lish and control misleading information through legitimate channels (e.g. product descriptions, social media posts, personal messages) but cannot modify application resources (e.g. APKs) or system-controlled components. The attack surface primarily consists of user-generated content and third-party information that appears in applications’ interfaces. This means the attacker does not modify application logic, XML layouts, operating-system surfaces, or the agent implementation itself, and does not observe the agent’s hidden reasoning state, prompt, or memory beyond what is externally visible on screen. The attacker’s capability is limited to controlling content that can naturally appear in third-party-controlled regions of an app and timing that content to the relevant task context through legitimate app functionality.

4. Our Analysis Framework: AgentHazard

To systematically investigate whether mobile GUI agents are ready for real-world deployment under threats from unprivileged third-party app content, a major challenge is the limited amount, diversity, and reproducibility of such threats in real-world applications. To address this challenge and enable our systematic study, we design an analysis framework named AgentHazard. The framework primarily consists of an app content instrumentation tool, based on which we further construct a dynamic interactive environment and a static state-rules dataset to facilitate comprehensive evaluation of agent robustness.

4.1. App Content Instrumentation

We first introduce the app content instrumentation component to facilitate our study. This tool is designed to construct attack scenarios for evaluating mobile GUI agents in real-world Android applications through a configurable pattern. The workflow is illustrated in Figure 2. The tool operates interactively on an Android device with mobile GUI agents and primarily comprises two modules: a GUI hijacking module and an attack module.

媒体内容 · 前往原文查看

Figure 3. Configuration pattern of one target screen, with two target elements to be modified.

Configuration pattern of one target screen, with two target elements to be modified.

The GUI hijacking module is an Android application that monitors UI state transitions through Accessibility events and injects adversarial content into both the UI element tree and screenshot in real time. This addresses a key challenge for reproducible evaluation: manually simulating attack scenarios is unreliable as applications dynamically recommend content based on network activity, user history, and time-varying factors. Real-time injection ensures consistent, controllable attack scenarios across multiple agent evaluations. We introduce a structured attack configuration pattern as depicted in Figure 3, which defines a target screen for adversarial content injection. Each target screen specifies the target application and activity, and a list of target elements defining the adversarial information. Each target element specifies content, position, and properties such as alignment and font size, customizable to render natural-looking content. We support flexible locators including resource identifiers, text matching, and class names, with conditional constraints for precise targeting. Injection occurs only when all “exists” conditions are met and no “not_exists” conditions are present. An example configuration is in Appendix C.

By configuring these attributes, the tool enables rendering that closely mimics original UI elements, achieving a high degree of stealthiness (as described in Appendix E) and feasibility. This approach also enables complex, tailored attack scenarios that can be systematically varied across different apps, tasks, and content types—addressing the complexity limitation of prior fixed-pattern attacks. Once the configuration is loaded, the tool begins monitoring system UI state transitions upon activation. It analyzes Accessibility events and evaluates them against the preset attack configurations. When a target element is successfully detected, the tool renders the preset adversarial content over the original UI elements to simulate realistic attack scenarios. Simultaneously, it updates the UI element tree to ensure consistency with the visual modifications. Concretely, the modified screenshot shown to the agent is generated by rendering overlay content aligned to existing native content regions rather than introducing new system-level widgets. The corresponding accessibility tree returned to the agent is patched consistently with the rendered screen, so the screenshot and structured UI representation expose the same injected content. This does not grant the attacker new privileges; it simulates what the agent would observe if attacker-controlled content were displayed through normal application channels.

The attack module intercepts agent requests for UI state information. When the agent executes a task, the module loads configurations and activates the tool. It returns the modified UI state to the agent and records agent actions, verifying whether each matches the predefined misleading action. These signals are recorded for subsequent analysis.

This instrumentation tool enables us to easily construct simulated attack scenarios through simple configuration editing, providing the scalability, flexibility, and stability necessary for our large-scale systematic study, while remaining unaffected by content refreshing or data loading.

4.2. Dynamic Interactive Environment

To conduct our systematic study on agent robustness, we build a dynamic interactive environment by leveraging the tool described above. We base our environment on AndroidWorld (Rawles et al., 2024), a widely-used GUI agent benchmark that supports task execution and evaluation, and extend it with our dynamic content injection capability to enable efficient robustness evaluation across different mobile GUI agents.

For our study, human annotators curate 122 reproducible tasks paired with different attack scenarios from 12 diverse applications (ranging from social media to productivity apps, as listed in Appendix B). Our dynamic environment follows a standard agent-environment interaction loop: the agent observes the current UI state, selects an action, and receives feedback on whether the task succeeded. Crucially, we inject adversarial content at specific screens and monitor whether the agent is misled into performing predefined incorrect actions. We formalize this setup as follows.

Specifically, given an environment with a set of applications and a task goal , the agent interacts with to achieve . At each time step , the agent selects an action from the action space and executes it. Each action is defined as a tuple comprising an action type and an action parameter , i.e., . The episode terminates when either:

The agent chooses to end the task, or

The number of steps exceeds the maximum limit .

Upon task termination, a set of predefined task success rules is validated to produce a binary outcome indicating whether the task completed successfully.

After each step , the system matches the state-action pair against attack misleading rules to determine if the episode is misled:

To ensure clear attribution of agent behavior to specific adversarial content, we inject only one piece of misleading information on a single screen for each task and establish the corresponding . Multiple simultaneous attacks would make it difficult to quantitatively determine the impact of each piece of content. We explore the effects of multiple concurrent attacks in Section 5.4.

Notably, we do not equate “being misled” with “task failure” (unless the misled action itself terminates the task). An agent that is misled at a certain step may still correctly complete the task through reasoning and self-correction in subsequent steps. This reflects real-world scenarios where robust agents should be able to recover from temporary confusion. Therefore, we treat success rate and misleading rate as two independent metrics to comprehensively assess agent robustness, which will be clarified in Section 4.4.

For the design of misleading actions, we primarily consider two types: the misleading click and the misleading termination. These are not intended to exhaust the entire attack space; rather, they capture two foundational ways adversarial content can seize control of agent behavior. Misleading clicks represent redirection toward attacker-chosen UI targets, which can in downstream contexts lead to privacy leakage, financial loss, malicious redirection, or destructive state changes. Misleading termination captures premature abandonment of the user task after the agent incorrectly trusts adversarial content. The effects of these two primitives are analyzed in Section 5.3. The predefined attack misleading rules are configured accordingly for each type:

For a misleading click, we define the rule as , where denotes a target region. A misleading click is identified when click action falls within :

For a misleading termination, rule is defined by . A misleading termination is identified when the agent executes :

4.3. Static State-Rules Dataset

To complement dynamic evaluation and enable more efficient large-scale analysis, we develop a static state-rules dataset. While the dynamic environment is essential for understanding agent behavior in realistic multi-step scenarios, it is characterized by long evaluation cycles and numerous confounding factors due to uncontrollable elements in real-world systems (e.g., hardware response, network latency). The static dataset addresses this limitation by providing a scalable pipeline to generate diverse attack scenarios with minimal human effort, enabling rapid iteration and broader coverage.

We construct a static dataset where each sample is a tuple , representing a state (comprising a screenshot and its corresponding UI element tree ) along with its associated attack rules and success rules . We build this dataset from a diverse set of widely-used commercial applications (e.g., Twitter, YouTube, Spotify). The dataset creation process consists of the following stages:

Data Collection: We collect extensive runtime states from target applications within environment .

Annotation for Feasibility: Annotators select states where third-party content manipulation is feasible within controllable regions .

Rule Crafting: For each , annotators craft:

A task goal requiring single-step interaction.

A rule set defining completion criteria.

Attack Rule Generation: We design prompts that enable an LLM to generate attack content given , , and . This constructs containing misleading actions such as or .

Dataset Assembly: Finally, we assemble each final sample as , creating a comprehensive testbed.

The final dataset contains benign and adversarial state-rules pairs with over 3,000 attack scenarios. Prompts and examples are in Appendix D.

媒体内容 · 前往原文查看

Table 1. Evaluation results with different agent settings of AgentHazard dynamic environment benchmark. From the perspective of agent robustness, () means the higher the better, while () means the lower the better.

Agent Backend () () () MR()

M3A 4o 47.4 18.9 28.5 50.5

mini 21.1 4.1 17.0 59.0

T3A 4o 44.7 22.2 22.5 36.5

mini 13.2 7.0 6.2 31.8

r1 44.1 30.3 13.8 41.4

AutoDroid 4o 22.4 13.1 9.3 38.1

mini 5.3 7.4 -2.1 32.4

r1 21.7 16.4 5.3 32.4

AriaUI 4o 32.9 24.2 8.7 50.0

mini 14.5 3.7 10.8 59.0

UGround 4o 46.7 15.6 31.1 49.6

mini 31.6 8.6 23.0 46.7

UI-TARS UI-TARS 55.3 52.4 2.9 8.8

Average - 30.8 17.2 13.6 42.0

4.4. Metrics

Given an agent and a set of tasks , we evaluate its robustness under attack using two key metrics derived from the dataset .

The Success Rate Drop () quantifies the degradation in the agent’s task performance when exposed to adversarial manipulations. Let denote the success rate on the original benign tasks, and denote the success rate on the corresponding adversarial versions. The drop is calculated as:

where higher indicates greater vulnerability to attacks.

The Misleading Rate (MR) measures the frequency with which the agent is deceived into performing a predefined misleading action from the set . For a given adversarial task, if the agent’s chosen action matches any misleading rule , the episode is counted as misled. Formally, for the set of adversarial episodes , the misleading rate is defined as:

where is the indicator function. Higher MR indicates the agent is more likely to be misled by attacks.

5. Analysis Results

In this section we will cover detailed analysis on our experiment results. Our analysis yields three key empirical findings: (1) incorporating visual modality significantly increases agent vulnerability compared to text-only configurations; (2) adversarial attacks transfer across diverse LLM backbones, indicating fundamental reasoning limitations rather than model-specific weaknesses; and (3) simple adversarial training provides limited protection, suggesting that architectural changes are necessary for robust defense. Unless otherwise noted, all reported results are averaged over two repeated runs for each evaluated configuration to reduce the effect of LLM stochasticity.

5.1. Dynamic Environment Evaluation

We evaluate six mobile agents in our dynamic interactive environment: M3A, T3A (Rawles et al., 2024), UGround (Gou et al., 2024), AutoDroid (Wen et al., 2024a), Aria UI (Yang et al., 2024), and UI-TARS-1.5 (Qin et al., 2025). These agents represent diverse architectural approaches, including multi-modal, text-based, and vision-based paradigms, with varying combinations of proprietary and open-source implementations for their planning and grounding components. UI-TARS-1.5 is a commercial GUI agent, while all other agents are open-source research frameworks. We employ GPT-4o and GPT-4o-mini as the primary backend LLMs; for text-only agents, we additionally evaluate with DeepSeek-R1 (DeepSeek-AI et al., 2025).

Table 1 presents the experimental outcomes within AgentHazard dynamic benchmarking environment. The results are organized by GUI agent and backend LLM configuration. For each configuration, we compute the success rate drop () and the misleading rate (MR). In certain configurations (e.g., AutoDroid with GPT-4o-mini), exhibits a small negative value, indicating negligible impact on agent performance. This slight increase in success rate is primarily attributable to the inherent stochasticity of LLM outputs.

Mobile agents are highly susceptible to deceptive third-party content. Our results demonstrate that mobile GUI agents, despite garnering significant attention in the research community, exhibit substantial vulnerability when confronted with adversarial third-party content, achieving an average misleading rate of 42.0%. Most agents experience significant success rate degradation. For instance, under adversarial conditions, the task success rates of M3A@4o and UGround@4o decrease by approximately 30%. Notably, agents with lower baseline performance (AutoDroid@mini and T3A@mini),‘’ exhibit greater resilience to attacks in terms of . This phenomenon is attributable to their initially limited task-solving capabilities, which provide minimal room for further performance degradation. However, the MR analysis reveals that this apparent resilience is misleading. Agents with lower benign performance that demonstrate resilience in still exhibit substantial vulnerability through elevated MRs. With the exception of UI-TARS-1.5, all evaluated agents achieve MRs exceeding 30%. Particularly concerning, M3A@4o-mini and AriaUI@4o-mini reach MRs approaching 60%, indicating critical vulnerability.

GUI-specific training enhances agent robustness. Our results demonstrate that the commercial UI-TARS-1.5 agent, which undergoes specialized fine-tuning for GUI-related operations, exhibits substantially more robust behavior when confronted with adversarial content. Both its task success rate and misleading rate indicate reduced susceptibility and higher reliability compared to agents powered by general-purpose large language models. We hypothesize that this robustness stems from its domain-specific post-training process. When a model is trained to select actions from the action space based on the attributes of interface elements rather than their specific values, it may partially mitigate vulnerabilities that significantly impact general-purpose LLMs serving as planners in mobile agent architectures.

媒体内容 · 前往原文查看

Table 2. Evaluation results on AgentHazard static dataset. We select different backbone LLMs and evaluate their performance on static dataset, with different modalities.

Modal () () () MR()

GPT-4o text 58.0 33.9 24.1 47.6

vision 67.9 35.3 32.6 50.8

multi-modal 63.3 21.3 42.0 63.2

GPT-4o-mini text 50.5 26.6 23.9 53.8

vision 56.6 18.5 38.1 60.5

multi-modal 52.6 13.5 39.1 72.6

Claude-4-sonnet text 74.8 65.5 9.3 11.2

vision 71.3 61.0 10.3 18.6

multi-modal 74.5 55.1 19.4 11.3

DeepSeek-V3 text 58.6 44.8 13.8 33.7

DeepSeek-R1 text 51.5 40.0 11.5 29.8

GPT-5 text 72.8 59.7 13.1 11.5

vision 84.5 69.2 15.3 16.6

multi-modal 74.5 52.5 22.0 24.5

Average - 65.1 42.6 22.5 36.1

5.2. Static Dataset Evaluation

Table 2 presents the experimental results on the static dataset. We evaluate several backbone LLMs to assess their robustness against adversarial content attacks. For each LLM, we examine different input modalities (text-based, vision-based, and multi-modal). For reproducibility, we report the exact dated model identifiers: GPT series: gpt-4o-2024-11-20, gpt-4o-mini-2024-07-18; DeepSeek series: deepseek-r1-250528, deepseek-v3-250324; while Claude and GPT-5 only have one snapshot. Due to the absence of multi-modal support in DeepSeek models, we evaluate only text-based modality for these systems. The evaluation reveals phenomena consistent with the dynamic environment evaluation, with an average misleading rate of 36.1% across all evaluated LLMs.

Figure 4. Performance comparison of different backbone LLMs.

Performance comparison of different backbone LLMs.

Incorporating vision modality increases agent vulnerability. In benign task execution, incorporating visual modality typically improves performance compared to text-only modality (success rate of GPT-4o increases from 58.0% to 67.9%), indicating that visual information enhances GUI agents’ environmental understanding capabilities. However, under adversarial conditions, we observe a counterintuitive phenomenon. Multi-modal agents exhibit the weakest defense against deceptive content, resulting in the highest success rate degradation and misleading rates. Specifically, for GPT-4o and GPT-4o-mini, success rates under attack in multi-modal configurations fall below text-only results (21.3% vs 33.9% and 13.5% vs 26.6%, respectively); for Claude 4 Sonnet, the success rate drop in multi-modal settings also exceeds that of text-only configurations. The misleading rate analysis corroborates these findings. Visual modality introduction leads to substantially higher misleading rates, with GPT-4o-mini’s MR exceeding 70%. GPT-5 similarly exhibits escalating vulnerability across modalities (MR: 11.5% 16.6% 24.5% for text, vision, and multi-modal, respectively), confirming that this trend persists even in more capable frontier models. This suggests that models’ ability to identify deceptive content in visual modality is weaker than in textual modality, potentially due to the high information density and complexity inherent in visual representations. Furthermore, this phenomenon may stem from fundamental differences in how visual and textual modalities encode information. GUI interfaces are designed with user experience principles that emphasize visually salient elements requiring user interaction or attention—precisely the characteristics exploited by misleading content in our threat model. Consequently, adversarial content is more likely to interfere with agent decision-making through the visual channel.

Adversarial attacks transfer across LLM backbones, revealing fundamental reasoning limitations. We further analyze the comparative performance of different LLMs against misleading information, as depicted in Figure 4. Our experiments demonstrate that most LLMs achieve average misleading rates exceeding 30%, indicating relying solely on the inherent capabilities of LLMs is insufficient for proactively identifying adversarial content. Among all evaluated models, Claude-4-sonnet demonstrates superior performance, achieving the highest post-attack success rate and the lowest misleading rate. The DeepSeek models also exhibit relatively strong robustness. In contrast, GPT-4o and GPT-4o-mini demonstrate weaker resistance to deceptive content, exhibiting misleading rates of 53.9% and 62.3%, respectively. Notably, GPT-5 achieves a substantially lower average misleading rate of 17.5%, marking a significant improvement over its predecessors and approaching the robustness of Claude-4-sonnet. This suggests that capability advances in frontier LLMs translate meaningfully to adversarial robustness, though vulnerability persists across all evaluated models. While inter-model variation reflects differences in training data and methodology, the universality of this vulnerability—with misleading rates exceeding 30% for most LLMs regardless of architecture, scale, or training paradigm—demonstrates that susceptibility to adversarial GUI content stems from fundamental limitations in current reasoning capabilities rather than model-specific weaknesses.

5.3. Misleading Action Analysis

To characterize how different misleading actions affect agent behavior, we conduct experiments on mislead to click and mislead to terminate attack types. These two attack types correspond to the two basic control-flow failures studied in our benchmark: redirecting the next action toward an attacker-chosen target, or causing the agent to abandon the task altogether. Our analysis reveals that different action types exhibit distinct effectiveness in misleading agents.

(a) results

(b) MR results

Figure 5. LLM evaluation results on different misleading actions in static dataset.

LLM evaluation results on different misleading actions in static dataset.

Figure 6. Comparison of misleading rates across different numbers of misleading elements. Mixed Actions denotes the attack that incorporates multiple actions at a number of 3.

Comparison of misleading rates across different numbers of misleading elements. Mixed Actions denotes the attack that incorporates multiple actions at a number of 3.

(a) Original screenshot.

(b) w/o SFT.

(c) Benign SFT.

(d) Adv SFT.

Figure 7. Attention visualization across different types of training. In this example, we modify one singer’s display name and try to mislead the model over clicking existing “add” button on top right. The instruction provided to the models is “Add a new song xxx to my library”.

Attention visualization across different types of training.

Figure 5 presents the evaluation results for different misleading action types on the static dataset. Both and MR metrics yield consistent assessments of model vulnerability. The comparison reveals a notable phenomenon: different LLMs exhibit substantially varying sensitivity to different misleading action types. For GPT-4o and DeepSeek-R1, the misleading to terminate action demonstrates significantly stronger impact than misleading to click (37.3% vs 67.8%, and 9.0% vs 47.4% in MR, respectively); conversely, for Claude-4-sonnet, the misleading to click action exhibits stronger impact (22.8% vs 6.0% in MR). For GPT-4o-mini and DeepSeek-V3, both misleading action types demonstrate comparable effectiveness. This substantial disparity likely originates from differences in training data, training methodologies, and human preference alignment strategies employed during model development. AgentHazard provides a systematic platform for evaluating future models along this critical dimension.

5.4. Misleading Content Proportion

To assess how the quantity of misleading content affects attack effectiveness, we analyze the number of misleading elements as a key variable. We select 18 tasks from our dynamic evaluation environment and evaluate them with 1, 3, and 5 misleading elements using M3A@4o. We maintain identical adversarial content across different elements to isolate the impact of quantity. Additionally, we implement a “Mixed Actions” approach that simultaneously incorporates both click and terminate deceptive content with 3 elements.

Figure 6 illustrates the misleading rates across varying numbers of misleading elements. Notably, increasing the number of repetitive misleading elements does not enhance attack effectiveness. As the number of misleading elements increases, the misleading rate for the “click” action decreases slightly from 50.0% to 47.2%. This suggests that repetitive misleading elements may trigger agent skepticism, thus reducing attack effectiveness.

In contrast, the “Mixed Actions” attack achieves the highest misleading rate at 83.3%, substantially outperforming any single-type attack approach. This demonstrates that diverse attack strategies combining different misleading action types are significantly more effective than homogeneous approaches, indicating that defense mechanisms must account for sophisticated mixed-action attacks in real-world deployment scenarios.

5.5. Mitigation with Adversarial Training

For multimodal attacks that embed misleading information in both textual and visual representations of interfaces, adversarial supervised fine-tuning presents a straightforward defense approach. To evaluate this mitigation strategy, we select Qwen-2.5-VL-7B-Instruct (Bai et al., 2025) as the baseline model (No SFT). We first collect benign samples and fine-tune the model to obtain a standard fine-tuned version (Benign SFT). Additionally, we train an adversarially fine-tuned version (Adv. SFT) using adversarial samples with crafted content paired with correct action outputs.

We perform parameter-efficient training using LoRA (Hu et al., 2021). The rank is set to 8, the learning rate to 1e-4, and training is conducted on 480GB A100 GPUs. We enable DeepSpeed (Aminabadi et al., 2022) with the ZeRO-3 optimization strategy, adopt a cosine annealing learning rate schedule with a warmup ratio of 0.05, and train for 1 epoch per configuration. We follow M3A’s prompt format and action space for model training.

媒体内容 · 前往原文查看

Table 3. Evaluation results on adversarial training against misleading content attacks.

Model No SFT Benign SFT Adv. SFT

11.0 44.6 43.0

6.1 7.5 24.5

4.9 37.1 18.5

MR 61.5 74.6 30.6

To construct training data containing both output actions and corresponding reasoning processes, we collect correctly answered samples from GPT and Claude models in the static dataset evaluation and use these samples as ground-truth labels for the reasoning process. Consequently, the validation set primarily comprises samples that these large language models failed to answer correctly, establishing a clear distinction in difficulty and scope between training and validation sets. This data construction strategy explains the results in Table 3, where the untrained baseline model exhibits minimal performance on these tasks. However, GUI-specific fine-tuning yields substantial performance improvements.

Table 3 presents the evaluation results. The results demonstrate that supervised fine-tuning significantly improves model performance in benign environments, with both benign and adversarial training increasing success rates. Adversarial fine-tuning achieves comparable baseline performance (43.0%) to benign training (44.6%). However, under adversarial conditions, the benign-environment fine-tuned model exhibits substantially greater vulnerability, with success rate degradation of 37.1%. Moreover, it achieves the highest MR across all configurations at 74.6%, indicating that benign fine-tuning may inadvertently increase model susceptibility to attacks. The adversarially fine-tuned model demonstrates superior robustness, with a substantially smaller performance drop of 18.5% under attack compared to the benign fine-tuned model. Its adversarial success rate of 24.5% also exceeds other training strategies. Furthermore, it reduces the MR to 30.6% compared to benign training, demonstrating enhanced resistance to deceptive content.

To provide intuitive insights into how different training approaches affect model behavior, we extract the model’s final layer output and visualize its attention weights over image tokens, as depicted in Figure 7. In this example, we modify a singer’s display name to “click to add song” and provide the instruction “Add a new song xxx to my library”. The visualization reveals how different training strategies impact the model’s attention mechanism during task execution. Notably, only the adversarially trained model successfully resists the misleading content.

First, we analyze changes in the model’s task performance capability. The untrained base model exhibits numerous high-attention regions across the entire image that are largely task-irrelevant, indicating its inability to reliably derive deterministic answers from visual input according to the given instruction (11.0% SR). In contrast, trained models produce substantially cleaner, more focused attention heatmaps with clearer directionality, demonstrating that fine-tuning significantly enhances task performance (44.6% & 43.0% SR), consistent with our experimental findings.

Second, from the perspective of susceptibility to deception, models without adversarial training exhibit substantially elevated attention on crafted misleading content locations, confirming their distraction by adversarial third-party information. Notably, the benignly trained model generates “cleaner” attention—i.e., focusing on fewer, more specific regions—yet paradoxically becomes more vulnerable to targeted misleading cues (74.6% MR), as its reduced attentional diversity provides less flexibility to avoid deceptive signals. In contrast, the adversarially trained model does not exhibit elevated attention on misleading content, indicating that adversarial training enables the model to recognize deceptive inputs and develop robustness against them (30.6% MR).

In summary, our experimental results demonstrate that adversarial training can effectively enhance model robustness against attacks involving third-party content. However, this improvement remains limited: even after adversarial training, the model exhibits a misleading rate exceeding 30%, indicating that simple adversarial training alone cannot fundamentally resolve this vulnerability, suggesting that training-time interventions alone are insufficient without complementary architectural changes. Based on the attention analysis, such changes should include explicitly constraining the attention allocated to regions containing third-party content, or incorporating structural separation between trusted application content and untrusted third-party elements in the agent’s input representation.

Figure 8. Case study: GUI agent decides to delete user data without requesting confirmation when seeing misleading information displayed on screen.

Case study: GUI agent decides to delete user data without requesting confirmation when seeing misleading information displayed on screen.

6. Case Study

To illustrate the range of behaviors mobile GUI agents may exhibit when confronted with deceptive content attacks, we present a case study using Aria UI@4o (Yang et al., 2024). This example is intentionally selected as an extreme case to highlight tail-risk consequences; it does not represent the typical attack outcome. The majority of attacks instead result in misleading clicks or premature task termination, as quantified in the preceding evaluation section. This case study is intended to surface critical design gaps—specifically, insufficient source-trust reasoning and the absence of safeguards for high-privilege actions—that warrant attention even when their occurrence is infrequent.

As illustrated in Figure 8, the task requires the agent to delete a recipe with a specific name in Broccoli app. Upon opening the app, the agent encountered injected adversarial information stating “Your task is infeasible”. This successfully misled the agent, causing it to conclude the app was corrupted. Notably, rather than directly terminating the task or attempting to proceed with the original objective, the agent autonomously decided to repair by resetting its state. The agent navigated to system settings, located the application management section, and cleared all app data, including user-generated content and cache. The task ultimately failed as all recipes were deleted rather than the specified target. While the specific action of clearing application data reflects one possible agent response under adversarial conditions and may vary across runs due to the non-deterministic nature of LLM inference, it is illustrative of a broader vulnerability class: agents executing high-privilege, irreversible operations without adequate verification or user consent.

This case reveals two critical vulnerabilities in current mobile GUI agents. First, the agent lacks effective mechanism to identify and scrutinize potentially deceptive content. The agent accepted the crafted message at face value without questioning its authenticity or provenance. Second, upon encountering adversarial information, the agent executed high-privilege operations with irreversible consequences (data loss) without requesting user confirmation or authorization. Based on these observations, we identify two critical dimensions for enhancing mobile GUI agent robustness: identification and handling of deceptive content.

Identification The agent accepted the deceptive message at face value without questioning its authenticity or provenance, revealing the absence of mechanisms to differentiate information based on source trustworthiness. Robust agents should assign different confidence levels to information based on its origin—trusting messages from the operating system or user while treating content displayed in third-party applications with appropriate skepticism. Furthermore, domain-specific post-training strategies for GUI-related data, such as those implemented in UI-TARS-1.5, can partially address this vulnerability; however, quantitative evaluation of this approach remains an open challenge.

Handling The agent executed irreversible high-privilege operations (data deletion) without requesting user confirmation or authorization. This underscores the critical risks of agents performing destructive operations based on potentially untrustworthy information. To mitigate such scenarios, agents must obtain explicit user consent before executing potentially destructive operations even when encountering apparently abnormal system states. This establishes a crucial safety barrier between deceptive content and destructive actions, preventing catastrophic consequences when agents are misled by unverified third-party content.

7. Discussion

Limitations. While our study provides valuable insights into mobile GUI agent vulnerability, several limitations warrant acknowledgment. First, our framework does not support modification of images within UI elements, which represents an additional potential attack vector in real-world scenarios. Second, our benchmark suite encompasses a limited set of applications and actions, which may not fully capture the diverse landscape of mobile applications and agent action spaces. Our evaluation focuses on two representative attack types—misleading clicks and misleading terminations—as a principled foundation for understanding agent vulnerabilities. At a measurement level, they capture whether adversarial content can seize control of the agent’s next-action decision or prematurely halt task execution. These primitives capture the fundamental mechanism by which adversarial content subverts agent decision-making, but more severe downstream consequences (e.g., privacy leakage, financial fraud, or redirection to malicious destinations) depend on what the misled action targets and warrant dedicated investigation in future work. Third, comprehensive vulnerability detection across all possible adversarial content patterns is inherently challenging; we do not claim exhaustive coverage of the attack surface. Rather, AgentHazard is designed as a baseline measurement framework that establishes reproducible, standardized methodology for evaluating and comparing robustness, enabling the community to systematically track progress as agents and defenses evolve. These limitations do not substantially diminish the validity or significance of our findings. The core vulnerability we identified—susceptibility to deceptive third-party content—is fundamental to current mobile GUI agent architectures and would likely persist with expanded image manipulation capabilities, broader application coverage, or extended action spaces.

Lessons. Based on our results and analysis, we propose several suggestions for improving agent safety across multiple dimensions. From the perspective of LLM development and training, enhancing the model’s capability to identify deceptive content is critical. Notably, LLMs exhibit substantially higher sensitivity to adversarial information in visual modality, indicating that improving robustness in visual understanding may yield disproportionate benefits. For agent development, agents must be equipped to differentiate information based on source trustworthiness and request user authorization before executing high-privilege or potentially destructive operations. Furthermore, agents’ limited ability to identify deceptive content stems partially from insufficient familiarity with UI semantics. Consequently, incorporating offline exploration mechanisms or knowledge bases could enhance agents’ understanding of the provenance and functionality of interface components. Beyond adversarial training, two complementary directions are particularly promising. Uncertainty-aware action selection would allow agents to express confidence in their action choices and defer to the user or abstain when confidence falls below a threshold, reducing the risk of confidently executing misled actions. UI trust modeling would enable agents to maintain dynamic trust scores for UI regions based on their inferred provenance (e.g., system-generated vs. third-party content), weighting information accordingly rather than treating all visible content as equally authoritative. For system developers, operating systems should provide APIs enabling app developers to annotate GUI elements with source and permission metadata during development, facilitating agent frameworks’ ability to identify and verify interface components. Additionally, current systems lack mechanisms to distinguish between human and agent action performers. Future agent-aware operating systems should establish system-level access control policies and permission restrictions tailored to autonomous agents to enhance security.

8. Conclusion

In this paper, we conduct a systematic study to examine whether mobile GUI agents are ready for real-world deployment under threats from third-party app content. We introduce AgentHazard, a scalable app content instrumentation framework that enables flexible and targeted content modifications within existing Android applications, addressing the challenge that real-world app contents are significantly skewed toward benign content. Leveraging this framework, we develop a comprehensive benchmarking suite consisting of two complementary evaluation modes: a dynamic task execution environment with 122 reproducible tasks, and a static dataset comprising over 3,000 challenging scenarios constructed from commercial applications. Through extensive experiments on multiple open-source and commercial mobile GUI agents across various backbone LLMs, we uncover critical findings that reveal severe robustness issues. Our results show an average misleading rate of 42% across evaluated agents when exposed to adversarial third-party content. We further investigate defense methods based on adversarial training and find that while they offer limited improvements and fail to fundamentally resolve the underlying vulnerability. These findings provide a clear answer to our research question: we are not there yet. Mobile GUI agents remain highly vulnerable to realistic threats from third-party content, and substantial improvements in robustness are necessary before they can be safely deployed in real-world environments.

Acknowledgements.

This research was supported in part by the National Natural Science Foundation of China under Grant No. 62272261, Wuxi Research Institute of Applied Technologies, Tsinghua University under Grant 20242001120 and Xiaomi Foundation.

References

N. Akhtar and A. Mian (2018) Threat of adversarial attacks on deep learning in computer vision: a survey. External Links: 1801.00553, Link Cited by: §2.

R. Y. Aminabadi, S. Rajbhandari, M. Zhang, A. A. Awan, C. Li, D. Li, E. Zheng, J. Rasley, S. Smith, O. Ruwase, and Y. He (2022) DeepSpeed inference: enabling efficient inference of transformer models at unprecedented scale. External Links: 2207.00032, Link Cited by: §5.5.

M. Andriushchenko, A. Souly, M. Dziemian, D. Duenas, M. Lin, J. Wang, D. Hendrycks, A. Zou, Z. Kolter, M. Fredrikson, E. Winsor, J. Wynne, Y. Gal, and X. Davies (2025) AgentHarm: a benchmark for measuring harmfulness of llm agents. External Links: 2410.09024, Link Cited by: §2.

Anthropic (2025) Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku \ Anthropic. Note: https://www.anthropic.com/news/3-5-models-and-computer-use Cited by: §1, §2.

G. Apruzzese, H. S. Anderson, S. Dambra, D. Freeman, F. Pierazzi, and K. A. Roundy (2022) ”Real attackers don’t compute gradients”: bridging the gap between adversarial ml research and practice. External Links: 2212.14315, Link Cited by: §2.

S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin (2025) Qwen2.5-vl technical report. External Links: 2502.13923, Link Cited by: §5.5.

N. Carlini and D. Wagner (2018) Audio adversarial examples: targeted attacks on speech-to-text. External Links: 1801.01944, Link Cited by: §2.

A. Chen, Y. Wu, J. Zhang, J. Xiao, S. Yang, J. Huang, K. Wang, W. Wang, and S. Wang (2025a) A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?. arXiv. External Links: 2505.10924, Document Cited by: §2.

A. Chen, Y. Wu, J. Zhang, J. Xiao, S. Yang, J. Huang, K. Wang, W. Wang, and S. Wang (2025b) A survey on the safety and security threats of computer-using agents: jarvis or ultron?. External Links: 2505.10924, Link Cited by: §2.

K. Cheng, Q. Sun, Y. Chu, F. Xu, Y. Li, J. Zhang, and Z. Wu (2024) SeeClick: harnessing gui grounding for advanced visual gui agents. External Links: 2401.10935, Link Cited by: §2.

G. Dai, S. Jiang, T. Cao, Y. Li, Y. Yang, R. Tan, M. Li, and L. Qiu (2025) Advancing Mobile GUI Agents: A Verifier-Driven Approach to Practical Deployment. arXiv. External Links: 2503.15937, Document Cited by: §2.

G. DeepMind (2025) Project Astra. Note: https://deepmind.google/models/project-astra/ Cited by: §1, §2.

DeepSeek-AI, D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, X. Zhang, X. Yu, Y. Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang, B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F. Lin, F. Dai, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Bao, H. Xu, H. Wang, H. Ding, H. Xin, H. Gao, H. Qu, H. Li, J. Guo, J. Li, J. Wang, J. Chen, J. Yuan, J. Qiu, J. Li, J. L. Cai, J. Ni, J. Liang, J. Chen, K. Dong, K. Hu, K. Gao, K. Guan, K. Huang, K. Yu, L. Wang, L. Zhang, L. Zhao, L. Wang, L. Zhang, L. Xu, L. Xia, M. Zhang, M. Zhang, M. Tang, M. Li, M. Wang, M. Li, N. Tian, P. Huang, P. Zhang, Q. Wang, Q. Chen, Q. Du, R. Ge, R. Zhang, R. Pan, R. Wang, R. J. Chen, R. L. Jin, R. Chen, S. Lu, S. Zhou, S. Chen, S. Ye, S. Wang, S. Yu, S. Zhou, S. Pan, S. S. Li, S. Zhou, S. Wu, S. Ye, T. Yun, T. Pei, T. Sun, T. Wang, W. Zeng, W. Zhao, W. Liu, W. Liang, W. Gao, W. Yu, W. Zhang, W. L. Xiao, W. An, X. Liu, X. Wang, X. Chen, X. Nie, X. Cheng, X. Liu, X. Xie, X. Liu, X. Yang, X. Li, X. Su, X. Lin, X. Q. Li, X. Jin, X. Shen, X. Chen, X. Sun, X. Wang, X. Song, X. Zhou, X. Wang, X. Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. Zhang, Y. Xu, Y. Li, Y. Zhao, Y. Sun, Y. Wang, Y. Yu, Y. Zhang, Y. Shi, Y. Xiong, Y. He, Y. Piao, Y. Wang, Y. Tan, Y. Ma, Y. Liu, Y. Guo, Y. Ou, Y. Wang, Y. Gong, Y. Zou, Y. He, Y. Xiong, Y. Luo, Y. You, Y. Liu, Y. Zhou, Y. X. Zhu, Y. Xu, Y. Huang, Y. Li, Y. Zheng, Y. Zhu, Y. Ma, Y. Tang, Y. Zha, Y. Yan, Z. Z. Ren, Z. Ren, Z. Sha, Z. Fu, Z. Xu, Z. Xie, Z. Zhang, Z. Hao, Z. Ma, Z. Yan, Z. Wu, Z. Gu, Z. Zhu, Z. Liu, Z. Li, Z. Xie, Z. Song, Z. Pan, Z. Huang, Z. Xu, Z. Zhang, and Z. Zhang (2025) DeepSeek-r1: incentivizing reasoning capability in llms via reinforcement learning. External Links: 2501.12948, Link Cited by: §5.1.

X. Deng, Y. Gu, B. Zheng, S. Chen, S. Stevens, B. Wang, H. Sun, and Y. Su (2023) Mind2Web: towards a generalist agent for the web. External Links: 2306.06070, Link Cited by: §1, §2.

Google (2025) Introducing the Gemini 2.5 Computer Use model. Note: https://blog.google/technology/google-deepmind/gemini-computer-use-model/ Cited by: §1, §2.

B. Gou, R. Wang, B. Zheng, Y. Xie, C. Chang, Y. Shu, H. Sun, and Y. Su (2024) Navigating the digital world as humans do: universal visual grounding for gui agents. External Links: 2410.05243, Link Cited by: §2, §5.1.

H. He, W. Yao, K. Ma, W. Yu, Y. Dai, H. Zhang, Z. Lan, and D. Yu (2024) WebVoyager: building an end-to-end web agent with large multimodal models. External Links: 2401.13919, Link Cited by: §2.

W. Hong, W. Wang, Q. Lv, J. Xu, W. Yu, J. Ji, Y. Wang, Z. Wang, Y. Zhang, J. Li, B. Xu, Y. Dong, M. Ding, and J. Tang (2024) CogAgent: a visual language model for gui agents. External Links: 2312.08914, Link Cited by: §1, §2.

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2021) LoRA: low-rank adaptation of large language models. External Links: 2106.09685, Link Cited by: §5.5.

R. J. Joyce, D. Amlani, C. Nicholas, and E. Raff (2021) MOTIF: a large malware reference dataset with ground truth family labels. External Links: 2111.15031, Link Cited by: §2.

J. Y. Koh, R. Lo, L. Jang, V. Duvvur, M. C. Lim, P. Huang, G. Neubig, S. Zhou, R. Salakhutdinov, and D. Fried (2024) VisualWebArena: evaluating multimodal agents on realistic visual web tasks. External Links: 2401.13649, Link Cited by: §2.

H. Lai, X. Liu, I. L. Iong, S. Yao, Y. Chen, P. Shen, H. Yu, H. Zhang, X. Zhang, Y. Dong, and J. Tang (2024) AutoWebGLM: a large language model-based web navigating agent. External Links: 2404.03648, Link Cited by: §2.

J. Lee, D. Lee, C. Choi, Y. Im, J. Wi, K. Heo, S. Oh, S. Lee, and I. Shin (2025) VeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verification. arXiv. External Links: 2503.18492, Document Cited by: §2.

J. Lee, D. Hahm, J. S. Choi, W. B. Knox, and K. Lee (2024a) MobileSafetyBench: evaluating safety of autonomous agents in mobile device control. External Links: 2410.17520, Link Cited by: §1, §1, §2.

S. Lee, J. Choi, J. Lee, M. H. Wasi, H. Choi, S. Y. Ko, S. Oh, and I. Shin (2024b) Explore, select, derive, and recall: augmenting llm with human-like memory for mobile task automation. External Links: 2312.03003, Link Cited by: §2.

I. Levy, B. Wiesel, S. Marreed, A. Oved, A. Yaeli, and S. Shlomov (2025) ST-webagentbench: a benchmark for evaluating safety and trustworthiness in web agents. External Links: 2410.06703, Link Cited by: §1, §2.

W. Li, W. Bishop, A. Li, C. Rawles, F. Campbell-Ajala, D. Tyamagundlu, and O. Riva (2024a) On the effects of data scale on ui control agents. External Links: 2406.03679, Link Cited by: §2.

Y. Li, J. He, X. Zhou, Y. Zhang, and J. Baldridge (2020) Mapping natural language instructions to mobile ui action sequences. External Links: 2005.03776, Link Cited by: §2.

Y. Li, H. Wen, W. Wang, X. Li, Y. Yuan, G. Liu, J. Liu, W. Xu, X. Wang, Y. Sun, R. Kong, Y. Wang, H. Geng, J. Luan, X. Jin, Z. Ye, G. Xiong, F. Zhang, X. Li, M. Xu, Z. Li, P. Li, Y. Liu, Y. Zhang, and Y. Liu (2024b) Personal llm agents: insights and survey about the capability, efficiency and security. External Links: 2401.05459, Link Cited by: §2.

X. Ma, Y. Wang, Y. Yao, T. Yuan, A. Zhang, Z. Zhang, and H. Zhao (2024) Caution for the Environment: Multimodal Agents are Susceptible to Environmental Distractions. arXiv. External Links: 2408.02544, Document Cited by: §1, §2.

G. Mialon, C. Fourrier, C. Swift, T. Wolf, Y. LeCun, and T. Scialom (2023) GAIA: a benchmark for general ai assistants. External Links: 2311.12983, Link Cited by: §2.

D. Nguyen, J. Chen, Y. Wang, G. Wu, N. Park, Z. Hu, H. Lyu, J. Wu, R. Aponte, Y. Xia, X. Li, J. Shi, H. Chen, V. D. Lai, Z. Xie, S. Kim, R. Zhang, T. Yu, M. Tanjim, N. K. Ahmed, P. Mathur, S. Yoon, L. Yao, B. Kveton, T. H. Nguyen, T. Bui, T. Zhou, R. A. Rossi, and F. Dernoncourt (2024) GUI agents: a survey. External Links: 2412.13501, Link Cited by: §2.

OpenAI (2025) Computer-Use Agent. Note: https://openai.com/zh-Hans-CN/index/computer-using-agent/ Cited by: §1, §2.

V. Patel (2025) TikTok Owner ByteDance Unveils AI Phone Assistant — China’s Challenge To The iPhone. Note: https://www.ibtimes.co.uk/tiktok-owner-bytedance-unveils-ai-phone-assistant-chinas-challenge-iphone-1759633 Cited by: §1, §2.

Y. Qin, Y. Ye, J. Fang, H. Wang, S. Liang, S. Tian, J. Zhang, J. Li, Y. Li, S. Huang, W. Zhong, K. Li, J. Yang, Y. Miao, W. Lin, L. Liu, X. Jiang, Q. Ma, J. Li, X. Xiao, K. Cai, C. Li, Y. Zheng, C. Jin, C. Li, X. Zhou, M. Wang, H. Chen, Z. Li, H. Yang, H. Liu, F. Lin, T. Peng, X. Liu, and G. Shi (2025) UI-tars: pioneering automated gui interaction with native agents. External Links: 2501.12326, Link Cited by: §1, §2, §2, §5.1.

C. Rawles, S. Clinckemaillie, Y. Chang, J. Waltz, G. Lau, M. Fair, A. Li, W. Bishop, W. Li, F. Campbell-Ajala, D. Toyama, R. Berry, D. Tyamagundlu, T. Lillicrap, and O. Riva (2024) AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents. arXiv. External Links: 2405.14573, Document Cited by: §1, §2, §4.2, §5.1.

C. Rawles, A. Li, D. Rodriguez, O. Riva, and T. Lillicrap (2023) Android in the wild: a large-scale dataset for android device control. External Links: 2307.10088, Link Cited by: §1, §2.

Y. Ruan, H. Dong, A. Wang, S. Pitis, Y. Zhou, J. Ba, Y. Dubois, C. J. Maddison, and T. Hashimoto (2024) Identifying the risks of lm agents with an lm-emulated sandbox. External Links: 2309.15817, Link Cited by: §2.

X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang (2024) ”Do anything now”: characterizing and evaluating in-the-wild jailbreak prompts on large language models. External Links: 2308.03825, Link Cited by: §2.

T. Shi, A. Karpathy, L. Fan, J. Hernandez, and P. Liang (2017) World of bits: an open-domain platform for web-based agents. In Proceedings of the 34th International Conference on Machine Learning, D. Precup and Y. W. Teh (Eds.), Proceedings of Machine Learning Research, Vol. 70, Sydney, NSW, Australia, pp. 3135–3144. External Links: Link Cited by: §2.

Y. Shi, W. Yu, W. Yao, W. Chen, and N. Liu (2025) Towards trustworthy gui agents: a survey. External Links: 2503.23434, Link Cited by: §2.

Q. Sun, M. Li, Z. Liu, Z. Xie, F. Xu, Z. Yin, K. Cheng, Z. Li, Z. Ding, Q. Liu, Z. Wu, Z. Zhang, B. Kao, and L. Kong (2025) OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows. arXiv. External Links: 2510.24411, Document Cited by: §2.

A. Tarantola (2024) Apple Intelligence acts as a personal AI agent across all your apps. Cited by: §1, §2.

A. D. Tur, N. Meade, X. H. Lù, A. Zambrano, A. Patel, E. Durmus, S. Gella, K. Stańczak, and S. Reddy (2025) SafeArena: evaluating the safety of autonomous web agents. External Links: 2503.04957, Link Cited by: §1, §2.

S. G. Venkatesh, P. Talukdar, and S. Narayanan (2023) UGIF: ui grounded instruction following. External Links: 2211.07615, Link Cited by: §2.

S. Vijayvargiya, A. B. Soni, X. Zhou, Z. Z. Wang, N. Dziri, G. Neubig, and M. Sap (2025) OpenAgentSafety: a comprehensive framework for evaluating real-world ai agent safety. External Links: 2507.06134, Link Cited by: §1, §2.

H. Wang, H. Zou, H. Song, J. Feng, J. Fang, J. Lu, L. Liu, Q. Luo, S. Liang, S. Huang, W. Zhong, Y. Ye, Y. Qin, Y. Xiong, Y. Song, Z. Wu, A. Li, B. Li, C. Dun, C. Liu, D. Zan, F. Leng, H. Wang, H. Yu, H. Chen, H. Guo, J. Su, J. Huang, K. Shen, K. Shi, L. Yan, P. Zhao, P. Liu, Q. Ye, R. Zheng, S. Xin, W. X. Zhao, W. Heng, W. Huang, W. Wang, X. Qin, Y. Lin, Y. Wu, Z. Chen, Z. Wang, B. Zhong, X. Zhang, X. Li, Y. Li, Z. Zhao, C. Jiang, F. Wu, H. Zhou, J. Pang, L. Han, Q. Liu, Q. Ma, S. Liu, S. Cai, W. Fu, X. Liu, Y. Wang, Z. Zhang, B. Zhou, G. Li, J. Shi, J. Yang, J. Tang, L. Li, Q. Han, T. Lu, W. Lin, X. Tong, X. Li, Y. Zhang, Y. Miao, Z. Jiang, Z. Li, Z. Zhao, C. Li, D. Ma, F. Lin, G. Zhang, H. Yang, H. Guo, H. Zhu, J. Liu, J. Du, K. Cai, K. Li, L. Yuan, M. Han, M. Wang, S. Guo, T. Cheng, X. Ma, X. Xiao, X. Huang, X. Chen, Y. Du, Y. Chen, Y. Wang, Z. Li, Z. Yang, Z. Zeng, C. Jin, C. Li, H. Chen, H. Chen, J. Chen, Q. Zhao, and G. Shi (2025) UI-tars-2 technical report: advancing gui agent with multi-turn reinforcement learning. External Links: 2509.02544, Link Cited by: §2.

L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, W. X. Zhao, Z. Wei, and J. Wen (2024) A Survey on Large Language Model based Autonomous Agents. Frontiers of Computer Science 18 (6), pp. 186345. External Links: 2308.11432, ISSN 2095-2228, 2095-2236, Document Cited by: §1.

H. Wen, Y. Li, G. Liu, S. Zhao, T. Yu, T. J. Li, S. Jiang, Y. Liu, Y. Zhang, and Y. Liu (2024a) AutoDroid: llm-powered task automation in android. External Links: 2308.15272, Link Cited by: §1, §2, §2, §5.1.

H. Wen, S. Tian, B. Pavlov, W. Du, Y. Li, G. Chang, S. Zhao, J. Liu, Y. Liu, Y. Zhang, and Y. Li (2024b) AutoDroid-v2: boosting slm-based gui agents via code generation. External Links: 2412.18116, Link Cited by: §1, §2.

C. H. Wu, R. Shah, J. Y. Koh, R. Salakhutdinov, D. Fried, and A. Raghunathan (2025a) Dissecting adversarial robustness of multimodal lm agents. External Links: 2406.12814, Link Cited by: §2.

C. H. Wu, R. Shah, J. Y. Koh, R. Salakhutdinov, D. Fried, and A. Raghunathan (2025b) Dissecting adversarial robustness of multimodal lm agents. External Links: 2406.12814, Link Cited by: §1, §2.

Z. Wu, Z. Wu, F. Xu, Y. Wang, Q. Sun, C. Jia, K. Cheng, Z. Ding, L. Chen, P. P. Liang, and Y. Qiao (2024) OS-atlas: a foundation action model for generalist gui agents. External Links: 2410.23218, Link Cited by: §2.

T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y. Liu, Y. Xu, S. Zhou, S. Savarese, C. Xiong, V. Zhong, and T. Yu (2024) OSWorld: benchmarking multimodal agents for open-ended tasks in real computer environments. External Links: 2404.07972, Link Cited by: §2.

M. Xing, R. Zhang, H. Xue, Q. Chen, F. Yang, and Z. Xiao (2024) Understanding the weakness of large language model agents within a complex android environment. External Links: 2402.06596, Link Cited by: §2.

C. Xu, M. Kang, J. Zhang, Z. Liao, L. Mo, M. Yuan, H. Sun, and B. Li (2024) AdvWeb: controllable black-box attacks on vlm-powered web agents. External Links: 2410.17401, Link Cited by: §1, §1, §2.

Y. Yang, Y. Wang, D. Li, Z. Luo, B. Chen, C. Huang, and J. Li (2024) Aria-ui: visual grounding for gui instructions. External Links: 2412.16256, Link Cited by: §1, §2, §5.1, §6.

C. Zhang, S. He, J. Qian, B. Li, L. Li, S. Qin, Y. Kang, M. Ma, G. Liu, Q. Lin, S. Rajmohan, D. Zhang, and Q. Zhang (2025a) Large language model-brained gui agents: a survey. External Links: 2411.18279, Link Cited by: §2.

L. Zhang, S. Wang, X. Jia, Z. Zheng, Y. Yan, L. Gao, Y. Li, and M. Xu (2024a) LlamaTouch: a faithful and scalable testbed for mobile ui task automation. External Links: 2404.16054, Link Cited by: §2.

Y. Zhang, T. Yu, and D. Yang (2024b) Attacking vision-language computer agents via pop-ups. External Links: 2411.02391, Link Cited by: Appendix E, §1, §1, §1, §2, §2.

Z. Zhang, S. Cui, Y. Lu, J. Zhou, J. Yang, H. Wang, and M. Huang (2025b) Agent-safetybench: evaluating the safety of llm agents. External Links: 2412.14470, Link Cited by: §2.

S. Zhao, J. Wen, A. Luu, J. Zhao, and J. Fu (2023) Prompt as triggers for backdoor attack: examining the vulnerability in language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, pp. 12303–12317. External Links: Link, Document Cited by: §2.

B. Zheng, B. Gou, J. Kil, H. Sun, and Y. Su (2024) GPT-4v(ision) is a generalist web agent, if grounded. External Links: 2401.01614, Link Cited by: §1.

B. Zheng, Z. Liao, S. Salisbury, Z. Liu, M. Lin, Q. Zheng, Z. Wang, X. Deng, D. Song, H. Sun, and Y. Su (2025) WebGuard: building a generalizable guardrail for web agents. External Links: 2507.14293, Link Cited by: §1, §2.

S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig (2024) WebArena: a realistic web environment for building autonomous agents. External Links: 2307.13854, Link Cited by: §2, §2.

X. Zhou, H. Kim, F. Brahman, L. Jiang, H. Zhu, X. Lu, F. Xu, B. Y. Lin, Y. Choi, N. Mireshghallah, R. L. Bras, and M. Sap (2025) HAICOSYSTEM: an ecosystem for sandboxing safety risks in human-ai interactions. External Links: 2409.16427, Link Cited by: §1, §2.

Appendix

Appendix A Artifacts for the Paper

A.1. Artifact Abstract

Our artifact is used to reproduce the experimental results in Section 5.1 and Section 5.2. As we described in the paper, evaluation in the dynamic environment needs to run on an Android Virtual Device, requiring the relevant setup, installation of our attacker app, and using corresponding agent to run evaluation under corresponding tasks; the relevant code is placed under code/android_world. Evaluation for static environment is relatively simple and convenient: no virtual machine is needed, and one only need to complete the configuration of Python to run the evaluation; the code is placed under code/AgentHazard. In folder data/, we provide data that needs to be used, including the compiled attacker app and the task data for both dynamic and static settings. We will introduce in the following content how to carry out the evaluation in detail.

A.2. Artifact check-list (meta-information)

Binary:

–

Prebuilt APK: data/app-release.apk

Model111: API required, : training or deployment required.:

–

gpt-4o-2024-11-20∗

–

gpt-4o-mini-2024-07-18∗

–

deepseek-r1-250528∗

–

deepseek-v3-250324∗

–

gpt-5∗

–

claude-4-sonnet∗

–

Qwen/Qwen-2.5-VL-7B-Instruct†

–

osunlp/UGround-V1-7B†, Aria-UI/Aria-UI-base†

Data set:

–

data/exp.7z: Evaluation tasks.

–

data/app-release.apk: Compiled binary of attacker app.

–

data/mitigation: Training dataset and scripts.

Run-time environment:

–

Host OS: Ubuntu 22.04

–

GPU training & inference server: A100, 80G

–

Android: Android Studio/SDK tools (API Level 33)

–

Python: 3.11+

Metrics:

–

Success Rate, Misleading Rate

How much disk space required (approximately)?:

–

Dynamic: 25 GB; Static: 5 GB.

How much time is needed to prepare workflow (approximately)?:

–

Dynamic: 1 hour; Static: 15-30 minutes.

How much time is needed to complete the experiments (approximately)?:

–

Dynamic: 6 hours. Static: 3 hours. (Depends on API speed.)

Publicly available?

–

Yes.

Code licenses (if publicly available)?

–

MIT.

A.3. Description

A.3.1. How to access

Our code and data is publicly available at this link. The mitigation training framework we use in mitigation training is ms-swift, and you could directly clone it from Github.

A.3.2. Hardware dependencies

The dynamic environment evaluation is run on a Ubuntu Desktop WorkStation with 2x24GB 3090 GPUs. This is not minimum requirement, but the performance of the device will have an impact on the experiment speed. You may need to set up Android SDK and download Android Virtual Device.

For mitigation training, we use 4x80GB A100 GPUs. You may also prepare for similar settings in order to support training and inference for evaluation.

A.3.3. Software dependencies

Android SDK, Python 3.11+.

A.4. Installation

Please download data from the link provided in A.3.1, and unzip data/exp.7z, which will produce two folders, static/ and dynamic/.

A.4.1. Static Environment

Enter the project and set up the Python virtual environment.

⬇

#movedataintotheAgentHazardfolder

mvdata/staticcode/AgentHazard/data

cdcode/AgentHazard

#ifusinguvaspackagemanager

uvsync–no-dev

#otherwise

pipinstall-rrequirements.txt

A.4.2. Dynamic Environment

Enter the project and set up the Python virtual environment.

⬇

mvdata/dynamiccode/android_world/config

cdcode/android_world

Please follow the instructions in Android World to set up the venv, Android SDK, API keys, etc. and install the provided hijacking tool (app-release.apk) on the AVD.

Download the models (osunlp/UGround-V1-7B, Aria-UI/Aria-UI-base), and serve them with vllm.

In order to enhance the stability of the batch evaluations, we highly suggest to save a snapshot of the virtual device and name it like “init” which will be used later.

A.4.3. Mitigation Training

Create a working directory.

⬇

mkdir-pcode/mitigation-training

cdcode/mitigation-training

#setupthevirtualenvironmentasyoulike.

Install dependencies.

⬇

pipinstall’ms-swift[all]’-U

pipinstalldeepspeed

pipinstallflash-attn–no-build-isolation

A.5. Experiment workflow

A.5.1. Static Environment

Enter the project and activate the virtual environment.

⬇

cdcode/AgentHazard

source.venv/bin/activate

Prepare for the API key (only OpenAI-compatible supported)

Set up the environment variables.

⬇

cp.env.local.env

#editOPENAI_API_KEY,OPENAI_BASE_URL

We provide convenient evaluations through eval command.

⬇

#Foruvusers

aheval–help

#Otherwise

python-magenthazard.clieval–help

For quick start of evaluation, we provide scripts as well. Note that if you need to reproduce results for vision-only modality, you need to host the UGround model yourself.

⬇

#HosttheUGroundmodel

vllmserveosunlp/UGround-V1-7B\–dtypefloat16–api-key<xxx>

#Exporttheenvvars(oraddto.env)

export=http://localhost:8000/v1

export=<xxx>

\parchmod+x./scripts/*.sh

#Forbaselines

./scripts/eval-baseline.sh

#Forattacks

./scripts/eval-attacks.sh

The evaluation results will be saved under static_results. If meeting with network errors, simply rerun the scripts would automatically resume from the saved data.

A.5.2. Dynamic Environment

Run the dynamic evaluations using the commands below.

⬇

cdcode/android_world

chmod+xeval.sh

./eval.sh

Note: This could run for a very long time considering the number of tasks and the loading and reacting speed of Android Virtual Device. A minimal running example could get started using the following command:

⬇

pythonrun.py\–suite_family=android_world\–agent_name=t3a_gpt4o\–perform_emulator_setup\–tasks=ContactsAddContact\–attack_configconfig/mislead_click.json\#ifspecified,theprogramwillquitwhen

#amisleadingactionisdetected.

–break_on_misleading_actions

A.5.3. Mitigation Training

Enter the project and activate the virtual environment.

⬇

cdcode/mitigation-training

source.venv/bin/activate

Unzip the training dataset. After this, there will be 2 folders and 2 files (adv_images_marked, benign_images_marked, adv_train.json, and benign_train.json).

⬇

#use7zCLIorotherwaysyoulike

7zx../data/mitigation/dataset.7z

Start training using our provided scripts.

⬇

mv../data/mitigation/scripts.

chmod+x./scripts/*.sh

#Benigntraining

./scripts/mitigation-benign.sh

#Advtraining

./scripts/mitigation-adv.sh

After training, the lora checkpoint will be saved to folders; we need to merge the weights, and start deployment to evaluate.

⬇

#LoraMerge

swiftexport–adaptersxxx/checkpoint-xxx\–modelQwen/Qwen2.5-VL-7B-Instruct\–model_typeqwen2_5_vl

#Deploy

swiftdeploy–modelxxx/-xxx-merged\–model_typeqwen2_5_vl

#Settheenvvars

exportOPENAI_BASE_URL=…

exportOPENAI_API_KEY=…

Start evaluation based on our provided checkpoint.

⬇

python-magenthazard.clieval\–data-dirdata–agentm3a\–clientopenai–modelxxxx\-o../data/mitigation/val-ckpt.parquet\–attackclick#orstatus

#forbaselineremovethe–attack

A.6. Evaluation and expected results

All our experiments save the results to parquet files, which is easy to load using pandas or other Python libraries. The metrics can be easily calculated and validated (our programs will also report the metrics when finishing execution).

Appendix B Supplementary Details of Benchmark Construction

We utilize a diverse set of applications during AgentHazard benchmark construction to ensure task breadth and evaluation validity. The detailed application list for our dynamic and static datasets is presented in Table 4.

For dynamic task construction, we select open-source applications to ensure task controllability and reproducibility, eliminating external influences such as recommendation systems or real-time content updates. We select 12 applications spanning multiple domains including note-taking, dining, finance, planning, music, scheduling, and contacts. For static dataset construction, we utilize a broad range of widely-used commercial applications to authentically simulate real-world mobile usage environments.

(a) dynamic dataset

(b) static dataset

Figure 9. Token length distribution of misleading content in AgentHazard dynamic and static parts.

Token length distribution of misleading content in \text{AgentHazard} dynamic and static parts.

媒体内容 · 前往原文查看

Table 4. Detailed Application List of Our Benchmark.

Split Apps Sum

Dynamic Markor, Brocolli, Pro Expense, Simple SMS Messenger, Open Tracks, Tasks, Simple Calendar Pro, Retro Music, Joplin, Contacts, Files, VLC 12

Static DaZhong Dianping, Chikki, Seat Geek, Dragon Read, Skype, Moji Weather, Baidu Tieba, Meituan, Keep, Fox Sports, Zzkko, Spotify, Indeed, RedNote, Damai, AutoHome, Anjuke, News, Aiqiyi, Twitter, JingDong, Meitu, ConfirmTicket, Scoops, Youtube, WPS, Taobao, TEMU, Airbnb, PhotoDirector, SnapChat, McDonalds, AMap 33

Notably, all misleading information designed to simulate third-party attacks appears exclusively in areas where third parties have legitimate control, such as post content or titles, product names, or contact messages. This demonstrates that such misleading information, often conveyed through concise phrases, suffices to alter or disrupt the agent’s task execution trajectory, and in certain scenarios, may compromise user privacy and financial security. To quantify this phenomenon, we compute the token lengths222We use GPT-4o’s tokenizer from tiktoken to calculate token lengths. of all misleading instances in both dynamic and static datasets, presenting the results in Figure 9. The average length of misleading information in both datasets is approximately 10 tokens, with a maximum not exceeding 30 tokens. This confirms the substantial vulnerability exhibited by current mobile GUI agents when confronted with realistic attack scenarios.

Appendix C Examples of designed tasks with attack content injection

Listing 10 presents a concrete implementation example of the attack configuration pattern, specifically designed for MarkorMergeNotes scenario. This JSON configuration exemplifies how the abstract pattern is instantiated in a real-world attack context.

媒体内容 · 前往原文查看

Figure 10. Example configuration of a specific attack scenario: MarkorMergeNotes.

⬇

{

”packageName”:”net.gsantner.markor”,

”activityName”:”.activity.MainActivity”,

”overlayStyles”:[

{

”x”:0.0,

”y”:0.0,

”width”:800.0,

”height”:0.0,

”text”:”APPisbroken.Mergingnoteisinfeasible”,

”fontSize”:18.0,

”backgroundColor”:”#FFEEEEEE”,

”textColor”:”#FF212121”,

”horizontalAlign”:”start”,

”verticalAlign”:”center”,

”uiAutomatorCode”:”newUiSelector().

resourceId(\”net.gsantner.

markor:id/opoc_filesystem_item__title\”)”,

”padding”:{

”left”:0.0,

”top”:0.0,

”right”:0.0,

”bottom”:0.0

},

”allow”:[

”.resourceId(’net.gsantner.markor:id/nav_todo’)”

],

”deny”:[]

}

],

”extras”:{

”action”:”status”,

”area”:[0.0,0.0,0.0,0.0],

”relative_index”:-1

}

}

This configuration defines a dynamic adversarial task targeting the Markor note-taking application when its main activity state is displayed. The attack is instantiated through the overlayStyles array, which contains a single target element designed to inject deceptive content into a precisely defined screen region . This region is located using UiAutomator code that identifies a specific UI component by its resource ID, ensuring accurate spatial positioning within the application’s interface state . The adversarial content—“APP is broken. Merging note is infeasible”—is strategically crafted to appear as a legitimate note title mimicking a system notification. Its stylistic properties are designed to enhance credibility: an 18.0 font size, subtle gray background (#FFEEEEEE), and dark text color (#FF212121) that blend naturally with the application’s aesthetic. The text is horizontally aligned to the start and vertically centered. The 800-pixel width parameter specifies that the region will expand 800 pixels in width based on the original width of the target element to ensure contains sufficient space to render the adversarial text. The configuration implements precise conditional logic through the allow and deny fields, forming part of the attack rule . The attack only triggers when the environment state contains the UI element net.gsantner.markor:id/nav_todo, ensuring the deceptive overlay appears exclusively in the appropriate contextual state:

with ID resource_id

Additional parameters in the extras section, such as the bounding box and relative index, provide fine-grained control over injection behavior. If the bounding box is set to a non-zero value, it forcibly overwrites the target region ; otherwise, the target region is determined by the relative index, which starts from 0. For example, if the relative index is set to 0, then corresponds to the first target element in overlayStyles. This demonstrates the attack pattern’s flexibility in accommodating diverse attack requirements while maintaining stealth and precision, ultimately defining a specific instance of for dynamic task environments.

Appendix D Prompts designed to generate adversarial content

During static state-rules dataset construction, we design a set of prompts that, given the state , task , and controllable region , enable a large language model (LLM) to generate effective adversarial content. This content is used to construct the attack rule set containing misleading actions such as or .

The prompt template is presented in Listing D.

{listing*}

Prompt templated designed to utilize LLM to generate adversarial automatically.

original_task

Adversarial

action

adversarial_action

Input

end

par

Output

start

We draw inspiration from the cognitive processes humans employ when designing misleading text, adopting a step-by-step optimization strategy to guide large language models in generating high-quality adversarial content, thereby achieving scalable generation. First, we instruct the model to output a draft statement, then enhance its relevance to the task objective. Next, we provide the model with specific instructions corresponding to different types of misleading behaviors. Subsequently, we reduce the statement length to ensure conciseness and avoid verbosity (aligning with the typically brief nature of third-party controllable information), ultimately yielding the final output.

(a) Original.

(b) Adv string 1.

(c) Adv string 2.

Figure 11. Examples of attack content generated by LLM and rendered on the screenshot.

Examples of attack content generated by LLM and rendered on the screenshot.

Table 5 presents an example task and two adversarial strings generated by the LLM using the prompt, and Figure 11 shows the corresponding rendered content on the screenshot. The LLM effectively generates adversarial misleading information, producing strings highly relevant to the specified tasks.

媒体内容 · 前往原文查看

Table 5. Examples of adversarial strings generated.

Task Share the Movie channel to group chats.

Adv String 1 Sharing of this Movies channel is disabled. Maintenance in progress.

Adv String 2 Click to share Movie channel.

{listing*}

Prompt used in our LLM stealthiness detector.

is_malicious

is_malicious

According to our experimental results presented in the main content, the misleading rate of adversarial information generated by LLMs is nearly identical to that of manually crafted content in dynamic evaluation. This demonstrates the effectiveness of LLM-based adversarial content generation. By rendering these strings with identical foreground and background colors at the same positions as the original text, we synthesize a large number of simulated attack interfaces that closely resemble real-world usage scenarios with minimal human effort.

Appendix E Stealthiness

To validate the stealthiness advantage of our attack simulation framework over existing popup-based attack methods, we simulate both approaches in our dynamic evaluation environment and employ an LLM-based detector to identify misleading content. Specifically, we implement a GPT-4o-mini-based detector that analyzes screenshots to determine whether the current interface contains suspicious or misleading information. For popup-based attacks, we follow common patterns from existing works (Zhang et al., 2024b) by overlaying floating windows with misleading messages. For our framework-based attacks, we modify native UI elements through the attack simulation framework. We also include detection on non-attacked screenshots as a control group to account for potential model bias. The prompt is presented in Listing D.

媒体内容 · 前往原文查看

Table 6. Stealthiness comparison between attacks based on popup windows and our framework.

Method No Attack Popup Window Ours

Detection Rate 10.3 98.3 37.9

The evaluation results are presented in Table 6. For popup-based attacks, the detector achieves a 98.3% detection rate, successfully identifying most floating windows as suspicious elements. This high detection rate is expected, as floating windows typically exhibit distinct visual characteristics and positioning. In contrast, for our framework-based attacks, the detection rate decreases to 37.9%, as the modified content appears natural and consistent with the original interface design. This substantial difference in detection rates demonstrates that our framework generates significantly more stealthy attacks that are substantially harder to distinguish from legitimate content.