BTL-UI: Blink-Think-Link Reasoning Model for GUI Agent

Dear Reviewer RGD1,

Thank you for your valuable suggestions. We sincerely appreciate your insights. We arrange our responses in the order your questions are raised, and hope they clarify your concerns.

Response to W1

We appreciate the reviewer's question regarding the selection of RFT dataset samples. As found in previous work [1], RFT requires higher data quality than SFT. We employ instruction action checks and blink calibration to select high-quality data for Blink data.

1. Is the Blink Phase Redundant with Attention in VLMs?

While existing VLMs use attention mechanisms to weight visual features of each patch, the Blink Phase explicitly models the biological dynamics of human visual attention. Specifically, saccadic eye movements rapidly locate task-relevant regions in discrete intervals. Unlike implicit attention weights in VLMs, Blink generates structured ROI annotations (<blink> tags) that mimic the "spotlight" of human visual focus. Moreover, previous approaches [2,3] have proven that Chain-of-Thought (CoT) can elicit reasoning in LLM. CoT demonstrates a strong ability to solve complex problems by explicitly outputting the reasoning process step by step, rather than directly answering. In dynamic GUI interaction tasks, BTL-UI mirrors how human visual attention combines stimulus salience and goal relevance by locating several key elements related to the user instruction and outputs them to promote further deep reasoning in the Think Phase. As shown in the ablation study (Table 5-a), if the model output is in the format of <think>...</think>/n<link>...</link> without Blink Phase, the success rate (SR) decreased by 3.8%. Therefore, we argue that the Blink Phase serves a distinct role that is not simply equivalent to built-in attention in existing VLMs.

2. If Qwen2.5-VL-32B Can Extract Elements, Why Not Use It Directly?

This is a fair and insightful question. We clarify our rationale below:

• We don't adopt Qwen2.5-VL-32B extract elements. Instead, in the first stage of Blink data generation, we extract candidate UI elements and their bounding boxes and types using OmniParser, which is an off-the-shelf rule-based UI parser.
• In the second stage, we use Qwen2.5-VL-32B as the analysis model, filtering and prioritizing UI elements from the first stage, based on task instructions and interaction history, to generate ROI annotations. Then, only data samples that can pass the blink calibration are selected as high-quality data. Specifically, each sample will generate several candidate elements with bboxes during the Blink data generation process. At the same time, the coordinates of the click and long-press actions are annotated in the original dataset. We check if the coordinates of the annotated click are within the bboxes.

In summary, we don't adopt Qwen2.5-VL-32B extract elements directly; instead, we produce high-quality data through a pipeline of elemental extraction, analysis, and verification. We will clarify this more explicitly in Section 3.2 and Figure 3 of the revision.

3. Do Performance Gains Come Merely from In-Domain Data?

We suppose the performance gains stem from the BTL framework, not just domain overlap. Recent studies [1] suggest that RFT only requires a small amount of high-quality data to stimulate the model's capabilities. And the goal of RFT is to stimulate the model's capabilities rather than inject new knowledge [4,5,6]. Therefore, the quality of training data is more important than the domain. Moreover, as we discussed in the first point and the comparison with the recent RFT-based GUI Agent on various benchmarks, BTL-UI achieves more competitive performance, which is due to the BTL framework, inspired by cognitive studies.

Response to W2

We thank the reviewer for pointing out the absence of direct comparison with UI-TARS and for highlighting the potential discrepancies in baseline numbers.

1. Comparison with UI-TARS

We acknowledge that UI-TARS is a relevant baseline, as it focuses on GUI interaction and shares benchmarks like ScreenSpot and AndroidControl. Our initial experimental comparison did not include UI-TARS due to oversight. We will add a comparison and detailed analysis with high-performing baselines, including UI-TARS and UGround-v1. A brief comparison is shown below:

• Screenspot: As disclosed in their technical report, UI-TARS is a strong GUI Agent developed based on Qwen2-VL with training on 50B tokens. While our BTL-UI is developed based on Qwen2.5-VL, with RFT on 4K samples. Although the overall grounding performance of BTL-UI is still lower than that of UI-TARS, it shows certain advantages on the subtasks of Mobile and Web. |Method|Size|MT|MI|DT|DI|WT|WI|Avg| |-|-|-|-|-|-|-|-|-| |UI-TARS|2B|95.2|79.1|90.7|68.6|87.2|78.3|84.7| |BTL-UI|3B|96.3|77.3|88.2|57.9|80.0|68.9|80.0| |UI-TARS|7B|94.5|85.2|95.9|85.7|90.0|83.5|89.5| |BTL-UI|7B|97.1|83.8|90.2|70.7|88.7|84.5|87.2|

• AndroidControl: As disclosed in their official repository, UI-TARS adopts the recent five images as input on multi-step interaction tasks for both training and inference, which is computationally expensive. However, in recent methods, a single image training and inference are adopted. Therefore, to ensure comparability, we reproduce the performance of UI-TARS based on the released checkpoint and prompt (). Although the overall performance of BTL-UI is still lower than that of UI-TARS, under the same experimental settings (single image), BTL-UI is superior to UI-TARS. |Method|Size|Images|High| |---|---|---|---| |UI-TARS|2B|5|68.9| |UI-TARS|3B|1|53.6| |BTL-UI|3B|1|63.4| |UI-TARS|7B|5|72.5| |UI-TARS*|7B|1|58.2| |BTL-UI|7B|1|69.2|

2. Clarification on OS-Atlas Score

Thanks for giving us the opportunity to clarify the performance issues regarding OS-Atlas.

• OS-Atlas is a GUI foundational model developed based on InternVL-2 and Qwen2-VL with Continuous Pre-Training (CPT) on GUI-specific tasks. OS-Atlas-Base (50.9 on AndroidControl-Low) is the model with CPT and without additional Supervised Fine-Tuning (SFT). Thus, as shown in 'Method' in Table 4, the experimental results of OS-Atlas-Base are zero-shot (ZS).
• Moreover, the results of OS-Atlas (85.2) in UI-TARS have been fine-tuned on the training set of AndroidControl.

For a more comprehensive comparison and to enhance the readability of the article, we will add experimental results of OS-Atlas-SFT to reduce misunderstandings. We hope our response clears things up for you.

Response to W3

Thanks for your thoughtful observation. While biological inspiration is not new in AI, our BTL-UI distinguishes itself through two unique dimensions:

1. Novelty in Domain-Specific Adaptation: Most prior biologically inspired methods [7] target general vision/language tasks. In contrast, we are the first to adapt biological visual cognition to GUI interaction, a domain with distinct challenges: dynamic interfaces, multi-modal elements including text, icons, and buttons, and task-dependent relevance. This domain shift is non-trivial, as GUI agents require modeling human-like interaction logic, which demands reinterpreting biological principles for goal-driven, interactive scenarios.
2. Mechanistic Depth Beyond Analogy: Our framework does not stop at superficial analogy to attention. Instead, it operationalizes specific biological processes:

• Blink Phase mirrors how human visual attention combines stimulus salience and goal relevance by locating several key elements related to the user instruction and outputs them to promote further deep reasoning in the Think Phase.
• BTL Reward Integrates process-outcome reward to simulate the “action-evaluation” loop in human cognition, where aided actions trigger re-focusing (Blink) on new ROIs. This mirrors how humans adjust visual attention based on interaction feedback, a mechanism absent in SFT of VLM.
• As shown in the ablation study (Table 5-a), if the model output is in the format of <think>...</think>/n<link>...</link> without Blink Phase, the success rate (SR) decreased by 3.8%.

In summary, our goal is not to replicate human cognition, but to abstract useful behavioral priors in GUI interaction tasks to improve the agent's visual perception and reasoning capabilities to enhance generalization.

Response to Q1

Thanks for raising this important question again. We have discussed this 'Response to W1'. Please kindly refer to it.

Response to Q2

Thanks for raising this insightful question again. We have discussed this 'Response to W1' and 'Response to W3'. Please kindly refer to them.

Response to Q3

Thanks for highlighting the relevance of UI-TARS as a strong baseline in the GUI agent domain again. We have discussed this 'Response to W2'. Please kindly refer to it.

[1] Liu Z, Sun Z, Zang Y, et al. Visual-rft: Visual reinforcement fine-tuning[J]. arXiv preprint arXiv:2503.01785, 2025.

[2] Wei J, Wang X, Schuurmans D, et al. Chain-of-thought prompting elicits reasoning in large language models[C]. Advances in Neural Information Processing Systems, 2022, 35: 24824-24837.

[3] Guo D, Yang D, Zhang H, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning[J]. arXiv preprint arXiv:2501.12948, 2025.

[4] Gandhi K, Chakravarthy A, Singh A, et al. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars[J]. arXiv preprint arXiv:2503.01307, 2025.

[5] Yue Y, Chen Z, Lu R, et al. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025[J]. arXiv preprint arXiv:2504.13837, 2025.

[6] Shah D J, Rushton P, Singla S, et al. Rethinking reflection in pre-training[J]. arXiv preprint arXiv:2504.04022, 2025.

[7] Fukushima K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological cybernetics, 1980, 36(4): 193-202.

Response to 'On Is the Blink Phase Redundant with Attention in VLMs?'

We appreciate the reviewer's concern regarding the potential overstatement of our framework’s novelty. We agree to localizing the relevant interface element itself, which is a well-established technique. However, our intention is not to claim innovation in isolated components, but rather to present the framework as a coherent and cognitively inspired processing pipeline that explicitly models human-like GUI interaction through structured perception, reasoning, and action stages, which can be seen in recent studies [1,2,3]. Each phase is individually supervised and functionally distinct, yet collectively optimized to support interpretable and generalizable predictions. We believe this end-to-end design, grounded in cognitive principles and validated through stage-wise ablations, is what sets our approach apart from prior work. We will revise our wording to better reflect this system-level contribution without overstating novelty at the component level.

[1] Li Y, Liu Z, Li Z, et al. Perception, reason, think, and plan: A survey on large multimodal reasoning models[J]. arXiv preprint arXiv:2505.04921, 2025.

[2] Sapkota R, Cao Y, Roumeliotis K I, et al. Vision-language-action models: Concepts, progress, applications and challenges[J]. arXiv preprint arXiv:2505.04769, 2025.

[3] Liu H, Guo D, Cangelosi A. Embodied intelligence: A synergy of morphology, action, perception and learning[J]. ACM Computing Surveys, 2025, 57(7): 1-36.

Response to 'On Comparison with UI-TARS'

Thanks for raising the question about the calculation of the average scores! We want to emphasize that the average scores in the first table are not the simple average of the six listed numbers, but the overall performance of ScreenSpot. The details of ScreenSot are shown as follows:

The average score of BTL-UI-7B can be calculated simply: $\frac{97.1 \times 273 + 83.8 \times 229 + 90.2 \times 194 + 70.7 \times 140 + 88.7 \times 230 + 84.5 \times 206}{1272} \approx 87.2$ The same rules apply to the calculation of UI-TARS and Qwen2.5-VL.

Dear Reviewer 96rE,

Thank you for your valuable feedback. We truly value your insights and are fully committed to addressing these points to enhance the quality of our work.

Response to W1

We appreciate the reviewer's question regarding the selection of RFT dataset samples. As found in previous work [1], RFT requires higher data quality than SFT. We adopt instruction action check and blink calibration to select high-quality data for Blink data.

1. Instruction Action Check: We found that there are samples in AndroidControl and GUI-Odyssey, where the labeled action doesn't match the instruction. To filter out high-quality samples, we use regular matching to extract keywords (e.g, click, scroll, and swipe) from the annotated actions and instructions, and verify their consistency.
2. Blink Calibration: Each sample will generate several candidate elements with bboxes during the Blink data generation process. At the same time, the coordinates of the click and long-press actions are annotated in the original dataset. Then, we check if the coordinates of the annotated click are within the bboxes. Only data samples that can be matched are selected as high-quality data.

Finally, we sample training data from source datasets after instruction action check and blink calibration, following a fixed random seed (2025) to ensure reproducibility. Moreover, we will release the training data to promote further study.

Response to W2

Thanks for seeking clarification regarding how bounding boxes are obtained in the first stage of our Blink data generation pipeline.

To clarify, the bounding boxes used in the Blink Phase are not provided by the original datasets, such as AndroidControl and GUI-Odyssey, as most of these datasets lack detailed UI element annotations. Moreover, as we discussed in 'Response to W1', these datasets only provide the coordinates of the click, but not the bounding boxes. Instead, we extract candidate UI elements and their bounding boxes using an off-the-shelf rule-based UI parser, which we refer to as OmniParser.

We will clarify this more explicitly in Section 3.2 and Figure 3 of the revised manuscript. Thank you again for highlighting the need for a clearer explanation of this step.

Response to W3

We appreciate the reviewer’s concern about the data quality from the second stage of the pipeline and its applicability to larger models.

1. Regarding data quality: The second stage, which uses Qwen2.5-VL-32B as the analysis model, filters and prioritizes UI elements based on task instructions and interaction history to generate ROI annotations. As we discussed in 'Response to W1', the data samples that pass the blink calibration process are selected as high-quality data. Moreover, recent studies [1] suggest that RFT only requires a small amount of high-quality data to stimulate the model's capabilities. Therefore, we suppose the capability of Qwen2.5-VL-32B has less influence on the final data quality.
2. Regarding applicability to larger models: The mainstream view of recent approaches [2,3,4] is that RFT can further stimulate the model's ability rather than injecting new knowledge into it. From this perspective, the capacity of base models and data quality are the key to determining the upper bound. As discussed in point 1, Qwen2.5-VL-32B is adopted as the analysis model to produce high-quality data, but not the knowledge distillation similar to a teacher-student network. Moreover, GUI-R1 adopted Qwen2.5-VL-7B to rollout high-quality data to RFT Qwen2.5-VL-3B and Qwen2.5-VL-7B, and both the performance improvements are solid. In summary, we believe that using the Blink data to RFT Qwen2.5-VL-32B, we can gain similar performance improvement.

Response to W4

Thanks for your careful reading and helpful feedback on writing clarity and presentation.

1. We will correct all spelling errors and inconsistencies, including:

• Replacing “BLink data”, “Blink Data”, and “Blikn data” with the consistent term “Blink data” throughout the paper.
• Fixing “archives an SR” to the correct phrasing “achieves an SR”.

2. We will also ensure that all acronyms (e.g., GRPO: Group Relative Policy Optimization) are properly defined when first introduced in the main text (Section 3.4) and included in the glossary of symbols in the appendix.

These revisions will be reflected in the updated version. We appreciate the reviewer’s attention to detail in helping improve the clarity and professionalism of the paper.

Response to Q1

We appreciate the reviewer’s follow-up request for more detailed information on the dataset used for RFT. We fully agree that a clear understanding of the data construction pipeline is essential for reproducibility and transparency. Regarding the confusion you mentioned, we have already responded point by point. We will clarify this more explicitly in Section 3.2 and Figure 3 of the revision.

[1] Liu Z, Sun Z, Zang Y, et al. Visual-rft: Visual reinforcement fine-tuning[J]. arXiv preprint arXiv:2503.01785, 2025.

[2] Gandhi K, Chakravarthy A, Singh A, et al. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars[J]. arXiv preprint arXiv:2503.01307, 2025.

[3] Yue Y, Chen Z, Lu R, et al. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?, 2025[J]. arXiv preprint arXiv:2504.13837, 2025.

[4] Shah D J, Rushton P, Singla S, et al. Rethinking reflection in pre-training[J]. arXiv preprint arXiv:2504.04022, 2025.

Dear Reviewer 96rE

Thanks for your constructive question and strong support. We really appreciate how carefully you considered our responses and your willingness to reconsider the paper's score. Your thoughtful feedback has been incredibly helpful in refining our research. We will continue working to conduct further experiments to enhance our work based on your valuable feedback.

Best regards,

Authors

Dear Reviewer NRM3,

Thank you for your valuable suggestions. We sincerely appreciate your insights. We arrange our responses in the order your questions are raised, and hope they clarify your concerns.

Response to W1

We sincerely thank you for raising this important point regarding latency, practical real-time optimization, and scalability, especially in the context of real-world GUI applications requiring fast and reliable response times. We respond to each sub-point in detail below:

1. Latency Benchmarks: As discussed in the Conclusion and Limitations (Section 5), while our current work focuses on performance metrics like grounding accuracy and task success rate, we recognize the need for explicit latency analysis. We will supplement this by quantifying end-to-end latency across the BTL pipeline. A brief comparison is shown in the following: |Model|Size|Format|Tokens|Latency/ms| | |-|-|-|-|-|-| | | | | | bf16 (H100)|fp8 + medusa (4090)| |BTL|7B|Link|58.4|1231|412| |BTL|7B|Think + Link|127.7|2425|864| |BTL|7B|Blink + Think + Link|185.1|4247|1462|

As shown in the table, after quantizing the model and using Medusa [1] to accelerate it, the BTL can achieve real-time response. In future work, we will aim to decrease the number of output tokens to achieve real-time optimization.

2. Real-Time Optimization: As discussed in the first point, model quantization and Medusa, which is a simple LLM Inference acceleration framework with multiple decoding heads, can achieve real-time optimization. Moreover, as shown in our ablation study (Table 5-b), the BTL framework supports adaptive $\lambda$ scaling to reduce the number of ROIs in the Blink Phase, which can balance performance and efficiency.

3. Scalability Testing: Beyond H100 experiments, we have tested on consumer hardware, specifically the RTX 4090, with quantized 7B or real-time interaction, ensuring the framework adapts to constrained resources.

Response to W2

Thanks for this important observation. We would like to clarify that while our experiments include structured benchmarks, they explicitly evaluate multi-step planning and long-term goal adaptation, with design choices in the BTL framework tailored to address such scenarios.

1. The BTL Reward mechanism is designed to support continuous re-derivation of long-term goals through its process-oriented supervision. The Blink Reward provides fine-grained guidance at each step by evaluating ROI relevance to the current sub-goal, while the Link Reward ensures alignment with the global objective. And our ablation study (Table 5-a) confirms that the integration of process and outcome rewards is critical for complex tasks. Removing Blink Data or BTL Reward leads to significant drops in AndroidControl-High SR (from 69.2% to 65.6% or 65.4%), indicating that process supervision (e.g., Blink’s ROI guidance) is essential for maintaining alignment with long-term goals during multi-step execution.
2. BTL-UI inherently supports GUI multi-trajectory prediction through its GRPO-based optimization framework, enabling it to explore and evaluate multiple potential interaction paths for complex tasks. Specifically, during policy optimization, the model generates a group of completions ($\{o_1,o_2,...,o_n\}$) for each GUI task instance, each representing a distinct interaction trajectory spanning Blink, Think, and Link Phases. This multi-trajectory generation allows the model to consider diverse strategies—such as alternative ROI selections in the Blink phase or varying reasoning steps in the Think phase—before determining the optimal path. In summary, while our current benchmarks include structured tasks, their design incorporates complex nested scenarios, and the BTL framework, via its process-outcome reward integration, demonstrates effectiveness in handling intermediate steps and adapting to long-term objectives.

Response to W3

Thanks for pointing out this important issue regarding reproducibility. We acknowledge that the initial submission omitted several key implementation details due to space constraints. In the revised version, we will include a comprehensive appendix section detailing the following:

We will clearly state this in the revised Implementation Details(Section 4.1). Moreover, we will release the training code, data, and corresponding checkpoints to ensure reproducibility.

Response to Q1

Thank you for this profound observation, which prompts us to clarify the fundamental nature of the BTL framework and its distinction from mere regularization. We contend that the Blink-Think-Link structure is not merely a regularization technique but a biologically inspired cognitive paradigm that aligns AI-driven GUI interaction with human cognitive mechanisms, thereby addressing the limitations of existing approaches. We address your concerns as follows:

1. Information Transmission Loss: The BTL framework is designed to minimize inter-stage information bottlenecks through deliberate architectural choices. First, the Blink Phase's ROI predictions (<blink> tags) are explicitly fed into the Think Phase as structured inputs, ensuring that visual attention signals directly guide reasoning. Second, the BTL Reward mechanism ensures consistency across stages: the Dual Format Reward verifies structural coherence, while Blink and Link Rewards jointly monitor visual grounding and action execution, preventing information loss through multi-stage alignment. Our ablation study (Table 5) shows that the absence of Blink Data or BTL Reward leads to significant performance drops (e.g., SR decreases from 69.2% to 65.6% without Blink Data), confirming that inter-stage information transmission is both efficient and critical.
2. Local Optimization Constraints: BTL-UI avoids local optima by globally optimizing the entire pipeline. The GRPO-based policy update (Equation 9) maximizes a joint objective that integrates rewards from all stages, ensuring Blink, Think, and Link Phases are co-optimized rather than tuned independently. The BTL Reward’s process-outcome integration (combining Blink’s granular reward and Link’s outcome feedback) further guides the model toward global Optimization. For instance, in the high-level task of AndroidControl, only the full BTL framework (with all stages and rewards) achieves the highest SR (69.2%), outperforming ablated versions that optimize stages in isolation. This demonstrates that the framework escapes local optima by leveraging cross-stage dependencies.
3. Task Decomposition Rigidity: We appreciate the reviewer's insightful comments. Indeed, the fixed Blink-Think-Link decomposition may not be suitable for all GUI tasks (e.g., highly reactive or parallel-perception settings). However, the current structure is parameterized: for example, when no relevant ROI exists, the Blink Phase can output <blink> None </blink>, effectively collapsing to Think-Link. Moreover, the BTL framework is inherently extensible and can support adaptive or task-specific decompositions. Specifically, the Blink Phase can be conditioned on the instruction context to modulate its granularity of attention, focusing broadly or narrowly depending on whether the query is high-level or low-level. In summary, BTL’s innovation lies in its cognitive alignment with human-GUI interaction, using structured phases to mimic biological processes rather than imposing arbitrary regularization. This design not only improves performance but also enhances interpretability and alignment with human behavior, addressing critical gaps in existing GUI agents.

Response to Q2

We appreciate the reviewer’s forward-looking question regarding the long-term applicability of our BTL-UI framework. We believe our approach is particularly well-suited for scaling to more complex benchmarks and real-world applications for the following reasons:

1. The core Blink-Think-Link paradigm mirrors the human cognitive process of visual attention, reasoning, and action, an architecture that naturally adapts to more complex GUI environments. As interfaces grow more intricate, such as nested menus and multi-app workflows, the Blink Phase can dynamically locate task-relevant ROIs. This ensures the framework does not rely on fixed element patterns but instead generalizes to novel, complex layouts.
2. The GRPO-based optimization supports multi-trajectory exploration, enabling the model to evaluate diverse interaction paths (e.g., alternative ROI selections or reasoning steps) for complex tasks. This capability—generating and comparing multiple completions to maximize long-term advantage—allows BTL-UI to adapt to unexpected changes in complex applications. Therefore, we suppose the BTL framework remains useful as benchmarks and applications grow more complex, with empirical results validating its scalability across diverse task types.

[1] Cai T, Li Y, Geng Z, et al. Medusa: Simple llm inference acceleration framework with multiple decoding heads, 2024[J]. arXiv preprint arXiv: 2401.10774, 2024.

Dear Reviewer gfqS,

Thank you for your valuable feedback. We sincerely appreciate your insights and are fully committed to addressing these points to improve the quality of our work.

Response to W1

Thank you for pointing out the need for a more comprehensive baseline comparison. Our initial experimental comparison did not include UI-TARS due to oversight. We will add a comparison and detailed analysis with high-performing baselines, including UI-TARS and UGround-v. We address each of the concerns below:

• Screenspot: As disclosed in their technical report, UI-TARS is a strong GUI Agent developed based on Qwen2-VL with training on 50B tokens. While our BTL-UI is developed based on Qwen2.5-VL, with RFT on 4K samples. Although the overall grounding performance of BTL-UI is still lower than that of UI-TARS, it shows certain advantages on the subtasks of Mobile and Web.

• AndroidControl: As disclosed in their official repository, UI-TARS adopts the recent five images as input on multi-step interaction tasks for both training and inference, which is computationally expensive. However, in recent methods, a single image training and inference are adopted. Therefore, to ensure comparability, we reproduce the performance of UI-TARS based on the released checkpoint and prompt (*). Although the overall performance of BTL-UI is still lower than that of UI-TARS, under the same experimental settings (single image), BTL-UI is superior to UI-TARS.

• Why don't we use UI-TARS as the baseline? When we conducted experiments on UI-TARS, we found that it has a weak ability to follow instructions, as it cannot output in the specified format according to the designed prompt, but instead adheres to its original output format. Furthermore, BTL-UI requires deep reasoning in the Think Phase. We suppose the space for deep reasoning of UI-TARS with large-scale data SFT and DPO is relatively limited. Therefore, we finally developed BTL-UI based on a foundational model Qwen2.5-VL.

Response to W2

Thanks for highlighting the importance of the Screenspot-Pro benchmark. We would like to clarify that we have already evaluated our BTL-UI model on ScreenSpot-Pro, and the corresponding results are provided in Table 6 of the Appendix (Section A). Specifically, compared to existing baselines, our BTL-UI achieves SOTA performance on this benchmark across both 3B and 7B model scales. Furthermore, a brief comparison with UI-TARS is shown in the following:

We appreciate the opportunity to emphasize this result in the main paper. In the revised version, we will highlight the ScreenSpot-Pro results more clearly in the main body (e.g., by merging them into Table 2 or Table 3) and discuss our model’s performance and robustness in this more challenging setting.

Response to W3

We appreciate the reviewer’s concern regarding the agent's generalization capability beyond offline datasets. Indeed, evaluating GUI agents in online interactive environments is critical for demonstrating robustness and avoiding overfitting to dataset-specific trajectories. While our current evaluation focuses on standardized offline benchmarks (e.g, Screenspot series, AndroidControl, and GUI-Odyssey) adopted by recent methods, we emphasize that several components of the BTL-UI framework were explicitly designed to mitigate overfitting:

• Our Blink Phase guides attention toward semantically relevant UI regions rather than relying on memorized element positions.
• The BTL Reward introduces the Process Reward Model (PRM), which encourages generalizable decision-making rather than imitating fixed trajectories.
• The proposed BTL-UI is optimized by Group Reward Policy Optimization (GRPO), a relative advantage-based policy optimization method that does not directly depend on supervision of ground-truth actions, further reducing the risk of overfitting to offline data, and demonstrating stronger generalization. We acknowledge, however, that online evaluation is an important next step. In our future work, we will integrate BTL-UI into online testing platforms, such as OSWorld [1] and AndroidWorld [2], to assess real-time adaptability. We will include a discussion of this direction in the revised manuscript’s limitations section.

Response to Q1

Thank you for your insightful question regarding the potential dependency of our model on the proposals generated by OmniParser during the Blink phase. We clarify as follows:

1. The accuracy of BTL-UI is not strictly upper bounded by OmniParser-generated proposals. The two-stage Blink Data Generation pipeline (Section 3.2) decouples raw element extraction from task-relevant filtering. While OmniParser extracts initial UI elements, the second stage leverages Qwen2.5-VL-32B to dynamically analyze, filter, and prioritize these elements based on task instructions and interaction history, effectively refining OmniParser’s outputs. Only data samples that can pass the blink calibration ('Response to W1' to Reviewer 96rE) are selected as high-quality data for RFT.
2. At inference time, BTL-UI is free to predict any region it deems relevant. In particular, the Blink Reward (Eq. 6) awards a non‑zero reward even when the model outputs an empty blink set ($P_i=\varnothing$), provided that either no ground‑truth ROI exists on‑screen or the chosen action belongs to the non‑interactive action set $\mathcal{A}^*$. This design explicitly prevents the model from being forced to conform to OmniParser proposals when they are incomplete.
3. Importantly, since the BTL-UI has the ability to understand screenshots and ground elements, the model operates independently, guided by its learned policy from BTL Reward over screenshots and instructions. Furthermore, the visualization (Appendix C) demonstrates that the BTL-UI is capable of discovering novel ROIs and reasoning beyond annotated ground-truth. We agree this limitation should be more explicitly discussed, and we will add a clarification in the revision to ensure readers are aware that the OmniParser is used to extract UI elements from data structures but does not bound the model’s capability to discover or reason about novel UI regions.

Response to Q2

Please kindly refer to 'Response to W1' and 'Response to W2'.

Response to Q3

We appreciate the reviewer’s concern about the Link Reward’s potential overfitting to offline datasets and its generalization to real-world online environments. Here, we justify the design and provide supporting evidence, along with plans for online validation:

1. Justification of the Link Reward Design: The Link Reward’s strict "all-or-nothing" criterion (reward=1 only if both action type and arguments are exactly correct) is intentionally designed to avoid reward hacking and ensure the semantic consistency between the agent’s intent and execution, which is critical for GUI interactions where even minor errors can derail tasks. Due to its focus on structural correctness rather than memorizing dataset-specific patterns, we suppose BTL-UI does not inherently overfit to offline data. In addition, since RFT is not supervised by the ground-truth directly during training like SFT, RFT has a lower risk of overfitting.
2. Planned Online Environment Experiments: Our current experiments on offline benchmarks adopted by recent methods, including AndroidControl and GUI-Odyeesy, that mimic online variability. Moreover, BTL-UI inherently supports GUI multi-trajectory prediction through its GRPO-based optimization framework, enabling it to explore and evaluate multiple potential interaction paths for complex tasks. We suppose these suggest the BTL framework generalizes beyond static offline data, as it handles dynamic state transitions analogous to real-world online interactions. However, due to resource limitations, we are unable to conduct experiments in an online environment further. We will add the absence of online evaluation to the "Limitations" section of the revision and list it as further work.

Respone to Q4

We appreciate the reviewer's suggestion to improve the clarity of our Blink Reward formulation. In response, we will revise Section 3.3 to define all variables involved in the reward computation explicitly. Specifically:

• $i$indexes a given training sample.
• $x$indexes the predicted region-of-interest (ROI) bounding boxes $p_i^x \in P_i$
• $y$indexes the ground-truth ROI boxes $g_i^y \in G_i$ in Section 3.2 We will also revise the definition of index_math in Eq. (5) to clearly reflect these index roles. Moreover, we will include a brief inline explanation of the Hungarian matcher:

...we adopt the Hungarian matcher, a classical assignment algorithm used to compute the optimal one-to-one matching between predicted and ground-truth bounding boxes based on IoU scores, under a given threshold τ.

[1] Xie T, Zhang D, Chen J, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments[J]. Advances in Neural Information Processing Systems, 2024, 37: 52040-52094.

[2] Rawles C, Clinckemaillie S, Chang Y, et al. Androidworld: A dynamic benchmarking environment for autonomous agents[J]. arXiv preprint arXiv:2405.14573, 2024.