Newshot

绕过 ADB:面向 AI 智能体的 0.32ms Android UI 树与 A* 路由(Rust/Kotlin)

Translation(model translation, not the source's own text)

DroidPath Agentic Benchmarks

本基准测试套件将 DroidPath(Shadow DOM + Layer 3 A Spatial Graph Router)* 与自主 AI 智能体使用的两种主流 Android 自动化范式进行对比:

  • Pure Vision(截图 + VLM 循环)—— 每一步都捕获全分辨率 PNG 截图,并发送给视觉语言模型以预测 (x, y) 坐标。
  • Appium / uiautomator dump XML 循环 —— 每一步都通过 ADB 运行 uiautomator dump,并将原始 XML 层级结构(约 25–35 KB)输入 LLM。
  • DroidPath(Shadow DOM + Layer 3 A) —— 通过持久化的 loopback TCP socket(android-daemon)流式传输增量 NDJSON 无障碍差异,并在本地 Rust 中通过 A 执行学习到的多跳路径,无需中间 LLM 轮次。

Measured Results(Live Unlocked Pixel 10a Hardware + Replay Suite)

透明性说明:以下硬件原语延迟和端到端(E2E)Settings 导航计时是在一台已解锁的 Google Pixel 10a(61061JEA307110,Android com.android.settings 根屏幕,54 个实时 UI 节点)上通过 USB ADB 实时测得的,使用 node scripts/benchmark_agentic.mjs --mode live --json 对编译后的 target/release/droidpath-mcp 运行。回放表中的多步 Pure Vision 与 Appium/XML 对比运行真实的 droidpath-mcp stdio JSON-RPC 调用,针对 60 节点 loopback Shadow DOM 守护进程,并配合 Methodology & Token Model 下记录的显式 token 与成本公式。

1. Live Hardware Probe(已解锁 Google Pixel 10a · com.android.settings,54 个节点)

Primitive / MethodStatusMeasured LatencyPayload SizeNotes
DroidPath android_get_ui_tree(稳态)ok0.32 ms10,742 bytes(约 2,686 tok)来自本地 Shadow DOM 流的 54 个实时 UI 节点(比 uiautomator 快 7,575×)
DroidPath android_get_ui_tree(冷连接)ok41.89 ms10,742 bytes(约 2,686 tok)首次 TCP 连接 + 实时快照(比 uiautomator 快 58×)
DroidPath Hybrid Vision( include_vision: true )ok452.33 ms38,581 bytes JPEG(540×1212)54 个实时 UI 节点 + 压缩 JPEG(q45,比原始 PNG 小 6.1×)
adb exec-out screencap -p(原始 PNG)ok658.1 ms235,258 bytes PNG仅视觉捕获——无结构化 UI 树
adb exec-out uiautomator dump /dev/tty(XML)ok2,423.9 ms29,357 bytes(约 7,340 tok)需要停止 UiAutomation 守护进程;文本 token 数比 Shadow DOM 多 2.7×

2. Live End-to-End Hardware Navigation(Google Pixel 10a · 真实种子水合 → 预热 A*)

Task(Settings → Target)Cold-Start(Seed Hydration)Cold StatusWarm A* GraphWarm StatusLive Success Rate
Settings → Display378.6 msarrived296.2 ms(1 step)arrived100%
Settings → Battery655.9 msarrived642.8 ms(1 step)arrived100%
Settings → About phone(需要滚动)775.7 msarrived744.1 ms(1 step)arrived100%
实时套件平均值603 ms100%(3/3)561 ms100%(3/3)100%

3. Local MCP Runtime Micro-Benchmarks(实测)

MetricMeasured Value
tools/list JSON-RPC 延迟(9 个工具)0.25 ms
android_graph_stats JSON-RPC 延迟0.42 ms
启动时加载的预水合 Seed Graph20 vertices / 19 edges
紧凑 Shadow DOM 树(60 节点模拟 Settings 屏幕)4,537 bytes(约 1,135 tokens)

4. Architecture & Cost Comparison(5 个 WebArena 风格多步任务 · Replay + API Cost Model)

ArchitectureAvg LLM Calls / TaskAvg Tokens / TaskAvg Est. Task TimeEst. Cost / 1,000 TasksAdvantage of DroidPath
DroidPath (Shadow DOM + A Router) *1.039 (warm A )* / ~1,135 (cold tree)464 ms (561 ms live warm)$0.12100% live E2E success · 192× cheaper than Vision
Pure Vision (Screenshots + VLM)4.27,6739,954 ms$23.02Baseline
Appium / uiautomator XML4.229,4008,946 ms$88.20735× more expensive than Warm DroidPath

Per-Task Breakdown

TaskHopsDroidPath (Calls / Tokens / Time)Pure Vision (Calls / Tokens / Time)Appium XML (Calls / Tokens / Time)
Enable Night Light (Display)31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms
Open Wi-Fi Settings (Network)41 call · 39 tok · 580 ms5 calls · 9,135 tok · 11,850 ms5 calls · 35,000 tok · 10,650 ms
Navigate to Battery Saver31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms
Open App Notifications31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms
Scroll to About Phone31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms

Methodology & Token Model

  • Text tokens: ceil(utf8_bytes / 4)(JSON/XML 的标准 BPE 近似值)。
  • Cost / 1,000 tasks: 按 $3.00 / 1M 输入 tokens 透明计算(标准 Claude Sonnet 级 API 定价):(avg_tokens_per_task 1,000 / 1,000,000) $3.00。
  • Vision image tokens: Anthropic 公布的 (W × H) / 750 公式,针对缩放到长边 1568px 的 1080×2400 Android 截图(706 × 1568 / 750 = 1,477 个图像 tokens + ~350 个提示/工具 tokens = 1,827 tokens/步)。
  • Opt-in Hybrid Vision (include_vision: true): 降采样至 540px 宽的 JPEG(质量:45,在解锁的 Pixel 10a Settings 上实测 38.6 KB,而原始 PNG 为 235.3 KB)严格按需启用,仅在 agent 明确需要视觉消歧时使用(例如 canvas/icons),在 FLAG_SECURE 屏幕上仍严格禁止。

Reproduce on Your Device


        # Build release binary first (optional — falls back to target/debug)
cargo build --release -p droidpath-mcp

# Replay benchmark (zero device required)
node scripts/benchmark_agentic.mjs

# Live physical device / emulator probe (measures real screencap -p & uiautomator dump)
node scripts/benchmark_agentic.mjs --mode live
      

The translation is machine-generated and individual terms may be off. Nothing is rewritten or cut; the original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗