Newshot

Bypassing ADB: 0.32ms Android UI trees and A* routing for AI agents (Rust/Kotlin

Original text(via HN AI)

DroidPath Agentic Benchmarks

This benchmark suite compares DroidPath (Shadow DOM + Layer 3 A Spatial Graph Router)* against the two dominant Android automation paradigms used by autonomous AI agents:

  • Pure Vision (Screenshot + VLM loop) — captures a full-resolution PNG screenshot on every step and sends it to a Vision-Language Model to predict (x, y) coordinates.
  • Appium / uiautomator dump XML loop — runs uiautomator dump over ADB on every step and feeds the raw XML hierarchy (~25–35 KB) to an LLM.
  • DroidPath (Shadow DOM + Layer 3 A) — streams incremental NDJSON accessibility diffs over a persistent loopback TCP socket (android-daemon) and executes learned multi-hop paths locally in Rust via A without intermediate LLM turns.

Measured Results (Live Unlocked Pixel 10a Hardware + Replay Suite)

Transparency Note: Hardware primitive latencies and End-to-End (E2E) Settings navigation timings below were measured live over USB ADB on an unlocked Google Pixel 10a (61061JEA307110, Android com.android.settings root screen, 54 live UI nodes) using node scripts/benchmark_agentic.mjs --mode live --json against compiled target/release/droidpath-mcp. Multi-step Pure Vision and Appium/XML comparisons in the replay table run real droidpath-mcp stdio JSON-RPC calls against a 60-node loopback Shadow DOM daemon paired with explicit token and cost formulas documented under Methodology & Token Model.

1. Live Hardware Probe (Unlocked Google Pixel 10a · com.android.settings, 54 Nodes)

Primitive / MethodStatusMeasured LatencyPayload SizeNotes
DroidPath android_get_ui_tree (steady-state)ok0.32 ms10,742 bytes (~2,686 tok)54 live UI nodes from local Shadow DOM stream ( 7,575× faster than uiautomator )
DroidPath android_get_ui_tree (cold connect)ok41.89 ms10,742 bytes (~2,686 tok)First TCP connect + live snapshot ( 58× faster than uiautomator )
DroidPath Hybrid Vision ( include_vision: true )ok452.33 ms38,581 bytes JPEG (540×1212)54 live UI nodes + compressed JPEG (q45, 6.1× smaller than raw PNG)
adb exec-out screencap -p (raw PNG)ok658.1 ms235,258 bytes PNGVisual capture only — no structured UI tree
adb exec-out uiautomator dump /dev/tty (XML)ok2,423.9 ms29,357 bytes (~7,340 tok)Requires stopping UiAutomation daemon; 2.7× more text tokens than Shadow DOM

2. Live End-to-End Hardware Navigation (Google Pixel 10a · Real Seed Hydration → Warm A*)

Task ( Settings → Target )Cold-Start (Seed Hydration)Cold StatusWarm A* GraphWarm StatusLive Success Rate
Settings → Display378.6 msarrived296.2 ms (1 step)arrived100%
Settings → Battery655.9 msarrived642.8 ms (1 step)arrived100%
Settings → About phone (scroll required)775.7 msarrived744.1 ms (1 step)arrived100%
Average across live suite603 ms100% (3/3)561 ms100% (3/3)100%

3. Local MCP Runtime Micro-Benchmarks (Measured)

MetricMeasured Value
tools/list JSON-RPC latency (9 tools)0.25 ms
android_graph_stats JSON-RPC latency0.42 ms
Pre-hydrated Seed Graph loaded at boot20 vertices / 19 edges
Compact Shadow DOM tree (60-node mock Settings screen)4,537 bytes (~1,135 tokens)

4. Architecture & Cost Comparison (5 WebArena-Style Multi-Step Tasks · Replay + API Cost Model)

ArchitectureAvg LLM Calls / TaskAvg Tokens / TaskAvg Est. Task TimeEst. Cost / 1,000 TasksAdvantage of DroidPath
DroidPath (Shadow DOM + A Router) *1.039 (warm A )* / ~1,135 (cold tree)464 ms (561 ms live warm)$0.12100% live E2E success · 192× cheaper than Vision
Pure Vision (Screenshots + VLM)4.27,6739,954 ms$23.02Baseline
Appium / uiautomator XML4.229,4008,946 ms$88.20735× more expensive than Warm DroidPath

Per-Task Breakdown

TaskHopsDroidPath (Calls / Tokens / Time)Pure Vision (Calls / Tokens / Time)Appium XML (Calls / Tokens / Time)
Enable Night Light (Display)31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms
Open Wi-Fi Settings (Network)41 call · 39 tok · 580 ms5 calls · 9,135 tok · 11,850 ms5 calls · 35,000 tok · 10,650 ms
Navigate to Battery Saver31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms
Open App Notifications31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms
Scroll to About Phone31 call · 39 tok · 435 ms4 calls · 7,308 tok · 9,480 ms4 calls · 28,000 tok · 8,520 ms

Methodology & Token Model

  • Text tokens: ceil(utf8_bytes / 4) (standard BPE approximation for JSON/XML).
  • Cost / 1,000 tasks: Computed transparently at $3.00 / 1M input tokens (standard Claude Sonnet-class API pricing): (avg_tokens_per_task 1,000 / 1,000,000) $3.00.
  • Vision image tokens: Anthropic's published (W × H) / 750 formula for a 1080×2400 Android screenshot scaled to 1568px long edge (706 × 1568 / 750 = 1,477 image tokens + ~350 prompt/tool tokens = 1,827 tokens/step).
  • Opt-in Hybrid Vision (include_vision: true): Downscaled 540px-wide JPEG (quality: 45, measured 38.6 KB on unlocked Pixel 10a Settings vs 235.3 KB raw PNG) is strictly opt-in and only used when an agent explicitly needs visual disambiguation (e.g., canvas/icons), remaining strictly blocked on FLAG_SECURE screens.

Reproduce on Your Device


        # Build release binary first (optional — falls back to target/debug)
cargo build --release -p droidpath-mcp

# Replay benchmark (zero device required)
node scripts/benchmark_agentic.mjs

# Live physical device / emulator probe (measures real screencap -p & uiautomator dump)
node scripts/benchmark_agentic.mjs --mode live
      

The text above was extracted automatically from the source page and may miss images, tables or code blocks. The original is authoritative.

Published by HN AI
This site only summarises and aggregates; copyright stays with the original author. Headlines and summaries follow the source — where they differ, the original wins.
Read the original ↗