Distributed Inference Bench
Open /bench

Benchmark on real phones. Route what’s fastest.

Distributed Inference Bench runs LLMs across physical Android devices and edge nodes. Compare TTFT, tokens/sec, thermal/battery impact, and route inference to the best target — in real time.

Open the Bench Explore Features
TTFT
First-token latency per device & model
Tokens/sec
Sustained throughput under load
Thermals & Battery
Temperature, throttle, drain
/bench
/bench preview

Replace the mock image with a screenshot of your actual /bench view for authenticity.

Qualcomm
Samsung
Google
MediaTek
Arm
OpenAI

Built for real‑world, on‑device inference

Measure what matters on phones, not just GPUs. Reproduce runs, compare models, and pick the best device dynamically.

Transparent metrics

TTFT, tokens/sec, error rate, temperature, battery drain — unified across devices and models.

Reproducible runs

Pin prompts, seeds, adapters, and versions. Export CSVs. Compare apples‑to‑apples.

Smart routing

Send requests to the best target in real time based on health and performance.

Device control

Per‑device adapters, tags, and schedules. Handle thermal throttling gracefully.

Clean UI

Landing mimics your /bench aesthetics — cards, tables, crisp typography, dark mode.

Simple deploy

Single HTML file. Drop into Cloudflare Pages, link /bench, and you’re live.

Live Bench

Embed a public view, or keep private and link out. Below is a placeholder iframe — point it to /bench or a read‑only view.

Embed placeholder
Set the iframe src to your public bench URL if desired.
Open /bench

How it works

Agents run on phones/edge nodes. The gateway orchestrates jobs, collects metrics, and surfaces insights in the Bench UI. Route traffic programmatically via API or use the UI to compare and dispatch.

  • • Connect Android devices (USB/TCP).
  • • Register models and adapters per device.
  • • Run benchmarks; export and compare.
  • • Route inference to the fastest healthy target.
Launch /bench Talk to us
Metric
TTFT (P95)
Metric
Tokens/sec
Health
Thermals
Fleet
Availability

Charts are decorative placeholders. Swap with real images or a lightweight chart embed later.

Want a demo or a custom device setup?

We can provision specific phones, build adapters, and share a read‑only Bench for your team.

[email protected]

Or just hit /bench to explore on your own.