AI is going where the devices are. A 14MB foundation model — needle (45M params, 28MB RAM, confidence-gated tool calling) — is the new proof point that capable agents can run on phones, wearables, smart-home hubs, and robots with no cloud dependency.
This comparison ranks the leading tiny / edge AI models and runtimes of 2026 — from needle's 14MB breakthrough to the 3-4B sweet-spot models (Phi, Gemma 3n, SmolLM2) and the runtimes that ship them (llama.cpp, ExecuTorch) — with star data verified August 15, 2026.
| # | Tool | ⭐ Stars | Type | Best For |
|---|---|---|---|---|
| 1 | cactus-compute/needle | 5,709 | 14MB Foundation Model | 45M params in 28MB RAM with confidence-gated tool calling — the category-defining edge model |
| 2 | jzhang38/TinyLlama | 9,019 | 1.1B Pretrained LLM | Open 1.1B Llama pretrained on 3T tokens — the research baseline for tiny LLMs |
| 3 | microsoft/Phi-4-mini | HF | 3.8B Compact Model | Microsoft's mini-model line — frontier-competitive quality at 3.8B, CPU-friendly |
| 4 | google/gemma-3n | HF | 1B / 4B Multilingual | 27 languages, 140+ countries, tiny enough for phones — Google's edge bet |
| 5 | HuggingFaceTB/SmolLM2 | HF | 135M–1.7B Family | Hugging Face's small-model family — the default starting point for edge experiments |
| 6 | Meta MobileLLM | HF | 125M–1.5B | Layer-wise weight sharing — Meta's research line for phone-class models |
| 7 | ggml-org/llama.cpp | 123,930 | Universal Edge Runtime | Runs everything from 45M to 400B on CPU/GPU — the runtime edge AI actually ships on |
| 8 | pytorch/executorch | 4,918 | On-Device Inference Runtime | PyTorch's official runtime for mobile/embedded — powers Meta's on-device AI |
| 9 | win4r/AirLLM | 30,690 | 70B on 4GB GPU | The other end of the spectrum — extreme memory compression for big models |
| 10 | Qualcomm AI Hub | — | Edge Model Zoo + Tooling | Optimized model zoo for Snapdragon devices — the commercial edge path |
needle (5,709⭐, +662/day, +1,929/week) is the category-defining release: a 45M-parameter foundation model that fits in 28MB of RAM — small enough for phones, wearables, smart-home hubs, and robots. Its killer feature is confidence-gated tool calling: the model knows when it doesn't know, and escalates to the cloud only when needed. That hybrid pattern (edge-first, cloud-escalation) is the architectural answer to latency, privacy, and cost.
✔ 14MB = runs anywhere; confidence gating = cloud costs only when needed
✖ 45M params caps deep reasoning — it's a router + tasker, not a thinker
For tasks that need real reasoning, the 1–4B class is the practical floor. Phi-4-mini (3.8B) delivers frontier-adjacent quality on CPU. Gemma 3n (1B/4B) covers 27 languages and 140+ countries — the global edge play. SmolLM2 (135M–1.7B) is Hugging Face's experiment-friendly family. TinyLlama (9,019⭐) remains the research baseline everyone benchmarks against.
✔ 3-4B models run on today's phones; quality is good enough for most tasks
✖ Multimodal + long context still push past phone-class memory budgets
Models come and go; the runtime layer compounds. llama.cpp (123,930⭐) is the universal edge runtime — GGUF quantization makes 45M–400B models portable across CPU/GPU. ExecuTorch (4,918⭐) is PyTorch's on-device answer, shipping Meta's mobile AI. AirLLM (30,690⭐) attacks from the other side: running 70B models on 4GB GPUs via block-wise quantization. The edge stack is now: tiny model + quantized runtime + local-first tool calling.
✔ Runtime ecosystems (GGUF, delegates) make model swaps cheap and portable
✖ Fragmentation: no single runtime covers every silicon vendor well
| Use Case | Best Tool | ⭐ | Why |
|---|---|---|---|
| Ultra-tiny edge agents (wearables, robots) | needle | 5,709 | 14MB, 28MB RAM, confidence-gated tool calling |
| Phone-class reasoning | Gemma 3n | HF | 1B/4B, 27 languages, on-device friendly |
| CPU-only quality | Phi-4-mini | HF | 3.8B with frontier-adjacent quality |
| Research baseline | TinyLlama | 9,019 | 1.1B trained on 3T tokens |
| Universal runtime | llama.cpp | 123,930 | GGUF quantization, any model, any device |
| PyTorch on-device | ExecuTorch | 4,918 | Meta's official mobile/embedded runtime |
Start with needle on a Raspberry Pi-class device, escalate to the cloud only when confidence is low. Fine-tune on RunPod or Vast.ai GPUs.
RunPod GPUs → Vast.ai GPUs →Every edge-first system needs a cloud fallback for hard cases. Put the escalation API behind Cloudflare Workers for edge distribution.
Cloudflare →Ship models to devices with ExecuTorch or llama.cpp and keep the fleet monitored with Better Stack observability.
Better Stack →Disclosure: some links above are affiliate links (we may earn a commission at no extra cost to you).
| Revenue Path | Potential | Why |
|---|---|---|
| GPU hosting affiliate (fine-tuning) | MEDIUM | Every edge model starts as a cloud-trained model — RunPod/Vast.ai refs fit |
| Cloud infrastructure affiliate (escalation layer) | MEDIUM | Hybrid edge+cloud architectures need Workers/functions — Cloudflare/Vercel |
| Observability affiliate (device fleets) | LOW | Fleet monitoring — Better Stack/UptimeRobot/Datadog |