Bonsai 27B Runs a 27-Billion-Parameter Model on a Phone: What On-Device AI Changes for SaaS

PrismML released Bonsai 27B on July 14, 2026: a 27-billion-parameter multimodal model compressed to roughly 3.9GB that runs locally on an iPhone 17 Pro at about 11 tokens per second, under an Apache 2.0 licence. The 1-bit build retains more than 90 percent of full-precision performance across 15 benchmarks. If that holds up outside the vendor's own testing, the interesting consequence for SaaS teams is not a faster chatbot — it is that a whole class of features you currently pay per token for could stop touching a server at all.

Key takeaways

• Bonsai 27B shipped July 14, 2026 under Apache 2.0, built on the Qwen3.6-27B base model.
• The 1-bit build lands around a 3.9GB deployed footprint; a ternary 1.58-bit build trades size for higher fidelity.
• PrismML reports over 90 percent of full-precision performance across 15 benchmarks, and roughly 11 tokens per second on an iPhone 17 Pro.
• The technique is quantization-aware training, not post-training rounding — the model learns to be good at 1 bit rather than being squeezed afterwards.
• The near-term SaaS impact is cost and privacy, not capability. Frontier models still win on hard reasoning.

What did PrismML actually ship?

Bonsai 27B is a compressed build of Qwen3.6-27B, released in two variants. The 1-bit build is the aggressive one, at roughly 3.9GB deployed. The ternary build constrains weights to minus one, zero and one — about 1.58 bits — and comes out slightly larger with higher fidelity.

The model is multimodal, accepting images as well as text, and PrismML positions it for reasoning, coding and agentic workflows rather than as a toy. Weights are on Hugging Face in GGUF format, and the Apache 2.0 licence means commercial use without a negotiation.

The performance claim worth repeating carefully: over 90 percent of the full-precision model's scores across 15 benchmarks. That is PrismML's own number. It is a strong claim, it is specific enough to be falsifiable, and independent replication is the thing to watch over the next few weeks rather than something to assume.

Why is quantization-aware training the actual news?

This is the part that separates Bonsai from the long history of disappointing compressed models.

Most quantization is post-training. You take a finished model and round its weights down to fewer bits. It is cheap, it is fast, and it degrades quality in ways that get dramatically worse the further you push — which is why 1-bit post-training quantization has historically produced models that technically run and practically do not work.

Quantization-aware training inverts that. PrismML retrains the model with quantization in the loop, so it learns to perform well as a 1-bit model rather than despite being one. The constraint is present during training instead of applied afterwards as damage.

That distinction is why the retention numbers are plausible where earlier 1-bit results were not. It also means the approach generalises: if it works on a 27B model, the same pipeline applies to whatever base model comes next. The technique is more durable news than this particular release.

What does this change for SaaS teams in practice?

Three things, in descending order of how soon they matter.

Cost structure changes first. Every AI feature you ship today has a per-token marginal cost that scales with usage — which is why so many products quietly cap their AI features or gate them behind a higher tier. A capable model running on the user's device has a marginal inference cost of zero to you. For high-frequency, low-complexity features — classification, summarisation, autocomplete, extraction, tagging — that is a materially different unit economic.

Privacy posture changes second. On-device inference means customer data does not leave the device. For anyone selling into healthcare, legal, financial services or the EU, "the model runs locally and your data never reaches our servers" is a sales answer that currently requires an enterprise contract and a lot of paperwork to approximate.

Offline capability changes third, and it is niche until it isn't. Field tools, travel apps, anything used in a warehouse or a basement or on a plane inherits a capability it simply could not have before.

What does not change: frontier reasoning. A 27B model at 1-bit precision running at 11 tokens per second is not competing with a frontier model on hard multi-step reasoning, and it is not going to. The realistic architecture is hybrid — local model handles the constant, cheap, private work, and hard requests escalate to a hosted model. That routing layer is the engineering problem this release creates.

How does this fit the wider open-weight picture?

It fits a pattern that has been building all year. The open-weight tier keeps absorbing capability that used to be exclusively hosted, and each release moves the line of what you can run yourself.

Bonsai is the extreme end of that trend on the size axis rather than the capability axis, which makes it complementary to the models we have covered previously — MiniMax M3 and Inkling from Thinking Machines both push on what open weights can do, where Bonsai pushes on where they can run.

If you are already thinking about running inference yourself rather than renting it, the self-hosted AI stack guide covers the infrastructure side, and the rest of this year's model coverage is in the AI tools library.

What should you actually do about it this month?

Not migrate. The sensible move is smaller and cheaper than that.

Audit your AI features and separate them into two buckets: the ones that need real reasoning and the ones that are pattern-matching wearing a trench coat. Most products have more in the second bucket than they expect — the classification step, the tag suggestion, the summary of a short document, the "did the user mean X" check.

Then price the second bucket. Multiply your per-call cost by your actual monthly call volume. If that number is uncomfortable, you now have a concrete reason to prototype a local path rather than a theoretical one.

Then test Bonsai against your real inputs, not benchmarks. Benchmark retention of 90 percent across 15 tests tells you very little about whether it handles your specific document format, your domain vocabulary, or your users' typos. This is the whole reason we test tools against real work rather than reproducing vendor tables.

And if the answer is that you are shipping a genuinely new local-AI capability, the hard part becomes explaining why it matters. "It runs on your device" is an architecture decision, not a benefit, until someone shows a user what changes because of it — which is the entire job of AI tool video production.

Frequently asked questions

Can I actually run Bonsai 27B on my phone today?

The weights are published on Hugging Face in GGUF format under Apache 2.0, and PrismML reports roughly 11 tokens per second on an iPhone 17 Pro with the 1-bit build at about 3.9GB. Running it takes a compatible local inference app or your own integration rather than an app-store download, and performance on older or lower-memory devices will be worse. Treat the iPhone 17 Pro figure as the top of the range, not the typical experience.

Is a 1-bit model good enough for production features?

For narrow, well-defined tasks — classification, extraction, summarisation of short text, tagging — quite possibly, and PrismML's reported retention of over 90 percent across 15 benchmarks supports that. For open-ended reasoning, long multi-step agentic work, or anything where a subtle error is expensive, it is not a frontier-model replacement. The realistic pattern is routing cheap frequent work locally and escalating the hard cases.

What is the difference between the 1-bit and ternary builds?

The 1-bit build is the smallest, at roughly 3.9GB deployed, and it is the one that makes phone deployment possible. The ternary build constrains weights to minus one, zero and one, which works out to about 1.58 bits per weight — slightly larger on disk, slightly higher fidelity in output. If you have the memory headroom, start with ternary and drop to 1-bit only if you need the footprint.