Cactus Hybrid
Post-trained models that know when they're wrong and request help from the cloud. Simple tasks stay on-device, complex requests route to a larger model.
What is the capital of France?
How it works
Method
- The probe
- A small head inside the checkpoint reads the model while it answers and scores the result with confidence = 1 − p(wrong). It ships as eleven extra tensors; the base weights stay byte-identical to the stock model.
- The score
- Every generation returns a confidence between 0 and 1 as structured data. It is never parsed out of the answer text, so the answer itself is unchanged.
- The route
- Pick a threshold. Above it, show the on-device answer. Below it, send the same prompt to a bigger model. One if-statement, no separate router model.
- The trade-off
- Raise the threshold for more accuracy and more cloud calls; lower it to keep more on device. The curve is per benchmark, so you can set it per feature.
- Engines
- Stock transformers commands work unchanged with trust_remote_code. GGUF and MLX builds are published alongside.
Results
How much has to leave the device
Proportion of queries handed off to the cloud on each benchmark.
| Benchmark | Handoff |
|---|---|
| ChartQA Charts | 15–20% |
| LibriSpeech Speech | 25–30% |
| MMBench Vision | 30–35% |
| GigaSpeech Speech | 30–35% |
| MMAU Audio | 30–35% |
| MMLU-Pro Reasoning | 45–55% |
FP16 · % of queries handed off · lower is better
Share of queries Gemma 4 E2B Hybrid hands to Gemini 3.1 Flash-Lite to match Flash-Lite's score. Lower is better: everything else runs on device. Source: model card.
Models
Checkpoints
- Gemma 4 E2B Hybrid
The smallest Gemma. Matches Flash-Lite on most benchmarks by handing off 15–35% of queries.
(opens model card on Hugging Face) - Gemma 4 26B-A4B Hybrid
Mixture-of-experts with 4B active parameters, for laptops and desktops.
(opens model card on Hugging Face) - Your model
We post-train additional checkpoints on demand. Tell us the base model and the bigger model you want to hand off to.