Splash: A Local Engine Built Around the Model

Most inference engines are built to run any model for maximum flexibility. Our approach turns this assumption upside down: the engine is built around the model for maximum efficiency. Its kernels, draft model, and memory plan are specialized for the model it serves. This design runs our industry-leading inference platform on data-center GPUs, where we are the fastest provider on Artificial Analysis for five frontier open models.

Today we bring the same technology to Apple silicon with Splash, our open-source inference engine for the Mac. It delivers 2× the decode speed of the next-fastest engine we measured on Qwen3.8-27B and stays ahead at every context length we tested, to 32K tokens. At four parallel subagents, the speedup increases to almost 4×.

Run It Now

Splash needs an M3 or newer Mac on macOS 26.4 or later with at least 36 GB of unified memory, and Homebrew. We recommend 48 GB or more.

brew install incoai/tap/splash
splash serve --model incoai/Qwen3.8-27B-Splash

That's all you need to get started. During the initial download, the engine is automatically tuned to your system configuration. When it prints its Ready line, call it from another terminal or attach your agent:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "incoai/Qwen3.8-27B-Splash",
    "messages": [{"role": "user", "content": "Explain speculative decoding in one sentence."}]
  }'

splash opencode

splash claude

splash codex

splash hermes

No other setup or configuration is needed. The server can be queried via OpenAI Chat Completions, Responses, and Anthropic Messages, with streaming, tool calls, JSON Schema output, and image input.

Splash in LM Studio Bionic

inco.ai×

We're also excited to share that we've worked with the good people at LM Studio to bring you Splash on day zero! LM Studio integrates Splash as a first-class inference engine, and we're excited to continue to ship updates and support new models together with the LM Studio team. If you don't have LM Studio Bionic yet, get it here.

To use it, grab the latest LM Studio Bionic, head to Settings → Runtime, and download Splash. Then download Qwen3.8-27B, and you'll be able to enjoy fast inference on your Mac for fully local agentic work.

The Engine Is Built Around the Model

Splash supports a small set of models, two today, and is specialized for each of them. The runtime, scheduler, cache, and API are shared. Everything else is purpose-built for each model for peak efficiency: fused shape-specific kernels, trained draft models, an optimized memory plan, and pinned performance baselines. We take care of all the complexity while you enjoy unprecedented performance in a single Splash package, rather than a raw MLX or Transformers checkpoint.

By design, there is no generic multi-model runtime, no fallback path, and nothing to tune manually. The only settings are ceilings such as --max-memory and --max-context, to support concurrent applications running on your Mac.

Scheduling and Memory

Splash batches requests as they arrive and balances prefill against decode, so new requests start quickly and running ones keep streaming. Attention state lives in a paged KV cache indexed by prefix, so a request that shares a prefix with an earlier one reuses those pages instead of recomputing them. For hybrid models, Splash also snapshots the Gated DeltaNet (GDN) state at prefix boundaries, so cache reuse covers the recurrent layers too instead of replaying them.

For a given model, the weights and the draft have fixed sizes and each request's state has a known cost per token, so Splash sets a memory budget at startup: Metal's recommended working-set limit minus those costs. For Qwen3.8-27B that means 15 GiB of weights and a 1.2 GiB draft before any KV cache. That is where the 48 GB we recommend above comes from: it leaves room for an editor and a browser while a task runs. As contexts grow, Splash reclaims cached state when it must and adapts scheduling under memory pressure.

DFlash 2 in the Decode Path

Speculative decoding is the decode path in Splash, not an option. Every supported model ships with its own DFlash 2 draft, trained for it. The draft proposes a block of tokens in one pass, and more of them survive verification than with the model's own multi-token prediction. The DFlash 2 post has the numbers for Qwen3.8-27B.

Three parts of the engine are built around that draft.

The decode step runs as one unit. When output is not constrained by a schema, Splash submits drafting, verification, acceptance, and state updates together instead of step by step, so the draft's speedup reaches the API instead of being spent on per-step overhead.

Speculation lives inside the batch. Each request carries its own draft state, which advances with its accepted tokens, so speculation works with concurrency and cache reuse instead of against them.

The draft stays small on long contexts. It uses sliding-window attention, and on a long prefill Splash computes only the draft state that generation and the prefix snapshots need, so draft memory is bounded however long the context grows.

Shape-specialized, Automatic Kernel Generation

Prefill and decode are different workloads: prefill runs large batches, and draft verification runs small blocks. Each gets its own 4-bit matrix multiplication and attention kernels, written for that workload and for the model's exact dimensions. That is too much to write and tune by hand, which is why our in-house kernel agents do it. Among what they produce:

• Decode kernels that read the 8-bit KV cache directly and reuse it across query heads and verification tokens.
• GDN prefill kernels that keep the recurrent state on-chip.
• Dedicated kernels for the routed experts in mixture-of-experts models.

Everything ships precompiled, with presets for GPU family, core count, and workload, so there is nothing for you to compile or tune manually.

How Splash Compares

An agent reads a repository, edits files, runs tests, and feeds the results back into a conversation that keeps growing. Its subagents fan out into concurrent requests that share that history. We measured four parts of that loop:

• Decode: how fast tokens come back as the conversation grows.
• Prefill: how long a cold read of a repository takes.
• Cache reuse: what the next turn costs once the context is loaded.
• Concurrency: how fast requests run together, and how many fit at once.

All engines ran on the same 48 GB M5 Pro, serving Qwen3.6-35B-A3B and Qwen3.8-27B over HTTP at their recommended settings. The prompts are a fixed set of coding tasks from NVIDIA's SPEED-Bench, up to 32K tokens, with a 1,024-token output limit. Reasoning was on for both models, the 27B at its medium level, and decode figures include reasoning tokens. Single-request figures are medians.

oMLX is the closest comparison: a general-purpose server with batching and cache reuse across many models. Lily and uzu serve one request at a time, and uzu's Qwen3.8-27B package has no draft. Because each engine ran at its own recommended settings, these are end-to-end comparisons. They show the combined effect of specializing the engine for its model, not the contribution of any one part.

Decode Stays Ahead as the Context Grows

Splash decodes 1.7× and 2.0× as fast as the next-fastest engine on Qwen3.6-35B-A3B and Qwen3.8-27B, and the lead holds at every prompt length we tested. On short prompts that is 210 and 74 tokens/s. At 32K it is 143 and 54 tokens/s. For an agent, that means a session deep into its history runs faster on Splash than a fresh one does on any other engine we tested. The draft trained for the model and the kernels tuned to its shapes hold that lead.

Prefill Leads at Every Prompt Length

Prefill is the wait before an agent's first action on a repository, and Splash leads at every prompt length we tested. With 32K prompts it processes about 2,000 input tokens/s on the 35B and 360 on the 27B. Compared with oMLX, the wait for the first token drops from 29 to 17 seconds on the 35B and from 317 to 96 seconds on the 27B. That is a big cut, but reading a large repository cold is still expensive for every engine we tested, ours included. An agent pays that cost when it first reads a repository, not on every turn.

Cache Reuse Is 7× Faster

The next turn is the common case for an agent. It sends back the same context with a little added, and the unchanged part is already loaded. This is where Splash's lead is largest. With a 32K context cached, Splash returns the first token in 123 ms on the 35B and 282 ms on the 27B, instead of the 17 and 96 seconds of the cold read. oMLX also hits its cache on every prompt, and Splash still reaches the first token 6.6× and 7.3× as fast.

On replay, Lily took 22 seconds on the 35B with no cache hits, and uzu took 112 seconds on the 27B, the same as its first pass. Ollama's cache did not hit on these long prompts. Without cache reuse, every turn pays the cold read again.

Concurrency Widens the Gap

When an agent fans out into four requests at once, Splash's lead grows. Its combined decode throughput is 2.0× and 3.9× that of the next-fastest engine, up from 1.7× and 2.0× for a single request. On short prompts that is 357 tokens/s on the 35B and 170 on the 27B. With 32K prompts the gap is wider still: 236 tokens/s against oMLX's 62 on the 35B, a 3.8× advantage. Four subagents share one GPU, and batching plus the draft keep all of them moving.

More requests fit, too. We sent 16 concurrent 32K requests to the 27B on this 48 GB machine. A general-purpose memory policy accepted nine of them on its first pass. Splash accepted and completed all 16. We did not test whether another engine could be configured to do the same. That is the memory plan at work: because the model's sizes are known in advance, Splash can budget nearly all of the machine's memory safely.

The Bottom Line

Splash rethinks local model inference from the ground up by building the engine around the model. Compared to a general-purpose engine on the same Mac, Splash provides superior performance across decode, prefill, cache reuse, and higher levels of subagent concurrency. These gains come from automatic kernel generation, specialized draft models, and a hardware-aware memory plan.

This is the same technology powering Inco's industry-leading inference platform on data-center GPUs, the fastest provider on Artificial Analysis for Kimi K3, MiniMax M3, GLM 5.3, GLM 5.3 Flash, and DeepSeek V4.1 Flash. DFlash draft models run in SGLang, vLLM, TensorRT-LLM, and llama.cpp, and have been downloaded more than 6 million times. Splash now brings those techniques to Apple silicon. One idea, from the largest data centers to a laptop in your living room.

Splash is out today, open source under Apache-2.0 at github.com/incoai/splash, with two supported models and more on the way. If you want a model on it, including your own fine-tunes, or you are building agents that run on your users' machines, write to us at contact@inco.ai. We are also hiring.

The engine is built around the model.

Citation

Please cite this post as:

@misc{inco2026splash,
  title  = {{Splash: A Local Engine Built Around the Model}},
  author = {{Inco AI}},
  year   = {2026},
  month  = {September},
  url    = {https://inco.ai/blog/splash/}
}