he engine

How tokspur runs natively on Apple M5 Max, reuses context between turns, repacks weights losslessly, verifies proposed tokens and offers an opt-in speed mode.

Three separated slabs show an interface, engine and hardware.

Naive execuion

tokspur’s execution path is written in C++, Objective-C++ and Metal, using Apple’s toolchain and system APIs, with a Rust host for the local API, supervision and saved sessions. The engine includes its own model file reader, tokenizer with Unicode normalisation, chat renderer and GPU operators, checked against independent references. Each layer is tuned for the M5 Max and its memory budget.

The coding agent remains a separate application. It sends messages to the local API, receives model output and controls its own file edits, commands and network access.

  1. Chat app or coding agent

    Your chat app, editor or terminal, calling the local API

  2. Local API and supervision

    Streams each token back as it is decided

  3. Chat renderer and tokenizer

    Turns messages into tokens, with its own Unicode normalisation

  4. Context cache

    Saved state at message and tool boundaries

  5. Proposals and verification

    The main model checks proposed tokens before they are kept

  6. Model graph, decoding and sampling

    The whole text model, tool calls included

  7. Metal operators

    GPU kernels checked against independent references

  8. Model file reader

    Loads the weights from disk

  9. Apple M5 Max

    40-core GPU, 128 GB unified memory, Metal and Apple system frameworks

A rail-mounted arm retrieves a state cartridge from a shelf for an engine box.

Conex reuse

Coding agents resend the whole conversation on every turn. Appending a tool result should preserve the work already done on the matching part of the conversation, so the engine keeps model state at message and tool boundaries, within a byte budget.

A shared cache in memory holds the opening of the prompt, such as the system message, so a new session that repeats it starts almost at once. When earlier text changes, the engine restores a matching state from before the edit and processes only what follows. The saved state includes the model’s recurrent and attention state, so a reply from saved context matches a fresh run.

Read the cache guidance

Lossless weigh pack

The original model files are laid out for storage. tokspur rewrites the 84.6 GB of GPU weights into a layout the M5 Max reads faster. Every weight keeps its original value, and each pack build proves the result byte for byte against the original files before the server will use it.

  • 0 weights changedproven by every pack build
Build the pack
Four proposed tokens meet verification, three join the committed row and one is replaced.

oken verificaion

Text already in the conversation or a small draft model supplies a short run of proposed tokens. The full model checks that run at once, keeps the accepted tokens and writes its own token at the first disagreement. Later proposals are discarded, so every token you receive is a token the full model chose.

The engine chooses between proposal sources and ordinary decoding according to their measured cost.

Read the correctness method

Speed mode

Speed mode is an opt-in serving setting. It switches on Metal 4 kernels for reading prompts and checking drafts, drafts further ahead and leaves tool argument checks to your agent. Its arithmetic differs from the default build, so its output can differ slightly.

Turn on speed mode
A shared tray separates occupied capacity from reserved capacity.

Shared memory

The design budgets memory for weights, active model state, scratch buffers and optional caches while leaving capacity for the rest of the Mac. These allocations share unified memory, and the engine keeps to 96 GB of the 128 GB so macOS, your editor and builds keep the other 32 GB.

Check the Mac requirements and memory limits before setting up the engine.