Configuration

Profile

tokspur reads the hardware profile in profiles/m5-max-128.json when you run tokspur configure and records the port, the profile and the model identifier in .tokspur/configured.json. tokspur plan prints the selected profile. The build record pins the profile file, so tokspur serve refuses to start with Profile drift after an edit. Change the port with tokspur configure --port and keep a record of it alongside the build version so a change can be reversed.

tokspur reads only the port, the model identifier and the 128 GB memory requirement from the profile. The other values describe the limits and formats that the engine is built around.

The profile file also holds api_memory_ceiling_gib (108), temporary_ssd_cache_gib (0) and persistent_cache (false), and tokspur plan prints them. Nothing in tokspur serve applies them, so they set no memory limit and no cache size. The 96 GB ceiling in the engine applies, and the options under saved context control caching.

Default engine

Native MTP is built into the engine and selected by default. Its weights are supplied through the pinned shared Q8 sidecar. The default uses the engine’s baseline kernels.

Drafter options

Choose native MTP or DFlash2 as the draft model. Prompt lookup adds suggestions from matching text when using MTP. The main model checks the proposed tokens before keeping them.

Native MTP

The default --drafter mtp uses the pinned 8-bit shared sidecar. It drafts 2 tokens ahead by default. Set --mtp-depth 3 for 3-token lookahead.

DFlash2

Select --drafter dflash2 and pass the model directory with --dflash2-dir. Set --dflash2-max-drafts to a value between 1 and 7 for the number of proposals per round. Its dense projections use 8-bit q8_0 by default, or 16-bit bf16 with --dflash2-precision bf16. These settings affect the drafter. The main model keeps its own precision.

See the DFlash2 model files and the speed mode command.

Prompt lookup

Prompt lookup proposes matching code and text already in context. With native MTP, choose --mtp-lookup bypass or --mtp-lookup hybrid to use those matches alongside model-generated drafts. The default is off. Lookup settings apply to MTP only.

Paths and credentials

Adopted model files stay where they are. Below the checkout, .tokspur/adoption.json records the verified files, .tokspur/configured.json records the port and profile, .tokspur/derived/tokspur-pack-v2.asa.gguf holds the lossless GPU weight pack, and the private .tokspur/state/ directory (mode 0700) holds the model link, the API key, the prefix checkpoints, temporary working files and the process and lock receipts. Keep the key out of shell transcripts, screenshots and logs, and keep the state directory out of version control.

Memory and context

On the 128 GB Mac the engine holds its process to 96 GB for weights, active model state, scratch allocations and optional caches, and leaves the rest for macOS and development work. The engine refuses further work if its footprint passes that ceiling or macOS reports memory pressure. The ceiling is a limit and not a reservation, so the engine does not allocate it up front. Retain headroom for your normal tools and keep the Mac’s global memory limits unchanged.

Saved context

Reuse depends on a matching prefix and unchanged model settings. Changing earlier messages or their rendering invalidates the saved state after the change, and the engine then restores a saved prefix that still matches instead of recomputing it.

A request that appends to the conversation continues from the state already in memory. A shared prefix cache in host memory covers the opening of the prompt, such as the system message, and --no-shared-prefix-cache turns it off. Prefix checkpoints on the SSD cover longer prefixes.

SSD checkpoints default to off. Pass --checkpoint-cap-gib 8 to enable an 8 GiB budget, or choose up to 32 GiB. A value of 0 turns them off. They contain unencrypted model state derived from prompts, so include their location in your storage and backup policy. The server writes a checkpoint only while the volume keeps --checkpoint-floor-gib free (20 by default). They live in an owned tokspur-checkpoints-v2/ subdirectory of .tokspur/state/checkpoints/ unless --checkpoint-dir names another parent, and --checkpoint-policy chooses the eviction order, recompute-cost or lru. Checkpoints stay on disk across restarts and apply again when the model, worker, Metal library and numerical settings match. The shared prefix cache and the conversation in memory start empty after a restart.

Read context reuse for the behaviour, and measurement conditions for comparing cold and warm requests.

Speed mode

Speed mode is an opt-in setting for tokspur serve. It switches on the Metal 4 kernels for reading prompts and checking drafts, drafts 3 tokens ahead with the adopted MTP sidecar and leaves tool argument checks to your agent. The current default uses the retained baseline kernels. Speed mode changes the arithmetic, so its output can differ slightly from the default.

.venv/bin/tokspur serve --speed-preset

For the fastest measured generation, use the DFlash2 draft model instead of the MTP sidecar. Download the pinned files listed under speed mode drafter into a directory of their own, then pass it to the server.

.venv/bin/tokspur serve --speed-preset --drafter dflash2 \
  --dflash2-dir /absolute/path/to/Qwen3.8-Flash-Next-DFlash2 \
  --dflash2-max-drafts 7 --dflash2-vocab-slice 0 --native-dflash2-verify-rows auto

The Metal 4 kernels need macOS 27 or later. Speed mode keeps the same 96 GB ceiling and context limit. Saved context stays on with the MTP sidecar, and the DFlash2 drafter starts without it. To return to the default, stop the server and start it again without the extra options. Read the engine notes for what changes and Results for what it measures.

Local operation

tokspur serve runs in the foreground and verifies the pinned sources before it loads the model. To change the port, stop the server and run tokspur configure --port again. Before an update, keep the previous build and configuration so you can restore them. If a process remains after shutdown, use shutdown troubleshooting.