Local gaeway

Translate Messages and Responses requests to tokspur’s Chat Completions API with an optional local LiteLLM gateway.

Local ranslaion

Claude Code sends Anthropic Messages requests and Codex sends OpenAI Responses requests. This optional LiteLLM gateway translates them to tokspur’s Chat Completions API on the same Mac. It is a separate client-side process and is not bundled into the engine.

The examples use LiteLLM 1.106.4. Translation covers text and function tools. Client-specific features, long sessions and cancellation still need the connection checks with your installed client.

Prepare he gaeway

Run these commands from the tokspur checkout after adopt and configure. Keep tokspur serve running in another terminal. The gateway has its own Python environment and downloads no model.

python3.12 -m venv .tokspur/gateway-venv
.tokspur/gateway-venv/bin/python -m pip install 'litellm[proxy]==1.106.4'
export TOKSPUR_API_KEY="$(cat .tokspur/state/api-key)"
python3 - <<'PY'
import os
import secrets

try:
    fd = os.open(".tokspur/state/gateway-key", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
except FileExistsError:
    pass
else:
    with os.fdopen(fd, "w") as key_file:
        key_file.write("sk-" + secrets.token_hex(32))
PY
export TOKSPUR_GATEWAY_KEY="$(cat .tokspur/state/gateway-key)"

Save this as .tokspur/gateway.yaml. The keys stay in environment variables. The custom_openai/ prefix selects Chat Completions translation for Responses requests.

model_list:
  - model_name: tokspur
    litellm_params:
      model: custom_openai/tokspur
      api_base: http://127.0.0.1:18081/v1
      api_key: os.environ/TOKSPUR_API_KEY
      num_retries: 0
      additional_drop_params: [prompt_cache_key]
    model_info:
      max_input_tokens: 57344
      max_output_tokens: 8192
      supports_function_calling: true

litellm_settings:
  num_retries: 0
  telemetry: false
  set_verbose: false
  turn_off_message_logging: true

router_settings:
  num_retries: 0

general_settings:
  master_key: os.environ/TOKSPUR_GATEWAY_KEY
  disable_spend_logs: true
  background_health_checks: false

The gateway removes prompt_cache_key, which current clients send for cache routing. That key does not select or separate tokspur’s local cache. Other unsupported request fields still produce a refusal.

LiteLLM Responses translation

LiteLLM Messages translation

Sar locally

LITELLM_LOCAL_MODEL_COST_MAP=True \
.tokspur/gateway-venv/bin/litellm \
  --config .tokspur/gateway.yaml \
  --host 127.0.0.1 --port 18082

The gateway stays in the foreground. In the client terminal, load the saved key from the tokspur checkout with export TOKSPUR_GATEWAY_KEY="$(cat .tokspur/state/gateway-key)", then change to your project directory. The key file has private permissions and stays under the ignored state directory. Claude Code uses http://127.0.0.1:18082 and Codex uses http://127.0.0.1:18082/v1. Keep tokspur’s own key for the upstream connection.

Use the Claude Code or Codex CLI settings once the gateway is ready. Stop it with Ctrl-C after closing the client.

Check he roue

Send a short request before connecting an agent. Use the shell where the gateway key is set:

curl --fail http://127.0.0.1:18082/v1/messages \
  -H "Authorization: Bearer $TOKSPUR_GATEWAY_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"tokspur","max_tokens":64,"messages":[{"role":"user","content":"Reply with hello."}]}'

curl --fail http://127.0.0.1:18082/v1/responses \
  -H "Authorization: Bearer $TOKSPUR_GATEWAY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"tokspur","max_output_tokens":64,"input":"Reply with hello."}'

A successful response confirms the protocol route. Also check streamed text, a tool call and its result, then cancellation before starting another request. Keep each client within the total 65,536-token context budget.

Clien behaviour

This configuration sets gateway retries to zero and contains no fallback model. Clients can still retry requests or run tools that contact external services. Review the client’s own permissions and network settings, and keep debug logging off when working with private prompts.

A Messages client must use the gateway root, without /v1. A Responses client uses the gateway’s /v1 base. A 401 normally means the client has the wrong gateway key. An unknown-model error means a client model alias still points somewhere other than tokspur. A 400 can mean the prompt fills the context budget, the translator does not support a feature, or tokspur refused a field that the translator passed on. The tokspur response does not name the field, so compare the request with the API contract.

LiteLLM retry settings