Session Keeping

• Sticky load balancing — a load-balanced virtual model routes every request of a session to the target that served it first, which keeps provider prompt caches warm and model behavior consistent mid-conversation.
• Sticky API-key selection — when one provider has several keys, every request in a detected session uses the same key, preserving provider prompt-cache affinity.
• Threaded audit logs — the dashboard’s Audit Logs page groups a session’s requests into one thread: the latest request as the row, with an expander that unfolds the older requests beneath it.
• Per-session usage and cost — usage records carry the same scoped session id, so the dashboard and admin API can report the requests, tokens, and provider spend for one conversation or agent task.

​

How a session is identified

1. Session headers — a built-in registry covers the headers known tools send, plus gateway conventions:

   Header	Sent by
   X-Session-Id	OpenCode, Kilo Code, generic clients
   X-Claude-Code-Session-Id	Claude Code (documented for gateways)
   Session-Id / Session_id	Codex CLI (current / older), Roo Code
   X-Litellm-Session-Id	LiteLLM convention
   Helicone-Session-Id	Helicone convention
   Agent-Session-Id	Goose

2. Body fields — for clients that mark sessions in the request body: Anthropic metadata.user_id (Claude Code embeds its session UUID there, both current and legacy formats are parsed), session_id (OpenRouter convention), litellm_session_id, prompt_cache_key (Zed and other OpenAI clients that reuse a cache key per conversation), and /v1/responses conversation references.
3. Automatic detection — chat and responses requests with no explicit signal are grouped by their conversation opening: the model, system context, tools, and the messages through the first user turn. Follow-up requests resend that prefix unchanged, so an agent that simply replays its history (Aider, Cline, Continue, …) still gets a stable session id (auto-…) with no client changes.

​

Sticky load balancing

virtual_models:
  - source: "smart"
    strategy: round_robin
    session_affinity: false   # default: true
    targets:
      - model: "openai/gpt-4o"
      - model: "anthropic/claude-sonnet-5"

​

Threaded audit logs

​

Per-session usage and cost

curl -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  "http://localhost:8080/admin/usage/sessions?days=7&limit=50&offset=0"

{
  "entries": [
    {
      "session_id": "scoped-0123456789abcdef0123456789abcdef",
      "user_path": "/team/alpha",
      "requests": 8,
      "input_tokens": 24000,
      "output_tokens": 4100,
      "total_tokens": 28100,
      "input_cost": 0.42,
      "output_cost": 0.16,
      "total_cost": 0.58
    }
  ],
  "total": 1,
  "limit": 50,
  "offset": 0
}

​

Configuration

# Master switch for session identification
SESSION_KEEPING_ENABLED=true
# Content-based detection for requests with no explicit signal
SESSION_AUTO_DETECT=true
# The built-in header/body-field registry
SESSION_BUILTIN_RULES=true
# Extra session headers (numbered from 1), merged over the built-ins.
# The optional transform "session-uuid" extracts a session_<uuid> value.
SESSION_HEADER_1=X-My-Session
SESSION_HEADER_1_TRANSFORM=

session:
  enabled: true
  auto_detect: true
  builtin_rules: true
  headers:
    - header: X-My-Session