> ## Documentation Index
> Fetch the complete documentation index at: https://gomodel-refactor-unify-native-forwarding.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Session Keeping

> Group requests from one client session for sticky load balancing and threaded audit logs

Coding agents and chat apps send many requests that belong to one logical
session — one conversation, one agent task. GoModel detects that session and
uses it throughout routing and observability:

* **Sticky load balancing** — a load-balanced virtual model routes every
  request of a session to the target that served it first, which keeps provider
  prompt caches warm and model behavior consistent mid-conversation.
* **Sticky API-key selection** — when one provider has several keys, every
  request in a detected session uses the same key, preserving provider
  prompt-cache affinity.
* **Threaded audit logs** — the dashboard's Audit Logs page groups a session's
  requests into one thread: the latest request as the row, with an expander
  that unfolds the older requests beneath it.
* **Per-session usage and cost** — usage records carry the same scoped session
  id, so the dashboard and admin API can report the requests, tokens, and
  provider spend for one conversation or agent task.

Session keeping works with zero configuration. Defaults fit most setups; the
options below exist for the rare cases they don't.

## How a session is identified

The first matching signal wins:

1. **Session headers** — a built-in registry covers the headers known tools
   send, plus gateway conventions:

   | Header                      | Sent by                               |
   | --------------------------- | ------------------------------------- |
   | `X-Session-Id`              | OpenCode, Kilo Code, generic clients  |
   | `X-Claude-Code-Session-Id`  | Claude Code (documented for gateways) |
   | `Session-Id` / `Session_id` | Codex CLI (current / older), Roo Code |
   | `X-Litellm-Session-Id`      | LiteLLM convention                    |
   | `Helicone-Session-Id`       | Helicone convention                   |
   | `Agent-Session-Id`          | Goose                                 |

2. **Body fields** — for clients that mark sessions in the request body:
   Anthropic `metadata.user_id` (Claude Code embeds its session UUID there,
   both current and legacy formats are parsed), `session_id` (OpenRouter
   convention), `litellm_session_id`, `prompt_cache_key` (Zed and other OpenAI
   clients that reuse a cache key per conversation), and `/v1/responses`
   `conversation` references.

3. **Automatic detection** — chat and responses requests with no explicit
   signal are grouped by their conversation opening: the model, system
   context, tools, and the messages through the first user turn. Follow-up
   requests resend that prefix unchanged, so an agent that simply replays its
   history (Aider, Cline, Continue, …) still gets a stable session id
   (`auto-…`) with no client changes.

Body-based signals and automatic detection read request bodies up to 1 MiB
(chunked requests included); larger bodies fall back to header signals.

Client-provided session ids are scoped by [user path](/features/user-path),
including UUID-shaped values, so one tenant cannot collide with another by
reusing the same id.

## Sticky load balancing

When a request carries a session and resolves through a
[virtual model](/features/virtual-models) with several targets, the first
request picks a target via the redirect's strategy (`round_robin` or `cost`)
and later requests stick to it. If the pinned target becomes unavailable or
saturated, the strategy picks again and the session re-pins — a session is
never glued to a dead target, and the all-saturated honest-429 behavior is
unchanged.

Affinity is on by default and can be disabled per redirect:

```yaml theme={null}
virtual_models:
  - source: "smart"
    strategy: round_robin
    session_affinity: false   # default: true
    targets:
      - model: "openai/gpt-4o"
      - model: "anthropic/claude-sonnet-5"
```

The same flag is available in the dashboard's virtual model editor ("Session
keeping") and on the admin API (`session_affinity` on
`PUT /admin/virtual-models`).

Pins are in-memory per instance (like rate-limit counters): after a restart or
on another replica, the next request of a session simply re-pins. Idle
sessions expire after 6 hours.

API-key affinity is deterministic rather than stored, so the same ordered key
set selects the same key across replicas and restarts. Disable it per provider
with `session_sticky_keys: false`,
`<PROVIDER>[_SUFFIX]_SESSION_STICKY_KEYS=false`, or the dashboard's
**Session-sticky API keys**
checkbox. Requests with no detected session remain round robin.

## Threaded audit logs

Audit entries record the session id (`session_id`), and the Audit Logs page
groups them by default ("Group by session" toggle). Each thread shows its
latest request with a count badge; the expander on the left unfolds the older
requests. The badge counts the complete session. Filters select which sessions
appear and which matching request represents each thread.
`GET /admin/audit/sessions` returns each thread's `request_count` and latest
matching audit-log entry. `GET /admin/audit/log?session_id=…` returns all
requests in an expanded session.

Each grouped thread also has a usage-chart action. It opens Usage Analytics
with that session selected; expanding an individual request exposes the same
action on its session badge.

## Per-session usage and cost

Usage Analytics includes a full-width **Usage by Recent Session** table. Each
row represents one detected, user-path-scoped session and shows request count,
input/output/total tokens, and input/output/total cost. Session chips in that
table and the request log filter the whole usage page.

The admin API exposes the same bounded report:

```bash theme={null}
curl -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  "http://localhost:8080/admin/usage/sessions?days=7&limit=50&offset=0"
```

```json theme={null}
{
  "entries": [
    {
      "session_id": "scoped-0123456789abcdef0123456789abcdef",
      "user_path": "/team/alpha",
      "requests": 8,
      "input_tokens": 24000,
      "output_tokens": 4100,
      "total_tokens": 28100,
      "input_cost": 0.42,
      "output_cost": 0.16,
      "total_cost": 0.58
    }
  ],
  "total": 1,
  "limit": 50,
  "offset": 0
}
```

Requests and tokens include responses served by GoModel's local response
cache. Cost fields include provider-bound requests only, because cost stored on
a local-cache hit is the cost that was avoided, not money actually spent.
Results are ordered by latest session activity; `limit` defaults to 50 and is
capped at 200. The normal usage filters (`session_id`, `user_path`, `model`,
`provider`, `label`, and date range) apply.

## Configuration

Everything is on by default.

```bash theme={null}
# Master switch for session identification
SESSION_KEEPING_ENABLED=true
# Content-based detection for requests with no explicit signal
SESSION_AUTO_DETECT=true
# The built-in header/body-field registry
SESSION_BUILTIN_RULES=true
# Extra session headers (numbered from 1), merged over the built-ins.
# The optional transform "session-uuid" extracts a session_<uuid> value.
SESSION_HEADER_1=X-My-Session
SESSION_HEADER_1_TRANSFORM=
```

Or in `config.yaml`:

```yaml theme={null}
session:
  enabled: true
  auto_detect: true
  builtin_rules: true
  headers:
    - header: X-My-Session
```

Credential-bearing headers (`Authorization`, API-key headers, …) are rejected
as session sources, since the session id is persisted on audit and usage
records.

**When to change the defaults:** set `SESSION_AUTO_DETECT=false` if you only
want explicitly-tagged sessions grouped; set `session_affinity: false` on a
redirect when you prefer strict load spreading over cache affinity (for
example pure round-robin capacity balancing across identical deployments).