llama-server speaks the OpenAI API directly, and GoModel has a
dedicated llamacpp provider type for it: the API key is optional
(llama-server usually runs keyless), and provider-native endpoints such as
/health and /rerank are reachable through passthrough. Do not register
llama.cpp as an ollama provider: that type speaks Ollama’s native API, which
llama.cpp does not implement.
The same provider type fits LM Studio and any other plain OpenAI-compatible
local server.
Start llama-server first. Without --alias, the model ID in /v1/models is
the model file’s path — set an alias so requests can use a clean name:
Configure
The base URL is required and registers the provider — llama-server’s default port (8080) collides with GoModel’s own, so there is no default:These examples assume GoModel runs in Docker and llama-server is on the host
at
localhost:8081 — hence host.docker.internal. If GoModel runs on the
host directly, use http://localhost:8081/v1. Running several llama-server
instances? Register each under a suffixed name:
LLAMACPP_STUDIO_BASE_URL=... creates provider llamacpp-studio.Verify
GET /v1/models returns llama-server’s model IDs prefixed by provider name.
Embeddings
/v1/embeddings works through GoModel as long as llama-server can serve it:
the loaded model must use a pooling type other than none. Dedicated embedding
GGUFs usually declare pooling in their metadata; otherwise pass --pooling mean
(or cls/last). The --embeddings flag is optional — it restricts the
server to embeddings only:
Model classification
llama-server’s/v1/models listing carries no capability metadata, so GoModel
classifies its models by ID: names containing embed or matching well-known
embedding families (bge, e5, gte, minilm) are categorized as embedding
models, and names containing rerank as reranking models — namespaced IDs are
checked by their final path segment. For anything that stays unclassified,
declare modes (and context window or pricing) under the provider’s model
metadata — see Model metadata. Categories only
affect dashboard grouping and failover suggestions; /v1/embeddings routes to
any model the provider serves regardless of category.
Beyond chat and embeddings
- Multimodal input — image and audio input in chat messages works with
multimodal models when llama-server is started with a projector
(
--mmproj, auto-loaded with-hfwhen available). These requests flow through/v1/chat/completionsnormally. - Native endpoints — llama-server’s provider-native routes are reachable
through passthrough at
/p/llamacpp/...: OpenAI-shaped paths (chat/completions,embeddings, …) are served from the/v1base, everything else (/health,/props,/tokenize,/infill, …) from the server root. For a suffixed setup use the instance name, e.g./p/llamacpp-studio/health. - Reranking — llama-server serves reranking (start with a reranker model
and
--rerank --embedding --pooling rank), but GoModel has no rerank endpoint; reach it through passthrough:POST /p/llamacpp/rerank(or/p/llamacpp/v1/rerank).