OpenAI-compatible API

lean serve exposes an HTTP API shaped like OpenAI’s, so most clients and SDKs work by changing the base URL.

lean serve lean-agent-light --port 8080

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key="dummy")
resp = client.chat.completions.create(
    model="lean-agent-light",
    messages=[{"role": "user", "content": "Explain expert offloading in two sentences."}],
)
print(resp.choices[0].message.content)

The server holds one model and serves one request at a time. Concurrent requests queue rather than interleave.

Authentication

Without --api-key (or LEAN_API_KEY) the API is open, which is why the default bind address is loopback-only. With a key set, every /v1 route requires it:

Authorization: Bearer <key>

/health is always reachable, key or not, so process supervisors do not need credentials.

POST /v1/chat/completions

Request

Field	Type	Default	Notes
messages	array	required	{"role": "...", "content": "..."}. Roles: system, user, assistant.
model	string	server’s model	Accepted and echoed back; the server serves whatever it was launched with.
max_tokens	integer	model limit	Rejected with 400 if larger than the model’s context length.
temperature	number	0.7	0 is greedy.
top_p	number	unset	Nucleus sampling.
seed	integer	random	Fixes sampling for reproducible output.
stream	boolean	false	Server-sent events instead of one JSON body.
thinking	boolean	true	Extension. Set false to skip the <think> phase. Ignored by models without one.

Response

{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1769472000,
  "model": "lean-agent-light",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": "The user wants a short explanation...",
        "content": "Expert offloading keeps only the experts a token actually routes to in VRAM..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 24, "completion_tokens": 96, "total_tokens": 120 }
}

finish_reason is stop when the model emits an end-of-sequence token, or length when it hits max_tokens.

On thinking models, the <think> block is split out into reasoning_content rather than left inline in content. The field is omitted entirely when there is no reasoning to report, so clients that ignore it still see clean prose.

Streaming

With "stream": true the response is text/event-stream. The first chunk carries the role, subsequent chunks carry content (or reasoning_content) deltas, the last chunk carries finish_reason and usage, and the stream ends with the literal data: [DONE].

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Expert"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":24,"completion_tokens":96,"total_tokens":120}}

data: [DONE]

POST /v1/completions

The legacy raw-text endpoint. It exists mainly so evaluation harnesses that score prompts (lm-eval-harness and friends) can drive the runtime.

Field	Type	Default	Notes
prompt	string or array	required
max_tokens	integer	16	OpenAI’s legacy default, not the chat default.
temperature	number	0.7
top_p	number	unset
stop	string or array	unset	Stop strings, trimmed from the output.
seed	integer	random
logprobs	integer	unset	Return per-token logprobs. Only the top-1 alternative is reported.
echo	boolean	false	Include the prompt in the response, with its logprobs when scoring.

Setting max_tokens: 0 with echo and logprobs selects pure scoring mode: the server runs a single chunked all-position prefill and returns per-token prompt logprobs without generating anything.

Unlike the chat endpoint, this one applies no chat template and no repeat penalty. What you send is what the model sees.

GET /v1/models

Lists the one model this server is hosting, in OpenAI’s list shape.

{
  "object": "list",
  "data": [
    { "id": "lean-agent-light", "object": "model", "created": 1769472000, "owned_by": "leanmodels" }
  ]
}

GET /health

Returns ok with status 200. Unauthenticated.

Errors

Errors use OpenAI’s envelope:

{ "error": { "message": "max_tokens (999999) exceeds the model's maximum context length (262144)", "type": "invalid_request_error", "code": null } }

Status	When
400	Malformed body, or max_tokens beyond the model’s context length.
401	Missing or wrong bearer key when one is configured.
408	Generation exceeded --request-timeout-secs.
500	Generation failed. The server recovers; the next request is served normally.

Not supported yet

Being explicit, because these are the gaps clients trip over:

• Tool and function calling. tools, tool_choice, and tool_calls are not accepted or emitted. Agentic clients that require them (Codex, Cline, Claude Code) cannot drive the server today. Aider works because it edits through plain text rather than tool calls.
• Multiple completions. No n parameter; one choice per request.
• Concurrency. Requests are serialised behind a single model instance.
• Multiple models per server. One process serves one model. Run several processes on different ports if you need more.
• Multimodal content. content is a plain string; no image or audio parts.
• logit_bias, presence_penalty, frequency_penalty, and response_format are ignored if sent.