Quick Start

1. Install

Linux

curl -sSf https://leanmodels.ai/install.sh | sh

Windows (PowerShell)

irm https://leanmodels.ai/install.ps1 | iex

Both installers download the CUDA build by default - that is what runs models on an NVIDIA GPU. For the CPU-only build instead:

curl -sSf https://leanmodels.ai/install.sh | sh -s -- --cpu

& ([scriptblock]::Create((irm https://leanmodels.ai/install.ps1))) -Cpu

The CPU build is fine for lean pull and lean verify, but far too slow to serve these models.

Already have lean? Upgrade in place instead of reinstalling - it keeps whichever build kind you have and verifies the download before replacing the binary:

lean update

One binary, no Python, no Docker, no runtime services. lean talks to your GPU through CUDA and to models through the local filesystem. Both installers verify the download against a published SHA256 checksum and refuse to install if it does not match.

The CUDA build needs the CUDA Toolkit, not just a driver

lean links cuBLAS, cuRAND and NVRTC directly, and those ship with the CUDA Toolkit rather than with the NVIDIA display driver. If nvidia-smi works but you have never installed the toolkit, you have the driver and not these.

Your operating system resolves them before lean starts, so a missing one is not a lean error message — it is error while loading shared libraries on Linux, or a missing-DLL dialog on Windows. Both installers check and will tell you first.

lean needs an NVIDIA GPU. There is no macOS build: lean exists to run models larger than your VRAM by streaming experts off disk, and it has no Metal backend, so a mac build would be CPU-only.

2. Pull a model

Start with lean-agent-light. It is free, it is the fastest model in the catalog, and it runs on 12 GB of VRAM.

lean pull lean-agent-light

Models land in ~/.leanmodels/models/<name>/. A pull is resumable: re-running the same command after an interruption picks up where it left off, and already-verified files are skipped.

Paid models need an API key first:

lean login
lean pull lean-agent-middle

Checkout is not open yet

The paid models are packaged and verified, but there is no way to buy one today, so lean login has no key to accept. The free models above work now.

3. Chat with it

lean run lean-agent-light

This opens an interactive REPL. /stats prints cache and throughput numbers for the session, /clear resets the conversation, and Ctrl-D exits.

The first response is slower than the rest: the runtime is filling its VRAM expert cache and warming the page cache. Steady-state throughput is what the benchmarks report.

4. Serve it over HTTP

To use the model from an editor, an agent, or your own code, run it as an OpenAI-compatible server:

lean serve lean-agent-light --port 8080

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"user","content":"Write a haiku about GPUs."}]}'

See the API reference for the full request and response shapes, and Aider for a worked example of pointing a coding agent at it.

Checking your hardware

lean tune

lean tune detects your GPUs, system RAM, and free disk, then reports which models fit and what settings they will use. Run it before a large download.

Verifying a download

lean verify lean-agent-light

Walks every checksum in the model’s manifest and reports damaged or missing files. Anything damaged is marked for repair, so a subsequent lean pull re-fetches just those files rather than the whole model.