/v1/chat/completions protocol works as a provider: Ollama, llama.cpp, LM Studio, or Xynthis’s own on-device model. Chat then runs entirely on your machine, and since all memory already lives locally under ~/.xynthis/, nothing leaves it.
An Ollama preset ships in the default config (http://localhost:11434/v1, model llama3.3); if Ollama is running, xynthis models set ollama is all it takes. A capable local model like that runs the full agent loop, tools included. The rest of this page covers the other case: a small model that needs lite mode.
Pick a model for your machine
You are never locked into one model. The picker lists every configured provider and you select whichever fits your hardware — so pick what you can run. Built-in presets span the whole range:
If your laptop is low-spec, the honest answer is you don’t need a local model at all: a cloud provider does 100% of the model compute remotely, and Xynthis’s memory is already local and lightweight, so nothing heavy runs on your machine. If you’re logged into Claude or ChatGPT,
xynthis models set claude-fable (or codex-5-6) just works with zero download.
The tiny offline presets (qwen-05b, llama-1b, qwen-3b) all ship lite and are served on demand by Ollama; each needs a one-time ollama pull <model> and greys in the picker until it’s pulled. They’re for running fully offline on a modest machine.
Worked example: MiniCPM5-1B
MiniCPM5-1B (OpenBMB) is a dense 1-billion-parameter on-device model that is state-of-the-art in its class at agentic tool use and full conversation — so unlike apfel below, it ships non-lite and runs the whole agent loop. It’s the recommended local model when you want tools and offline. Theminicpm provider is a built-in preset:
http://127.0.0.1:11434/v1, model minicpm5-1b) with launch_command = "ollama serve", so serve brings the Ollama daemon up on demand and greys the entry in the picker until the model is actually served. It’s a one-time job to make the model available to Ollama (the weights are not downloaded automatically):
minicpm shows as available in the picker and runs with the full toolset. If the 1B model struggles under the full ~70-tool prompt, set lite = true on the [providers.minicpm] block in config.toml to route it through the lite path instead. The picker’s availability check is model-aware for Ollama: it stays greyed until the exact minicpm5-1b tag is served, even if Ollama is already up for a different model.
Worked example: apfel
apfel is a CLI that exposes Apple Intelligence (the Mac’s built-in foundation model) as an OpenAI-compatible server. Theapfel provider ships as a built-in preset, so on a Mac you only need to install the CLI and select it:
apfel --serve --port 8091 on demand and greys the entry in the picker until its port answers. The equivalent block, if you ever want to define it by hand for a different port, is:
lite = true: Apple’s on-device model has a small context window and no reliable tool calling. Lite mode keeps it usable (next section).launch_commanduses the absolute binary path. The app runs this command through/bin/sh -c, and GUI apps on macOS get a minimalPATHthat does not include Homebrew’s directories, so a bareapfelwould fail with “command not found”. Runwhich apfelin a terminal and use that path.
curl http://127.0.0.1:8091/v1/models shows the exact name to put in default_model.
What lite mode does
The full agentic prompt is large: the tool specifications alone (79 built-in tools under the daemon) make up most of a ~24k-token system prompt. A small local model can’t tool-call anyway, and on a RAM-tight Mac the resulting KV cache spikes memory hard. When the active provider haslite = true, the daemon (which the app and a plain xynthis chat use) runs a slimmed-down agent instead; in-process runs such as the bare xynthis REPL or xynthis chat --no-app ignore the flag:
- Two tool schemas are sent instead of the full set:
recallandweb_search. The model cannot edit files, run commands, or browse. - The system prompt shrinks to a short one.
- Pre-turn memory injection is capped, not skipped: a short brain-recall hint of up to roughly 1,200 characters is injected. Skill patterns are not.
XYNTHIS_LITE_MICRO=off to fall back to the stricter behaviour — no tool schemas and no memory injection at all.
The result is a fast, low-memory local chat. The trade-off is real: lite mode can recall and search the web, but it cannot act. For anything that needs the agent to edit, run, or browse, switch back to a full provider: xynthis models set anthropic, or one tap in the app’s model picker. Your memory is untouched either way; the brain keeps running and storing regardless of which chat model is active.
Auto-start from the app
When you pick a local provider in the app’s model picker and its server isn’t answering on its port, the app runs the provider’slaunch_command, waits for the port to come up, then switches. Providers without a launch_command just show as unavailable until you start the server yourself. A plain xynthis chat starts the daemon too, and the daemon runs a local provider’s launch_command when its port is not answering, so the CLI does start servers on that path. In-process runs (xynthis chat --no-app, --session, --new, the bare xynthis REPL, or xynthis resume) never start it: run the server yourself or put it under your own process manager.