> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xynthis.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Local models

> Run Xynthis fully offline against any OpenAI-compatible local server, with a worked apfel example.

Any local server that speaks the OpenAI `/v1/chat/completions` protocol works as a provider: Ollama, llama.cpp, LM Studio, or Xynthis's own on-device model. Chat then runs entirely on your machine, and since all memory already lives locally under `~/.xynthis/`, nothing leaves it.

An Ollama preset ships in the default config (`http://localhost:11434/v1`, model `llama3.3`); if Ollama is running, `xynthis models set ollama` is all it takes. A capable local model like that runs the full agent loop, tools included. The rest of this page covers the other case: a small model that needs **lite mode**.

## Worked example: apfel

apfel is a CLI that exposes Apple Intelligence (the Mac's built-in foundation model) as an OpenAI-compatible server. The `apfel` provider ships as a built-in preset, so on a Mac you only need to install the CLI and select it:

```bash theme={null}
brew install apfel
xynthis models set apfel
```

The preset already carries the right settings; serve launches `apfel --serve --port 8091` on demand and greys the entry in the picker until its port answers. The equivalent block, if you ever want to define it by hand for a different port, is:

```toml theme={null}
[providers.apfel]
kind = "openai-chat"
base_url = "http://127.0.0.1:8091/v1"
default_model = "apple-foundationmodel"
lite = true
launch_command = "/opt/homebrew/bin/apfel --serve --port 8091"
```

Port 8091 is deliberate: apfel's default collides with Ollama's 11434.

Two details matter here:

* **`lite = true`**: Apple's on-device model has a small context window and no reliable tool calling. Lite mode keeps it usable (next section).
* **`launch_command` uses the absolute binary path.** The app runs this command through `/bin/sh -c`, and GUI apps on macOS get a minimal `PATH` that does not include Homebrew's directories, so a bare `apfel` would fail with "command not found". Run `which apfel` in a terminal and use that path.

Single-model servers generally accept any model id; if yours is picky, `curl http://127.0.0.1:8091/v1/models` shows the exact name to put in `default_model`.

## What lite mode does

The full agentic prompt is large: the tool specifications alone (roughly 70 tools) make up most of a \~24k-token system prompt. A small local model can't tool-call anyway, and on a RAM-tight Mac the resulting KV cache spikes memory hard. When the active provider has `lite = true`, the agent runs as plain chat instead:

* No tool schemas are sent. The model cannot edit files, browse, run commands, or search memory.
* The system prompt shrinks to a short one.
* Pre-turn memory injection is skipped entirely: no brain recall, no skill patterns. Remembered facts do not surface in lite conversations.

The result is a fast, low-memory local chat. The trade-off is real: lite mode is conversation only. For anything that needs the agent to act or remember, switch back to a full provider: `xynthis models set anthropic`, or one tap in the app's model picker. Your memory is untouched either way; the brain keeps running and storing regardless of which chat model is active.

## Auto-start from the app

When you pick a local provider in the app's model picker and its server isn't answering on its port, the app runs the provider's `launch_command`, waits for the port to come up, then switches. Providers without a `launch_command` just show as unavailable until you start the server yourself. The CLI does not auto-start servers; run the server first, or put it under your own process manager.