Use your own model

How it works

Ollama, MLX LM, and LM Studio expose an OpenAI-compatible API on your Mac. Ollama and LM Studio are called by the browser; MLX is called through an authenticated bridge in the local application server because the stock MLX server does not advertise browser CORS support.

Ollama

1. 1Install from ollama.com and start the app.
2. 2For a 36 GB Mac, pull qwen3.5:9b for the live demo or qwen3.5:27b-q4_K_M for a slower quality pass.
3. 3Allow this site to reach it by setting OLLAMA_ORIGINS, then restarting Ollama.
4. 4Open Local models, press Test connection, then select a model. The choice is saved automatically for this signed-in profile.

# install a tool-capable model
ollama pull qwen3.5:9b

# macOS / Linux - allow browser access, then restart Ollama
export OLLAMA_ORIGINS="http://localhost:8080,http://127.0.0.1:8080"
ollama serve

# Windows (PowerShell)
setx OLLAMA_ORIGINS "http://localhost:8080,http://127.0.0.1:8080"

# check it's up
curl http://localhost:11434/v1/models

Ollama's OpenAI-compatible endpoints live at http://localhost:11434/v1 - /models and /chat/completions, including tools support. Full reference: ollama/docs/openai.md.

MLX LM on Apple silicon

1. 1Update your existing environment with pip install -U mlx-lm.
2. 2Start one 4-bit model on 127.0.0.1:8081. Port 8080 is reserved for this application.
3. 3Keep the application server running on the same Mac; it provides the authenticated same-origin bridge.
4. 4Open Local models, choose MLX LM, and test the connection.

# fast, recommended showcase model
mlx_lm.server \
  --model mlx-community/Qwen3.5-9B-MLX-4bit \
  --host 127.0.0.1 \
  --port 8081

# inspect the endpoint from another terminal
curl http://127.0.0.1:8081/v1/models

LM Studio

1. 1Install LM Studio and download a model from the Discover tab.
2. 2Open the Developer tab and press Start Server. The default port is 1234.
3. 3In the server settings, switch on Enable CORS.
4. 4Open Local models, choose LM Studio, test the connection and pick your model.

# optional CLI
lms server start --cors

# check it's up
curl http://localhost:1234/v1/models

LM Studio serves /v1/models, /v1/chat/completions, /v1/completions and /v1/embeddings on http://localhost:1234, with tool calling on supported models. Full reference: lmstudio.ai/docs.

Which model should I pick?

• 3–4B: fast smoke testing. Expect malformed JSON and weaker multi-step decisions; the deterministic fallback keeps the demo safe.
• 9B 4-bit: recommended for a 36 GB M4 showcase. It leaves useful memory headroom while providing materially better tool and instruction following.
• 27B 4-bit: the quality option for negotiation, using roughly 17–20 GB for common local packages before runtime overhead. Run one turn at a time.
• Whatever you pick, it must support tool calling - otherwise it can talk about products but never search for them.

When something doesn't work

• “Couldn't reach …” - the server isn't running. For Ollama/LM Studio also re-check CORS; for MLX keep both local servers running and use port 8081.
• Models list is empty - you haven't downloaded a model yet, or LM Studio hasn't loaded one into the server.
• It answers but never searches - the model doesn't support tools. Switch to one that does.
• Very slow first reply - the model is loading into memory. The second question will be much quicker.
• Different port - direct providers may use another loopback port. The authenticated MLX bridge intentionally stays fixed to the server's configured loopback URL.