Bring your own LLM

The three routes

The workspace default applies to every governed answer that doesn’t pick a model explicitly — chat (incl. streaming), the REST query APIs, MCP tools, and agent runs. An explicit per-request model always wins, and an explicit “offline” pick stays offline. Model choice can never widen access: the ACL filter runs before synthesis, so any model only sees documents the caller was already entitled to read.

Configuration API

The Settings card is the intended surface, but everything it does rides two admin endpoints (session-cookie auth — these are dashboard routes, not machine-token routes):

# Read the current workspace config (the key itself is never returned)
curl -b "$SESSION_COOKIE" https://your-workspace.example.com/api/admin/llm-config
# → { "defaultModelId": "byo",
#     "byo": { "endpoint": "https://llm.example.com/v1", "model": "llama3.1", "hasKey": false },
#     "posture": { "allowPrivate": false, "deployment": "saas" } }

# Save a self-hosted default (apiKey semantics: omit = keep sealed key, "" = clear, value = replace)
curl -b "$SESSION_COOKIE" -X POST https://your-workspace.example.com/api/admin/llm-config \
  -H 'content-type: application/json' \
  -d '{ "defaultModelId": "byo",
        "byo": { "endpoint": "https://llm.example.com/v1", "model": "llama3.1" } }'

# Dry-run the saved config (1-token completion + informative /models lookup)
curl -b "$SESSION_COOKIE" -X POST https://your-workspace.example.com/api/admin/llm-config/validate

# A governed answer through whatever the workspace default resolves to (machine token)
curl -X POST https://your-workspace.example.com/api/v1/query \
  -H "authorization: Bearer $AIVM_MACHINE_TOKEN" -H 'content-type: application/json' \
  -d '{ "question": "What changed in pricing last quarter?" }'

What Validate actually checks

Self-hosted servers: requirements

All four common servers speak the same OpenAI-compatible surface, so they all use the same config — endpoint + model id, key optional:

None of these ship TLS. Put a reverse proxy (Caddy, nginx, Cloudflare Tunnel) or a tailnet in front for HTTPS. Azure OpenAI is deliberately not covered by the generic path (different auth header, deployment-scoped URLs, no /v1/models) — use a managed model or an OpenAI-compatible gateway for now.

Network posture: SaaS vs your own environment

How your provider key is handled

• Sealed with AES-256-GCM before it is stored, bound to your workspace (the seal’s associated data is the tenant id — another workspace’s Brain cannot open it).
• Write-only: no read path returns it — the config API reports only hasKey: true. Replacing it is typing a new one; clearing it is saving an empty value.
• Key rotation is supported operator-side (versioned sealing keys), and saving a key on a deployment without a sealing key configured fails loudly (503) rather than storing anything readable.
• Never logged, never in telemetry, never in the audit ledger — the ledger records the model id of each answer, not provider credentials.

What still runs on platform models

Your workspace model governs answer synthesis. Background intelligence still runs on AIVM-managed models: document digestion at upload, onboarding genesis, domain classification for access control, and evaluation judging. On a deployment with no platform key at all, each degrades safely rather than fabricating — digestion keeps the raw content, genesis refuses, classification falls back to rules. Routing these through your BYO provider is on the roadmap.

Embeddings are configured per deployment (not per workspace): AIVM_EMBED_PROVIDER supports OpenAI-compatible and Ollama embedding endpoints with AIVM_EMBED_MODEL, AIVM_EMBED_DIM, AIVM_EMBED_BASE_URL, and AIVM_EMBED_API_KEY. Changing the embedding model changes the vector space — it’s a re-index decision for the operator, which is why it isn’t a workspace toggle.

Workspace admins: configure all of this under Settings → Models & providers.