Models

The model nodes (LLM, TTS, STT, Embed and Rerank) do not hard-code any model. Pick a model on a node and the node reshapes to it: its inputs, its outputs and its settings follow what that model can do.

The model list is read live

Boltjar asks every provider it can use which models it offers now:

Ollama, whenever it answers: the chat and embedding models installed there, with what each one can do (tools, images, thinking) and its context size.
xAI, Anthropic and OpenAI, once their key is in Settings › AI Providers: the chat models the key reaches.
Every OpenAI-compatible endpoint you add in Settings › AI Providers (below).

The list refreshes:

in the background when the server starts, so a start never waits on a provider;
when you add or remove a key or an endpoint, and when you pull or remove an Ollama model in Settings › AI Providers;
when you press Refresh at the foot of a model picker;
when the editor reads a list that went stale, as it loads or when you open a model picker: Ollama’s list goes stale after 30 seconds and every other provider’s after 3 hours, so a model you pulled in a terminal joins the list when you next open a picker;
before validation reports a model as missing: its provider is asked again, at most once every 30 seconds, and validation waits up to 10 seconds for the answer.

Nothing refreshes on a timer. A provider that cannot be reached keeps the last list it gave, so the pickers work offline, and the list is kept in user/data/models-cache.json between starts.

The TTS, STT and Rerank pickers list the models that have a manifest (below): the ones that ship with Boltjar and the ones you add. Those lists are not read from the providers.

OpenAI-compatible endpoints

OpenRouter, Groq, LM Studio, llama.cpp, vLLM and many other servers speak OpenAI’s API. To use one, open Settings › AI Providers, stay on AI Providers, and press Add endpoint on the OpenAI-compatible card:

Field	What to type
Name	A short lowercase id, such as `openrouter`. Its models show up as `openrouter/<model>`.
Base URL	The address with its version path, such as `https://openrouter.ai/api/v1` or `http://localhost:1234/v1`. A bare host gets `/v1`.
API key	The server’s key, or nothing for a local server that takes none. It is saved as a secret (`OPENROUTER_API_KEY` for `openrouter`), never in the endpoint list.

Its models join the list once you save it. A server that reports only model names gives text in and out with no tool calls; one that also reports what each model takes, as OpenRouter does, gets images, tools and JSON output where a model supports them. A manifest can add what a server does not report.

No model is picked for you

Boltjar never chooses a model on its own, because a model can cost money. A model node starts with nothing picked: its picker says select a model, and when no provider is connected yet it offers Add a connection, which opens Settings › AI Providers.

An LLM with nothing picked answers with the offline mock (free, no network), so a graph runs before you connect anything. Every other model node (TTS, STT, Embed, Rerank) needs a model before the graph can turn On. A cloud model whose key is missing says which key to add.

The model you pick is saved in the workflow file. Share that file and it keeps the model’s name, never your key: on another computer it runs on that person’s key, or asks them for one.

What a manifest adds

A manifest is a small TOML file that describes one model: who serves it, what it takes in, what it can do, and which settings it has. A listed model with a manifest takes the manifest’s label, settings and capabilities; a listed model without one gets its settings from what the provider reports. A manifest for a model its provider no longer lists stays in the picker, marked unavailable, with the reason.

id = "ollama/llama3.1:8b"          # unique; the picker stores this
provider = "ollama"                # who serves it
model = "llama3.1:8b"              # the name the provider knows it by
label = "Llama 3.1 8B (Ollama)"
summary = "Local 8B chat model with tools. Needs Ollama."
kind = "llm"                       # llm, tts, stt, embed or rerank
context = 131072
inputs = ["text"]                  # add "image" for a model that reads images
outputs = ["text"]
tools = true                       # can call Tool nodes
thinking = false                   # has a thinking trace
json = true                        # can be held to JSON output

[[params]]
name = "temperature"
type = "float"                     # float, int, text, bool or select
default = 0.8
min = 0.0
max = 2.0
step = 0.05

[[params]]
name = "num_ctx"
type = "int"
default = 8192
min = 256
max = 131072

What the node does with a model’s description, from a manifest or from its provider:

inputs grows ports on the LLM: an image input for a model that reads images.
[[params]] become knobs on the node, and like every knob each one can be converted to an input. A select param lists its options.
tools lets the LLM offer wired Tool nodes to the model.
thinking adds a thinking knob, shaped by thinking_style: effort (none, low, medium, high), level (off or adaptive), budget (a token count) or bool. The trace comes out of the LLM’s reasoning output.
json adds a JSON output toggle.

A param only reaches the provider when Boltjar sends that setting to it. The Ollama call sends temperature, top_p, top_k, num_ctx, seed and keep_alive. xAI, OpenAI and the OpenAI-compatible endpoints get temperature, top_p, max_completion_tokens, max_tokens and seed. Anthropic gets max_tokens, temperature, top_p (only when temperature is not set), top_k and stop_sequences. For TTS and STT, the params are the provider’s own settings, such as the voice.

Providers

Provider	Serves	Needs
`ollama`	chat models and embeddings on your machine, listed live	Ollama running, the model pulled
`xai`	Grok chat models (listed live), voices, transcription	`XAI_API_KEY`
`anthropic`	Claude models, listed live	`ANTHROPIC_API_KEY`
`openai`	OpenAI chat models, listed live	`OPENAI_API_KEY`
an endpoint’s name	the models of an OpenAI-compatible server, listed live	the endpoint, added in Settings › AI Providers
`google`	Gemini chat models, through a manifest you add (none ships, and the list is not read live)	`GOOGLE_API_KEY`
`fish`	voices and transcription	`FISH_API_KEY`
`elevenlabs`	voices and transcription	`ELEVENLABS_API_KEY`
`http`	reranking, through a service you run	the service’s address, in the Rerank node’s `endpoint` setting

A cloud chat model without its key answers with a mock reply, so a graph still runs; a voice model without its key fails with an error that names the key.

Where manifests come from

Manifests load when the server starts, from three places, in this order:

boltjar/nodes/core/models/: the ones that ship. Each model node’s page lists them.
packs/<pack>/models/: models a node pack brings. A pack cannot replace a model that is already declared.
user/models/: yours. They load last, and one with the same id as a shipped manifest replaces it.

A manifest with a mistake is skipped with a warning in the server log; the rest still load.

Add a model

A model your providers already list needs nothing: it is in the picker. Pull an Ollama model (ollama pull llama3.1:8b in a terminal, or from the Ollama card in Settings › AI Providers) and it joins the list on its own; Refresh in a picker brings it in at once.

Write a manifest to give a model a label, its settings or a capability its provider does not report, or to add a model no list covers, such as a voice:

Copy the closest manifest from boltjar/nodes/core/models/ into user/models/.
Change id, model, label and the capabilities to match the new model. For an Ollama model, model is the exact name Ollama lists.
Restart Boltjar. The model shows up in the picker of the matching node.