RAG with Vectors

Runs locally

Needs Ollama with two models: bge-m3 for the embeddings and gemma4:12b for the answers. No key; the handbook never leaves your machine.

A small online shop keeps its rules in a team handbook: leave, expenses, on-call, returns. Nobody reads 1,500 words to find out whether unused leave carries over, so this graph lets the team ask the handbook instead, and the answer quotes the rule it used.

It is retrieval-augmented generation in two branches of one graph. The first splits the handbook into chunks, embeds each one and stores it. The second embeds your question, finds the closest chunks, and hands them to the model with the question.

Download the graph and save it in user/graphs/, or build it below. To try it on the same handbook as this page, download the sample handbook too: a made-up shop’s rules, about 1,500 words. The graph is MIT-0: copy it and license what you build from it however you like.

1. Get ready

ollama pull bge-m3
ollama pull gemma4:12b

Save the handbook as user/data/files/handbook.md: the sample above, or your own team’s, in text or Markdown. That folder is the only one the file nodes read.

The downloaded graph picks no model, because a model can cost money and Boltjar never picks one for you. It stays Off until both Embed nodes have one, and its problems list says pick a model for Embed for each. Pick bge-m3 (Ollama) on Embed chunk and on Embed question, and a chat model such as Gemma 4 12b (Ollama) on the LLM; without that one, the LLM answers with a mock reply.

2. The indexing branch

Node	Settings
Manual, named Index	none
Read File, named Handbook	path handbook.md
Chunk, named Chunks	size 800, overlap 120, by chars
Vector Store	none
Vectors, named Clear	operation clear, namespace handbook
For-each, named Each chunk	none
Embed, named Embed chunk	model: pick bge-m3 (Ollama)
Vectors, named Add	operation index, namespace handbook

From	To
Index trigger	Clear trigger
Vector Store vectors	Clear vectors
Clear trigger	Each chunk trigger
Handbook text	Chunks text
Chunks out	Each chunk list
Each chunk each	Embed chunk trigger
Each chunk item	Embed chunk text
Embed chunk trigger	Add trigger
Vector Store vectors	Add vectors
Embed chunk embedding	Add embedding
Each chunk item	Add text
Add trigger	Each chunk loop

Read it as a loop: Clear empties the handbook namespace, then For-each hands out the chunks one at a time. Each chunk is embedded and added, and Add’s trigger goes back into loop to release the next chunk. Clearing first means a second run replaces the handbook instead of adding it twice, so an edited rule never answers next to its old wording.

1. Clear empties the handbook namespace first, then fires Each chunk.
2. The Vector Store: the one store that Clear, Add and Excerpts all use.
3. The chunks go into Each chunk’s list.

The Vector Store’s wire to Clear crosses the one from Handbook to Chunks, because Clear takes its trigger from Index above and its store from below. The wire along the top is the loop’s wire back, shown next.

1. item: one chunk at a time, into Embed chunk and into Add’s text.
2. Embed chunk turns the chunk into a vector for Add.
3. The wire back: Add’s trigger into loop releases the next chunk. It closes the loop, so it crosses the wires that go out.

The Vector Store’s wire comes into Add from below and crosses the embedding and text wires, because Add takes its store between its trigger and its embedding.

Clear takes no embedding: a Vectors node shows its embedding input only for search and index.

3. The question branch

Node	Settings
Chat Input, named Question	placeholder Ask the handbook...
Embed, named Embed question	model: pick bge-m3 (Ollama)
Vectors, named Excerpts	operation search, namespace handbook, top k 4, min score 0.3
Template, named Prompt	the text below
LLM	model: pick Gemma 4 12b (Ollama), or another model you pulled
Chat	none

Answer from the handbook excerpts below in plain text, with no markdown. Write two parts: a one-sentence answer, then the rule you used, quoted on its own line starting with "Handbook:". If they do not cover it, say so.

Excerpts:
{excerpts}

Question: {question}

Asking for the quote keeps the answer honest: you can see which rule it came from, and an answer the handbook does not support has nothing to quote. “No markdown” matters because the Chat node shows the reply as plain text, so **bold** would show its asterisks.

From	To
Question trigger	Embed question trigger
Question text	Embed question text
Embed question trigger	Excerpts trigger
Vector Store vectors	Excerpts vectors
Embed question embedding	Excerpts embedding
Excerpts trigger	Prompt trigger
Excerpts results	Prompt excerpts
Question text	Prompt question
Prompt trigger	LLM trigger
Prompt out	LLM prompt

Plus the four Chat wires from the first graph: Question into user and user_trigger, the LLM into reply and reply_trigger.

The Prompt’s tags are named after the nodes their wires come from: Excerpts results becomes {excerpts}, Question text becomes {question}. The question fires each node once down the line, through the Prompt to the LLM, so one question runs the model once.

The question branch in two halves. First the search:

1. Embed question turns the question into a vector.
2. Excerpts’ vectors takes the Vector Store from the indexing branch, left out of this picture.

Wires cross here in two places: the Vector Store’s wire crosses Embed question’s trigger, because Excerpts takes its store between its trigger and its embedding, and Question text rises to Embed question and to the Prompt past Question trigger on its way to the Chat.

Then the answer:

1. The closest chunks land on the Prompt’s excerpts.
2. The question itself lands on question, on the wire from Question text.

4. Run it

Once the models are picked, press On. The Manual trigger fires as the graph turns On, so indexing starts right away; press Fire on the Index node to run it again after you edit handbook.md. Then ask a question in the Question node, for example Can I carry unused leave into next year?

1. The question, and the answer the model gave from the handbook’s excerpts: one sentence, then the rule it used after Handbook:.

Each hit reaches the prompt with its text and its score. If the answer says the handbook does not cover something it does, check that Ollama was running when the Index node fired: when Ollama does not answer, Embed gives an empty vector and the search finds nothing, so press Fire on the Index node again once it runs. A bge-m3 that is not pulled keeps the graph Off instead, and its problems list says to pull the model in Settings › AI Providers.

Change it

• More or fewer hits. Raise top k for broader answers, or min score to keep only close matches.
• Many files. Put a List Dir and a second For-each in front of Read File to index a folder.
• A sharper order. Put a Rerank between Excerpts and the Prompt. It needs a rerank model picked on it and a rerank service running over HTTP; Boltjar does not bundle one.
• Remember the source. Set Add’s metadata to ref = handbook.md, and delete with that ref removes the file’s chunks later.