Chunk

Datapulledcore.data.chunk

Split text into overlapping chunks to embed and store. The output is a list; feed it through For-each.

Pulled. It runs when a node that fires reads its output, once per turn.

Inputs

Port	Type	Notes
text	text

Outputs

Port	Type	Notes
out	list

Knobs

Right-click a knob and pick Convert to input to drive it from a wire.

Knob	Kind	Default	Notes
size size	number	800	50 to 8000, step 50
overlap overlap	number	120	0 to 2000, step 10
by	choice	chars	chars, words

Example

Splits a document before embedding it. With size 800 and overlap 120 by characters, each chunk starts 680 characters after the one before, so a sentence cut at one edge is whole in the next chunk. Wire out into For-each and embed one chunk at a time, as the RAG guide does.

More in Data

• Build JSON: Build a JSON object from wired inputs: each input’s port name becomes a key holding that input’s value.
• Compute: Derive a value from an expression over the wired inputs.
• Format List: Turn a json list into text: render each item with a {field} template and join with a separator (one per line by default).
• Get: Read the value at a dotted or bracket path (user.name, items.0.id).
• List: Assemble wired values into one list (the reverse of For-each).
• Parse: Parse JSON text into a structured value.
• Sentences: Split text into a list of sentences (each ends on . ! ? … or a new line).
• Split JSON: Split a json object onto one output per configured key (the reverse of Build JSON): each key routes its value.
• Stringify: Turn a value into JSON text, indented two spaces.
• Tag Parse: Read inline [bracket] tags in text: emit a mood (the highest-priority mood tag), the display text with its tags shown as emoji, cut short or removed, and an action.
• Template: Assemble text from {tag} pipes.