Three native tools that chain together in any multi-step job. Output from one step pipes directly to the next via published properties.
1. Document Chunker
Split documents into right-sized pieces for embedding. Embedding models have token limits (~8K) — a 50-page document won't fit. The chunker creates focused pieces so vector search returns precise matches, not entire documents.
6 strategies: tokens (with overlap), sentences, paragraphs, lines, pages, custom regex. Configurable chunk size, overlap, metadata, and max chunks.
Formats: text, markdown, HTML, CSV, JSON.
2. Embeddings
Convert text into dense numerical vectors that capture semantic meaning. Texts with similar meaning produce vectors close together in vector space — "cancel my subscription" matches "stop my monthly plan" even though no words overlap.
Models: text-embedding-3-small (fast), text-embedding-3-large (accurate, cross-language). Compatible with any OpenAI-compatible API.
Alternative: Hugging Face feature-extraction for free embeddings.
3. Vector Store
Store, query, and manage vectors across 7 providers. Upsert embeddings with metadata, query by semantic similarity with top-K retrieval and metadata filtering, delete by ID or filter.
Operations: upsert, query, delete, list_indexes.
The local provider uses JSON files + cosine similarity. Zero dependencies. The entire RAG pipeline runs on a single machine with nothing installed beyond Java.
How It Chains
Ingestion: Document Chunker → Embeddings → Vector Store (upsert). Attach to a trigger file and new documents are automatically indexed.
Retrieval: Embeddings (query) → Vector Store (query) → Any LLM (Claude, GPT, Gemini, Mistral, Groq, DeepSeek, xAI, Ollama) generates an answer grounded in your actual documents.