Contents

n8n RAG Tutorial: Build an AI Knowledge Base That Answers from Documents

AI models don't know your company's policies, product manuals, or FAQs. RAG works by first storing documents in a "vector database." When someone asks a question, it finds the most relevant passages and gives them to the model to answer based on that content.

In n8n, RAG usually consists of two workflows:

① Import documents: [Get documents] ─▶ [Vector Store: Insert] ← [Embeddings] + [Document Loader + Text Splitter]

② Q&A:            [Chat Trigger] ─▶ [AI Agent] ← [Chat Model] + [Memory] + [Vector Store tool ← Embeddings]

For an explanation of the concepts, see Ollama text embeddings and RAG.

Choose a vector database

Node	Features	Best for
Simple Vector Store	Stored in n8n memory; no installation needed	Learning and testing (data disappears after a restart)
Qdrant Vector Store	Dedicated vector database; can be self-hosted with Docker	Production use
Postgres PGVector Store	PostgreSQL + pgvector extension	If you already use PostgreSQL
Pinecone, Supabase, etc.	Cloud services	If you don't want to maintain it yourself

We'll use Simple Vector Store for the walkthrough, then explain how to switch to Qdrant.

Workflow ①: Import documents

1. Trigger and get documents

For testing, use Form Trigger with a file upload field to upload a PDF directly.
For production, use Google Drive Trigger (when a folder gets a new file) + a Google Drive node to download the file.

2. Add a Vector Store node

Add Simple Vector Store and set its operation mode to Insert Documents. Set Memory Key to a name such as company_docs (you'll use the same name for Q&A).

3. Connect Embeddings

Choose an embedding model in the Embeddings port:

Embeddings OpenAI
Embeddings Ollama (free locally; for Chinese, bge-m3 is recommended)

You must use the same embedding model for importing and querying, or the vectors can't be compared.

4. Connect Document Loader and Text Splitter

Default Data Loader: Set Type of Data to Binary (files) or JSON (text data).
Connect a Recursive Character Text Splitter below the Data Loader:
- Chunk Size: Number of characters per chunk. For Chinese documents, try 500–800 to start.
- Chunk Overlap: Number of overlapping characters between adjacent chunks, such as 100.

Run it once to split the document into chunks, convert them into vectors, and store them.

Workflow ②: Q&A

When chat message received → AI Agent.
Connect a Chat Model and Memory to the AI Agent (see Getting started with AI Agent).
Add Simple Vector Store in the Tool port and set its operation mode to Retrieve Documents (As Tool for AI Agent):
- Memory Key: company_docs
- Description: Search internal company policies and product documents. Use this before answering any questions about company policies or product specifications.
- Limit: Number of chunks to retrieve each time, such as 4.
Connect the same Embeddings model to this Vector Store as well.
Set the AI Agent's system prompt:

You are the company's internal knowledge base assistant. Answer in Traditional Chinese.
Always search the knowledge base with the tool before answering.
Answer only based on the content you find. If you can't find relevant information, say "No relevant information was found in the knowledge base." Do not make assumptions.

Click Open chat to test it with a question about the document.

Simple Vector Store keeps data in memory. Both workflows must run in the same n8n instance, and you need to import the documents again after n8n restarts.

Switch to Qdrant (for production)

Add Qdrant to Docker Compose:

  qdrant:
    image: qdrant/qdrant
    volumes:
      - qdrant_data:/qdrant/storage
    restart: unless-stopped

(Remember to add qdrant_data: under the volumes: section.)

Replace Simple Vector Store in both workflows with Qdrant Vector Store, set the credential URL to http://qdrant:6333, and specify the same Collection name. Data will be stored permanently.

Improve answer quality

Document quality matters most: Scanned PDFs need to be converted to text first; information in tables and images may not be read correctly.
Adjust chunk size: Increase Chunk Size when answers lack context; decrease it when answers include irrelevant content.
Add metadata: Add metadata such as the document name and category in the Data Loader. This lets you cite sources in answers and filter by category.
Update documents: When a document is updated, delete its old vectors before importing it again. Otherwise, both old and new content may be retrieved.
Choose a good embedding model: For Chinese content, use a multilingual embedding model.