Evaluation metric example: RAG document relevance
Needs changesNeeds changes. This template uses 1 thing(s) that n8n 3.0 removes: AI Agent v1 (old agent modes). 1 breaking item(s) found. Static check, last run 2026-10-19.
What this workflow does
This AI workflow has 19 nodes and runs manually, when you click “Execute workflow” (plus 2 other triggers). Along the way it connects Evaluation, Google Sheets, Embeddings OpenAI and OpenAI and 1 more.
How it runs
Trigger: Manually, when you click “Execute workflow” (plus 2 other triggers).
- Get dataset (Google Sheets): uses Google Sheets
- AI Agent: runs an AI agent, with model: OpenAI Chat Model; 1 tool (Simple Vector Store)
- Match chat format (Edit Fields (Set)): sets 1 field: chatInput
- Remove Duplicates: removes duplicate items
- Evaluating? (Evaluation): check if evaluating
- Simple Vector Store: stores documents in the Simple vector store, with embeddings: Embeddings OpenAI; document loader: Default Data Loader
- Extract documents (Edit Fields (Set)): sets 1 field: documents
- Calculate doc relevance metric (OpenAI): uses OpenAI
- Set metrics (Evaluation): set metrics
Services it uses
Before you upgrade to n8n 3.0
1 thing in this workflow is removed in n8n 3.0. Fix it before you upgrade:
- AI Agent v1 (old agent modes) in AI Agent (v1.9): replace with Latest AI Agent node (Tools Agent). For SQL Agent, use a Postgres or MySQL tool with a recent AI Agent.
| Removed in 3.0 | Replace with |
|---|---|
| AI Agent v1 (old agent modes) AI Agent (v1.9) | Latest AI Agent node (Tools Agent). For SQL Agent, use a Postgres or MySQL tool with a recent AI Agent. |
Nodes used (15)
- Evaluation Trigger
n8n-nodes-base.evaluationTrigger - Evaluation
n8n-nodes-base.evaluation - Chat Trigger
@n8n/n8n-nodes-langchain.chatTrigger - Edit Fields (Set)
n8n-nodes-base.set - No Operation, do nothing
n8n-nodes-base.noOp - Google Sheets
n8n-nodes-base.googleSheets - Remove Duplicates
n8n-nodes-base.removeDuplicates - Simple Vector Store
@n8n/n8n-nodes-langchain.vectorStoreInMemory - Embeddings OpenAI
@n8n/n8n-nodes-langchain.embeddingsOpenAi - Default Data Loader
@n8n/n8n-nodes-langchain.documentDefaultDataLoader - Recursive Character Text Splitter
@n8n/n8n-nodes-langchain.textSplitterRecursiveCharacterTextSplitter - Manual Trigger
n8n-nodes-base.manualTrigger - OpenAI
@n8n/n8n-nodes-langchain.openAi - AI Agent
@n8n/n8n-nodes-langchain.agent - OpenAI Chat Model
@n8n/n8n-nodes-langchain.lmChatOpenAi
From the author
AI evaluation in n8n This is a template for n8n's evaluation feature. Evaluation is a technique for getting confidence that your AI workflow performs reliably, by running a test dataset containing different inputs through the workflow. By calculating a metric…