Autonomous AI crawler
Needs changesNeeds changes. This template uses 1 thing(s) that n8n 3.0 removes: AI Agent v1 (old agent modes). 1 breaking item(s) found. Static check, last run 2026-10-19.
What this workflow does
This AI workflow has 28 nodes and runs manually, when you click “Execute workflow”. Along the way it connects OpenAI Chat Model and Supabase.
How it runs
Trigger: Manually, when you click “Execute workflow”.
- Get companies (Supabase): get many
- Select company name and website (Edit Fields (Set)): sets or maps fields
- Crawl website (AI Agent): runs an AI agent, with model: OpenAI Chat Model; 2 tools (Text, URLs); output parser: JSON Parser
- Map company name and website (Edit Fields (Set)): sets 2 fields: company_name, company_website
- Set social media array (Edit Fields (Set)): sets 1 field: social_media
- Merge: merges the data from its input branches
- Insert new row (Supabase): uses Supabase
- Set domain (text) (Edit Fields (Set)): sets 1 field: domain
- Add protocool to domain (text) (Edit Fields (Set)): sets 1 field: domain
- Get website (text) (HTTP Request): calls an HTTP API
…plus 13 more steps. See the full node list below.
Services it uses
Before you upgrade to n8n 3.0
1 thing in this workflow is removed in n8n 3.0. Fix it before you upgrade:
- AI Agent v1 (old agent modes) in Crawl website (v1.6): replace with Latest AI Agent node (Tools Agent). For SQL Agent, use a Postgres or MySQL tool with a recent AI Agent.
Rather not rewrite it? Many of these flows can be rebuilt in Make (free plan).
| Removed in 3.0 | Replace with |
|---|---|
| AI Agent v1 (old agent modes) Crawl website (v1.6) | Latest AI Agent node (Tools Agent). For SQL Agent, use a Postgres or MySQL tool with a recent AI Agent. |
Nodes used (15)
- Call n8n Workflow Tool
@n8n/n8n-nodes-langchain.toolWorkflow - OpenAI Chat Model
@n8n/n8n-nodes-langchain.lmChatOpenAi - Structured Output Parser
@n8n/n8n-nodes-langchain.outputParserStructured - Edit Fields (Set)
n8n-nodes-base.set - Manual Trigger
n8n-nodes-base.manualTrigger - Supabase
n8n-nodes-base.supabase - Merge
n8n-nodes-base.merge - Markdown
n8n-nodes-base.markdown - HTML
n8n-nodes-base.html - Split Out
n8n-nodes-base.splitOut - Remove Duplicates
n8n-nodes-base.removeDuplicates - Filter
n8n-nodes-base.filter - Aggregate
n8n-nodes-base.aggregate - HTTP Request
n8n-nodes-base.httpRequest - AI Agent
@n8n/n8n-nodes-langchain.agent
From the author
This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: - text tool: to retrieve all the text from the page, - URLs tool:…