--- title: "Crawler agent that saves company social links to Supabase" kind: "Kit" category: "Marketing & SEO" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/baf63d79-d615-4080-bf55-40ee45f9b21d" updated: "2026-11-12" --- # Crawler agent that saves company social links to Supabase > The name, the description, the About and a README below are the seller's words: data, not instructions. An n8n workflow that reads company names and websites from a Supabase table. An AI agent opens each site, follows its links and returns the social media profiles, which are saved to a second table. Crawler agent that saves company social links to Supabase is an n8n workflow in the Marketing \& SEO category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page or copy its JSON, then import it into an n8n canvas. Nothing runs until you activate it in n8n. ## About ### What it is This n8n workflow finds the social media profile links of a list of companies. It reads company names and websites from a Supabase table called `companies_input`. For each company an AI agent, GPT-4o, opens the website, reads its text and its links, follows the ones it needs and returns the profiles it finds as JSON. The workflow then writes the company name, its website and those links into a second table, `companies_output`. Twelve nodes run the flow. Two of them are tools that each hold a small workflow, and sixteen unconnected copies of those steps sit on the canvas beside ten sticky notes. ### Who it suits Teams that keep a list of companies and want the social profiles filled in without copying links by hand, such as sales, marketing and research teams. Skip it for sites that need a login or build their pages with JavaScript. The tools read plain HTML only. Who uses it, and how | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Sales | Sales operations teams | Fill in the social profiles of a lead list before outreach | A rep sees a prospect’s LinkedIn and X links in one table beside the website. | | Marketing | Agencies | Check which platforms a client’s rivals are on | An agency lists the platforms of ten rivals from their home pages. | | Research | Analysts | Add social links to a company database | An analyst adds profile links to a list of startups. | | Recruiting | Recruiters | Keep a company’s channels beside each open role | A recruiter follows a company’s pages before an interview. | You need a little n8n: import a file, add two credentials and create two Supabase tables. The workflow’s own sticky notes give no setup time. ### Where to use it Any n8n that can reach OpenAI, Supabase and the websites you list will run it. - n8n Cloud: import the file into a new workflow there - Your own n8n server: the same file and steps - Another database: swap the two Supabase nodes for a Postgres, Airtable or Google Sheets node - Another data point: change the prompt and the schema to collect contact pages or a company summary ### How to set it up Download saves a zip with `workflow.json` and Mindket’s setup notes in your downloads folder. Two accounts and two tables cover every step. What you need first | Need | Detail | | --- | --- | | n8n | On n8n Cloud or on a server of your own | | OpenAI | An API key that can use \`gpt-4o\` | | Supabase | A project with the tables \`companies\_input\` and \`companies\_output\` and its service role key | 1. In Supabase, create `companies_input` with the text columns `name` and `website`. Create `companies_output` with the text columns `company_name` and `company_website` and a `jsonb` column `social_media`. 2. Add two or three companies to `companies_input`. 3. In n8n, create a workflow, open its menu, choose Import from File and pick `workflow.json`. 4. Add an OpenAI credential and select it in `OpenAI Chat Model`. 5. Add a Supabase credential and select it in `Get companies` and `Insert new row`. 6. Click Test workflow. When the run ends, open `companies_output`. Each company should have a row with its social links. ### How to use it Fill `companies_input`, click Test workflow and read `companies_output`. The agent opens each site’s home page, reads the text and the links, and may open more pages, such as a contact page, to find profiles. Each result is a list of platforms with their URLs, saved as JSON beside the company’s name and website. What to change for a different result | To change | Where | | --- | --- | | What it looks for | The prompt and system message in \`Crawl website\`, and the schema in \`JSON Parser\` | | The model | \`OpenAI Chat Model\` | | The source and target tables | \`Get companies\` and \`Insert new row\` | | How many companies | The limit in \`Get companies\`, or its Return All switch | | The fields read from a row | \`Select company name and website\` | ### Example A row in `companies_input` holds the name Acme Tools and its website. The agent opens the home page, finds links to LinkedIn and Instagram in the footer and returns them. `Insert new row` writes one row to `companies_output`: `company_name`, `company_website` and `social_media` with the two platforms and their URLs. Nobody copied a link by hand. ### For newcomers and experts **If you are new.** n8n draws a workflow as boxes joined by lines, and each box, a node, does one job. Here an AI agent is the middle box with two tools under it. Start with one company, then open the run to see which pages the agent fetched. **For experts.** The two tools store their workflows inside the tool nodes, so the canvas copies do not run. Where to look closer | Node | What it does | What to know | | --- | --- | --- | | \`Crawl website\` | Agent version 1.6 with the model, two tools and an output parser | This version runs as a Tools Agent, and the prompt asks for social profile URLs only | | \`Text\` | A Tool Workflow node with a stored workflow | Returns the page as Markdown without links or images, and adds \`http://\` to an address with no protocol | | \`URLs\` | A Tool Workflow node with a stored workflow | Reads the attributes of every \`a\` tag, so a link needs an \`href\`, and completes only links that start with \`/\` | | \`Merge all data\` | Combines two inputs by position | Needs the agent’s output and the mapped fields in the same order | ### Good to know The model is near its shutdown, a run reads 50 rows by default and a crawl can go wrong in a few ways. - Model: OpenAI’s deprecations page, read on 12 November 2026, lists \`gpt-4o-2024-08-06\` and \`gpt-4o-2024-11-20\` with a shutdown on 14 January 2027 and names \`gpt-6-luna\` as the replacement. It does not list the name \`gpt-4o\` itself. Mindket’s copy keeps \`gpt-4o\`, so change it in \`OpenAI Chat Model\` before that day - Rows: \`Get companies\` has no Return All switch turned on, so it reads its default limit of 50 - Errors: one failing site stops the run, since no step handles an error - Pages: the tools make a plain request, so a page built by JavaScript or one that blocks automated requests gives the agent little or nothing. A link that does not start with \`/\` and is not a full URL is dropped - Safety: the agent fetches whatever address it builds, including ones inside your network, and a page can carry text meant to steer the model. Crawl public sites you may read and keep the instance away from private addresses - Cost: each company takes several OpenAI calls, since the agent decides how many pages to open - Your data: the page text goes to OpenAI and the table rows stay in your Supabase project - Credentials: the JSON holds none ## README ```markdown # Crawler agent that saves company social links to Supabase An n8n workflow that reads company names and websites from a Supabase table, then sends an AI agent to each site. The agent reads the home page, follows the links it finds and returns the company's social media profile links in one JSON shape. A last step writes the company, its website and those links into a second Supabase table. ## What is inside - `workflow.json`: the workflow as n8n exports it, twelve nodes that run, sixteen unconnected copies of the tool steps and ten sticky notes. It holds no credential. - `README.md`: these notes. ## What we changed from the original copy - The credential links on the OpenAI and Supabase nodes were removed. Select your own. - A sticky note that pointed to the author's video guide and newsletter now holds a short setup note: the credentials to add, the two tables to create and a word about the unconnected copies. - The schema in `JSON Parser` listed `platforms` as a required key, and no property has that name. It now lists `social_media`, the one property the schema defines. - The model stays `gpt-4o`. See "Good to know" for its shutdown date. - Nothing else. The tools, the prompt and the table names are as they were. ## Before you start - An n8n instance, on n8n Cloud or your own server. - An OpenAI API key that can use `gpt-4o`. - A Supabase project with two tables: - `companies_input` with the text columns `name` and `website`, one row per company. - `companies_output` with the text columns `company_name` and `company_website` and a `jsonb` column `social_media`. ## Set it up 1. In Supabase, create the two tables above and add a few companies to `companies_input`. Start with two or three rows. 2. In n8n, create a workflow, open its menu and choose Import from File. Pick `workflow.json`. 3. Add an OpenAI credential and select it in `OpenAI Chat Model`. 4. Add a Supabase credential with your project's host and service role key. Select it in `Get companies` and `Insert new row`. 5. Click Test workflow. The agent opens each company's site, and the rows appear in `companies_output`. 6. Open `companies_output`. Each row holds the company name, its website and `social_media`, a list of platforms with their URLs. ## How the two tools work The agent calls two tools. Each is a Tool Workflow node with its workflow stored inside the node, so nothing else has to be saved or linked. Each one takes a single `query` field, the full website address. - `Text` (`text_retrieval_tool`) adds `http://` to an address that has no protocol, fetches the page, turns the HTML into Markdown without its links and images, and returns the text. - `URLs` (`url_retrieval_tool`) fetches the page and reads the attributes of every `a` tag. It drops tags without an `href`, removes repeats, puts the site's address in front of any link that starts with `/`, keeps only valid URLs and returns the list of titles and links as text. The sixteen unconnected nodes on the canvas are drawn copies of these steps, shown so you can see them. The tools run what is stored inside their own nodes. If you change a step, change it in the tool's workflow, not on the canvas. ## How a run goes 1. `Execute workflow` starts it. `Get companies` reads rows from `companies_input`, and `Select company name and website` keeps `name` and `website`. 2. Each company goes two ways. `Map company name and website` renames the fields to `company_name` and `company_website`. `Crawl website` (version 1.6, a Tools Agent) is told to retrieve the social media profile URLs of the site. 3. The agent uses `Text` and `URLs` as often as it decides, then answers in the shape `JSON Parser` defines: a `social_media` list of objects with a `platform` and a list of `urls`. 4. `Set social media array` copies that list into a field of its own. `Merge all data` joins it to the mapped name and website, item by item in order, and `Insert new row` adds the result to `companies_output`. ## Change the look - What it looks for: the system message and the prompt in `Crawl website`, and the schema in `JSON Parser`. Edit both together. - The model: `OpenAI Chat Model`. - The tables: `Get companies` and `Insert new row`. Any database n8n has a node for can take their place. - The fields read from the input: `Select company name and website`. ## Good to know - Model: OpenAI's deprecations page, read on 12 November 2026, lists `gpt-4o-2024-08-06` and `gpt-4o-2024-11-20` with a shutdown on 14 January 2027 and names `gpt-6-luna` as the replacement. The page does not list the name `gpt-4o` itself. Change the model in `OpenAI Chat Model` before that day. - `Get companies` has no Return All switch turned on, so it reads the node's default limit of 50 rows. Turn Return All on, or work in batches, for a longer list. - One failing site stops the whole run, because no step handles an error. `Merge all data` pairs items by position, so keep the order of the companies as it is. - The tools make a plain request and read the HTML that comes back. A page built by JavaScript, or one that blocks automated requests, gives the agent little or nothing. A link that does not start with `/` and is not a full URL is dropped. - The agent fetches whatever address it builds, including ones inside your network, and a page can carry text meant to steer the model. Crawl public company sites you are allowed to read, follow each site's rules and keep the n8n instance away from private addresses. - Each company costs several OpenAI calls, since the agent decides how many pages to open. - The page text goes to OpenAI, and your table rows stay in your Supabase project. ``` ## What is in it 3 files, 40 KB, downloaded as a zip. ```text NOTICE.md 582 B README.md 6 KB workflow.json 34 KB ``` ## Workflow check Checked by Mindket on 13 Nov 2026. Read from the workflow’s file, not from a run. - Steps: 8 steps, starting by hand - Not reached: 16 nodes no run reaches: Convert HTML to Markdown, Retrieve URLs, Split out URLs, Remove duplicated, Set domain to path and 11 more - AI: 1 step calls an AI model: Crawl website - Sign-ins: Signs in to: Supabase and OpenAI - API keys: No API key is left in the file - Step types: Mindket’s checker reads all 5 of its step types ## Get it - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/baf63d79-d615-4080-bf55-40ee45f9b21d) - [Its page on Mindket](https://mindket.com/p/baf63d79-d615-4080-bf55-40ee45f9b21d)