Crawler agent that saves company social links to Supabase

Crawler agent that saves company social links to Supabase

An n8n workflow that reads company names and websites from a Supabase table, then sends an AI agent to each site. The agent
reads the home page, follows the links it finds and returns the company's social media profile links in one JSON shape. A last
step writes the company, its website and those links into a second Supabase table.

What is inside

• workflow.json: the workflow as n8n exports it, twelve nodes that run, sixteen unconnected copies of the tool steps and ten
  sticky notes. It holds no credential.
• README.md: these notes.

What we changed from the original copy

• The credential links on the OpenAI and Supabase nodes were removed. Select your own.
• A sticky note that pointed to the author's video guide and newsletter now holds a short setup note: the credentials to add,
  the two tables to create and a word about the unconnected copies.
• The schema in JSON Parser listed platforms as a required key, and no property has that name. It now lists social_media,
  the one property the schema defines.
• The model stays gpt-4o. See "Good to know" for its shutdown date.
• Nothing else. The tools, the prompt and the table names are as they were.

Before you start

• An n8n instance, on n8n Cloud or your own server.
• An OpenAI API key that can use gpt-4o.
• A Supabase project with two tables:
• companies_input with the text columns name and website, one row per company.
• companies_output with the text columns company_name and company_website and a jsonb column social_media.

Set it up

1. In Supabase, create the two tables above and add a few companies to companies_input. Start with two or three rows.
2. In n8n, create a workflow, open its menu and choose Import from File. Pick workflow.json.
3. Add an OpenAI credential and select it in OpenAI Chat Model.
4. Add a Supabase credential with your project's host and service role key. Select it in Get companies and Insert new row.
5. Click Test workflow. The agent opens each company's site, and the rows appear in companies_output.
6. Open companies_output. Each row holds the company name, its website and social_media, a list of platforms with their URLs.

How the two tools work

The agent calls two tools. Each is a Tool Workflow node with its workflow stored inside the node, so nothing else has to be saved
or linked. Each one takes a single query field, the full website address.

• Text (text_retrieval_tool) adds http:// to an address that has no protocol, fetches the page, turns the HTML into
  Markdown without its links and images, and returns the text.
• URLs (url_retrieval_tool) fetches the page and reads the attributes of every a tag. It drops tags without an href,
  removes repeats, puts the site's address in front of any link that starts with /, keeps only valid URLs and returns the
  list of titles and links as text.

The sixteen unconnected nodes on the canvas are drawn copies of these steps, shown so you can see them. The tools run what is
stored inside their own nodes. If you change a step, change it in the tool's workflow, not on the canvas.

How a run goes

1. Execute workflow starts it. Get companies reads rows from companies_input, and Select company name and website keeps
   name and website.
2. Each company goes two ways. Map company name and website renames the fields to company_name and company_website. `Crawl
   website` (version 1.6, a Tools Agent) is told to retrieve the social media profile URLs of the site.
3. The agent uses Text and URLs as often as it decides, then answers in the shape JSON Parser defines: a social_media list
   of objects with a platform and a list of urls.
4. Set social media array copies that list into a field of its own. Merge all data joins it to the mapped name and website,
   item by item in order, and Insert new row adds the result to companies_output.

Change the look

• What it looks for: the system message and the prompt in Crawl website, and the schema in JSON Parser. Edit both together.
• The model: OpenAI Chat Model.
• The tables: Get companies and Insert new row. Any database n8n has a node for can take their place.
• The fields read from the input: Select company name and website.

Good to know

• Model: OpenAI's deprecations page, read on 12 November 2026, lists gpt-4o-2024-08-06 and gpt-4o-2024-11-20 with a shutdown
  on 14 January 2027 and names gpt-6-luna as the replacement. The page does not list the name gpt-4o itself. Change the model
  in OpenAI Chat Model before that day.
• Get companies has no Return All switch turned on, so it reads the node's default limit of 50 rows. Turn Return All on, or
  work in batches, for a longer list.
• One failing site stops the whole run, because no step handles an error. Merge all data pairs items by position, so keep the
  order of the companies as it is.
• The tools make a plain request and read the HTML that comes back. A page built by JavaScript, or one that blocks automated
  requests, gives the agent little or nothing. A link that does not start with / and is not a full URL is dropped.
• The agent fetches whatever address it builds, including ones inside your network, and a page can carry text meant to steer the
  model. Crawl public company sites you are allowed to read, follow each site's rules and keep the n8n instance away from private
  addresses.
• Each company costs several OpenAI calls, since the agent decides how many pages to open.
• The page text goes to OpenAI, and your table rows stay in your Supabase project.