The Scenario
An SEO consultant has just kicked off an engagement with a B2B software company preparing for a platform migration. Before anything else can happen — redirect mapping, content decisions, canonical structure — the consultant needs to know what pages actually exist. The client's team forwarded a sitemap XML file that was last updated eight months ago and visibly missing entire product sections. The consultant needs a crawl-based URL inventory in an Excel workbook, today, so the technical audit can begin.
The bad version:
- Download Screaming Frog, set it up, configure the crawl settings, wait for the crawl to finish.
- Export to CSV, open in Excel, clean up the columns, remove the metadata fields the client does not need.
- Discover the crawl stopped at 500 URLs because the free tier has a limit. Upgrade. Re-crawl. Wait again.
The first deliverable is due at the end of the week. The crawl setup is supposed to take an hour, not a day.
The Easy Way: One Prompt in SheetXAI
SheetXAI is an AI agent that lives inside your Excel workbook. Specify the domain and SheetXAI will use its built-in Supadata crawling integration to discover all URLs on the site and write them directly into the workbook — no separate crawler tool, no CSV export.
Crawl the website in cell A1 using Supadata's web map tool and write all discovered page URLs into column A of this Excel sheet, starting at row 2
What You Get
- Column A populated with every URL Supadata discovered on the site, starting at row 2.
- The full inventory written directly into the workbook — no intermediate export step.
- Data ready for the next audit phase: redirect mapping, content categorisation, or gap analysis.
- Cell A1 preserved for the domain reference, making it easy to re-run the crawl for a different site.
What If the Data Is Not Quite Ready
The site has multiple subdomains that each need their own inventory
The client operates a marketing site, a docs site, and a help centre on separate subdomains.
Crawl each domain listed in column A using Supadata and write the discovered URLs for each domain into a separate worksheet named after the domain — write the URL count for each in row 1 of each sheet
You want to filter the inventory to only URLs with certain path patterns
The redirect mapping only needs to cover the /product/ and /features/ sections.
Crawl https://example.com using Supadata, write all discovered URLs into column A, then filter the list to keep only URLs containing '/product/' or '/features/' and write the filtered set into column B
You need to compare the crawl results against the client's existing sitemap
The client provided 180 URLs they believe are live. The crawl may reveal gaps or additions.
Crawl https://example.com using Supadata and write all discovered URLs into column A — then compare against the URLs already in column C and write 'New - not in sitemap' in column B for any URL found by the crawl that is not in column C, and 'In sitemap only' in column D for any URL in column C not found by the crawl
You want the URL inventory plus a structural breakdown in one pass
Crawl https://example.com using Supadata, write all URLs into column A, then write the total URL count in cell C1, the count of URLs containing '/blog/' in C2, the count containing '/product/' in C3, and the count of root-level pages in C4
The pattern: building the structural summary into the same crawl step means the audit kickoff deck is already started before you have left the spreadsheet.
Try It
Get the 7-day free trial of SheetXAI and open an Excel workbook, type a client domain into cell A1, and ask it to crawl and map the site in one operation. Once you have the URL inventory, look at the spoke on scraping competitor pricing pages to pull content from the pages that matter most.