The Scenario
You are a developer trying to understand how a competitor's product search feature works under the hood. Their search results page makes a series of API calls you cannot see from the browser source — the data loads dynamically and the network tab shows a handful of XHR requests that look like the real API.
You want a structured list of every network request the page makes: URL, method, status code, and a preview of the response. Your plan is to use that list to reverse-engineer their data endpoints and understand what they are querying.
You have tried a few approaches. Browser DevTools recordings do not export cleanly. HAR file parsing is its own project. Capturing this in an automated, repeatable way is exactly the kind of thing that sounds like an afternoon of tooling work.
The bad version:
- Open the competitor's page in Chrome DevTools, go to the Network tab, interact with the search, then manually scroll through 80 or 120 captured requests trying to identify the relevant XHR calls — most of them are tracking pixels and analytics pings
- Try to export as a HAR file, write a script to parse it, realize the JSON structure varies by Chrome version, spend 45 minutes on the parser alone
- Eventually produce a list of endpoints, but it is tied to one session, one search query, and you have no easy way to re-run it systematically across different queries or competitor pages
The research question is a business priority. The tooling problem it requires is not something you want to sink a full day into.
The Easy Way: One Prompt in SheetXAI
SheetXAI is an AI agent that lives inside your Excel workbook. It reads the URL you specify and through its built-in ScrapingAnt integration it runs an extended scrape that captures all XHR and Fetch requests made by the page during rendering — then writes them into your workbook as structured rows.
Open the SheetXAI sidebar and ask:
Scrape the URL in cell A1 using ScrapingAnt's extended JSON output mode and write all captured XHR/Fetch requests into my sheet — one row per request with columns: Request URL, Method, Status Code, Response Preview
SheetXAI sends the URL through ScrapingAnt's extended scrape mode, which instruments the headless browser to log all network activity during page load and interaction. Every XHR and Fetch request appears as its own row. Request URL, method, status code, and a truncated response preview fill in across the four columns.
What You Get
- One row per captured XHR or Fetch request
- Column A: full request URL
- Column B: HTTP method (GET, POST, etc.)
- Column C: HTTP status code
- Column D: first 200 characters of the response body as a preview
Tracking pixels and static asset requests (images, CSS, fonts) are filtered out so you see only the data-fetching network calls.
What If the Data Is Not Quite Ready
The relevant API calls only fire after you interact with the search — not on initial page load
Scrape the URL in cell A1 with ScrapingAnt extended mode, simulate typing a search query into the main search input before capturing network requests, then write all XHR/Fetch calls that fire after the interaction into my sheet
You want to compare network calls across three different competitor pages
For each URL in column A (rows 1 through 3), run a ScrapingAnt extended scrape, capture all XHR/Fetch requests, and write them into separate sheets named after the domain — one sheet per competitor URL
Some responses are gzipped and the preview is unreadable
For each XHR request captured from the URL in cell A1, write the request URL into column A and method into column B; for the response preview in column D, attempt JSON parsing and write the first top-level key-value pairs if parseable, otherwise write "Binary or compressed — inspect manually"
Full reverse-engineering session in one pass: capture, filter, classify, and prioritize
Scrape the URL in cell A1 with ScrapingAnt extended JSON mode; capture all network requests; filter to only XHR and Fetch calls; write Request URL, Method, Status Code, and Response Preview into columns A through D; in column E classify each request as "Data API," "Auth," "Analytics," or "Other" based on the URL pattern and response shape; sort by column E so Data API calls appear first
The classification step saves the manual review work of sorting through 80 requests looking for the three that matter.
Try It
If you have a competitor page you are trying to reverse-engineer — Get the 7-day free trial of SheetXAI and put the URL in cell A1 of an Excel workbook. Ask it to run a ScrapingAnt extended scrape and log every XHR request into your sheet. Related: the spoke on scraping SaaS pricing pages, and the full ScrapingAnt hub for more use cases.