How to scrape Google Shopping prices with Python

To scrape Google Shopping prices with Python you need two steps: a search that turns a product name into a Google Shopping product_id, and a product request that returns every seller for that ID with price, shipping and total price. This guide builds both into one script, saves the sellers to CSV, handles the errors you will actually see, and works out what it costs.

We use the CrawlerData API, so there is no browser or proxy to manage. Your script sends JSON and gets JSON back. Everything below was run against the live API on 12 October 2026, and the output excerpts are real (trimmed).

What you need

• Python 3.9 or newer and the requests library (pip install requests).
• A CrawlerData API key. New accounts get 50 free credits. A search costs 1 credit, and a sellers request costs 1 credit per 10 sellers returned (rounded up, minimum 1).
• The key in an environment variable, so it never ends up in your code:

export CRAWLERDATA_API_KEY="YOUR_API_KEY"

Every request is a POST to https://api.crawlerdata.com/v1/jobs with an Authorization: Bearer header and a JSON body that names the endpoint, its parameters and a queue:

{
  "endpoint": "google-shopping-search",
  "params": { "query": "sony wh-1000xm5", "country": "us" },
  "queue": "live"
}

On the live queue the response holds the result. On fast or regular you get a job ID back immediately and fetch the result later. All three queues cost the same. If you leave queue out, the API uses fast, so for a simple script set live explicitly.

Step 1: a request helper with error handling

Most of the work in a scraper is deciding what to retry. The API gives you three kinds of outcome:

The third case is the one to handle well. A job that does not complete comes back failed with a readable error, such as "Google temporarily blocked the request. Please retry.", and "credits": 0. Retry it and you pay only for the attempt that succeeds.

import csv
import os
import sys
import time

import requests

API_URL = "https://api.crawlerdata.com/v1/jobs"
API_KEY = os.environ["CRAWLERDATA_API_KEY"]
HEADERS = {"Authorization": f"Bearer {API_KEY}"}


class CrawlerDataError(Exception):
    pass


def run_job(endpoint, params, retries=2):
    """Run one live job and return its result. Retries failed jobs and 429/5xx."""
    body = {"endpoint": endpoint, "params": params, "queue": "live"}
    error = None
    for attempt in range(retries + 1):
        try:
            resp = requests.post(API_URL, json=body, headers=HEADERS, timeout=200)
        except requests.RequestException as exc:
            error = f"network error: {exc}"
        else:
            if resp.status_code in (400, 401, 402):
                # Bad params, bad key or no credits: retrying will not help.
                raise CrawlerDataError(f"HTTP {resp.status_code}: {resp.json()['error']['message']}")
            if resp.status_code == 429 or resp.status_code >= 500:
                error = f"HTTP {resp.status_code}"
            else:
                resp.raise_for_status()
                data = resp.json()
                if data["job"]["status"] == "completed":
                    return data["result"]
                error = data["job"]["error"]  # failed jobs cost 0 credits
        if attempt < retries:
            time.sleep(5 * 2 ** attempt)
    raise CrawlerDataError(f"{endpoint} failed after {retries + 1} attempts: {error}")

The timeout=200 is deliberate. A live request runs the whole crawl before it answers, so give it room. A short client timeout would abandon jobs that were about to succeed, and an abandoned job can still finish on our side and be charged.

Step 2: search Google Shopping for a product ID

The Google Shopping API endpoint is google-shopping-search. It takes query, country (one of 31 two-letter codes, default us) and language (default en), and returns the products Google Shopping shows for the query.

def search(query, country="us"):
    result = run_job("google-shopping-search", {"query": query, "country": country})
    return result.get("shopping_results", [])

Our search for sony wh-1000xm5 in the US came back in about 4 seconds with 23 results. Here is the first one, trimmed:

{
  "position": 1,
  "product_id": "11726664398501807381",
  "title": "Sony WH-1000XM5 Noise Canceling Wireless Headphones",
  "price": { "value": 198, "currency": "USD", "raw": "$198.00" },
  "old_price": { "value": 287, "currency": "USD", "raw": "$287" },
  "merchant": "Walmart",
  "rating": 4.6,
  "reviews": 15000,
  "delivery": "Free delivery",
  "link": "https://www.google.com/search?ibp=oshop&q=gid:11726664398501807381&prds=gpcid:11726664398501807381,…"
}

The price and merchant on a search result belong to one offer, the one Google chose to show on the card. To see every seller, you need the product_id.

Two things to watch in search results:

• The same model can appear under several product IDs. Our results included at least five listings for the WH-1000XM5 with different IDs, from different sellers and at different prices. Check titles before you pick one.
• Store product_id as a string. IDs in our results were 19 or 20 digits long, and some are larger than a 64-bit signed integer can hold.

Step 3: get every seller for the product

The Google Shopping Sellers API endpoint is google-shopping-product. It takes the product_id, plus country and language, and returns the product and all of its sellers in one request.

def sellers(product_id, country="us"):
    result = run_job("google-shopping-product", {"product_id": product_id, "country": country})
    if result.get("error"):
        raise CrawlerDataError(f"{product_id}: {result['error']}")
    return result

The error check is there for IDs Google does not recognise. We sent a made-up ID and got a completed job with "sellers_count": 0, an empty sellers list and "error": "Product not found". That is a successful request, so it cost 1 credit, the minimum for a sellers request. Validate IDs before you put them in a daily job.

For our product the request returned all 22 sellers in one response, sorted by total price. Two of them, trimmed:

{
  "sellers_count": 22,
  "sellers": [
    {
      "position": 1,
      "merchant": "Best Buy",
      "merchant_rating": 4.6,
      "offer_title": "Sony - WH-1000XM5 Wireless Noise Cancelling Over-the-Ear Headphones - Black",
      "price": { "value": 198, "currency": "USD", "raw": "198.00 USD" },
      "shipping": { "value": 0, "currency": "USD", "raw": "0.00 USD" },
      "total_price": { "value": 198, "currency": "USD", "raw": "198.00 USD" },
      "delivery": "Free delivery by October 20",
      "availability": "In Stock",
      "returns": null,
      "link": null
    },
    {
      "position": 7,
      "merchant": "eBay",
      "merchant_rating": null,
      "offer_title": "Sony Wh-1000xm5 Wireless Noise Canceling Headphones (b-stock) - Black",
      "price": { "value": 215, "currency": "USD", "raw": "215.00 USD" },
      "shipping": { "value": 20, "currency": "USD", "raw": "20.00 USD" },
      "total_price": { "value": 235, "currency": "USD", "raw": "235.00 USD" },
      "delivery": "Delivery by October 20 $6.57",
      "availability": "In Stock",
      "returns": null,
      "link": null
    }
  ]
}

Things worth knowing before you use these numbers:

• Identify an offer by merchant and offer_title. Together they tell you who sells which listing.
• shipping can be null. Two sellers in our result had no shipping value. In that case total_price equals price, which does not prove that shipping is free.
• Keep delivery too. It is the text Google shows. In the eBay row above it mentions a different amount from the shipping value, so if a shipping difference decides something, look at both.
• Check offer_title for condition. Words like “b-stock”, “refurbished” or “used” show up in the seller’s own title, as in the eBay row above.

Step 4: write the sellers to CSV

Prices come as objects with value, currency and raw. For a spreadsheet you want plain numbers, so the writer takes value and keeps the currency in its own column.

def value(money):
    return money["value"] if money else None


def write_csv(path, result):
    fields = ["position", "merchant", "merchant_rating", "price", "shipping",
              "total_price", "currency", "availability", "delivery", "offer_title"]
    with open(path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=fields)
        writer.writeheader()
        for s in result["sellers"]:
            writer.writerow({
                "position": s["position"],
                "merchant": s["merchant"],
                "merchant_rating": s["merchant_rating"],
                "price": value(s["price"]),
                "shipping": value(s["shipping"]),
                "total_price": value(s["total_price"]),
                "currency": (s["total_price"] or s["price"] or {}).get("currency"),
                "availability": s["availability"],
                "delivery": s["delivery"],
                "offer_title": s["offer_title"],
            })

An empty cell in shipping means Google showed no shipping value. Do not fill it with 0 unless you have checked.

Step 5: put it together

def main():
    query = sys.argv[1]
    country = sys.argv[2] if len(sys.argv) > 2 else "us"

    products = search(query, country)
    if not products:
        sys.exit(f"No Google Shopping results for {query!r} in {country}")
    for p in products[:5]:
        price = p["price"]["raw"] if p["price"] else "n/a"
        print(f'{p["position"]:>2}  {p["product_id"]}  {price:>10}  {p["title"][:60]}')

    product = products[0]
    result = sellers(product["product_id"], country)
    path = f'sellers_{product["product_id"]}_{country}.csv'
    write_csv(path, result)
    print(f'\n{result["sellers_count"]} sellers for "{result["product"]["title"]}" -> {path}')


if __name__ == "__main__":
    main()

Save it as google_shopping_sellers.py and run it:

python google_shopping_sellers.py "sony wh-1000xm5" us

Our run printed this:

 1  11726664398501807381     $198.00  Sony WH-1000XM5 Noise Canceling Wireless Headphones
 2  13793769760861274890     $259.99  Sony Wh-1000XM5 Wireless Noise-Cancelling Over-the-Ear Headp
 3  4698748532434137820     $119.99  Sony Wh-1000xm5/s Wireless Industry Leading Noise Canceling
 4  16951088828645575361     $299.99  Sony WH-1000XM5 Noise-Canceling Wireless Over-Ear Headphones
 5  14815663489606098776     $281.99  Sony WH 1000XM5 Noise Canceling Wireless Over Ear Headphones

22 sellers for "Sony WH-1000XM5 Noise Canceling Wireless Headphones" -> sellers_11726664398501807381_us.csv

And the first rows of the CSV:

position,merchant,merchant_rating,price,shipping,total_price,currency,availability,delivery,offer_title
1,Best Buy,4.6,198,0,198,USD,In Stock,Free delivery by October 20,Sony - WH-1000XM5 Wireless Noise Cancelling Over-the-Ear Headphones - Black
2,Walmart,4.3,198,0,198,USD,In Stock,Free delivery,"Sony Wh-1000xm5 The Best Wireless Noise Canceling Headphones, Silver"
3,Sony,4.8,198,0,198,USD,In Stock,Free delivery between Wednesday - Thursday,Sony WH-1000XM5 Premium Wireless Noise Cancelling Headphones Black

Picking products[0] is fine for a demo. In real use, look up the ID once per product, check it is the listing you mean, and save it.

What it costs

A Google Shopping search is 1 credit. The sellers request returns every seller in one call, with no pagination or follow-up requests, and costs 1 credit per 10 sellers returned, rounded up, with a minimum of 1. Prices are the same on every queue, and failed jobs are not charged. Credit packs run from $1.00 per 1,000 credits (Starter, $50 for 50,000) to $0.30 per 1,000 (Ultimate). The pricing page has all four packs.

For this script:

• First lookup of our product: 4 credits, 1 for the search and 3 for the sellers request (22 sellers, so ceil(22 / 10) = 3).
• Every later check of the same product: 3 credits while it has 21 to 30 sellers, because you reuse the saved product_id and skip the search. A product with 8 sellers costs 1 credit per check, and one with 12 costs 2.
• Monitoring 1,000 products once a day for 30 days: assuming about 21 to 30 sellers per product (3 credits per check), that is 1,000 searches to set up plus 30,000 sellers requests × 3 credits, so 1,000 + 90,000 = 91,000 credits. That is $91 at the Starter rate; packs are bought whole, so in practice 2 Starter packs (100,000 credits) for $100. At the Ultimate rate it is about $27.30.
• The same 1,000 products in three countries: each country is a separate request, so three times the credits.

Purchased credits are valid for 6 months, with no subscription. Results that complete but are empty, such as an unknown product ID, are charged like any other completed request.

Running it every day

• Save each product_id once and reuse it, so a daily check is a single sellers request.
• Store the check time with every row. Prices and sellers change through the day, and a timestamp turns a folder of CSVs into a price history.
• Add markets as separate checks. country takes one of 31 two-letter codes, and prices come back in that market’s currency.

Next steps

If you are tracking prices for your own catalog, the competitor price monitoring page shows how the same two endpoints fit a daily workflow. For brand compliance work, see MAP monitoring. If you also need specs, images and descriptions, the Google Shopping Product API returns them for the same product_id. And if you are still choosing a provider, our Google Shopping API pricing comparison works through cost per product with all sellers.