Keep a second SERP provider: a 5-minute SerpApi failover

Every SERP API vendor can have an outage, change a price, retire a parameter or turn up in the news. None of that is a reason to leave one. It is a reason to know, before it happens, what your code does when search calls start failing.

This guide puts a second provider behind your existing SerpApi calls in about five minutes. It uses SerpApi as the primary because many teams run it, and CrawlerData’s /search.json as the fallback because it accepts the same query parameters and returns SerpApi-shaped JSON, so only the base URL and the key change. Nothing here is a criticism of SerpApi.

Why keep a fallback

If your product shows search results, tracks rankings or feeds an agent, the search call is on your critical path. The ordinary reasons to want a second source are an outage, a rate limit you hit during a spike, a billing problem or an API change.

Vendors also appear in the news. Two things have been reported about Google’s case against SerpApi, and we link the sources instead of summarising the case:

• Search Engine Journal reported on August 12, 2026 that Google filed an amended complaint on August 10, after a July 20 court order dismissed its earlier DMCA claims.
• SerpApi wrote on August 25 that it has asked the court to dismiss the amended complaint, and told customers that “nothing about your service changes.”

We do not know how the case will end, and this post does not try to predict it. The point is narrower: when a vendor you depend on is in the news, “what would we do?” should already have an answer.

The five-minute setup

1. Create a CrawlerData API key at app.crawlerdata.com. New accounts with a verified work email get 50 free credits.
2. Put both keys in environment variables: SERPAPI_API_KEY and CRAWLERDATA_API_KEY.
3. Call this function instead of calling SerpApi directly:

import os

import requests

PROVIDERS = [
    {"name": "serpapi", "url": "https://serpapi.com/search.json", "key": os.environ["SERPAPI_API_KEY"]},
    {"name": "crawlerdata", "url": "https://api.crawlerdata.com/search.json", "key": os.environ["CRAWLERDATA_API_KEY"]},
]


def search(q, **params):
    """Try each provider in order. Fail over on a timeout, a connection error, any non-2xx status or a body that is not JSON."""
    last_error = None
    for p in PROVIDERS:
        try:
            res = requests.get(
                p["url"],
                params={"engine": "google", "q": q, **params, "api_key": p["key"]},
                timeout=30,
            )
            res.raise_for_status()
            return p["name"], res.json()
        except (requests.RequestException, ValueError) as e:
            last_error = f"{p['name']}: {e}"
    raise RuntimeError(f"All providers failed. Last error: {last_error}")


def normalize(data):
    """The fields both providers return under the same names."""
    return [
        {"position": r.get("position"), "title": r.get("title"), "link": r.get("link"), "snippet": r.get("snippet")}
        for r in data.get("organic_results", [])
    ]

Use it like this:

provider, data = search("best crm software", gl="us", hl="en")
for r in normalize(data)[:3]:
    print(provider, r["position"], r["title"], r["link"])

The function tries each provider in order, moves on after a timeout, a connection error, any non-2xx status (a 429 or a 5xx included) or a body that is not JSON, and tells you which provider answered. How a provider reports an empty or failed search inside a 200 response differs, so decide for yourself whether that should fail over. For “no results” you probably do not want it to.

What carries over, and what does not

/search.json takes the parameters most SerpApi calls use: engine, q, gl, hl, location, uule, device, start and api_key. An Authorization: Bearer header works too. In the run we did for this guide, a google search returned organic_results (each with position, title, link and snippet), related_questions, related_searches, search_information and ai_overview (with text_blocks and references), plus search_metadata and search_parameters.

The engines are google (the News, Images, Shopping and Local tabs through tbm), google_shopping, google_news, google_images, google_local and google_autocomplete. SerpApi’s engines for other search engines are not part of /search.json, and neither is a parameter that is not listed above. Run a sample of your real queries through it before you rely on it.

Test the failover before you need it

Force the primary to fail and confirm the second one answers:

PROVIDERS[0]["url"] = "https://invalid.invalid/search.json"
provider, data = search("what is a serp api", gl="us", hl="en")
print(provider, len(normalize(data)))  # crawlerdata 5

Then run 20 of your real queries through both providers and compare the results. Two providers fetch Google separately, so positions and snippets can differ slightly between them, and between runs of the same provider. Store the provider name with every result, so a position change right after a failover is not mistaken for a ranking change.

Running it

• Alert when the fallback serves a request. A silent failover hides the outage you want to know about.
• Exercise the fallback. A fallback you never call is one you have not tested. Send it a few requests a week.
• Add a cooldown if you need one. If the primary is down for an hour, going straight to the fallback for a minute at a time avoids waiting 30 seconds on every request.
• Cost. You pay each provider for the requests it serves. CrawlerData is prepaid credits with no subscription, valid for 6 months, and failed requests are not charged, so an idle fallback costs nothing until it is used.

Next steps

The SerpApi-compatible API page lists the engines and parameters with Python and Node.js examples, CrawlerData vs SerpApi compares the two on billing and features, and the Google Search API page lists every field of the native response.