Operating the API

Amazon Data API error codes

Standard HTTP status codes with a machine-readable code and a message written for whoever is reading the log at 3am.

Error shape

Any non-422 error
{
  "error": {
    "code": "rate_limited",
    "message": "Rate limit exceeded: 15 requests/second on the Growth plan.",
    "status": 429
  }
}

Validation failures use FastAPI's detail array instead, so each problem is tied to a specific field:

422 Unprocessable Entity
{
  "detail": [
    {
      "loc": ["query", "page"],
      "msg": "Input should be greater than or equal to 1",
      "type": "greater_than_equal",
      "input": "0"
    }
  ]
}

Full reference

StatusCodeCauseRetry?
400 bad_request The request was malformed — usually a query string that could not be parsed. No — fix the request
401 unauthorized Missing API key. Send it as `x-api-key` or `Authorization: Bearer <key>`. No — fix the request
403 forbidden The key is valid but revoked, or your plan does not include this endpoint. No — fix the request
404 not_found Unknown endpoint path. Check the API reference for the exact spelling. No — fix the request
422 validation_error A parameter failed validation. The `detail` array names the offending field. No — fix the request
429 rate_limited You exceeded your plan's per-second rate limit, or your monthly credits are exhausted. Yes, with backoff
500 internal_error Something broke on our side. These are logged and alerted on. Yes, with backoff
502 upstream_error Amazon returned an unusable response. Safe to retry with backoff. Yes, with backoff
503 endpoint_disabled This endpoint has been taken offline for maintenance. Nothing is charged; check the status page. Yes, with backoff
504 upstream_timeout The upstream request exceeded the time budget. Safe to retry. Yes, with backoff

The three you will actually see

429 — rate limited

You exceeded your plan's per-second ceiling. The response carries Retry-After: 1. Rejected requests cost nothing, so a tight retry loop is not expensive — just make sure it backs off rather than hammering.

429 — quota exhausted

Different code, same status. Your monthly credits are gone and your plan has no overage allowance. The message includes your reset date. This one will not resolve on retry — upgrade, or wait for the period to roll.

Telling them apart
if response.status_code == 429:
    code = response.json()["error"]["code"]
    if code == "rate_limited":
        time.sleep(float(response.headers.get("Retry-After", 1)))
        return retry()
    if code == "quota_exhausted":
        alert_ops("AmazonCrawler credits exhausted")  # retrying won't help
        raise QuotaError(response.json()["error"]["message"])

502 / 504 — upstream trouble

Amazon returned something unusable or took too long. These are transient and uncharged. Retry with exponential backoff — three attempts clears the overwhelming majority.

What you are never charged for

  • 5xx errors — anything that failed on our side or upstream.
  • 429s — requests the rate limiter rejected before doing any work.
  • 422s — validation caught before we reach Amazon.
  • 401 / 403 / 404 — rejected at the edge.

Only a request that reached upstream and came back with a body consumes credits. Check X-Credits-Cost on any response if you want to verify it.

A retry policy that behaves

client.py 
import random, time, requests

RETRYABLE = {429, 500, 502, 503, 504}

def call(path, params, *, tries=5):
    for attempt in range(tries):
        r = session.get(f"https://amazoncrawler.com/v1{path}", params=params, timeout=30)

        if r.ok:
            return r.json()

        if r.status_code == 429 and r.json()["error"]["code"] == "quota_exhausted":
            raise QuotaError(r.json()["error"]["message"])

        if r.status_code not in RETRYABLE:
            r.raise_for_status()

        # exponential backoff with jitter, honouring Retry-After
        wait = float(r.headers.get("Retry-After", 2 ** attempt))
        time.sleep(wait + random.uniform(0, 0.3))

    raise RuntimeError(f"{path}: exhausted {tries} attempts")
Persistent 5xx across many requests usually means an incident rather than your code. Check the status page before digging.