HTTP 406 Not Acceptable: What It Means and How to Fix It
HTTP 406 Not Acceptable is a status code meaning the server has no version of the resource that matches the Accept, Accept-Language, or Accept-Encoding headers in the request, and will not send a default instead. Some server and firewall configurations also return 406 for requests they consider suspicious, which is why scrapers see it more often than browsers do.
- Code
- 406
- Name
- Not Acceptable
- Class
- 4xx client error
- Retry?
- No, fix the cause first
What causes a 406 error?
- →An
Acceptheader that asks only for a type the server does not produce, such asapplication/jsonon an HTML page. - →An
Accept-EncodingorAccept-Languagevalue the server refuses to fall back from. - →A firewall rule that answers blocked requests with 406 instead of 403.
- →Library defaults that send unusual or empty negotiation headers.
How do you fix a 406 error when web scraping?
- →Send browser-like negotiation headers, for example
Accept: text/html,application/xhtml+xml,*/*;q=0.8and anAccept-Languagevalue. - →Add
*/*with a low q-value so the server can fall back to any type it has. - →Compare your request with the browser’s in DevTools. If only your client gets a 406, the server is filtering on headers.
- →Do not retry the same headers. Change them, then try once.
How do you fix a 406 error on your own server?
- →Serve a sensible default representation rather than a 406 when negotiation fails. RFC 9110 allows it.
- →Use 403 for security blocks so clients can tell a refusal from a format mismatch.
How do you handle a 406 error in a retry loop?
406 is not in RETRYABLE, so raise_for_status() raises on the first response instead of spending retries on a request that will fail the same way. Fix the cause, then send the request again.
import random
import time
import requests
RETRYABLE = {408, 429, 500, 502, 503, 504, 520, 521, 522, 523, 524}
def fetch(url: str, max_attempts: int = 5) -> requests.Response:
for attempt in range(max_attempts):
try:
response = requests.get(url, timeout=(10, 60))
except requests.Timeout:
time.sleep(2**attempt + random.uniform(0, 1))
continue
if response.status_code not in RETRYABLE:
response.raise_for_status()
return response
retry_after = response.headers.get("Retry-After", "")
backoff = 2**attempt + random.uniform(0, 1)
time.sleep(min(int(retry_after) if retry_after.isdigit() else backoff, 60))
raise RuntimeError(f"Gave up on {url} after {max_attempts} attempts")
How does Context.dev handle a 406 error?
Every scrape runs in a real browser, so requests carry the Accept, Accept-Language, and Accept-Encoding headers a browser sends. If a site needs something specific, sharedParams.headers sends extra headers to the target (requests with custom headers skip the cache).
See what the web scraping API does on every request, or read how to fix HTTP errors in web scraping for a longer walkthrough.
Frequently asked questions about a 406 error
What causes a 406 Not Acceptable error?
Content negotiation failed: the server has nothing that matches your Accept headers and will not send a default. Firewalls that use 406 for blocked requests are the other common cause.
How do I fix a 406 error in Python requests?
Set an explicit Accept header that includes */*, add Accept-Language, and compare the request with what a browser sends.
Is 406 the same as 415?
No. A 406 is about the response format you asked for. A 415 Unsupported Media Type is about the format of the body you sent.
Which status codes are related to 406?
Sources
Last reviewed