What Is a 520 Status Code?
A 520 status code is a Cloudflare-specific error, "Web server returns an unknown error", shown when the origin server sends Cloudflare an empty, unknown, or unexpected response. It is not part of the HTTP standard. Common causes are crashing applications, firewalls blocking Cloudflare IPs, response headers over 128 KB, and HTTP/2 misconfiguration at the origin.
- Code
- 520
- Name
- Web Server Returns an Unknown Error
- Class
- Cloudflare extension
- Retry?
- Yes, with backoff
What causes error 520?
- →The origin application crashed while handling the request.
- →A firewall or security plugin on the origin blocking Cloudflare’s IP addresses.
- →Response headers larger than 128 KB, often from too many cookies.
- →An empty or malformed response, HTTP/2 enabled at the origin without working support, or Authenticated Origin Pulls enabled without the origin accepting client certificates.
How do you fix error 520 when web scraping?
- →Retry with backoff. Many 520s come from origins that crash under load and recover.
- →Clear cookies for the site. A cookie jar that keeps growing can push the origin’s response headers over the limit.
- →If every request fails, the origin is broken for everyone. Wait, or fetch a cached copy instead.
How do you fix error 520 on your own server?
- →Check origin error logs for crashes at the times of the 520s.
- →Allow all Cloudflare IP ranges in your firewall and security plugins.
- →Reduce response header size, and check HTTP/2 to Origin settings in Cloudflare.
- →As a temporary workaround, set the DNS record to DNS-only and reproduce the error without Cloudflare in the path.
How do you handle error 520 in a retry loop?
fetch() treats 520 as temporary. It waits for Retry-After when the server sends a number of seconds, otherwise backs off exponentially with jitter, caps every wait at 60 seconds, and gives up after five attempts.
import random
import time
import requests
RETRYABLE = {408, 429, 500, 502, 503, 504, 520, 521, 522, 523, 524}
def fetch(url: str, max_attempts: int = 5) -> requests.Response:
for attempt in range(max_attempts):
try:
response = requests.get(url, timeout=(10, 60))
except requests.Timeout:
time.sleep(2**attempt + random.uniform(0, 1))
continue
if response.status_code not in RETRYABLE:
response.raise_for_status()
return response
retry_after = response.headers.get("Retry-After", "")
backoff = 2**attempt + random.uniform(0, 1)
time.sleep(min(int(retry_after) if retry_after.isdigit() else backoff, 60))
raise RuntimeError(f"Gave up on {url} after {max_attempts} attempts")
How does Context.dev handle error 520?
Context.dev retries the fetch for you, and by default Scrape can reuse a capture made in the last 3 days (maxAgeMs), so a page that is briefly down may still come back from cache. If no capture is available, the failed output carries an error_code and message inside an HTTP 200 response, and a request where every output fails is not charged.
See what the web scraping API does on every request, or read how to fix HTTP errors in web scraping for a longer walkthrough.
Frequently asked questions about error 520
What does 520 code mean?
Cloudflare reached the origin server, but the origin answered with something Cloudflare could not use: an empty, malformed, or unexpected response.
Is error 520 my fault as a visitor?
Almost never. It is a problem between Cloudflare and the site’s own server. Clearing cookies helps only when oversized cookies are the cause.
How is 520 different from 521?
With a 520 the origin answered badly. With a 521 the origin refused the connection entirely.
Which status codes are related to 520?
Sources
Last reviewed