What Is a 524 Status Code?
A 524 status code is a Cloudflare error, "A timeout occurred", returned when Cloudflare connected to the origin but did not get an HTTP response before its proxy read timeout, 125 seconds by default. The connection works; the request takes too long, usually because of a heavy query, a large export, or an overloaded server.
- Code
- 524
- Name
- A Timeout Occurred
- Class
- Cloudflare extension
- Retry?
- Yes, with backoff
What causes error 524?
- →A long-running request, such as a large report, export, or unindexed query.
- →An overloaded origin with a long request queue.
- →A write to the origin that does not finish within Cloudflare’s 30-second proxy write timeout.
How do you fix error 524 when web scraping?
- →Retry with backoff, and avoid URLs that trigger heavy work, such as very large page sizes.
- →Break big requests into smaller ones, for example with pagination parameters.
- →Lower concurrency so your requests do not add to the queue.
How do you fix error 524 on your own server?
- →Poll long jobs instead of holding one request open: return a job ID, then let the client check status.
- →Log origin response times and fix the slow endpoints.
- →Move requests that legitimately take longer than 125 seconds to a DNS-only subdomain. Enterprise plans can raise the timeout up to 6,000 seconds.
How do you handle error 524 in a retry loop?
fetch() treats 524 as temporary. It waits for Retry-After when the server sends a number of seconds, otherwise backs off exponentially with jitter, caps every wait at 60 seconds, and gives up after five attempts.
import random
import time
import requests
RETRYABLE = {408, 429, 500, 502, 503, 504, 520, 521, 522, 523, 524}
def fetch(url: str, max_attempts: int = 5) -> requests.Response:
for attempt in range(max_attempts):
try:
response = requests.get(url, timeout=(10, 60))
except requests.Timeout:
time.sleep(2**attempt + random.uniform(0, 1))
continue
if response.status_code not in RETRYABLE:
response.raise_for_status()
return response
retry_after = response.headers.get("Retry-After", "")
backoff = 2**attempt + random.uniform(0, 1)
time.sleep(min(int(retry_after) if retry_after.isdigit() else backoff, 60))
raise RuntimeError(f"Gave up on {url} after {max_attempts} attempts")
How does Context.dev handle error 524?
Context.dev retries the fetch for you, and by default Scrape can reuse a capture made in the last 3 days (maxAgeMs), so a page that is briefly down may still come back from cache. If no capture is available, the failed output carries an error_code and message inside an HTTP 200 response, and a request where every output fails is not charged.
See what the web scraping API does on every request, or read how to fix HTTP errors in web scraping for a longer walkthrough.
Frequently asked questions about error 524
What is the Cloudflare 524 timeout?
Cloudflare waits 125 seconds by default for the origin to send a response after connecting. Enterprise customers can raise it to as much as 6,000 seconds.
How do I fix error 524?
Make the slow request faster, split it up, or switch to an asynchronous job pattern. Very long requests can also bypass Cloudflare on a DNS-only subdomain.
What is the difference between 524 and 504?
Both are gateway timeouts. A 524 is Cloudflare’s version, raised by Cloudflare itself. A 504 comes from any other gateway or proxy.
Which status codes are related to 524?
Sources
Last reviewed