• Per scrape request — options.timeout_seconds, default 30. Raise it whenever you raise the waiting settings.
• Microservice calls — 50 seconds. Exceeding this returns 504 Gateway Timeout, and retrying usually resolves it.

Most /user-templates endpoints call a microservice in the background, which is where 504 shows up most often.

• options.body (POST body) — 64 KB.
• max_captured — 50 by default, 200 maximum.
• max_body_bytes — 10 KB by default, 256 KB maximum.
• max_total_bytes — 512 KB by default, 2 MB maximum.

Values above the maximum are not rejected; they are silently clamped. If capture dropped data, network_dropped will be greater than zero.

Long-running requests such as /webhook/scrape-wp may stream:

• Expect Content-Type: text/event-stream.
• Keep the connection open until an event of type result arrives.
• Each line has the form data: <JSON> followed by a blank line.
• If the connection drops, continue with POST /webhook/check-job and the dispatch_id.

• Start polling at 2 s and back off; never poll more than once per second.
• Sending many URLs in one request with urls is both faster and cheaper than sending them separately.
• Split very large lists into moderate batches — several medium dispatches are easier to track than one enormous job.

Some URLs are blocked outright and return 403. Others trigger an extra_credit rule that multiplies the cost; that is applied silently, with no error.