`POST /webhook/unblocker`
Fetches one or many URLs. Anti-bot protection, JavaScript rendering and proxy rotation are
handled for you.
Authentication: the `x-api-key` header is required.
Cost: 5 credits per request.
The smallest working request
curl -X POST http://localhost:3000/webhook/unblocker \
-H "Content-Type: application/json" \
-H "x-api-key: js_YOUR_API_KEY" \
-d '{
"url": "https://example.com/products",
"selectors": [
{ "property_name": "title", "selector": "h1.product-title", "attribute_type": "TEXT" },
{ "property_name": "price", "selector": "span.price", "attribute_type": "TEXT" }
],
"mode": "browser",
"options": {
"humanize": true,
"wait_until": "domcontentloaded",
"wait_seconds": 2,
"use_proxy": true,
"retry_count": 2
}
}'Request body
- `url` (string) — a single target. Can be combined with `urls`.
- `urls` (string array) — multiple targets; the same selector set is applied to each.
- `selectors` (array) — extraction rules, see Selector Guide.
- `mode` (`"browser"` | `"http"`) — `browser` runs a real browser and executes
JavaScript; `http` sends a plain HTTP request and is faster and cheaper to run.
- `wait_until` (string) — page load strategy, see Rendering and Waiting.
- `actions` (array) — browser interactions, `browser` mode only, see Browser Actions.
- `options` (object) — rendering, proxy, timeout and capture settings.
- `max_items` (number) — upper bound on records returned for list-type results.
The options object
Every field is optional.
- `company_name` (string, auto) — request group identifier.
- `config_name` (string, auto) — HTTP config name.
- `humanize` (boolean, default `true`) — simulates human behaviour with mouse movement
and scrolling.
- `wait_until` (string, default `"domcontentloaded"`) — page load strategy.
- `wait_seconds` (number, default `1`) — fixed wait after load, in seconds.
- `wait_selector` (string) — waits until this CSS selector appears.
- `timeout_seconds` (number, default `30`) — per-request timeout.
- `use_proxy` (boolean, default `true`) — use the proxy pool.
- `headers` (string) — custom HTTP headers, one `key:value` per line.
- `retry_count` (number, default `2`) — retries after a failure.
- `debug` (string) — `"on_failure"` adds diagnostic detail to failed requests.
- `method`, `body`, `body_type` — sending an HTTP POST, see HTTP POST Requests.
- `capture_network`, `capture_options` — see Network Capture.
A fuller example body
```json
{
"url": "https://example.com",
"urls": ["https://example.com/page1", "https://example.com/page2"],
"selectors": [
{ "property_name": "title", "selector": "h1", "attribute_type": "TEXT" },
{
"property_name": "links[]",
"selector": "a.link",
"attribute_type": "ATTRIBUTE",
"attribute": "href"
},
{
"property_name": "card",
"selector": ".product-card",
"attribute_type": "OBJECT",
"children": [
{ "property_name": "name", "selector": ".name", "attribute_type": "TEXT" },
{ "property_name": "price", "selector": ".price", "attribute_type": "TEXT" }
]
}
],
"mode": "browser",
"wait_until": "domcontentloaded",
"actions": [
{ "type": "wait", "milliseconds": 1000 },
{ "type": "click", "selector": "#accept-cookies" },
{ "type": "scroll", "direction": "down", "amount": 800 }
],
"options": {
"humanize": true,
"wait_seconds": 1,
"timeout_seconds": 120,
"use_proxy": true,
"retry_count": 2,
"debug": "on_failure"
},
"max_items": 10
}
```Response
```json
{
"success": true,
"dispatch_id": "disp_abc123",
"message": "Scrape completed",
"company_id": 5,
"config_id": 12,
"routine_id": 34,
"job_id": 789,
"total_urls": 2,
"success_count": 2,
"failed_count": 0,
"execution_time": "4.2s",
"results": [
{
"url": "https://example.com/products",
"predefined_request_id": 1,
"success": true,
"data": { "title": "Awesome Product", "price": "$29.99" },
"job_id": 789,
"company_id": 5,
"config_id": 12,
"routine_id": 34
}
]
}
```The keys inside `results[].data` match the `property_name` values from your `selectors`
array exactly.
Errors
- `400` — invalid body or a rule violation, such as `method: "POST"` in browser mode.
- `401` — missing or invalid API key.
- `402` — not enough credit.
- `403` — the URL is blacklisted.
- `504` — timed out; raise `timeout_seconds` or retry.
browser or http?
- Server-rendered static HTML — `http`, faster and lighter.
- Content arrives via JavaScript — `browser`.
- You need clicks, forms or scrolling — `browser`.
- Posting to a JSON API — `http`; POST is only allowed there.
- Capturing network traffic — `browser`; capture only works there.
Sending many URLs in one request with the `urls` array is cheaper than sending them one by
one — the automatic +1 credit is charged once for the request, not per URL.