Turn any URL into clean JSON.
One API key. Give us a URL and a set of fields — we fetch the page, rotate the IP, run the browser if it needs one, and hand back structured data. No proxies to manage, no parser to babysit.
Data extraction API
Give us a URL and a set of fields. We fetch the page, apply your CSS selectors, and hand back clean JSON. No browser to run, no parsing code, no IP rotation to manage. The base URL for every endpoint is https://scrape.land.
There is one request you'll use most of the time: POST /v1/extract. The rest of this section shows that exact request, then transplants it into every language so you can plug it in fast.
Authentication
Send your API key in the X-Api-Key header on every request:
X-Api-Key: pb_live_YOURKEYAn Authorization bearer header works too, if that fits your stack better:
Authorization: Bearer pb_live_YOURKEYPOST/v1/extract
The canonical request. Send a url and a fields map, get a data object back with one key per field. This is the same request used in every language snippet below.
curl https://scrape.land/v1/extract \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example-product.com/item/42",
"fields": {
"title": "h1",
"price": ".price",
"image": {"css": "img.gallery", "attr": "src"},
"tags": {"css": ".tag", "all": true}
}
}'Response. Each key in data lines up with the field you asked for:
{
"url": "https://example-product.com/item/42",
"status": 200,
"data": {
"title": "Vintage desk lamp",
"price": "49.00",
"image": "https://example-product.com/img/a.jpg",
"tags": ["retro", "lighting", "brass"]
}
}How fields map to selectors
Each entry in fields is one of these forms:
- A CSS selector string. We return the text of the first matching element.
"title": "h1"returns the text inside the firsth1. - A selector@attr shorthand to read one attribute.
"image": "img.gallery@src"returns thesrc; use@textor no param for the text. - An XPath shorthand: any string starting with
/is treated as XPath."title": "//h1/text()","link": "//a/@href". - An object with
cssplus optional keys:attr(read an attribute instead of text),all(return every match as an array), orxpath(an XPath expression instead ofcss). They combine.
So this field map:
{
"title": "h1",
"price": ".price",
"image": "img.gallery@src",
"tags": {"css": ".tag", "all": true}
}produces this data object, field by field:
| Field | Selector / object | Returned value |
|---|---|---|
title | "h1" | text of the first h1 |
price | ".price" | text of the first .price |
image | "img.gallery@src" | the src attribute, as a string |
tags | {"css":".tag","all":true} | an array of every .tag text |
When a field comes back null
A field is null either because the page genuinely has no such element, or because the selector itself could not be used. Those mean very different things to you, so we tell them apart: if a selector could not be compiled, the response carries a field_errors object naming the field and why. The field is still present in data as null, the request is still a 200, and your other fields are unaffected — one unusable selector never sinks the rest of the extraction.
{
"data": {"title": "Example Domain", "price": null, "tags": null},
"field_errors": {
"tags": "selector did not compile (unknown pseudoclass or pseudoelement :is) — this field is null because the selector could not be used, not because the page lacks the element"
}
}No field_errors key means every selector compiled, so a null there is a real "not on the page". XPath is validated up front instead and returns 400.
- Newer selectors are not supported yet and will land in
field_errors::is(),:where(),:has(> x)in its relative form,:focus-within, namespaced selectors likesvg|circle. Everything from CSS3 works — descendant/child/sibling combinators,:nth-child(),:not(), attribute selectors including^= $= *=and theiflag. - Four selectors work here that no browser supports. They are Cascadia extensions for matching on text, which CSS itself cannot do — genuinely useful, but not standard CSS. A selector using them will not work in a browser, in devtools, or in any other scraping tool, so treat them as a convenience of this API rather than something to standardise on.
Selector Matches Example :contains(…)element whose text contains a substring h1:contains("Example"):matches(…)element whose text matches a regular expression p:matches(^Price:):matchesOwn(…)same, but only the element's own text, ignoring descendants h1:matchesOwn(^Example):haschild(…)element with a direct child matching a selector p:haschild(a)Note the argument types, which are the easy mistake:
:matches()and:matchesOwn()take a regex over text — they are not a spelling of:is().:matches(h1,h2)compiles and then matches nothing, because it is looking for the literal texth1,h2. Quote any argument containing spaces or punctuation::contains("Learn more"), not:contains(Learn more).
JavaScript pages: render & wait_for
If the page builds its content with JavaScript, add "render": true. We drive a real browser, let the page run, then read the DOM and apply your selectors. Pair it with "wait_for" set to a CSS selector to hold until that element appears before reading — the reliable way to wait for late-loading content:
curl https://scrape.land/v1/extract \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://example-product.com/item/42",
"render": true,
"wait_for": "h1.title",
"fields": {"title": "h1.title", "price": ".price"}
}'Other optional top-level params you can add to any extract request:
| Param | What it does |
|---|---|
country | ISO exit country, e.g. us. The request leaves from that country. |
session | Sticky session name. Reuse one exit IP across calls (logins, carts). |
render | true to run the page in a real browser before reading it. |
wait_for | In render mode, a CSS selector to wait for before reading the DOM. |
headers | true to include the response headers in the result. |
format | For /v1/fetch: html (default), text, markdown (LLM-ready Markdown — see below), or raw — exact bytes base64-encoded under base64 with the content_type, so you can pull a PDF, image, or any binary through the proxy intact (capped 7 MiB). |
screenshot | For /v1/fetch: true returns a base64 PNG of the rendered page (implies render). Add full_page: true for the whole scroll height. |
send_headers | An object of extra request headers to forward to the target (custom User-Agent, Authorization, …). |
cookies | A Cookie header value to send, e.g. to scrape behind a login. |
method / body | HTTP method (default GET) and request body. Use POST/PUT/… to hit JSON APIs or submit forms. Non-GET is never retried and isn't compatible with render. |
block_resources | In render mode, true skips images/fonts/media for a faster, lighter render (DOM unaffected, extraction still works). |
fingerprint | In render mode, true presents a browser identity coherent with country — timezone, locale, geolocation, navigator.languages, generic WebGL/canvas — so a render reads as one real client from that region and trips fewer bot blocks. A session keeps it stable. |
device | "mobile" renders as a phone (390×844 touch viewport, 3×, mobile Safari UA) for the mobile layout; default is 1280×800 desktop. Implies render. |
actions | In render mode, scripted steps before capture: scroll, click, wait, wait_for, fill. Great for infinite scroll and content behind clicks. |
extract_type | Auto-extract a normalized object for a known page type without a prompt or schema. See Auto-extract. |
metadata | true also returns a metadata object: title, description, canonical, lang, OpenGraph/Twitter tags, and parsed schema.org jsonld. Great for link previews and SEO — no selectors needed. |
links | true also returns a links array: every a href resolved to an absolute URL, deduped, http(s) only. Handy for crawling and discovery. |
screenshot, and 1 — cheaper than a plain render, and the same as a plain fetch — when you send block_resources. A render that fails is not billed at all. Unless you need a screenshot or the images themselves, send "block_resources": true: images, fonts and media are skipped, the DOM your selectors read is identical, on an image-heavy page it removes a large part of the transfer (measured over 14 pages: median 42%, best 77%, and around 23% even on image CDNs that serve URLs without a file extension), and it drops the cost from 5 units to 1. It applies even alongside screenshot — the resources really are blocked, so the screenshot you get back will be missing its images. Render capacity is also bounded — if every browser slot is busy you get a 503 with Retry-After: 1, so retry rather than treat it as a failure.AI extraction (describe fields in plain English)
prompt, a schema, or any extract_type preset — is available on Scale and above (Scale, Business, Business XL/XXL/Ultra). On Free, Starter, Growth and pay-as-you-go these return 403 with the plan you'd need. Selector-based fields extraction (CSS and XPath, below) works on every plan, including Free. Check GET /v1/capabilities with your key to see which you have.Don't want to write or maintain selectors? Send a prompt instead of fields: we fetch the page (all the params above still apply), convert it to Markdown, and a language model returns exactly the fields you described as JSON. Selectors break when a site's markup changes; a prompt adapts. Add an optional schema (a JSON object of key→type) to pin the exact output shape. Pass model to pick a strength tier: "fast" (the default — a median 1.5s, and the right choice for almost every extraction, since pulling stated fields off a page is a reading task) or "smart" (a median 3.4s, and better when the answer has to be derived from the page rather than read off it — combining a start time, a duration and a timezone, say — billed at a higher request-unit rate). An unrecognised name is a 400 listing the valid ones. If the tier you asked for is unavailable we answer on the other one rather than failing, bill the tier that actually served, and tell you by returning requested_model alongside model.
Asking a question instead of listing fields? A prompt like "what's this business about?" is a question, not a field list — but the model still has to invent key names for it (business, category, competitors_mentioned…) and you have to guess what it picked. Send "structured": false and you get a plain-language reply under answer instead, with no data key at all:
curl https://scrape.land/v1/extract -H "X-Api-Key: pb_live_YOURKEY" -H "Content-Type: application/json" -d '{
"url": "https://scrape.land/",
"prompt": "what's this business about?",
"structured": false
}'{
"url": "https://scrape.land/",
"status": 200,
"extracted_by": "ai",
"model": "fast",
"answer": "scrape.land is a web data-extraction API: you send a URL and it returns clean structured data, handling proxy rotation, retries, geo-targeting and JavaScript rendering for you. Pricing is flat per 1,000 delivered requests."
}data and answer are never both present, so a typed client branches on which one it got rather than type-switching a single field. structured defaults to true, so nothing you already send changes. It costs exactly the same request-units as structured extraction — it is the same page and the same model call. A schema or an extract_type pins a JSON shape and therefore implies structured output; sending either together with "structured": false is a 400 naming both fields rather than us silently guessing which you meant. It has no effect on selector-based fields, which never reach a model.
Need a list? Ask for "every product on the page" and you get the array back under data.items. If the page has no text (a JS-only page fetched without render), you get a 422 telling you to retry with "render": true rather than a response full of nulls.
<time datetime> are passed to the model, so an exact price, currency code or ISO publish date is available even when the page only renders "From $49" or "3 days ago". Data encoded somewhere else entirely (a rating in a CSS class name) is still better pulled with fields (CSS/XPath), which see the raw HTML. Mix both freely.curl https://scrape.land/v1/extract \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://books.toscrape.com/",
"prompt": "the page title, and the title and price (as a number) of the first book"
}'{
"url": "https://books.toscrape.com/",
"status": 200,
"extracted_by": "ai",
"model": "fast",
"data": {
"page_title": "Books to Scrape",
"first_book_title": "A Light in the Attic",
"first_book_price": 51.77
}
}Auto-extract (one word instead of a schema)
Auto-extract is AI extraction under the hood, so it needs the same Scale or above plan.
For common page types you don't even need a prompt. Pass extract_type and we apply a curated prompt + schema so you get a normalized object back: product, article, job, discussion, event, recipe, real_estate, and profile. Your own prompt/schema still win if you also send them.
Presets run on the default fast tier and typically answer in a couple of seconds. Pass "model": "smart" if a page is genuinely ambiguous — it takes about twice as long and costs more request-units. Synchronous requests get a 150s budget; past that you get a 504, so submit a slow one as an async job (POST /v1/jobs, 5-minute budget).
curl https://scrape.land/v1/extract \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
"extract_type": "product",
"model": "fast"
}'Getting the response headers
Sometimes you need the response headers, not just the body: a redirect Location, a Set-Cookie, a Content-Type. Add "headers": true and we include a headers object alongside data.
{
"url": "https://example-product.com/item/42",
"status": 200,
"data": {"title": "Vintage desk lamp"},
"headers": {
"content-type": "text/html; charset=utf-8",
"set-cookie": "sid=abc123; Path=/; HttpOnly",
"cache-control": "max-age=300"
}
}More endpoints
POST/v1/fetch
When you want the whole page instead of specific fields, use /v1/fetch. Set format to html (default), text (visible text), markdown, or raw (base64 bytes + content_type, for PDFs/images/binaries, up to 7 MiB). The same country, session, render, wait_for, and headers options apply.
curl https://scrape.land/v1/fetch \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example-product.com/item/42", "format": "html"}'LLM-ready Markdown
"format": "markdown" returns the page as clean Markdown under a markdown key — the shape RAG pipelines and LLM context windows actually want, so you don't ship raw HTML tags to a model and pay for them as tokens. Site chrome (nav, header, footer, aside, scripts and styles) is stripped, headings, lists, links, code blocks and tables are kept, and relative links and images are resolved to absolute URLs so a chunk still works once it's separated from its source page. It composes with every other option — country, session, render, wait_for, actions, fingerprint, and it works in /v1/batch too. It costs exactly one request-unit, the same as any other fetch — there is no Markdown or AI surcharge.
curl https://scrape.land/v1/fetch \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/blog/post", "format": "markdown", "render": true}'
# -> {"url":"...","status":200,"markdown":"# Post title\n\nBody text..."}POST/v1/batch
Fetch up to 20 URLs in one call. Send a urls array plus any shared params; you get one results entry per URL, in input order — each a normal fetch result or an {"url","error"}. Each URL is billed as a separate delivered request.
Bulk extraction: add fields or a prompt and every URL is extracted just like /v1/extract — each result carries a data object. Great for turning a list of URLs into rows in one call.
curl https://scrape.land/v1/batch \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{"urls": ["https://a.example/", "https://b.example/"], "format": "text"}'POST/v1/search
Run a web search and get back organic results — title, the real destination url, and a snippet — without parsing a SERP yourself. Send {"q": "...", "count": 10} plus any shared proxy params. Billed as one delivered request. count is capped at 10 — one result page from the search engine; ask for more and you still get 10.
curl https://scrape.land/v1/search \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{"q": "best web scraping api", "count": 5}'POST/v1/rank
Rank the links on a page by how relevant each is to a goal, so you can pick which sub-pages to fetch next instead of crawling everything. Give a url and a query; we fetch the page, read its links (anchor text and all), and an LLM returns them ordered with a score (0–1) and a short reason. It is stateless: it ranks the links a page already has and never fetches them. Add top_k to cap the list, model ("fast"/"smart"), and any shared fetch params. If the AI step fails you still get the links back in document order with a ranking_error — your code never breaks. Billed as one fetch plus the AI surcharge; a failed ranking bills only the fetch.
curl https://scrape.land/v1/rank -H "X-Api-Key: pb_live_YOURKEY" -H "Content-Type: application/json" -d '{"url": "https://example.com/blog", "query": "in-depth technical tutorials", "top_k": 10}'POST/v1/jobs — async & webhooks
For long-running work (a big batch, a slow render) run it asynchronously: submit a job, get an id back immediately, then poll it or receive the result on a webhook. Send your normal operation params plus type (fetch/extract/batch/search) and an optional webhook_url.
curl https://scrape.land/v1/jobs \
-H "X-Api-Key: pb_live_YOURKEY" -H "Content-Type: application/json" \
-d '{"type":"batch","urls":["https://a.example","https://b.example"],
"fields":{"title":"h1"},"webhook_url":"https://your.app/hook"}'
# -> {"id":"job_ab12","status":"queued"}
# poll it:
curl https://scrape.land/v1/jobs/job_ab12 -H "X-Api-Key: pb_live_YOURKEY"
# -> {"id":"job_ab12","type":"batch","status":"done","result":{}}
# list recent jobs (newest first, no result blobs):
curl "https://scrape.land/v1/jobs?limit=20" -H "X-Api-Key: pb_live_YOURKEY"When a job finishes we POST {"id","status","result"} to your webhook_url (a public https URL — internal/loopback is rejected). Jobs run for up to a few minutes; each is metered exactly like the synchronous call.
Verifying the webhook. Every delivery carries an X-Scrapeland-Signature header of the form t=<unix>,v1=<hex>, where the hex is HMAC-SHA256("<t>.<raw body>") keyed with your webhook secret. Sign the RAW body bytes (not a re-serialised copy), compare in constant time, and reject anything whose t is more than a few minutes old — that is what stops a captured delivery being replayed at you later. Ask support for the secret if you use webhook_url.
import hmac, hashlib, time
def verify(raw_body: bytes, header: str, secret: str, tolerance=300) -> bool:
parts = dict(p.split("=", 1) for p in header.split(",") if "=" in p)
ts, sig = parts.get("t", ""), parts.get("v1", "")
if not ts or not sig or abs(time.time() - int(ts)) > tolerance:
return False
expected = hmac.new(secret.encode(),
ts.encode() + b"." + raw_body, hashlib.sha256).hexdigest()
return hmac.compare_digest(expected, sig)GET/v1/capabilities
A cacheable endpoint so your code can feature-detect what a deployment supports before sending a request: available formats and features, whether render (and sub-options) and AI extraction are on, the AI models and extract_types, the live endpoints, and the limits. Both SDKs expose it as capabilities().
It works without a key, but send your X-Api-Key and the answer is scoped to your account. AI extraction is plan-gated (see AI extraction), so an unkeyed call can only tell you what the deployment supports, while a keyed call tells you what you can call: ai.enabled is then true only if your plan includes it, and extract_type/extract_types disappear with it. ai.configured reports the deployment, ai.min_plan names the plan you'd need, and ai.plan_scoped tells you whether the answer was personalised. Feature-detect on ai.enabled and you will never be told yes and then handed a 403.
{
"formats": ["html", "text", "markdown", "raw"],
"render": {"enabled": true, "screenshot": true, "actions": true,
"device": true, "block_resources": true, "fingerprint": true},
"ai": {"enabled": true, "configured": true, "min_plan": "scale",
"plan_scoped": true, "models": ["fast", "smart"], "default": "fast"},
"extract_types": ["product", "article", "job", "discussion",
"event", "recipe", "real_estate", "profile"],
"endpoints": ["/v1/fetch", "/v1/extract", "/v1/batch", "/v1/search",
"/v1/jobs", "/v1/account", "/v1/capabilities"],
"limits": {"batch_max_urls": 20, "links_max": 2000,
"search_max_count": 10, "raw_max_bytes": 7340032,
"max_response_bytes": 5242880,
"max_response_bytes_scope": "key"}
}Response size. limits.max_response_bytes is the largest single response we will return, and it depends on your plan (Free and Starter 2 MB, Pay As You Go and Growth 5 MB, Scale 10 MB, Business 25 MB). Real pages are nowhere near it — the median response we serve is about 40 KB and 99% are under 1.2 MB — so it only ever catches a video, an archive, or a dataset dump. A response over the limit is refused whole with a 413 naming the size, the limit and your plan; we never hand back a truncated page, because a page that silently lost its tail is billed as a success and parses as corrupt data. limits.max_response_bytes_scope tells you whose number you got: "key" when you sent an X-Api-Key (your own plan's limit), or "lowest_plan" when you didn't — the floor every account gets, so it is always safe to size against and presenting a key can only raise it.
GET/v1/account
Check your key's plan, remaining quota, prepaid credit, rate limit, and per-key budget in code (e.g. before a large job) instead of opening the dashboard. It only ever reports your own account. Both SDKs expose it as account().
{
"plan": "pro",
"included_requests_remaining": 421900,
"prepaid_credit_cents": 0,
"rate_limit_rps": 100,
"key_budget": {"max_requests": 0, "requests_used": 1234, "unlimited": true}
}The same request, in every language
One request, transplanted idiomatically. Each does the same thing: an HTTP POST to https://scrape.land/v1/extract with the X-Api-Key header and the JSON body, then parse the JSON response.
curl https://scrape.land/v1/extract \
-H "X-Api-Key: pb_live_YOURKEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://example-product.com/item/42",
"fields":{"title":"h1","price":".price"}}'import requests
r = requests.post("https://scrape.land/v1/extract",
headers={"X-Api-Key": "pb_live_YOURKEY"},
json={"url": "https://example-product.com/item/42",
"fields": {"title": "h1", "price": ".price"}},
timeout=60)
print(r.json()["data"])from scrapeland import ScrapelandClient
client = ScrapelandClient("pb_live_YOURKEY") # or $SCRAPELAND_API_KEY
data = client.extract("https://example-product.com/item/42",
{"title": "h1", "price": ".price"})
print(data)const res = await fetch("https://scrape.land/v1/extract", {
method: "POST",
headers: { "X-Api-Key": "pb_live_YOURKEY", "Content-Type": "application/json" },
body: JSON.stringify({
url: "https://example-product.com/item/42",
fields: { title: "h1", price: ".price" },
}),
});
console.log((await res.json()).data);package main
import ("bytes"; "encoding/json"; "fmt"; "net/http")
func main() {
body, _ := json.Marshal(map[string]any{
"url": "https://example-product.com/item/42",
"fields": map[string]string{"title": "h1", "price": ".price"},
})
req, _ := http.NewRequest("POST", "https://scrape.land/v1/extract", bytes.NewReader(body))
req.Header.Set("X-Api-Key", "pb_live_YOURKEY")
req.Header.Set("Content-Type", "application/json")
resp, _ := http.DefaultClient.Do(req)
defer resp.Body.Close()
var out map[string]any
json.NewDecoder(resp.Body).Decode(&out)
fmt.Println(out["data"])
}$ch = curl_init("https://scrape.land/v1/extract");
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_HTTPHEADER => ["X-Api-Key: pb_live_YOURKEY", "Content-Type: application/json"],
CURLOPT_POSTFIELDS => json_encode([
"url" => "https://example-product.com/item/42",
"fields" => ["title" => "h1", "price" => ".price"],
]),
CURLOPT_RETURNTRANSFER => true,
]);
$data = json_decode(curl_exec($ch), true)["data"];Python SDK & migrating from Zyte
Our Python client wraps both the extraction API and the raw tunnel, and ships a drop-in adapter for the Zyte API client. Already on Zyte? Change the import and your key — the query and response shapes match.
pip install scrapelandfrom scrapeland import ScrapelandClient
client = ScrapelandClient("pb_live_YOURKEY") # or $SCRAPELAND_API_KEY
# structured extraction:
res = client.extract("https://example-product.com/item/42", {"title": "h1"})
# raw tunnel fetch through a fresh exit IP:
r = client.get("https://api.ipify.org", country="us", session="job42")
# or hand the tunnel mapping to your existing requests/httpx code:
import requests
requests.get("https://api.ipify.org", proxies=client.proxies(country="de"))from scrapeland.zyte import ZyteAPI — url, httpResponseBody (base64), httpResponseHeaders, customHttpRequestHeaders, geolocation→exit country, sessionContext→sticky session, and iter()/AsyncZyteAPI all map. browserHtml/screenshot/auto-extraction aren't available through the raw tunnel — use the extraction API with render instead.Raw tunnel (advanced)
Most people want structured data and should use the extraction API above. If instead you need raw HTTP access to the page yourself, the raw tunnel is a forward proxy: point any HTTP client at it with your API key as the username, and every request exits from a fresh IP, with optional country, session, and protocol controls. Host: gateway.scrape.land:8080.
curl -x http://pb_live_YOURKEY:@gateway.scrape.land:8080 https://api.ipify.org
# -> an exit IP, different on each requestgateway.scrape.land:443, which corporate, university and hotel networks rarely filter. Identical behaviour and billing — only the port differs. Keep the http:// scheme: your key still travels in the username, exactly as on 8080.Control parameters
Append -name-value pairs to the key in the username, or send X-Proxy-* control headers.
| Goal | Username form | Header |
|---|---|---|
| Exit country | KEY-country-us | X-Proxy-Country: us |
| Sticky session (same IP) | KEY-session-abc123 | X-Proxy-Session: abc123 |
| Protocol | KEY-protocol-socks5 | X-Proxy-Protocol: socks5 |
| Anonymity (premium) | KEY-anonymity-elite | X-Proxy-Anonymity: elite |
| Max latency (ms) | KEY-maxlatency-3000 | X-Proxy-Max-Latency: 3000 |
Tunnel URL builder
Pick your options and copy the ready-to-run command. Replace YOURKEY with a key from your dashboard.
Using the tunnel from code
import requests
KEY = "pb_live_YOURKEY"
proxy = f"http://{KEY}-country-us:@gateway.scrape.land:8080"
r = requests.get("https://api.ipify.org?format=json",
proxies={"http": proxy, "https": proxy}, timeout=30)
print(r.json()) # {'ip': '...'} a US exit IPimport { HttpsProxyAgent } from "https-proxy-agent";
const KEY = "pb_live_YOURKEY";
const agent = new HttpsProxyAgent(`http://${KEY}-country-gb:@gateway.scrape.land:8080`);
const res = await fetch("https://api.ipify.org?format=json", { agent });
console.log(await res.json());p, _ := url.Parse("http://pb_live_YOURKEY-country-us:@gateway.scrape.land:8080")
client := &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(p)}}
resp, _ := client.Get("https://api.ipify.org")import { chromium } from "playwright";
const browser = await chromium.launch({
proxy: {
server: "http://gateway.scrape.land:8080",
username: "pb_live_YOURKEY-country-us",
password: "",
},
});
const page = await browser.newPage();
await page.goto("https://api.ipify.org");# settings.py
DOWNLOADER_MIDDLEWARES = {
"scrapeland.scrapy.ScrapelandMiddleware": 740,
}
SCRAPELAND_API_KEY = "pb_live_YOURKEY"
SCRAPELAND_COUNTRY = "us" # optional default for every request
# per-request override:
yield scrapy.Request(url, meta={"scrapeland": {"country": "de", "session": "job42"}})Errors
| Status | Meaning |
|---|---|
401 | Missing/invalid API key on the extraction API. Set X-Api-Key. |
407 | Missing/invalid API key on the raw tunnel. Set it as the tunnel username. |
402 | Quota exhausted. Top up your balance or upgrade. |
429 | Rate limit for your plan exceeded (see Plan limits). A Retry-After header says when to try again. |
403 | Destination not allowed (blocked by the SSRF/destination guard), or AI extraction requested on a plan below Scale. The message says which. |
413 | Request body, or the response, exceeds the size limit for your plan. The message names the size, the limit and the plan; see limits.max_response_bytes. |
502 | No working upstream matched your filters (try fewer constraints). |
503 | We're momentarily at capacity (most often every browser slot is busy on a render). Retry after the Retry-After header — this is not a failed request and is not billed. |
code, the plan you are on, and what to do about it:
{
"error": "AI extraction (prompt, schema, extract_type) is not included on the \"free\" plan — it needs \"scale\" or above. Upgrade on the Plans page of your dashboard, or use CSS/XPath \"fields\" extraction, which every plan includes.",
"code": "plan-upgrade-required",
"plan": "free",
"required_plan": "scale",
"fallback": "use CSS/XPath \"fields\" extraction, which every plan includes",
"capability": "AI extraction (prompt, schema, extract_type)"
}plan-upgrade-required (403), quota-exhausted (402), key-budget-exhausted (402), rate-limited (429), response-too-large (413) and email-unverified (403 — the free allowance is held until the account’s email address is confirmed; paid plans never see it). required_plan is present only when buying something actually lifts the limit — the format=raw ceiling, for instance, is the same on every plan, so it is omitted there rather than dangling an upgrade that changes nothing. fallback is always present.
The raw tunnel is an HTTP proxy and answers in text/plain (a JSON body would land inside your tunnel), so it carries the same code in the X-Scrapeland-Error response header instead. Same vocabulary, both surfaces — entitlements.refusal_codes in GET /v1/capabilities lists them.The X-Proxy-Country response header reports the exit country used. Target-site statuses (the site's own 200/403/404) are passed through unchanged.
Plan limits
Four things change with your plan: how fast you may send requests, how large a single response may be, whether AI extraction is available, and whether you get priority exits. They apply identically to the extraction API and the raw tunnel. GET /v1/capabilities with your key returns your own entitlements under entitlements, and GET /v1/account your own numbers.
| Plan | Included / month | Rate limit | Max response | Priority exits | AI extraction |
|---|---|---|---|---|---|
| Free | 1,000 | 5 req/s | 2 MB | no — high-latency pool | no |
| Starter — $99 | 600,000 | 50 req/s | 2 MB | yes | no |
| Growth — $249 | 1,600,000 | 100 req/s | 5 MB | yes | no |
| Scale — $499 | 3,400,000 | 200 req/s | 10 MB | yes | yes |
| Business — $999 | 7,200,000 | 1,000 req/s | 25 MB | yes | yes |
| Business XL — $1,999 | 15,000,000 | 1,000 req/s | 25 MB | yes | yes |
| Business XXL — $3,499 | 30,000,000 | 1,000 req/s | 25 MB | yes | yes |
| Business Ultra — $9,999 | 100,000,000 | 1,000 req/s | 25 MB | yes | yes |
| Pay as you go | Coming in 2027. Not sold as a plan today — start on Free or pick a subscription. Prepaid top-ups are already available on every plan. | ||||
Rate limit. Sustained requests per second, enforced per account across every key you own (a per-key limit would be no limit at all — keys are free to mint); an individual key can be throttled below it. Short bursts above it are tolerated. Over it you get a 429 with a Retry-After header — back off and continue, nothing is lost and nothing is billed. Each plan's rate is sized so a flat-out run against your whole monthly allowance lasts hours rather than minutes.
Max response. The largest single response we will return, exposed as limits.max_response_bytes in GET /v1/capabilities. Real pages are nowhere near it — the median response we serve is about 40 KB and 99% are under 1.2 MB — so it only catches a video, an archive or a dataset dump. Anything larger is refused whole with a 413 that names the size, the limit and your plan. We never return a truncated page, and a refused response is not billed.
Billing never exceeds what you've paid. Included requests are spent first. Past them, requests draw on prepaid credit at $0.20 / 1,000 — the balance counts down and floors at zero, at which point requests return 402 instead of continuing. There is no invoice after the fact and no way to run up a charge you did not authorise in advance.
Guides
Rotating vs sticky sessions
By default every request exits from a fresh IP (great for spreading load and avoiding rate limits). When you need the same IP across requests (logins, carts, multi-step flows) add a session param (or -session-NAME on the tunnel); we pin that session to one exit IP for about 10 minutes, then it rotates. Use a unique session name per logical user.
Handling rate limits & errors
- Target returns 429/403: retry. A new request rotates to a fresh IP. Add a small backoff.
- 402 from us: your quota/balance is exhausted. Upgrade or top up.
- 401/407 from us: the API key is missing or wrong (header for the API, username for the tunnel).
- 502 from us: no upstream matched your filters. Loosen
country/maxlatencyand retry.
import requests, time
KEY = "pb_live_YOURKEY"
def get(url, tries=4):
for i in range(tries):
proxy = f"http://{KEY}:@gateway.scrape.land:8080" # fresh IP each try
try:
r = requests.get(url, proxies={"http": proxy, "https": proxy}, timeout=30)
if r.status_code < 400:
return r
except requests.RequestException:
pass
time.sleep(2 ** i) # exponential backoff
raise RuntimeError("all retries failed")FAQ
Is there a free tier?
Yes — 1,000 requests, no card required. Paid plans start at $99/mo (600k requests = $0.165 / 1k) and the rate improves as the tier grows: $0.139 / 1k on Business ($999 / 7.2M) and $0.100 / 1k on Business Ultra ($9,999 / 100M). See Plan limits for the full ladder.
How am I billed?
Per 1,000 delivered requests. If a proxy is blocked or returns a 403/429, we retry another IP and never bill for it. You only pay once the page comes back.
Your plan's included requests are spent first. Past them, requests come out of prepaid credit at $0.20 / 1,000 — you top up in advance, the balance counts down, and it floors at zero. There is no invoice after the fact and no way to run up a bill you didn't authorise. Without a subscription, pay-as-you-go is $0.1165 / 1,000, the cheapest rate we sell.
Can I target a specific country?
Yes. On the extraction API add "country": "us" to the request body. On the raw tunnel add -country-us (any ISO code) to your key. See the tunnel URL builder.
How do I scrape pages that need JavaScript?
Add "render": true to your extract request, and "wait_for" with a CSS selector if content loads late. See render & wait_for.
Do you offer residential IPs?
Today the pool is rotating datacenter/ISP-grade IPs, self-validated continuously. Residential/mobile pools are on the roadmap (premium plans).
Is this legal?
These are standard tools; you're responsible for using them lawfully and respecting destination sites' terms. See our Terms.