Docs · Fetch a page (plain request)
Fetch a page (plain request)
A page from one plain request, without running JavaScript. The cheapest step. 1 credit.
https://api.lurkapi.com /v1/web/fetchWhen to use this
Pass a url and get the page as the site sends it, before any JavaScript runs. We request it the way a browser would. Most sites answer in 1–3 seconds, so start here. For pages built with JavaScript, use render.
If our browser got past a site's bot wall recently, this request reuses that pass, so it often works on walled sites too.
Price. Always 1 credit, and only when the page comes back. If the site blocks this step or can't be reached, the call fails with 502 upstream_error and costs nothing. Then /v1/web/scrape (Auto) can try the other steps for you, or /v1/web/custom the ones you list.
For AI models, use format=markdown, and add main=true to keep only the page's own content. Pages over 2 MB are cut off (truncated: true). The web scraping guide explains each step.
Example request
Copy one and replace YOUR_API_KEY with your API key (get one free). The code versions read the key from the LURKAPI_API_KEY environment variable instead.
curl "https://api.lurkapi.com/v1/web/fetch?url=https%3A%2F%2Fbooks.toscrape.com%2F&format=markdown" \
-H "x-api-key: YOUR_API_KEY"Paste it into a terminal, with your key in place of YOUR_API_KEY.
const params = new URLSearchParams({
url: "https://books.toscrape.com/",
format: "markdown",
});
const res = await fetch(`https://api.lurkapi.com/v1/web/fetch?${params}`, {
headers: { "x-api-key": process.env.LURKAPI_API_KEY },
});
const data = await res.json();
if (!data.success) throw new Error(`${data.code}: ${data.error}`);
console.log(data.content);Node 18+. Save it as lurk.mjs and run node lurk.mjs. It reads your key from LURKAPI_API_KEY.
import os
import requests
res = requests.get(
"https://api.lurkapi.com/v1/web/fetch",
params={
"url": "https://books.toscrape.com/",
"format": "markdown",
},
headers={"x-api-key": os.environ["LURKAPI_API_KEY"]},
timeout=95,
)
data = res.json()
if not data["success"]:
raise RuntimeError(f"{data['code']}: {data['error']}")
print(data["content"])Run pip install requests, save it as lurk.py and run python lurk.py. It reads your key from LURKAPI_API_KEY.
package main
import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"time"
)
func main() {
params := url.Values{}
params.Set("url", "https://books.toscrape.com/")
params.Set("format", "markdown")
req, _ := http.NewRequest("GET", "https://api.lurkapi.com/v1/web/fetch?"+params.Encode(), nil)
req.Header.Set("x-api-key", os.Getenv("LURKAPI_API_KEY"))
client := &http.Client{Timeout: 95 * time.Second}
res, err := client.Do(req)
if err != nil {
panic(err)
}
defer res.Body.Close()
var data map[string]any
if err := json.NewDecoder(res.Body).Decode(&data); err != nil {
panic(err)
}
if data["success"] != true {
panic(fmt.Sprintf("%v: %v", data["code"], data["error"]))
}
fmt.Println(data["content"])
}Save it as main.go and run go run main.go. It reads your key from LURKAPI_API_KEY.
<?php
$params = http_build_query([
'url' => 'https://books.toscrape.com/',
'format' => 'markdown',
]);
$context = stream_context_create(['http' => [
'header' => 'x-api-key: ' . getenv('LURKAPI_API_KEY'),
'timeout' => 95,
'ignore_errors' => true,
]]);
$body = file_get_contents('https://api.lurkapi.com/v1/web/fetch?' . $params, false, $context);
$data = json_decode($body, true);
if (!$data['success']) {
throw new Exception("{$data['code']}: {$data['error']}");
}
print_r($data['content']);Save it as lurk.php and run php lurk.php. It reads your key from LURKAPI_API_KEY.
require 'json'
require 'net/http'
uri = URI('https://api.lurkapi.com/v1/web/fetch')
uri.query = URI.encode_www_form(
'url' => 'https://books.toscrape.com/',
'format' => 'markdown',
)
req = Net::HTTP::Get.new(uri)
req['x-api-key'] = ENV.fetch('LURKAPI_API_KEY')
res = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https', read_timeout: 95) do |http|
http.request(req)
end
data = JSON.parse(res.body)
raise "#{data['code']}: #{data['error']}" unless data['success']
puts data['content']Save it as lurk.rb and run ruby lurk.rb. It reads your key from LURKAPI_API_KEY.
Parameters
Add them to the URL after a ?, as in ?format=markdown. Join several with &.
| Parameter | Description |
|---|---|
urlstringrequired | The page to fetch, as a full URL starting with http:// or https://. It must be a public site on the standard ports (80 or 443).
|
formatstringoptional | How content comes back: html, markdown (best for AI models) or plain text. JSON, XML and plain-text pages come back unchanged.
|
mainbooleanoptional | true keeps only the page's own content: its <h1> and its <main> element (or its articles if it has no <main>, else the whole page). Navigation, sidebars, headers, footers, hidden parts, skip links and cookie banners are left out. Works with format=markdown or text; format=html ignores it.
|
Example response
A real response, captured from the live API. Lists are trimmed to a couple of items, and long text such as media links is cut short with ….
Show the example response (1 KB)Hide
{
"success": true,
"credits_remaining": 9999,
"credits_charged": 1,
"status": 200,
"url": "https://books.toscrape.com/",
"title": "All products | Books to Scrape - Sandbox",
"content_type": "text/html",
"content": "[Books to Scrape](https://books.toscrape.com/index.html) We love being s…",
"truncated": false,
"via": "direct",
"captcha": null,
"attempts": [
{
"via": "direct",
"ms": 536,
"status": 200,
"wall": null,
"error": null
}
],
"billing": {
"mode": "fixed",
"credits": 1,
"breakdown": {
"base": 1
},
"note": null
}
}Response fields
Every field in a successful response, and what it means. A name with [] belongs to each item of a list: attempts[].via is the via of every item in attempts. null means there's no value.
Show all 22 fieldsHide
| Field | Type | Description |
|---|---|---|
| success | true | Always true here; errors have success: false. |
| credits_remaining | number | Your balance after this call. On anonymous playground calls: free tries left today. |
| credits_charged | number | Credits this call cost; 0 on free endpoints. On anonymous playground calls: tries used (1). |
| status | integer | The HTTP status the site gave the page: 200, or its own 404, 410 and so on. |
| url | string | The page's URL after any redirects. When via is unblocker, it's the URL you asked for, since that step doesn't report redirects. |
| title | stringcan be null | The page's title, from its <title>; null when it has none or isn't HTML. |
| content_type | stringcan be null | The page's Content-Type, like text/html; charset=utf-8; null if the site sent none. |
| content | stringcan be null | The page in the format you asked for: HTML, markdown or plain text. Null for binary files such as images, PDFs and archives. |
| truncated | boolean | The page was over 2 MB, so it was cut off there. |
| via | string | The step that got the page: direct or static (plain requests), browser (our browser), unblocker (a browser on separate IPs), residential (a residential IP) or browser_residential (our browser on a residential IP). |
| captcha | stringcan be null | The captcha solved to get the page (turnstile, datadome or recaptcha), which is charged. Null if none was solved. |
| attempts | object[] | Every step tried, in order. The last one got the page. |
| attempts[].via | string | The step tried. |
| attempts[].ms | integer | How long it took, in milliseconds. |
| attempts[].status | integercan be null | The HTTP status it got; null if no answer came back. |
| attempts[].wall | stringcan be null | What blocked it, like cloudflare challenge or datadome block; null if nothing did. |
| attempts[].error | stringcan be null | Why it failed when no answer came back; null if one did. |
| billing | object | What this call cost and why. |
| billing.mode | string | fixed: this endpoint's set price. auto and custom: the price of the step that got the page. |
| billing.credits | integer | What this call cost, the sum of breakdown. |
| billing.breakdown | object | Credits by what they paid for: base, plus on Auto or Custom unblock (a browser on separate IPs), premium (a residential IP) or premium_render (a browser on a residential IP), and captcha when one was solved. |
| billing.note | stringcan be null | On Auto and Custom, why the price varies; null on fixed-price endpoints. |
Cost and freshness
Each call costs 1 credit. Calls turned away before they reach the platform (invalid parameters, a key or balance problem, a rate limit) are free, and so are 5xx errors. A not_found is charged, since the platform was checked. How charging works.
Not cached. Every call fetches fresh data.
Errors
A failed call returns success: false, an error message, a code your program can check, and a docs link. These are the errors this endpoint can return. All error codes.
| Status | Code | Meaning |
|---|---|---|
| 401 | missing_api_key | No API key was sent. Put it in the x-api-key header. |
| 401 | invalid_api_key | This key doesn't exist or was revoked. |
| 402 | insufficient_credits | Not enough credits for this call. Buy a pack, or wait for tomorrow's free top-up. |
| 402 | over_max_credits | This call would cost more than the max_credits you set. The error says what it needs; raise max_credits to allow it. Not charged. |
| 403 | account_suspended | This account is suspended. Email support@lurkapi.com. Not charged. |
| 405 | method_not_allowed | Endpoints only take GET, with parameters in the URL. |
| 429 | rate_limited | Too many calls in a short time. Wait as many seconds as the retry-after header says, then retry. |
| 500 | internal_error | Something broke on our side. Retry. Not charged. |
| 502 | upstream_error | The platform didn't send a usable answer. Retry. Not charged. |
| 503 | upstream_busy | All our connections to the platform are busy. Retry in a few seconds. Not charged. |
| 504 | upstream_timeout | The platform took over 30 seconds to answer (90 for web pages). Retry. Not charged. |
Use it in Claude
Connect LurkAPI to Claude and ask in plain English. This endpoint is the web_fetch tool. For example:
Use LurkAPI's web_fetch tool with url "https://books.toscrape.com/", format "markdown", then summarize what you find.Claude calls web_fetch with arguments like these, and each call costs 1 credit:
{"url": "https://books.toscrape.com/","format": "markdown"}To save tokens, tool results leave out empty fields, keep one URL per image or video, and give transcripts as plain text unless you pass timestamps=true. A result over 45,000 characters comes in parts. Its _note gives the offset for the next part, and later parts are free for 15 minutes. Pass trim=false for the full response.
From the blog
All posts
· 5 min read
Reddit RSS feeds end November 13: how to keep yours working
Reddit turns off every RSS feed on November 13, 2026. Keep your feed URLs: a small Cloudflare Worker serves the same paths from LurkAPI. Change only the host.

· 4 min read
Best Facebook scrapers in 2026: prices and speed compared
LurkAPI is the best Facebook scraper: public Pages' posts at $0.19 per 1,000, about 1 s a call, with contacts, comments and events on the same key.

· 4 min read
TikTok Research API: who can get it, and what businesses use instead
Non-commercial researchers in the US, Europe, Canada and Brazil, after a review of about 4 weeks. Businesses can’t apply; they use a public-data API instead.