Skip to content

Start a website crawl

A website crawl API starts a crawl that collects content across multiple pages and returns a crawl response. AnyAPI, Context.dev, Firecrawl, Search1API and Spider provide this through Longwave. Published rates start at $0.005 per page (Firecrawl).

Add to Longwave

Start a website crawl with your agent

Paste this into Claude Code or a terminal. Then run /mcp to sign in.

providers
5
lowest price
$0.0013/result

Providers compared

Provider Price Access
AnyAPI anyapi.openapi.post.v1.run.web.crawl $1.25 per 1,000 results billed per result - $1.25 per 1,000 results, capped at $12.50 per 1,000 requests. Your key
Firecrawl firecrawl.web.crawl $0.005 per page Basic operation: 1 credit/page. Smallest published PAYG increment: $5 / 1,000 credits on Hobby; a paid plan is required. JSON, Question and Highlight formats add 4 credits/page. Your key
Search1API search1api.openapi.post.deepcrawl $0.02 per call 20 credits/request. Smallest top-up: $5 / 5,000 credits ($0.001/credit). Search/news add 1 credit per successfully crawled result when crawl_results is enabled. Your key
Context.dev Crawl a website with Context.dev context-dev.openapi.post.web.crawl Brand retrieval costs 10 credits. Monthly plans and plan-dependent overage rates apply; no standalone PAYG pack is published. Your key
Spider Crawl a website spidercloud.crawl Usage-dependent: $1/GB of pre-transformation bandwidth plus $0.0001 per CPU minute. No fixed price per API call or page. Your key

Prices are each vendor's published rate. Functions marked “No key needed” are billed at that rate in Longwave credits, with no markup. Functions marked “Your key” are billed to your own vendor account.

Run it

In Claude Code, Codex, Cursor or ChatGPT, add the Longwave MCP server once, then ask for the result. In Longwave, ask in any thread.

Questions people ask

Which providers can start a website crawl?

Through Longwave: AnyAPI, Firecrawl, Search1API, Context.dev and Spider. The table above compares their prices and access. To use one in particular, name it to your agent or give its function id.

Do I need an API key?

Providers marked "Your key" need your own key, added once in Longwave settings. Providers marked "Connect account" need you to sign in to that account.

Start a website crawl in Longwave

Ask any Longwave thread for the result, or add Longwave to Claude Code, Codex, Cursor or ChatGPT.

Add to Longwave