Firecrawl
In short
Firecrawl is a web scraping API that turns any URL, including JavaScript-heavy pages and PDFs, into clean markdown for AI agents, with an open-source script that builds llms.txt and llms-full.txt for a whole site. Firecrawl /scrape detects PDF and DOCX files from their URLs and returns them as markdown2.
At a glance
- Type
- Markdown tooling
- Operator
- Firecrawl
- Home page
- firecrawl.dev
- Pricing
- usage for 1 of 3 products; no published price for the other 2
- Last read
- 2026-11-05
What it does for agent readiness
- Firecrawl /scrape returns a page as clean markdown, structured data, screenshots or HTML, and its docs say it handles JS-heavy pages and dynamic apps.2
- Firecrawl marks its hosted llms.txt generator API at llmstxt.firecrawl.dev as deprecated and says it stopped maintaining it after 30 June 2025.3
- Firecrawl points users of llms.txt generation to the create-llmstxt-py repository, a Python script that builds llms.txt and llms-full.txt with Firecrawl and OpenAI APIs.4
- Firecrawl create-llmstxt-py discovers every URL on a site with the Firecrawl map endpoint before writing the two files.4
- Firecrawl gives a free tier of 1,000 credits every month with no credit card.5
- Firecrawl Hobby costs 19 USD a month billed monthly, or 16 USD a month billed annually, for 5,000 credits.5
- Firecrawl Standard costs 99 USD a month (83 USD billed annually) for 100,000 credits, and Growth 399 USD a month (333 USD annually) for 500,000 credits.5
- Firecrawl does not charge for a scrape that returns no result but charges 1 credit for a page that answers 403 or 404.5
- Firecrawl runs an MCP server at mcp.firecrawl.dev that accepts keyless or OAuth connections as well as API keys.6
Products and plans
| Product | What it does | Price | As of | Note |
|---|---|---|---|---|
| Scrape API | Converts one URL to markdown or structured data. | usage, from 0 USD 5 | 2026-11-05 | Free tier 1,000 credits a month; Hobby from 16 USD a month billed annually. |
| create-llmstxt-py | Open-source script that writes llms.txt and llms-full.txt for a site using Firecrawl map and scrape. | no published price | ||
| Firecrawl MCP server | MCP access to search and scrape for coding agents. | no published price |
Standards
Our checks
| Check | Domain | Result | Measured on | Method |
|---|---|---|---|---|
| Ora score (cached) | firecrawl.dev | 85, grade B | 2026-11-05 | GET https://ora.ai/api/score/firecrawl.dev (cached result, no scan triggered); measured_on is Ora's stored scannedAt 7 |
| Cloudflare isitagentready scan | firecrawl.dev | 53 percent, 8 of 15 checks, L5 Agent-Native | 2026-11-05 | POST https://isitagentready.com/api/scan with the default All Checks profile (every check except a2aAgentCard and ap2), one request every 3 seconds; percent = passed / counted over the four scored categories, as the page shows it 8 |
What it does not do
- Firecrawl does not change what a site serves to agents; it converts pages on the reader side.
- Firecrawl no longer maintains the hosted llms.txt generator API, so the generator now runs as a script with the user's own API keys.
Sources
- Firecrawl home page. https://www.firecrawl.dev/, read 2026-11-05.
- Firecrawl docs: Scrape. https://docs.firecrawl.dev/features/scrape, read 2026-11-05.
- Firecrawl llms.txt generator (llmstxt.firecrawl.dev). https://llmstxt.firecrawl.dev/, read 2026-11-05.
- GitHub firecrawl/create-llmstxt-py. https://github.com/firecrawl/create-llmstxt-py, read 2026-11-05.
- Firecrawl pricing. https://www.firecrawl.dev/pricing, read 2026-11-05.
- Firecrawl docs: MCP server. https://docs.firecrawl.dev/mcp-server, read 2026-11-05.
- Ora cached score for firecrawl.dev. https://ora.ai/api/score/firecrawl.dev, read 2026-11-05.
- isitagentready.com home page. https://isitagentready.com/, read 2026-11-05.
Last reviewed 2026-11-05