# AX Check: firecrawl.dev
Checked 2026-10-12.

Firecrawl's docs, pricing, and setup paths are all clear
21 of 23 checklist items passed: quickstarts, CLI/SDK/MCP install paths, and pricing all work without login. Only the homepage lacks a Markdown response.

## Onboarding needs a login

## Coding sessions
All three independent sessions completed the task. Two were reviewed on pricing and both correctly pulled tiered, credit-based pricing (Free, Hobby, Standard, Growth) straight from Firecrawl's pricing page, with one noting figures could shift and to check the live page.

### DeepSeek V4.1 Flash
[View public run](https://agents.withgauge.com/p/runs/3ec05121-6a8a-4a2a-96ca-759fb98527c1) · [Read transcript](https://www.ax-check.com/firecrawl.dev/sessions/deepseek.json)
Pricing was given with specific tiers, credit costs, and billing assumptions (free, Hobby, Standard, pay-as-you-go) pulled from Firecrawl's actual pricing page.

Evidence: README.md and final output list Free/Hobby $19 (or $16 annual)/Standard ~$83 tiers with credit-per-page assumptions, sourced from fetches of firecrawl.dev/pricing at seq 19, 29, 35.
#### End-to-end onboarding
- **Onboarding needs a login**: The agent never obtained a real Firecrawl API key. It searched the sandbox environment and filesystem for an existing key, found none, then built a working example script and SDK wiring. It only tested the script with no key (clean failure) and with a deliberately invalid dummy key 'fc-invalid-test-key', which correctly returned a 401 Unauthorized from Firecrawl's real API. This proves the HTTP wiring reaches the hosted API, but no authenticated operation was ever completed because no valid credential was available or obtainable without human intervention.
  Event 5:

  ```text
  PI_GATEWAY_API_KEY=<redacted>
  ```
  Event 95:

  ```text
  Firecrawl request failed: Unauthorized: Invalid token
  status: 401
  exit=1
  ```
  Event 97:

  ```text
  The only blocker is a valid `FIRECRAWL_API_KEY` — Firecrawl is a fully authenticated hosted API with no anonymous tier, and none is present in this environment.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No Firecrawl API key available in the sandbox**: Firecrawl requires an authenticated API key for every request (no anonymous tier). The sandbox environment only exposed an internal gateway key (PI_GATEWAY_API_KEY), not a Firecrawl credential, and no human supplied one during the session. This is a missing-credentials limitation of the test environment, not a product defect — the agent correctly identified it and stopped short of claiming false success.
  Event 5:

  ```text
  PI_GATEWAY_API_KEY=<redacted>
  ```
  Event 97:

  ```text
  Firecrawl is a fully authenticated hosted API with no anonymous tier, and none is present in this environment.
  ```

#### Suggested Changes
- **Add a free, key-free sandbox/demo endpoint for first-time evaluation**: The agent could not exercise the SDK end-to-end because every endpoint, including simple scrape/search, requires a paid-tier or free-tier API key with no anonymous trial path. Verify by having a fresh script call scrape() or search() with zero setup (no signup) and confirm it returns real data rather than a 401.
  Event 95:

  ```text
  Firecrawl request failed: Unauthorized: Invalid token
  status: 401
  exit=1
  ```
  Event 97:

  ```text
  no anonymous tier, and none is present in this environment
  ```

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/af72dd2a-ba8d-4592-b072-aa36c0b5a319) · [Read transcript](https://www.ax-check.com/firecrawl.dev/sessions/kimi.json)
Pricing was given as a tiered, credit-based breakdown (Free/Hobby/Standard/Growth) with the caveat that figures may be outdated and to check the live pricing page.

Evidence: get_final_output lists plan tiers with credit assumptions and a 'pricing changes, check link' caveat, though no web fetch was performed to verify the numbers (get_web_activity returned empty), so the figures rely on model memory rather than live confirmation.
#### End-to-end onboarding
- **Onboarding needs a login**: The agent never obtained a Firecrawl API key on its own. It checked the environment for credentials, found none, and the example script exited with an explicit error asking for FIRECRAWL_API_KEY. It then tested the script with a dummy key and got a 401 Unauthorized, confirming no authenticated operation was ever completed against the hosted API.
  Event 3:

  ```text
  env | grep -iE 'firecrawl|api_key'
  ```
  Event 19:

  ```text
  FIRECRAWL_API_KEY is not set. Get a key at https://firecrawl.dev and run: export FIRECRAWL_API_KEY=fc-...
  
  
  Command exited with code 1
  ```
  Event 77:

  ```text
  requests.exceptions.HTTPError: Unexpected error during scrape URL: Status code 401. Unauthorized: Invalid token - No additional error details provided.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No Firecrawl API key available in sandbox**: The sandbox environment had no FIRECRAWL_API_KEY set, and the agent has no self-service way to obtain one (sign-up is an external, human-gated step for this product). This is a missing-credentials limitation of the test environment, not a product defect: the SDK behaved correctly, failing with a clear 401 on an invalid key and a clear exit message when no key was present at all.
  Event 19:

  ```text
  FIRECRAWL_API_KEY is not set. Get a key at https://firecrawl.dev and run: export FIRECRAWL_API_KEY=fc-...
  ```
  Event 77:

  ```text
  requests.exceptions.HTTPError: Unexpected error during scrape URL: Status code 401. Unauthorized: Invalid token - No additional error details provided.
  ```
- **Installed SDK version's top-level API diverged from documented usage**: Agent error / SDK version mismatch: the agent first wrote code against FirecrawlApp.scrape_url, which is the commonly documented pattern, but the installed firecrawl-py v4.50.0 package's top-level FirecrawlApp class only exposes unrelated methods (parse, search_github, etc.), forcing multiple steps inspecting package internals to find the working firecrawl.v1.V1FirecrawlApp.scrape_url interface.
  Event 24:

  ```text
  AttributeError: type object 'Firecrawl' has no attribute 'scrape_url'
  ```
  Event 51:

  ```text
  ImportError: cannot import name 'FirecrawlApp' from 'firecrawl.v1' (/opt/freestyle/python/lib/python3.12/site-packages/firecrawl/v1/__init__.py). Did you mean: 'V1FirecrawlApp'?
  ```

#### Suggested Changes
- **Align top-level FirecrawlApp export with scrape_url usage**: In firecrawl-py 4.50.0 the top-level firecrawl.FirecrawlApp class does not implement scrape_url; only firecrawl.v1.V1FirecrawlApp does. A developer following the common FirecrawlApp(api_key=...).scrape_url(url) pattern hits an AttributeError immediately. Either restore scrape_url/crawl_url on the top-level class or update quickstart docs to reference firecrawl.v1.V1FirecrawlApp directly. Verify by running FirecrawlApp(api_key=...).scrape_url(url) and confirming no AttributeError.
  Event 24:

  ```text
  AttributeError: type object 'Firecrawl' has no attribute 'scrape_url'
  ```

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/6629ddc6-6093-4e81-a9c9-138bd87ed2bb) · [Read transcript](https://www.ax-check.com/firecrawl.dev/sessions/qwen.json)

#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: No real Firecrawl API key was ever obtained during the session. The agent found that the hosted API and the v2 SDK client both accept anonymous (keyless) requests on a rate-limited free tier, and used that keyless path to run a scrape operation successfully. Since no credentials were acquired or used, this does not meet the bar for verified self-service onboarding (which requires obtaining real credentials and using them), nor is it a login_required case since no blocker halted progress — the keyless path let the agent proceed anyway.
  Event 66:

  ```text
  === Scrape result (first 500 chars) ===
  |     |     |     |
  | --- | --- | --- |
  ...
  ```
  Event 59:

  ```text
  There's a v2-only client that supports the keyless free tier. Let me update the example to use it:
  ```
  Event 31:

  ```text
  raise ValueError('No API key provided')
  ValueError: No API key provided
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **Top-level SDK client requires a key despite API allowing anonymous use**: The main `Firecrawl` class (v1 compatibility shim) raises ValueError and refuses to run without an API key, even though the raw hosted API and the v2 FirecrawlClient both accept anonymous requests. This is a product/SDK inconsistency that forced the agent to dig into SDK internals and switch to a different client class rather than a credential or environment issue.
  Event 31:

  ```text
  raise ValueError('No API key provided')
  ValueError: No API key provided
  ```
  Event 41:

  ```text
  firecrawl.v2.utils.error_handler.UnauthorizedError: Unauthorized: Failed to scrape. Unauthorized: Invalid token - No additional error details provided.
  ```

#### Suggested Changes
- **Align the top-level Firecrawl client's key requirement with the v2 client's keyless support**: In firecrawl-py, the top-level `Firecrawl` class instantiates a v1 shim (`V1FirecrawlApp`) that hard-fails with 'No API key provided' when no key is set, while `firecrawl.v2.FirecrawlClient` happily runs keyless requests against the same hosted API. Either make the top-level class consistent with v2's keyless behavior or document the discrepancy in the SDK README/quickstart so new developers don't have to read client.py source to find the working client.
  Event 31:

  ```text
  File "/opt/freestyle/python/lib/python3.12/site-packages/firecrawl/v1/client.py", line 542, in __init__
      raise ValueError('No API key provided')
  ```
  Event 49:

  ```text
  # No API key is allowed: scrape, search, and interact fall back to the
      # keyless free tier (rate-limited per IP). The HTTP client only sends an
      # Authorization header when a key is present, and other methods return
      # 401 from the API until a key is provided.
  ```

### Task given to each agent
Help me build a simple example using Firecrawl. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: B · 84/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — Homepage answers Markdown requests

  ```text
  Homepage returned text/html even when text/markdown was requested; no Markdown representation served.
  ```

- **Pass** — llms.txt provides an actionable documentation index

  ```text
  llms.txt links CLI, MCP, skills, API reference, SDKs, quickstarts and docs index.
  ```

- **Pass** — llms.txt provides navigation guidance

  ```text
  llms.txt groups links under CLI, MCP, Skills, Docs, SDKs, Get started headings.
  ```

- **Pass** — llms.txt mentions offered API, MCP, and skills

  ```text
  llms.txt mentions CLI, MCP Server docs, and Firecrawl skills with links.
  ```

- **Pass** — A compact guide representation exists

  ```text
  SKILL.md is a standalone Markdown guide served at the site root and /agent-onboarding/.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  SKILL.md fetched directly with 200 and full agent onboarding workflow.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  SKILL.md is 6056 tokens, well under the 8000-token budget.
  ```

- **Skipped** — Product-docs links survive format changes

  ```text
  Homepage Markdown unsupported, so link preservation across formats cannot be judged.
  ```

- **Pass** — The compact guide is independently actionable

  ```text
  SKILL.md gives concrete install, credential, and verification steps across seven actionable paths.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Fetched Go and Rust quickstarts and API reference all returned HTTP 200.
  ```


### Onboarding
- **Pass** — Docs lead to a relevant quickstart

  ```text
  llms.txt links Go and Rust quickstarts with prerequisites, install, and runnable code.
  ```

- **Pass** — Installation commands are extractable

  ```text
  Rust quickstart shows Cargo.toml dependency; CLI docs show npm install and npx commands.
  ```

- **Pass** — Code examples are available without interaction

  ```text
  Go and Rust quickstarts include full code examples and sample responses inline.
  ```

- **Pass** — Prerequisites and auth boundaries are explicit

  ```text
  Quickstarts list language version prerequisites and API key with a free signup link.
  ```


### Pricing
- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders full plan tables and credit costs as static Markdown, no interaction needed.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  All plan prices stated: Free $0, Hobby $19, Standard $99, Growth $399, Scale $749 monthly.
  ```

- **Pass** — Pricing units and limits are explicit

  ```text
  Credits, concurrency, rate limits and per-endpoint credit costs are explicit per plan.
  ```

- **Pass** — Agents identify pricing and its assumptions

  ```text
  2 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Pricing was given with specific tiers, credit costs, and billing assumptions (free, Hobby, Standard, pay-as-you-go) pulled from Firecrawl's actual pricing page.
  
  Evidence: README.md and final output list Free/Hobby $19 (or $16 annual)/Standard ~$83 tiers with credit-per-page assumptions, sourced from fetches of firecrawl.dev/pricing at seq 19, 29, 35. Kimi K3: Pricing was given as a tiered, credit-based breakdown (Free/Hobby/Standard/Growth) with the caveat that figures may be outdated and to check the live pricing page.
  
  Evidence: get_final_output lists plan tiers with credit assumptions and a 'pricing changes, check link' caveat, though no web fetch was performed to verify the numbers (get_web_activity returned empty), so the figures rely on model memory rather than live confirmation. This behavioural item does not affect the fast grade.
  ```


### Activation
- **Pass** — An API reference or OpenAPI spec is reachable

  ```text
  API reference and OpenAPI specs reachable at docs.firecrawl.dev/api-reference/v2-introduction and endpoint pages.
  ```

- **Pass** — An MCP server is documented and well-formed

  ```text
  MCP server documented with keyless, OAuth, and API-key setup plus client configs.
  ```

- **Pass** — A CLI install path is documented

  ```text
  CLI install documented via npx firecrawl-cli and global npm install.
  ```

- **Pass** — SDK packages resolve on their registries

  ```text
  Registry lookups for firecrawl-py, firecrawl, and firecrawl-cli all returned HTTP 200.
  ```

- **Pass** — Agent skills are published

  ```text
  Agent skills published at firecrawl.dev/SKILL.md and agent-onboarding/SKILL.md.
  ```



[Full report data](https://www.ax-check.com/firecrawl.dev/report.json)