# AX Check: fireworks.ai
Checked 2026-10-18.

Fireworks.ai quickstart and pricing are usable with no login wall
Quickstart gives runnable code and explicit API key steps; pricing lists per-token and per-GPU rates publicly. 18 of 23 checklist items passed, none failed outright.

## Onboarding needs a login

## Coding sessions
All three independent sessions completed and found pricing without hitting a login wall, quoting per-million-token rates by model size plus GPU and fine-tuning billing, though two noted the figures should be double-checked against the live docs.

### DeepSeek V4.1 Flash
[View public run](https://agents.withgauge.com/p/runs/3d46623e-c095-4114-a6f2-a56f7cdf02f1) · [Read transcript](https://www.ax-check.com/fireworks.ai/sessions/deepseek.json)
Quoted per-1M-token rates from the docs with assumptions: Standard mode, input/cached/output split, batch 50% off, Priority ~1.25x, size tiers. Cost estimate used the real token counts. (seq 74, seq 87, seq 93, README.md)
Session findings unavailable.

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/5b7ebf7d-bd42-47e4-99b1-8e36f438710b) · [Read transcript](https://www.ax-check.com/fireworks.ai/sessions/kimi.json)
Gave per-1M-token prices by model size for serverless, per-GPU-second billing for dedicated deployments, and new-account free credit. It noted the figures were from memory and needed checking against the live pricing page. (seq 19)
#### End-to-end onboarding
- **Onboarding needs a login**: Agent never obtained a Fireworks API key. It confirmed the hosted endpoint is reachable (401 without auth) and installed the SDK, but the script failed immediately at client initialization because no FIREWORKS_API_KEY was present in the sandbox, and the agent had no self-service way to create one. No authenticated product operation was ever completed.
  Event 4:

  ```text
  ModuleNotFoundError: No module named 'fireworks'
  ```
  Event 17:

  ```text
  fireworks.FireworksError: The api_key client option must be set either by passing api_key to the client or by setting the FIREWORKS_API_KEY environment variable
  ```
  Event 19:

  ```text
  there is no `FIREWORKS_API_KEY` in this environment, and the API requires one (I confirmed the endpoint is reachable — it returns `401` without a key, and the SDK raises an auth error on init)
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No API key available to authenticate with Fireworks**: The sandbox environment had no FIREWORKS_API_KEY, and the agent had no mechanism to self-provision one. This is a missing-credentials limitation of the test environment, not a product defect: Fireworks' API correctly requires a key (confirmed via a 401 on the unauthenticated endpoint check) and the agent surfaced this clearly instead of faking success.
  Event 10:

  ```text
  401
  Successfully installed fireworks-ai-1.2.27 httpx-aiohttp-0.2.0
  
  ```
  Event 17:

  ```text
  raise FireworksError(
  fireworks.FireworksError: The api_key client option must be set either by passing api_key to the client or by setting the FIREWORKS_API_KEY environment variable
  ```

#### Suggested Changes
- **Add an API-key link to the SDK's missing-key error message**: When FIREWORKS_API_KEY is unset, the fireworks-ai Python SDK raises a generic FireworksError naming only the env var, forcing the agent to supply the key-creation URL from its own general knowledge rather than from the tool itself. Add a direct link to the key management page inside that error string. Verify by re-running the client with no key set and confirming the error text includes a key-creation link.
  Event 17:

  ```text
  fireworks.FireworksError: The api_key client option must be set either by passing api_key to the client or by setting the FIREWORKS_API_KEY environment variable
  ```

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/928025d5-232d-4044-a4d1-b6fe1f9c0e59) · [Read transcript](https://www.ax-check.com/fireworks.ai/sessions/qwen.json)
Pricing was given as per-model, per-million-token ranges by model size (~$0.10–$1.10/M small, ~$2–$8/M large). It also separated fine-tuning and dedicated GPU billing and said to check model cards. The ranges were not sourced or checked. (seq 50)
#### End-to-end onboarding
- **Onboarding needs a login**: Agent never obtained a real Fireworks API key. It inspected environment variables (finding only gateway/harness credentials unrelated to the Fireworks product), tried an invalid test key against the live API (got 403 as expected), and then wrote example scripts that require a human to export FIREWORKS_API_KEY before they can run. No authenticated product operation was performed.
  Event 4:

  ```text
  ALG_INFERENCE_MODEL=***REDACTED***
  PI_PROVIDER=***REDACTED***
  PI_MODEL=***REDACTED***
  ALG_INFERENCE_PROVIDER=***REDACTED***
  PI_GATEWAY_API_KEY=***REDACTED***
  ```
  Event 30:

  ```text
  HTTP 403
  ```
  Event 50:

  ```text
  Blocked on one thing: there is no `FIREWORKS_API_KEY` in this environment. I confirmed the API endpoint is reachable (invalid-key probe returns 403 as expected), so the examples will work the moment you provide a key from https://fireworks.ai/account/api-keys
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No Fireworks API key available in sandbox**: The sandbox only exposed internal gateway/harness credentials (PI_GATEWAY_API_KEY, ALG_INFERENCE_* vars), not a usable Fireworks product API key. This is a test-environment/credential limitation, not a product defect: the agent correctly identified it needed a real key from the Fireworks dashboard and did not fabricate one. It validated the API was live by sending a deliberately invalid key and observing a 403, then stopped short of live inference.
  Event 4:

  ```text
  PI_GATEWAY_API_KEY=***REDACTED***
  ```
  Event 30:

  ```text
  HTTP 403
  ```
  Event 50:

  ```text
  there is no `FIREWORKS_API_KEY` in this environment
  ```

#### Suggested Changes
None identified in this transcript.

### Task given to each agent
Help me build a simple example using Fireworks AI. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: B · 65/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — Homepage answers Markdown requests

  ```text
  Homepage returned text/html for a text/markdown request; no Markdown representation offered.
  ```

- **Failed** — llms.txt provides an actionable documentation index

  ```text
  https://fireworks.ai/llms.txt returns HTTP 404, so no documentation index exists.
  ```

- **Skipped** — llms.txt provides navigation guidance

  ```text
  No llms.txt body was returned (404), so navigation guidance cannot be judged.
  ```

- **Skipped** — llms.txt mentions offered API, MCP, and skills

  ```text
  No llms.txt body was returned (404), so API/MCP/skills mentions cannot be judged.
  ```

- **Pass** — A compact guide representation exists

  ```text
  Serverless quickstart served as text/markdown at /getting-started/quickstart.md, a standalone Markdown page.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  Quickstart gives concrete steps: create API key, install SDK, send chat completion via curl/Python.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  Markdown quickstart is 7716 tokens, under 8000, and essential workflow stays usable.
  ```

- **Skipped** — Product-docs links survive format changes

  ```text
  Homepage Markdown unsupported, so link preservation across format changes cannot be measured.
  ```

- **Pass** — The compact guide is independently actionable

  ```text
  Serverless quickstart gives API key, install, and runnable curl/Python/JS calls with expected output.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Fetched quickstart, API reference, MCP setup, and skills pages all returned 200.
  ```


### Onboarding
- **Pass** — Docs lead to a relevant quickstart

  ```text
  Docs quickstart pages give concrete first steps: create API key, install SDK, make first call.
  ```

- **Pass** — Installation commands are extractable

  ```text
  Install commands extractable: pip install --pre fireworks-ai, npm install openai, brew install firectl.
  ```

- **Pass** — Code examples are available without interaction

  ```text
  Full code examples in Python, JavaScript and curl appear inline without interaction.
  ```

- **Pass** — Prerequisites and auth boundaries are explicit

  ```text
  Quickstarts state API key creation in dashboard and export FIREWORKS_API_KEY before calls.
  ```


### Pricing
- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders serverless, training, and GPU rates as static text without interaction.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  Concrete per-token, per-GPU-hour, and per-training-token prices are stated publicly.
  ```

- **Pass** — Pricing units and limits are explicit

  ```text
  Units are explicit: USD per 1M tokens, per GPU minute/hour, per 1M training tokens.
  ```

- **Pass** — Agents identify pricing and its assumptions

  ```text
  3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Quoted per-1M-token rates from the docs with assumptions: Standard mode, input/cached/output split, batch 50% off, Priority ~1.25x, size tiers. Cost estimate used the real token counts. (seq 74, seq 87, seq 93, README.md) Kimi K3: Gave per-1M-token prices by model size for serverless, per-GPU-second billing for dedicated deployments, and new-account free credit. It noted the figures were from memory and needed checking against the live pricing page. (seq 19) Qwen 3.8 Max: Pricing was given as per-model, per-million-token ranges by model size (~$0.10–$1.10/M small, ~$2–$8/M large). It also separated fine-tuning and dedicated GPU billing and said to check model cards. The ranges were not sourced or checked. (seq 50) This behavioural item does not affect the fast grade.
  ```


### Activation
- **Pass** — An API reference or OpenAPI spec is reachable

  ```text
  API reference introduction and OpenAPI specs for quota endpoints are reachable.
  ```

- **Pass** — An MCP server is documented and well-formed

  ```text
  Docs MCP server documented with Claude Code and Cursor configs at docs.fireworks.ai/mcp.
  ```

- **Pass** — A CLI install path is documented

  ```text
  firectl CLI install documented via Homebrew, curl, wget, plus signin.
  ```

- **Pass** — SDK packages resolve on their registries

  ```text
  PyPI registry lookup for fireworks-ai returned HTTP 200.
  ```

- **Pass** — Agent skills are published

  ```text
  Agent Skills page documents research, configure, debug skills with install commands.
  ```



[Full report data](https://www.ax-check.com/fireworks.ai/report.json)