# AX Check: claude.com Checked 2026-09-19. Claude Code installs cleanly and pricing is stated upfront. 20 of 23 checks passed: quickstart gives copy-paste install commands, and pricing tables list exact per-model, per-seat costs with no login wall. ## Onboarding needs a login ## Coding sessions All three independent sessions completed successfully and located detailed pricing without hitting a login wall. Each pulled per-model per-token rates (Haiku, Sonnet, Opus) straight from the live docs, noting billing basis, caching and batch discounts, and that prices should be verified as they change. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/bb26b5c9-e323-4f04-9cbe-ca3ac73be94b) · [Read transcript](https://www.ax-check.com/claude.com/sessions/deepseek.json) Final output gives per-model MTok pricing table pulled from the live docs (seq 21/23 fetches) and notes assumptions: pay-as-you-go standard API tier, batch discount, prompt-cache write/read multipliers, and that the example call would cost 'well under a cent'. #### End-to-end onboarding - **Onboarding needs a login**: Agent never obtained real Anthropic API credentials. It checked the sandbox environment for ANTHROPIC_API_KEY/ANTHROPIC_AUTH_TOKEN, found none, and confirmed via the SDK's own authentication error that no key was available. No self-service credential acquisition (e.g., signup flow) was attempted or available in this sandboxed environment; the agent correctly stopped rather than fabricate a key. Event 16: ```text default key set: False --- False False ``` Event 63: ```text TypeError -> "Could not resolve authentication method. Expected one of api_key, auth_token, or credentials to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted" ``` Event 65: ```text This environment has no `ANTHROPIC_API_KEY` (or `ANTHROPIC_AUTH_TOKEN`), and the SDK confirms it ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Anthropic API key available in sandbox**: The sandbox environment contained no ANTHROPIC_API_KEY or ANTHROPIC_AUTH_TOKEN. This is a test-environment/credential limitation, not a product defect — a normal API key is expected for any Claude API user. The agent correctly identified this via the SDK's built-in authentication check rather than guessing or fabricating a key, and stopped as instructed by the task prompt. Event 16: ```text default key set: False --- False False ``` Event 63: ```text TypeError -> "Could not resolve authentication method. Expected one of api_key, auth_token, or credentials to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted" ``` #### Suggested Changes None identified in this transcript. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/b1b6d97d-9712-4a57-9bac-bdba45219f01) · [Read transcript](https://www.ax-check.com/claude.com/sessions/kimi.json) Final output gives per-model per-MTok pricing (Haiku ~$1/$5, Sonnet ~$3/$15, Opus ~$15/$75), notes pay-per-token billing basis, caching/batch discounts, and caveats that prices change and should be verified — assumptions are named. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained a real ANTHROPIC_API_KEY. It checked the environment, found none, and only tested the SDK wiring with a placeholder key that intentionally triggered a 401 auth failure. No authenticated operation against the hosted Claude API occurred; the agent asked the human to supply a key. Event 7: ```text ANTHROPIC_API_KEY set: Python 3.12.3 v22.23.2 anthropic 0.122.0 ``` Event 20: ```text ERROR: set ANTHROPIC_API_KEY first (get one at console.anthropic.com) exit=1 ``` Event 23: ```text SDK working; got expected auth failure with fake key: 401 ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No API key available in sandbox**: The environment had no ANTHROPIC_API_KEY and no other route to obtain one. This is a credential/environment limitation, not a product defect — the product requires a normal signup/API key step at console.anthropic.com, which is standard and expected. The agent correctly identified this and stopped rather than fabricating a working call. Event 9: ```text No `ANTHROPIC_API_KEY` is set. Let me check for any available credentials and then build the example. ``` Event 20: ```text ERROR: set ANTHROPIC_API_KEY first (get one at console.anthropic.com) exit=1 ``` Event 25: ```text There is no `ANTHROPIC_API_KEY` in this environment**, so I can't make a real call. ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/c31bcf5c-a253-43d6-a447-56e2a19f434f) · [Read transcript](https://www.ax-check.com/claude.com/sessions/qwen.json) Final output gives per-token pricing for Sonnet/Haiku/Opus and names assumptions (model choice, pay-per-token plan, caching/batch discounts, free trial credit availability). #### End-to-end onboarding - **Onboarding needs a login**: Agent never obtained a real Anthropic API key. It found only an internal gateway key (PI_GATEWAY_API_KEY) which returned a 401 when tried directly against api.anthropic.com. It wrote a working example script but could not run it successfully, and explicitly told the user a human must go to console.anthropic.com to get a key. Event 15: ```text provider=gauge-fireworks model=accounts/fireworks/models/qwen3p8-max 401 ``` Event 23: ```text TypeError: "Could not resolve authentication method. Expected one of api_key, auth_token, or credentials to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted" ``` Event 25: ```text There's **no `ANTHROPIC_API_KEY` in this environment**. The only credential present (`PI_GATEWAY_API_KEY`) is for a different gateway and returns 401 from Anthropic. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Anthropic API key available in sandbox**: The environment only exposed an internal PI_GATEWAY_API_KEY, not a real ANTHROPIC_API_KEY. Testing that key directly against the hosted Anthropic API returned 401, and running the example script failed with an SDK auth error. This is a missing-credentials issue tied to the test environment, not a product defect — a normal developer would supply their own key from the Anthropic console. Event 15: ```text provider=gauge-fireworks model=accounts/fireworks/models/qwen3p8-max 401 ``` Event 23: ```text TypeError: "Could not resolve authentication method. Expected one of api_key, auth_token, or credentials to be set. Or for one of the `X-Api-Key` or `Authorization` headers to be explicitly omitted" ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using Claude. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 84/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html even when text/markdown was requested; no Markdown representation offered. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists products, features, pricing, developer platform, solutions and support with descriptive links. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under clear headings (Products, Features, Pricing, Developer Platform, Solutions). ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt links Claude API, Developer documentation, and Model Context Protocol partnerships. ``` - **Pass** — A compact guide representation exists ```text llms.txt index and Markdown quickstart pages exist; docs pages served as text/markdown. ``` - **Pass** — A focused guide is directly retrievable ```text Quickstart.md and desktop-quickstart.md return Markdown with concrete install and first-session steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Quickstart Markdown is 3074 tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text Quickstart gives concrete install, login, session, and first-change steps with commands. ``` - **Pass** — Install and next-step links resolve ```text Quickstart and desktop-quickstart install/next-step links fetched successfully (200). ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs quickstart page gives step-by-step install, login, and first-session instructions for Claude Code. ``` - **Pass** — Installation commands are extractable ```text Quickstart lists extractable install commands: curl, brew, winget, plus claude --version. ``` - **Pass** — Code examples are available without interaction ```text Quickstart shows inline code blocks for install, login, and example prompts without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Quickstart states prerequisites (terminal, project, subscription/Console) and login/API-key auth steps. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders all plan prices and feature tables as static text without interaction. ``` - **Pass** — Prices are stated, not gated ```text Free $0, Pro $17/$20, Max from $100, Team $20/$25, Enterprise $20/seat all stated. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: per month, per seat/month, per MTok, per 1K searches, per session-hour. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Final output gives per-model MTok pricing table pulled from the live docs (seq 21/23 fetches) and notes assumptions: pay-as-you-go standard API tier, batch discount, prompt-cache write/read multipliers, and that the example call would cost 'well under a cent'. Kimi K3: Final output gives per-model per-MTok pricing (Haiku ~$1/$5, Sonnet ~$3/$15, Opus ~$15/$75), notes pay-per-token billing basis, caching/batch discounts, and caveats that prices change and should be verified — assumptions are named. Qwen 3.8 Max: Final output gives per-token pricing for Sonnet/Haiku/Opus and names assumptions (model choice, pay-per-token plan, caching/batch discounts, free trial credit availability). This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Claude API docs and Messages API reference reachable via platform.claude.com; llms.txt links Claude API. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP documented: claude mcp add commands, transports, OAuth, and connector directory with 824 servers. ``` - **Pass** — A CLI install path is documented ```text CLI install documented: curl install.sh, Homebrew, WinGet, plus claude --version verification. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup result for an Anthropic SDK or CLI package was supplied. ``` - **Pass** — Agent skills are published ```text Agent Skills published: SKILL.md format, skill-creator, and skills docs on claude.com and code.claude.com. ``` [Full report data](https://www.ax-check.com/claude.com/report.json)