# AX Check: openrouter.com Checked 2026-10-07. Agents can install, run, and price OpenRouter with no friction. 19 of 23 checks passed: quickstart, SDKs, API/MCP docs, and pricing all load without login. Only the root llms.txt index is missing. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed and correctly reported live per-token pricing pulled from the models endpoint, citing concrete rates like gpt-4o-mini at $0.15/$0.60 per million tokens plus fee assumptions such as BYOK and the ~5.5% platform fee. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/41cd826b-2b8b-4392-893b-94e4e95c3cd4) · [Read transcript](https://www.ax-check.com/openrouter.com/sessions/deepseek.json) Final output states per-token pricing with explicit assumptions (live catalog, specific models like gpt-4o-mini/gpt-4o, ~5.5% min $0.80 credit fee, pass-through no markup, caching affects rates). #### End-to-end onboarding - **Onboarding needs a login**: Agent found no OPENROUTER_API_KEY in the environment and could not self-provision one. It successfully queried the public, unauthenticated /models catalog but could not demonstrate any authenticated operation (chat completion) because the key was missing. The session ends with the agent explicitly stating inference is blocked pending a human-supplied key. Event 11: ```text env | grep -i '^PI_' | sed -E 's/(KEY|TOKEN|SECRET)=.*/\1=/' ``` Event 38: ```text BLOCKED: OPENROUTER_API_KEY is not set. The /models catalog is public, but chat completions require a key. exit=1 ``` Event 40: ```text Live **inference** is blocked: there is **no `OPENROUTER_API_KEY`** in this environment. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No OpenRouter API key available for authenticated calls**: The sandbox environment only exposed an internal PI_GATEWAY_API_KEY, not an OpenRouter key. This is a test-environment/credential-provisioning gap, not a product defect: OpenRouter's public /models endpoint worked fine unauthenticated, and the script failed cleanly with a clear error pointing to the missing key rather than crashing or hanging. Event 14: ```text PI_GATEWAY_API_KEY= ``` Event 38: ```text BLOCKED: OPENROUTER_API_KEY is not set. The /models catalog is public, but chat completions require a key. exit=1 ``` #### Suggested Changes None identified in this transcript. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/367fa94b-bb07-45b7-9e6d-8551fb1c8c89) · [Read transcript](https://www.ax-check.com/openrouter.com/sessions/kimi.json) README.md and final summary state per-token pricing pulled live from /api/v1/models (e.g. gpt-5-mini $0.25/$2.00 per 1M) alongside explicit assumptions: prepaid credits/BYOK, no subscription, :free tier rate limits. #### End-to-end onboarding - **Onboarding needs a login**: No OPENROUTER_API_KEY was present in the environment and the agent could not self-provision one. It correctly identified the missing credential, attempted an unauthenticated chat completion which returned a 401, and then stopped, stating that key creation requires a logged-in OpenRouter account it cannot create itself. No authenticated product operation (e.g. a successful chat completion or generation cost lookup) was ever demonstrated. Event 15: ```text No `OPENROUTER_API_KEY` is set anywhere — that matters, and I'll come back to it. ``` Event 32: ```text {"error":{"message":"No cookie auth credentials found","code":401}} ``` Event 47: ```text no `OPENROUTER_API_KEY` in this environment. Unauthenticated chat calls return `401 No cookie auth credentials found`, so I could verify everything except an actual completion. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing OpenRouter API key blocks authenticated calls**: The sandbox had no OPENROUTER_API_KEY and no way to self-serve one (account creation and key issuance require a logged-in human account). This is a credential/test-environment limitation, not a product defect — OpenRouter requiring account-based key issuance is normal auth behavior. The agent correctly detected this via env scan and a live 401 response, then stopped rather than faking success. Event 13: ```text PI_GATEWAY_API_KEY ``` Event 32: ```text {"error":{"message":"No cookie auth credentials found","code":401}} ``` Event 47: ```text The only steps I can't self-serve are account creation, payment, and key issuance. ``` #### Suggested Changes - **Fix sentinel pricing values in the public models endpoint**: GET /api/v1/models returns pricing.prompt and pricing.completion as "-1000000" (i.e. -$1 per token) for router/meta-models like openrouter/fusion and nvidia/switchyard, instead of null or omitting the field. This broke a naive cheapest-model sort in the example script. Fix by returning null or a documented sentinel for non-per-token pricing models, and verify by re-running a sort-by-prompt-price query against /api/v1/models and confirming router models no longer appear as the cheapest. Event 32: ```text typesafe/jev-router in=$-1000000.000 out=$-1000000.000 ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/e80ec5bb-7051-4688-bfd0-58c5c2f63fe4) · [Read transcript](https://www.ax-check.com/openrouter.com/sessions/qwen.json) Final output states prepaid-credit, per-token per-model pricing with the concrete example 'openai/gpt-4o-mini is $0.15/M input, $0.60/M output' sourced live from the /models endpoint, plus notes on free-tier variants and refunds as assumptions/context. #### End-to-end onboarding - **Onboarding needs a login**: Agent found no OPENROUTER_API_KEY in environment or in ~/.pi/agent/auth.json (empty object) and had no way to self-serve one (no browser/OAuth flow available in this shell environment). It built a correct example script and confirmed the public, unauthenticated /models endpoint works, but explicitly could not execute an authenticated chat completion. No credential was ever obtained or used against OpenRouter during the session. Event 27: ```text {} { "providers": { "gauge-fireworks": { ``` Event 39: ```text Set OPENROUTER_API_KEY first (https://openrouter.ai/keys). exit=1 ``` Event 42: ```text there is **no `OPENROUTER_API_KEY`** in this environment and `auth.json` is empty — so I could not execute an actual chat completion. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing OpenRouter API key prevents any authenticated call**: The sandbox had no OPENROUTER_API_KEY set and the pi auth store (~/.pi/agent/auth.json) was empty, so the agent could not run its own example script past the key check. This is a credential/environment limitation, not a product defect — OpenRouter requires a key or OAuth login by design, and no human supplied one in this session. Event 27: ```text {} { "providers": { "gauge-fireworks": { ``` Event 39: ```text Set OPENROUTER_API_KEY first (https://openrouter.ai/keys). exit=1 ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using OpenRouter. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 83/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — llms.txt provides an actionable documentation index ```text https://openrouter.com/llms.txt returned 404; no root index, though docs/llms.txt exists. ``` - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with 200 when requested with Accept: text/markdown. ``` - **Skipped** — llms.txt provides navigation guidance ```text No llms.txt body was fetched; the root path returned 404, so navigation guidance cannot be judged. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text No llms.txt body was fetched; API, MCP and skills mentions cannot be judged. ``` - **Pass** — A compact guide representation exists ```text Homepage negotiated Markdown and .md docs pages (quickstart.md, api_reference/overview.md) are served as text/markdown. ``` - **Pass** — A focused guide is directly retrievable ```text https://openrouter.ai/docs/quickstart.md returns a focused, actionable quickstart with install and first request. ``` - **Pass** — Equivalent instructions fit a token budget ```text Quickstart Markdown is 2809 tokens, well under the 8000-token budget and far below the HTML extraction. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown links to docs, API reference, llms.txt, pricing and skills, preserving doc routes. ``` - **Pass** — The compact guide is independently actionable ```text Quickstart gives concrete steps: install @openrouter/sdk, set OPENROUTER_API_KEY, run npx tsx chat.ts. ``` - **Pass** — Install and next-step links resolve ```text Fetched quickstart, API reference, MCP and skills pages all returned 200 with usable content. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs quickstart page gives concrete first steps: install SDK, send first message, stream. ``` - **Pass** — Installation commands are extractable ```text Quickstart shows extractable install commands: npm install @openrouter/sdk, pip install openrouter. ``` - **Pass** — Code examples are available without interaction ```text Quickstart provides Python, TypeScript, Shell and SDK code examples inline without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Prerequisites list Node.js 18+ and an API key created at openrouter.ai/settings/keys. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders a full plan table with fees, limits and features without interaction. ``` - **Pass** — Prices are stated, not gated ```text Plan fees 5.5% Standard, 8% Business and free-tier limits are stated openly. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: platform fee percentages, 50 req/day free, $25,000 BYOK allowance. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Final output states per-token pricing with explicit assumptions (live catalog, specific models like gpt-4o-mini/gpt-4o, ~5.5% min $0.80 credit fee, pass-through no markup, caching affects rates). Kimi K3: README.md and final summary state per-token pricing pulled live from /api/v1/models (e.g. gpt-5-mini $0.25/$2.00 per 1M) alongside explicit assumptions: prepaid credits/BYOK, no subscription, :free tier rate limits. Qwen 3.8 Max: Final output states prepaid-credit, per-token per-model pricing with the concrete example 'openai/gpt-4o-mini is $0.15/M input, $0.60/M output' sourced live from the /models endpoint, plus notes on free-tier variants and refunds as assumptions/context. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text API reference page documents OpenAPI spec in YAML and JSON at openrouter.ai/openapi.yaml and .json. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP guide documents remote server URL, per-client setup, OAuth, and full tool list. ``` - **Skipped** — A CLI install path is documented ```text No dedicated OpenRouter CLI install path documented; Ori harness is a separate tool. ``` - **Pass** — SDK packages resolve on their registries ```text Registry lookups returned HTTP 200 for npm @openrouter/sdk, @openrouter/agent, and pypi openrouter. ``` - **Pass** — Agent skills are published ```text OpenRouterTeam/skills repo publishes many Agent Skills with install instructions for multiple agents. ``` [Full report data](https://www.ax-check.com/openrouter.com/report.json)