# AX Check: coderabbit.ai Checked 2026-10-06. CodeRabbit's quickstart and pricing are clear and quick to find. Docs give a concrete sign-up-and-scan setup with install commands; pricing lists exact per-developer rates ($24/$48/$72, Enterprise custom) right on the page. ## Onboarding needs a login ## Coding sessions All three independent sessions completed and reviewed pricing. Two reported the $24/$48/$72 per-developer tiers plus Enterprise custom and usage-based add-on rates; one instead described a Free/Lite and ~$12-24/seat structure, showing some disagreement on exact figures, though all found pricing stated directly on the page without needing to log in. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/b8bd1dab-330c-4acb-b0e9-1f47e9257229) · [Read transcript](https://www.ax-check.com/coderabbit.ai/sessions/deepseek.json) Final output lists per-tier pricing ($24/$48/$72, Enterprise custom) with explicit assumptions: per-developer/annual billing, hourly review caps, 14-day trial, and $0.40/agent-minute metering, sourced from coderabbit.ai/pricing (seq 39). #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: No CodeRabbit account or API key was obtained during the session. The agent only read public docs and tested the hosted metrics API with a dummy key, which correctly returned 401. No authenticated operation against the real product was ever performed; the agent explicitly states it has no API key and cannot execute the example against real data. Event 94: ```text Credentials: I have no `CODERABBIT_API_KEY`, so I cannot execute the example against real data. I did not fake output — with a dummy key it returns `401`, which is the correct behavior. ``` Event 92: ```text 401 Unauthorized: check CODERABBIT_API_KEY. exit=1 ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service API key available for the hosted metrics endpoint**: The CodeRabbit hosted metrics API requires a real org-scoped API key tied to a signed-up account (and an Enterprise plan). The agent had no credentials and no path to generate one without human sign-up through the Git-platform-based quickstart, so it could not run an authenticated call against live data. This is a missing-credentials limitation of the test environment, not a product defect — the quickstart intentionally requires human account creation via GitHub/GitLab/etc. Event 94: ```text Credentials: I have no `CODERABBIT_API_KEY`, so I cannot execute the example against real data. ``` Event 63: ```text Use your existing GitHub, GitLab, Azure DevOps, Bitbucket Cloud, or Bitbucket Data Center account to [create your CodeRabbit account] ``` - **Metrics API is Enterprise-plan gated**: Even with a valid key, most CodeRabbit API endpoints including the metrics endpoint the agent built an example around require the Enterprise plan. This is documented product behavior (a deliberate plan restriction), confirmed by the API's own 403 response schema and the docs, and it limits how much of the API is testable at lower plan tiers. Event 49: ```text API plan requirements vary by endpoint. The [AI Deep Scan code findings API](/api-reference/security-scans-code-list) is available on all plans with a non-agentic API key and the `security:read` permission, other APIs require `enterprise` plan. ``` Event 57: ```text message: API access requires an Enterprise plan subscription code: FORBIDDEN ``` - **Default Python User-Agent blocked by Cloudflare edge (recovered)**: An initial dummy-key request against the real API returned a Cloudflare challenge error (code 1010) instead of the documented 401/403, because Python's default urllib User-Agent was blocked at the edge. The agent diagnosed this with curl variants and fixed it by setting a normal User-Agent header, after which the API returned the correct documented 401. This was a transient test-environment/client artifact, fully resolved within the session, not an unresolved product issue. Event 76: ```text 403 Forbidden: metrics require an Enterprise plan API key. Server said: error code: 1010 ``` Event 85: ```text Good — it was the default `Python-urllib` User-Agent being blocked by Cloudflare. I'll set a normal UA so the example actually works. ``` Event 92: ```text 401 Unauthorized: check CODERABBIT_API_KEY. exit=1 ``` - **No hosted API exists to actually run a code review**: CodeRabbit's hosted REST API is read-only/administrative (metrics, users, roles, audit logs, learnings) and has no endpoint to submit code and receive a review. Running an actual review requires the CLI, Git platform app, or IDE extension rather than the lightweight hosted-API/SDK path the task asked for. This is inherent product scope, not an agent error. Event 94: ```text There is no CodeRabbit SDK (`docs.coderabbit.ai/sdk` → 404). The hosted REST API exists, but it is read-only/administrative... It does not run code reviews. ``` #### Suggested Changes - **Add a 404-redirect or stub page at docs.coderabbit.ai/sdk clarifying no SDK exists**: A direct request to https://docs.coderabbit.ai/sdk returned 404 with no indication of whether an SDK exists elsewhere or is simply absent, forcing the agent to infer its nonexistence indirectly from the API docs. Add a short page at that path (or a redirect into /api) stating plainly that CodeRabbit has no SDK and pointing to the REST API and CLI as the available programmatic surfaces. Verify by requesting docs.coderabbit.ai/sdk and confirming it returns a 200 with that guidance instead of a 404. Event 17: ```text 404 https://docs.coderabbit.ai/sdk ``` - **Document the Cloudflare User-Agent requirement on the API reference page**: A request to the metrics endpoint with Python's default urllib User-Agent returned an opaque Cloudflare error ("error code: 1010") instead of the documented 401/403/etc. responses, which could mislead a developer into thinking their API key or plan was the problem. Add a note on docs.coderabbit.ai/api-reference/metrics-data-api (or the general API index) stating that requests need a standard User-Agent header or they may be blocked at the edge. Verify by making a raw urllib/curl request without a custom User-Agent and confirming the doc's guidance matches the behavior. Event 76: ```text 403 Forbidden: metrics require an Enterprise plan API key. Server said: error code: 1010 ``` - **Link pricing page from docs.coderabbit.ai instead of 404 on /pricing**: The docs site has no working pricing path at docs.coderabbit.ai/pricing (404), requiring a separate lookup at coderabbit.ai/pricing. Add a redirect or link from the docs domain's navigation/FAQ to the marketing pricing page so developers researching via docs do not hit a dead end. Verify by requesting docs.coderabbit.ai/pricing and confirming it resolves instead of 404. Event 17: ```text 404 https://docs.coderabbit.ai/pricing ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/f1e0eb9f-455c-4206-9b43-92034a143f25) · [Read transcript](https://www.ax-check.com/coderabbit.ai/sessions/kimi.json) Final output gives a pricing table (Essentials $24, Team $48, Advanced $72, Enterprise custom, Agent add-on $0.40/agent-min) with stated assumptions: billed annually, monthly +25%, source coderabbit.ai/pricing, and notes API is Enterprise-gated. #### End-to-end onboarding - **Onboarding needs a login**: Agent installed the real CodeRabbit CLI and ran doctor diagnostics confirming backend/websocket reachability, but every attempt to run an authenticated review failed because no account credentials (browser login or API key) were available in the sandbox. No resource was ever created or queried against the hosted product. Event 58: ```text [warn] Authentication Not signed in. Run `coderabbit auth login`. [warn] Auth environment Browser sign-in and stdin fallback are unavailable. Run `coderabbit auth login --agent` from an automation-friendly environment. ``` Event 62: ```text ✗ Non-interactive environment. Use --api-key for authentication. ``` Event 68: ```text No key was available in this environment, so I can't run an actual review. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No API key available for CLI authentication**: CodeRabbit CLI requires either interactive browser sign-in or an Agentic API key (cr-...) for non-interactive environments. The sandbox has neither, so `coderabbit review` cannot authenticate. This is a credential/environment limitation, not a product defect — the CLI correctly detected the non-interactive context and pointed to the right flag. Event 62: ```text Notice: Detected pi-dev environment. Use `coderabbit review --agent` for structured agent-friendly output. ✗ Non-interactive environment. Use --api-key for authentication. ``` Event 58: ```text [warn] Auth environment Browser sign-in and stdin fallback are unavailable. Run `coderabbit auth login --agent` from an automation-friendly environment. ``` #### Suggested Changes - **Add a self-service API key creation path documented in the CLI quickstart**: The CLI docs at docs.coderabbit.ai/cli.md mention `--api-key` as the fallback for headless/bot environments but assume an account already exists with a key generated via the browser-only app.coderabbit.ai dashboard. Adding a documented CLI or API command to mint an Agentic API key without first completing browser OAuth would let automated/agent environments self-onboard. Verify by running `coderabbit auth login --api-key` end-to-end from a fresh non-interactive shell with no prior browser session. Event 38: ````text If browser sign-in is blocked or unavailable, authenticate with an Agentic API key instead: ```bash theme={null} # US when no region is already saved cr auth login --api-key "cr-************" ``` ```` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/5f28d04a-027c-407e-bf24-82d650a71f2b) · [Read transcript](https://www.ax-check.com/coderabbit.ai/sessions/qwen.json) Final output gives tiered pricing (Free/Lite, Pro ~$12/seat individual ~$24/seat team, Enterprise custom) and names the basis — seat-based billing tied to GitHub/GitLab org and plan tier — plus flags that figures should be checked live since they change. #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent never obtained CodeRabbit credentials or an account. It only found a local PI_GATEWAY_API_KEY env var (unrelated to CodeRabbit) and confirmed no CodeRabbit CLI/SDK was installed. It wrote a local .coderabbit.yaml config file but never installed the GitHub App, authenticated with any CodeRabbit API, or performed any authenticated operation against the hosted product. Event 3: ```text PI_GATEWAY_API_KEY= ``` Event 13: ```text Successfully wrote 567 bytes to /sandbox/repo/.coderabbit.yaml ``` Event 8: ```text no CodeRabbit API key or GitHub/GitLab credentials are available ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No CodeRabbit account, API key, or connected repo available**: CodeRabbit is a hosted PR-review product that only activates via GitHub/GitLab App install plus an account; the sandbox had no CodeRabbit credentials, no SDK/CLI installed, and no remote repo connection. This is a missing-credentials/test-environment limitation rather than a product defect, since CodeRabbit's normal workflow requires a real repo and dashboard-based install. Event 8: ```text activation requires an account, a paid/trial plan, and a connected remote repo, none of which exist here ``` Event 6: ```text (no output) Command exited with code 1 ``` #### Suggested Changes - **Publish a lightweight API/SDK path for sandboxed or CI-only trials**: The agent found no standalone SDK or REST API it could call with just an API key, so it could only write a static .coderabbit.yaml with no way to trigger or observe a review. If a token-based API trial exists for headless environments, surface it near the configuration docs so agents without dashboard access can exercise a real review workflow and verify success via an API response. Event 8: ```text There's no standalone SDK or public REST API I can call with just an API key; activation requires an account, a paid/trial plan, and a connected remote repo ``` ### Task given to each agent Help me build a simple example using CodeRabbit. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 84/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a text/markdown request; no Markdown representation served. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links docs, changelog, guides, product pages and trust/status resources. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under Product, Learning hubs, Guides, Company and Documentation sections. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt links CLI, Slack agent, IDE and MCP-related blog and docs pages. ``` - **Pass** — A compact guide representation exists ```text Quickstart page served as text/markdown at docs.coderabbit.ai/getting-started/quickstart. ``` - **Pass** — A focused guide is directly retrievable ```text Quickstart gives concrete steps: sign up, add repos, then next actions. ``` - **Pass** — Equivalent instructions fit a token budget ```text Markdown quickstart is 907 tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be judged. ``` - **Pass** — The compact guide is independently actionable ```text Quickstart gives concrete steps: sign up with Git platform, add repositories, then next steps. ``` - **Pass** — Install and next-step links resolve ```text Quickstart and its .md variant fetched successfully; sign-up and repo links present. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs quickstart page gives 2-minute setup: sign up, add repos, then next steps. ``` - **Pass** — Installation commands are extractable ```text CLI docs show install commands: curl install script, brew install coderabbit, PowerShell irm. ``` - **Pass** — Code examples are available without interaction ```text CLI docs include inline bash examples like coderabbit review and cr auth login. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Quickstart requires Git platform account; CLI docs explain auth login and API key. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders plan cards with prices directly in HTML, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Essentials $24, Team $48, Advanced $72 per developer/month stated; Enterprise custom. ``` - **Pass** — Pricing units and limits are explicit ```text Per-developer monthly units, hourly review limits, and usage rates ($0.25/file, $0.40/min) explicit. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Final output lists per-tier pricing ($24/$48/$72, Enterprise custom) with explicit assumptions: per-developer/annual billing, hourly review caps, 14-day trial, and $0.40/agent-minute metering, sourced from coderabbit.ai/pricing (seq 39). Kimi K3: Final output gives a pricing table (Essentials $24, Team $48, Advanced $72, Enterprise custom, Agent add-on $0.40/agent-min) with stated assumptions: billed annually, monthly +25%, source coderabbit.ai/pricing, and notes API is Enterprise-gated. Qwen 3.8 Max: Final output gives tiered pricing (Free/Lite, Pro ~$12/seat individual ~$24/seat team, Enterprise custom) and names the basis — seat-based billing tied to GitHub/GitLab org and plan tier — plus flags that figures should be checked live since they change. This behavioural item does not affect the fast grade. ``` ### Activation - **Skipped** — An API reference or OpenAPI spec is reachable ```text No API reference or OpenAPI spec page was fetched; docs mention Enterprise API access only. ``` - **Pass** — An MCP server is documented and well-formed ```text Docs describe CodeRabbit as MCP client, connection setup, auth, and .coderabbit.yaml knowledge_base.mcp config. ``` - **Pass** — A CLI install path is documented ```text CLI docs give install script, Homebrew, Windows install, auth login, and review commands. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup result for a CodeRabbit SDK or packaged CLI package was supplied. ``` - **Pass** — Agent skills are published ```text CLI docs document `coderabbit skills` to preview and install verified CodeRabbit skills for coding agents. ``` [Full report data](https://www.ax-check.com/coderabbit.ai/report.json)