# AX Check: coder.com Checked 2026-10-26. Coder's quickstart and docs are clear, but Premium pricing is hidden Docs, install steps, and API reference are all readable without login. Community tier is free, but Premium and AI Premium show no actual price, just "annually per user" or "custom." ## Onboarding needs a login ## Coding sessions All three independent sessions completed and read the pricing page directly. Each reported Community as free, Premium as per-user annual billing with no published number, and AI Premium as custom pricing, with one session estimating Premium near $50/user/month and others flagging that self-hosted compute costs are extra. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/5f5b760c-1ab6-4216-baaa-058163d77e37) · [Read transcript](https://www.ax-check.com/coder.com/sessions/deepseek.json) Pricing came from the pricing page: Community is free, Premium is billed annually per user with no published price, and AI Premium is custom. It also noted that workspace compute costs are extra. (seq 41, seq 79) #### End-to-end onboarding - **Onboarding needs a login**: No credentials were ever obtained. Coder is self-hosted software with no public SaaS endpoint; the agent correctly identified that generating a session token requires an existing deployment at /settings/tokens, which it had no way to create without running a local 'coder server' stack (explicitly disallowed by the task). It wrote example code but never executed it against a real deployment, so no authenticated operation occurred. Event 79: ```text That requires installing Coder and running a local server (a local service stack), which your instructions tell me not to do. There's no hosted SDK/API I can hit without a deployment URL and a token generated on that deployment. ``` Event 79: ```text So I did **not** execute anything. ``` Event 79: ```text Stopping here as instructed. To proceed, give me an existing `CODER_URL` + token, and I'll run the example against it. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Coder has no hosted SaaS endpoint to call**: Product behavior, not agent error: Coder is self-hosted software requiring the user to run 'coder server' plus provisioners/Terraform templates before any workspace can exist. There is no public Coder Cloud API, so a 'use the hosted product via SDK/API' task cannot proceed without a human-provisioned deployment URL and token. The agent correctly detected this via docs and the pricing page rather than guessing. Event 79: ```text Coder (coder.com) is **self-hosted software, not a hosted/SaaS product**. There's no public Coder Cloud endpoint to call. ``` Event 48: ```text Can't start coder server : Address already in use Encountered an error running "coder server", see "coder server --help" for more information error: configure http(s): listen tcp 127.0.0.1:3000: ``` - **Quickstart requires a long-running local server**: Test-environment restriction, not a product defect: the official quickstart installs a binary and runs 'coder server' as a persistent local process before any API call works, which conflicts with the session's explicit instruction to avoid starting local service stacks or long-running commands. Event 79: ```text curl -fsSL https://coder.com/install.sh | sh coder server # long-running local service on 127.0.0.1:3000 ``` Event 79: ```text That requires installing Coder and running a local server, so this example was **not executed** in this environment. ``` #### Suggested Changes - **Offer a free hosted sandbox/demo deployment for quickstart testing**: The quickstart at coder.com/docs/get-started only documents installing and running a local 'coder server'; there is no way to obtain a token against a hosted instance for quick API/SDK trials. Adding a short-lived hosted trial deployment (with self-service token issuance) would let developers test the REST API/SDK without standing up infrastructure first. Verify by having a new user generate a token and call GET /api/v2/users/me successfully within minutes of signup. Event 79: ```text There's no public Coder Cloud endpoint to call. ``` Event 42: ```text Quickstart Generate a token on your Coder deployment by visiting: https://coder.example.com ``` - **Fix the broken /docs/api redirect**: Requesting https://coder.com/docs/api returned a 404, while the correct path is /docs/reference/api; add a redirect from the shorter, more guessable /docs/api path to the actual REST API reference page so developers landing there via search or memory don't hit a dead end. Event 15: ```text === https://coder.com/docs/api === HTTP 404 final=https://coder.com/docs/api ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/a77addcf-688a-48de-9e9c-9fefa1862f4d) · [Read transcript](https://www.ax-check.com/coder.com/sessions/kimi.json) Gave free open source plus about $50/user/month Premium, billed annually with volume tiers, and said to confirm at coder.com/pricing. (seq 6) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: Agent made a single directory-listing check for local Coder config/binary, found nothing, and immediately concluded the product requires a self-hosted server. No attempt was made to find or use a hosted Coder SaaS endpoint, obtain credentials, or call any API. No credentials were obtained and no authenticated operation was attempted. Event 3: ```text ls /sandbox/repo 2>/dev/null && ls ~/.config/coderv2 2>/dev/null; which coder 2>/dev/null; echo "---done" ``` Event 6: ```text There is no hosted, signup-with-an-API-key SaaS endpoint I can call. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Agent assumed self-hosted-only with no real investigation**: After a single failed local lookup for a coder binary/config, the agent declared the product has no accessible developer workflow and stopped without searching for API docs, a hosted Coder cloud offering, or any SDK usage path. This is an agent decision/self-limitation rather than a demonstrated product defect, since no actual quickstart page or API was consulted. Event 3: ```text ls /sandbox/repo 2>/dev/null && ls ~/.config/coderv2 2>/dev/null; which coder 2>/dev/null; echo "---done" ``` Event 6: ```text Coder (coder.com) is a **self-hosted** cloud development environment platform. Its quickstart requires running a control plane first ``` #### Suggested Changes - **Clarify hosted/SaaS access options on the quickstart landing page**: The agent concluded, without visiting any documentation, that Coder has no hosted SaaS endpoint and only supports self-hosted server/Docker setups requiring long-running processes. Verify the quickstart or landing page clearly surfaces any lightweight hosted trial or API-only path (if one exists) so agents evaluating the product via light SDK/API calls are not forced to assume a full server deployment is required. Event 6: ```text There is no hosted, signup-with-an-API-key SaaS endpoint I can call. ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/0b936e2c-394b-4f30-aa8a-281c387e133d) · [Read transcript](https://www.ax-check.com/coder.com/sessions/qwen.json) Listed the plans from the pricing page (Community free, Premium per user billed annually but unpublished, AI Premium custom/trial) and noted that self-hosted infrastructure costs come on top of the license. (seq 17, seq 26, seq 28) #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained real product credentials. It scanned environment variables and found no CODER_URL or CODER_SESSION_TOKEN, has no coder CLI installed, and concluded that Coder requires a self-hosted control plane with its own Postgres/TLS/IdP before any API or CLI call can be authenticated. It explicitly stopped rather than standing up a local stack (per instructions) and asked the product owner to supply a session token for an existing deployment. No authenticated operation against a hosted Coder instance was ever attempted. Event 4: ```text env | grep -iE 'coder|api|token|key' | sed 's/=.*/=/'; echo "---"; which coder terraform docker kubectl helm 2>/dev/null ``` Event 7: ```text PI_GATEWAY_API_KEY= GIT_CONFIG_KEY_0= --- /usr/bin/docker ``` Event 28: ```text Environment check: no `coder` CLI, no `CODER_URL` / `CODER_SESSION_TOKEN` / any Coder credentials, no Terraform or Kubernetes. Repo `/sandbox/repo` is empty (`.git` + `.gitignore` only). ``` Event 28: ```text **What unblocks me:** give me a `CODER_URL` + session token for an existing deployment (yours, or a free homelab instance you've stood up elsewhere). ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No hosted Coder instance or credentials available**: Coder is a self-hosted control plane, not a SaaS product with sign-up. The agent found no CODER_URL, session token, or coder CLI in the sandbox, and the docs confirm every entry point (CLI, REST API, Go SDK) requires a running deployment URL plus a session token obtained after installing a control plane. This is a missing-credentials/product-model blocker, not an agent error — the agent correctly identified it had nothing to authenticate against. Event 4: ```text env | grep -iE 'coder|api|token|key' | sed 's/=.*/=/' ``` Event 28: ```text it needs a persistent, stateful deployment (control plane + Postgres + reverse proxy + TLS + IdP) that I can't keep alive across a session ``` - **Quickstart requires a local service stack and long-running server**: The agent reviewed the Coder quickstart/install docs and determined that even the fastest path installs the Coder server plus Postgres and launches a workspace locally, which conflicts with the session's instruction to avoid local service stacks or long-running commands. This is product behavior (self-hosted architecture) rather than an agent mistake, and the agent correctly stopped instead of proceeding. Event 28: ```text The **Quickstart** (the "single machine as quickly as possible" path) still installs the Coder server + Postgres and launches a workspace — i.e. a local service stack, plus a long-running server process. That's exactly what you told me not to do, so I stopped instead of doing it. ``` #### Suggested Changes - **Fix the broken quickstart link at coder.com/docs/quickstart**: Requesting https://coder.com/docs/quickstart returned a 404-style 'Oops, something went wrong' page during this session, even though the install docs explicitly reference a 'Quickstart' path. Product owner should verify the correct quickstart URL (e.g. under /docs/install/) and update cross-links that point to /docs/quickstart so first-time visitors following the referenced path don't hit a dead end. Event 22: ```text Oops, something went wrong. Defeat the Bugs to return home. 404 ``` - **Publish a lightweight, token-based hosted trial path for the SDK/API**: The agent found no way to get a CODER_URL and session token without first standing up a full control plane (DNS, certs, database, identity provider), which blocked any SDK/API/CLI usage. Product owner could add a free hosted sandbox or trial deployment that issues a session token on signup, separate from the self-hosted install flow, so developers can exercise the REST API/Go SDK without deploying infrastructure. Verify by confirming a new user can obtain a CODER_URL + token and successfully call the API within a few minutes. Event 28: ```text give me a `CODER_URL` + session token for an existing deployment (yours, or a free homelab instance you've stood up elsewhere). With that I'll write and test a Terraform workspace template, drive it through the CLI/API ``` - **Publish Premium per-seat pricing instead of gating it behind a demo request**: The pricing page only exposes a numeric price for the free Community tier; Premium and AI Premium tiers show no published per-user figures and instead route to 'Get a demo' or 'Start a trial'. Product owner should add indicative per-user annual pricing to coder.com/pricing for the Premium tier so prospective customers can evaluate cost without a sales call; verify by checking that a price or price range renders on the page for each tier. Event 17: ```text Premium Annually per user For enterprises ready to achieve world-class security, scalability, and developer experience. G e t a d e m o ``` ### Task given to each agent Help me build a simple example using Coder. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 95/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with Accept: text/markdown, confirming Markdown negotiation. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links curated docs index, full corpus, and per-product indexes. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt organizes links under Coder, code-server, Platform, and Resources headings. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text llms.txt lists docs, pricing, and resources but no API, MCP, or skills surfaces. ``` - **Pass** — A compact guide representation exists ```text Homepage serves text/markdown; docs pages offer .md and Accept: text/markdown per llms.txt. ``` - **Pass** — A focused guide is directly retrievable ```text Quickstart retrievable as Markdown at /docs/get-started.md with concrete install and workspace steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Markdown quickstart is 4056 tokens, well under 8000 and below the 5408-token HTML extraction. ``` - **Pass** — Product-docs links survive format changes ```text Markdown quickstart preserves docs links to install, customize-template, architecture and agents pages. ``` - **Pass** — The compact guide is independently actionable ```text Quickstart gives concrete steps: install runtime, install Coder CLI, create template, launch workspace, connect IDE. ``` - **Pass** — Install and next-step links resolve ```text Install and next-step links resolve: /docs/install, customize-your-template, agents getting-started fetched successfully. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs link to a Quickstart with concrete install, template, workspace, and IDE steps. ``` - **Pass** — Installation commands are extractable ```text Quickstart gives extractable install commands: curl install.sh, winget install Coder.Coder, coder server. ``` - **Pass** — Code examples are available without interaction ```text Quickstart and CLI reference show inline code examples without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Prerequisites list CPU/RAM and terminal; first-user admin signup defines the auth boundary. ``` ### Pricing - **Failed** — Prices are stated, not gated ```text Community is Free, but Premium and AI Premium show no numeric price, only 'Annually per user' and 'Custom'. ``` - **Pass** — Pricing is readable without interaction ```text Pricing page fetched as Markdown shows Community, Premium, AI Premium tiers without interaction. ``` - **Pass** — Pricing units and limits are explicit ```text Pricing states per-user annual Premium, unlimited workspaces, and agent caps (5 vs unlimited). ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Pricing came from the pricing page: Community is free, Premium is billed annually per user with no published price, and AI Premium is custom. It also noted that workspace compute costs are extra. (seq 41, seq 79) Kimi K3: Gave free open source plus about $50/user/month Premium, billed annually with volume tiers, and said to confirm at coder.com/pricing. (seq 6) Qwen 3.8 Max: Listed the plans from the pricing page (Community free, Premium per user billed annually but unpublished, AI Premium custom/trial) and noted that self-hosted infrastructure costs come on top of the license. (seq 17, seq 26, seq 28) This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text REST API reference page fetched with token setup and curl example; API docs linked from docs index. ``` - **Skipped** — An MCP server is documented and well-formed ```text No MCP server documentation found in the fetched Coder documents. ``` - **Pass** — A CLI install path is documented ```text Quickstart documents installing the coder CLI via install.sh and winget, then coder server. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup for a Coder SDK or packaged CLI package was supplied. ``` - **Pass** — Agent skills are published ```text Quickstart links coder/skills setup skill and coder-templates agent skill on GitHub. ``` [Full report data](https://www.ax-check.com/coder.com/report.json)