# AX Check: netlify.com Checked 2026-10-06. Netlify's docs, CLI, and pricing are all readable without logging in 22 of 23 checklist items passed; only gap is llms.txt not mentioning Netlify's API, MCP server, or skills directly. ## Onboarding needs a login ## Coding sessions All three independent sessions completed the task and found pricing clearly stated. Each pulled matching tier details (Free, Personal $9, Pro $20, Enterprise custom) with credit costs for deploys, compute, and bandwidth spelled out plainly in the docs. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/f2300b60-963a-45a0-8738-d422a3f3fccc) · [Read transcript](https://www.ax-check.com/netlify.com/sessions/deepseek.json) README.md pricing section lists Free/Personal/Pro/Enterprise tiers with credit costs per workload type (deploys, compute GB-hour, bandwidth, requests) and notes these are current published tiers subject to change, citing netlify.com/pricing as source. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained Netlify credentials on its own. It checked the environment for a NETLIFY_AUTH_TOKEN or CLI config, found none, confirmed api.netlify.com returns 401 unauthenticated, and built/validated the deploy script and example files entirely locally (syntax check and --dry-run, which only hashes local files, no network calls). No authenticated call to the real Netlify API was ever made. Event 14: ```text --- token in env? --- PI_GATEWAY_API_KEY= --- netlify config in home --- ls: cannot access '/sandbox/.netlify': No such file or directory ``` Event 15: ```text --- connectivity test (api.netlify.com) --- http_code=401 ``` Event 87: ```text --- dry run --- Static files (SHA1): /index.html fc86ec9c94c8eb346a57905a7f509da42f6718af Functions (SHA256 of zip): hello 7822fb6fe4abf6040a56c065c5cc9a49bc1c8dd9b34b33a0c2e2a90ac6612dd3 ``` Event 102: ```text **No Netlify credentials are present.** There is no `NETLIFY_AUTH_TOKEN` (or CLI config) in this environment; `api.netlify.com` returns `401` unauthenticated. So I could **not actually deploy or verify against the hosted product** ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing Netlify auth token blocks any live deploy**: No NETLIFY_AUTH_TOKEN or ~/.netlify config exists in the sandbox, and api.netlify.com rejects unauthenticated requests with 401. This is a credentials gap in the test environment, not a product defect — Netlify requires a personal access token for API use by design. The agent correctly identified this, built the deploy script, and validated it locally via dry-run, but never executed an authenticated call against the hosted API. Event 15: ```text http_code=401 ``` Event 102: ```text There is no `NETLIFY_AUTH_TOKEN` (or CLI config) in this environment; `api.netlify.com` returns `401` unauthenticated. ``` #### Suggested Changes - **Clarify SDK deploy removal earlier in @netlify/api docs**: The @netlify/api README buries the fact that site deployment was removed in v7.0.0 under a 'Site deployment' section near the bottom, after showing createSite/deleteSite examples that imply full site lifecycle support. Consider adding a note near the top-level usage example (where `new NetlifyAPI(...)` and `listSites()` are shown) pointing to the REST deploy endpoints or CLI, so developers don't assume the SDK handles deploys. Verify by checking that a first-time reader of the top of the README understands deploy is out of scope for the SDK. Event 42: ```text ## Site deployment Support for site deployment has been removed from this package in version 7.0.0. You should consider using the [`deploy` command](https://cli.netlify.com/commands/deploy/) of Netlify CLI. ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/2cff63ad-0db9-42ec-8d9b-68cfe06ced17) · [Read transcript](https://www.ax-check.com/netlify.com/sessions/kimi.json) Final output breaks down Free/Pro/Business tiers with concrete limits (bandwidth, build minutes, function invocations, team members) as the stated assumptions behind each price. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained Netlify credentials. It checked the environment for a token, found none, and hit a 401 from the Netlify API on an unauthenticated request. It explicitly stated it could not complete the interactive OAuth flow needed for `netlify login` and stopped before any authenticated operation, scaffolding local files only. Event 4: ```text --- ``` Event 10: ```text 401 ``` Event 12: ```text The Netlify API is reachable but returns `401` — every hosted workflow (create site, deploy, manage functions/env vars) requires a personal access token, and there's no `NETLIFY_AUTH_TOKEN` in this environment. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Netlify auth token available for deployment**: The sandbox had no NETLIFY_AUTH_TOKEN and no CLI installed. The agent confirmed the API requires authentication (401 on an unauthenticated call to the sites endpoint) and correctly identified that `netlify login` needs an interactive browser OAuth flow it cannot perform. This is a normal credential/login requirement of the product, not a product defect, and it halted the agent before any real deploy could happen. Event 4: ```text --- ``` Event 10: ```text 401 ``` Event 12: ```text there's no `NETLIFY_AUTH_TOKEN` in this environment. So I'm blocked on the actual deploy step; I'll say so plainly rather than fake it. ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/6d7899b6-0615-43ee-8fcd-5c711d4e5bdd) · [Read transcript](https://www.ax-check.com/netlify.com/sessions/qwen.json) README.md and final summary give concrete tiered pricing (Free $0/300 credits, Personal $9, Pro $20, Enterprise custom) with explicit assumptions: credit rates per deploy/compute/bandwidth, auto-recharge off by default, and this example's workload (~1 deploy, 15 credits) relative to Free allowance. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained real Netlify credentials. It checked the environment for a token, found none, and confirmed api.netlify.com was reachable but required auth. It built a complete deploy pipeline (zip builder, REST API client, deploy/verify/teardown scripts) but every live-publish attempt failed without a token, and a deliberately bogus token also failed with 401. No authenticated operation against the hosted product was ever demonstrated. Event 4: ```text env | grep -i -E "netlify|token" | sed 's/=.*/=/'; ls ~/.netlify 2>/dev/null ``` Event 90: ```text FAILED: No Netlify credentials. Set NETLIFY_AUTH_TOKEN (personal access token from https://app.netlify.com/user/applications/personal-access-tokens). exit=1 === bogus token: FAILED: GET /user -> 401 Unauthorized {"code":401,"message":"Access Denied"} exit=1 ``` Event 124: ```text Blocked on one thing: there are no Netlify credentials in this environment. `npm run deploy` fails fast with a clear message (exit 1); with a bogus token it returns `GET /user -> 401 Unauthorized`. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Netlify personal access token available in the sandbox**: The environment had no NETLIFY_AUTH_TOKEN or similar credential, and there is no self-service way inside the sandbox to generate one (it requires a human to log into the Netlify dashboard and create a personal access token). This is a session/test-environment limitation, not a product defect — api.netlify.com itself was reachable and responded correctly (401 unauthenticated, 401 on a bad token), confirming the API behaves as documented. Event 12: ```text api.netlify.com -> 401 ``` Event 90: ```text FAILED: No Netlify credentials. Set NETLIFY_AUTH_TOKEN (personal access token from https://app.netlify.com/user/applications/personal-access-tokens). exit=1 ``` - **netlify npm package is now the CLI, not the standalone JS SDK**: The agent installed the `netlify` npm package expecting a JS client/SDK (as historically documented), but it instantiated as a constructor and failed; inspecting node_modules revealed `netlify` is now the CLI tool, and the old standalone client package name (`netlify-js-client`) no longer exists on the registry. This cost a round of install/debug time before pivoting to the REST API directly. This is an agent expectation mismatch stemming from package naming/ecosystem drift, recovered without further impact. Event 17: ```text TypeError: N is not a constructor at [eval]:1:57 ``` Event 21: ```text "name": "netlify", "version": "27.11.2", "description": "Netlify command line tool", ``` Event 25: ```text npm error 404 'netlify-js-client@*' is not in this registry. ``` #### Suggested Changes - **Clarify netlify npm package identity on its README/npm listing**: The `netlify` npm package (v27.11.2) describes itself as a command line tool, but a developer following older docs or intuition may expect a JS SDK/client, as happened here (constructor error at seq 17). Add a note on the npm package page or README clarifying it is CLI-only and pointing to the REST API as the documented path for programmatic/SDK-style usage, so developers don't burn an npm install cycle (~2 minutes per this session) discovering this. Event 17: ```text TypeError: N is not a constructor at [eval]:1:57 ``` Event 21: ```text "description": "Netlify command line tool", ``` ### Task given to each agent Help me build a simple example using Netlify. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 98/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — llms.txt mentions offered API, MCP, and skills ```text llms.txt lists platform/docs pages but never mentions the offered API, MCP servers, or skills. ``` - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown (200) for Accept: text/markdown, confirming Markdown support. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt (200) indexes platform, solutions, company and resources pages with descriptions. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt explains pages are available as Markdown via .md or Accept header. ``` - **Pass** — A compact guide representation exists ```text llms.txt states every page is available as Markdown via .md or Accept: text/markdown. ``` - **Pass** — A focused guide is directly retrievable ```text CLI guide fetched as Markdown at docs.netlify.com/.../get-started-with-cli/ with concrete install and auth steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text CLI guide Markdown is 6650 tokens, under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown retains docs, CLI, platform and pricing links; docs routes remain reachable. ``` - **Pass** — The compact guide is independently actionable ```text CLI guide gives install, auth, deploy, env-var and local-dev steps with concrete commands. ``` - **Pass** — Install and next-step links resolve ```text Homepage CLI link and docs install/next-step links fetched successfully (docs, CLI reference). ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text CLI guide gives concrete first steps: install netlify-cli, netlify login, netlify deploy. ``` - **Pass** — Installation commands are extractable ```text Installation commands shown: npm install -g netlify-cli and npm install netlify-cli --save-dev. ``` - **Pass** — Code examples are available without interaction ```text Code examples appear inline in fetched docs, e.g. netlify deploy and netlify.toml snippets. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Prerequisites (Node.js 18.14.0+) and auth token acquisition via netlify login or UI are explicit. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Netlify's pricing.md renders plan tiers and credit costs as plain text without interaction. ``` - **Pass** — Prices are stated, not gated ```text Free $0, Personal $9/month, Pro $20/month, Enterprise custom are all stated. ``` - **Pass** — Pricing units and limits are explicit ```text Credits per plan and per-feature credit costs (deploys, compute, bandwidth) are explicit. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: README.md pricing section lists Free/Personal/Pro/Enterprise tiers with credit costs per workload type (deploys, compute GB-hour, bandwidth, requests) and notes these are current published tiers subject to change, citing netlify.com/pricing as source. Kimi K3: Final output breaks down Free/Pro/Business tiers with concrete limits (bandwidth, build minutes, function invocations, team members) as the stated assumptions behind each price. Qwen 3.8 Max: README.md and final summary give concrete tiered pricing (Free $0/300 credits, Personal $9, Pro $20, Enterprise custom) with explicit assumptions: credit rates per deploy/compute/bandwidth, auto-recharge off by default, and this example's workload (~1 deploy, 15 credits) relative to Free allowance. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Docs link to Netlify API reference at open-api.netlify.com; Database API reference page fetched. ``` - **Pass** — An MCP server is documented and well-formed ```text Official hosted MCP server documented: HTTP endpoint netlify-mcp.netlify.app/mcp, OAuth on first use. ``` - **Pass** — A CLI install path is documented ```text CLI install documented: npm install -g netlify-cli, plus local dev-dependency install. ``` - **Pass** — SDK packages resolve on their registries ```text Registry lookup for npm:netlify-cli returned HTTP 200. ``` - **Pass** — Agent skills are published ```text Netlify skills published in netlify/context-and-tools repo with install via @netlify/skills. ``` [Full report data](https://www.ax-check.com/netlify.com/report.json)