# AX Check: ngrok.com Checked 2026-09-25. ngrok's quickstart and pricing are both clear and machine-readable. All 23 checklist items pass: install steps, code samples, and exact pricing ($0 to $20/mo tiers) are all stated plainly, no login needed. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4 Pro, Kimi K3, Qwen 3.8 Max) completed and produced pricing tables covering Free, Hobbyist/Personal/Pro, Pay-as-you-go, and Enterprise tiers, each pulling exact figures from ngrok.com/pricing and correctly naming the assumptions behind them like usage limits and add-ons. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/23231169-708b-4ddb-aec0-2c838d8f6563) · [Read transcript](https://www.ax-check.com/ngrok.com/sessions/deepseek.json) Final output gives a pricing table (Free/Hobbyist/Pay-as-you-go/Enterprise) sourced from ngrok.com/pricing and ties tiers to concrete assumptions (endpoint limits, transfer/request quotas, usage-based add-ons). #### End-to-end onboarding - **Onboarding needs a login**: Agent explored ngrok's hosted REST API (api.ngrok.com) and wrote a working Node.js script to call it, but never obtained a real API key. No env vars or config files contained credentials, and the agent did not attempt any self-service signup/API-key creation flow (e.g. via dashboard.ngrok.com). It tested with a placeholder key which was rejected by the API with a 403, then stopped and told the user to supply a real key. Event 7: ```text no ngrok env ``` Event 19: ```text --- ``` Event 66: ```text ✗ Blocked: GET /reserved_domains → HTTP 403: The API authentication you specified does not look like a valid credential. Your credential: 'placeholder'. API keys and instructions are available on your dashboard: https://dashboard.ngrok.com/api-keys ``` Event 68: ```text there is **no API key or authtoken anywhere in this environment** (checked env vars and config files; none exist) ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No ngrok API key available in the sandbox**: The environment had no NGROK_API_KEY, authtoken, or config file, and ngrok's dashboard requires a human login to generate a key. This is a missing-credential limitation of the test environment/session setup, not a product defect — the agent correctly identified it could not self-serve a key and stopped rather than faking success. Event 7: ```text no ngrok env ``` Event 61: ```text ✗ NGROK_API_KEY is not set. Create one at https://dashboard.ngrok.com/api-keys, then run: NGROK_API_KEY= node ngrok-api-example.mjs ``` Event 66: ```text ✗ Blocked: GET /reserved_domains → HTTP 403: The API authentication you specified does not look like a valid credential. Your credential: 'placeholder'. ``` - **Live tunneling requires a local listener, ruled out by session constraints**: The agent determined that ngrok's core tunnel functionality (agent SDK/CLI) requires running a local service and a long-running process, which the task's rules explicitly forbid. This is an inherent product-shape issue (tunnels forward to local ports) interacting with the session's own restrictions, not a bug in ngrok. Event 68: ```text these are how you actually create a **live tunnel**, and their entire job is to forward traffic to a **local listener**. That requires a local service stack + a long-running process, which violates your constraints, so I did not go down that path. ``` #### Suggested Changes - **Add an Ngrok-Version header requirement note to the API root response**: Calling https://api.ngrok.com/ with no headers returns a bare 400 requiring an Ngrok-Version header, discovered only through trial and error (seq 8, 18). Document this requirement directly on the api.ngrok.com landing/docs response or the main API docs page so new integrators do not have to guess the header via error messages alone. Event 8: ```text api.ngrok.com HTTP 400 ``` Event 18: ```text Your request has not specified an API version. Include the version you wish to use in the Ngrok-Version header. Supported versions: 2. ``` - **Publish a machine-readable OpenAPI spec at a discoverable URL**: The agent tried the conventional https://api.ngrok.com/openapi.json path and got a 404, forcing manual trial-and-error against undocumented endpoint paths like /domains (404) before finding /reserved_domains and /endpoints. Publishing a discoverable OpenAPI/Swagger document (or linking one clearly from the API root's error response) would let SDKs and agents self-discover valid endpoints without guessing. Event 51: ```text HTTP 404 size 72 paths: [] ``` Event 52: ```text {"status_code":404,"msg":"Not Found","details":{"path":"/domains"}} domains: HTTP 404 ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/da0840fa-d38c-4729-a0b0-17bbb319a755) · [Read transcript](https://www.ax-check.com/ngrok.com/sessions/kimi.json) Final output gives a pricing table (Free/Hobbyist/Pay-as-you-go/Enterprise with $ figures) sourced from ngrok.com/pricing (seq 13) and names assumptions like usage-credit consumption, transfer limits, and no-rollover credit. #### End-to-end onboarding - **Onboarding needs a login**: The agent could not obtain an ngrok authtoken or account without human intervention: signup redirected through login.ngrok.com with browser session handling and email verification, and no NGROK_AUTHTOKEN was present in the environment. It built and tested a runnable SDK script, but the script explicitly exits at the auth check without ever calling the ngrok API, so no authenticated product operation occurred. Event 25: ```text Headless signup is a dead end — ngrok account creation goes through `login.ngrok.com` with browser session handling and email verification, so I can't create an account programmatically. ``` Event 38: ```text Blocked: NGROK_AUTHTOKEN is not set. Get a free token at https://dashboard.ngrok.com/get-started/your-authtoken exit=1 ``` Event 23: ```text https://login.ngrok.com/signup final=https://login.ngrok.com/?error=session_mismatch&grant_id=iamsog_3JpfTEVMrIc2ye3hX9mu9awhelt code=200 ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service account/token creation without a browser login**: Programmatic signup at dashboard.ngrok.com/signup redirected to login.ngrok.com and returned a session_mismatch error, and the pricing/docs make clear account creation requires email verification in a browser. This is normal product authentication behavior, not a defect, but it stopped the agent from obtaining an authtoken and running the example end-to-end within the session. Event 23: ```text final=https://login.ngrok.com/?error=session_mismatch&grant_id=iamsog_3JpfTEVMrIc2ye3hX9mu9awhelt code=200 ``` Event 25: ```text Headless signup is a dead end — ngrok account creation goes through `login.ngrok.com` with browser session handling and email verification, so I can't create an account programmatically. ``` #### Suggested Changes - **Flag the ngrok agent quickstart (ngrok http 80) as long-running before showing it**: The agent noted that the canonical quickstart command starts a persistent local agent process, which conflicts with lightweight/no-long-running-command constraints. Add a note near the quickstart command in docs.ngrok.com's getting-started guide clarifying that `ngrok http 80` runs until manually stopped, so scripted/CI users know to reach for the SDK's forward() call instead. Verify by checking that the quickstart page now distinguishes the CLI long-running mode from the embedded SDK mode. Event 40: ```text ngrok's canonical quickstart (`ngrok http 80`) **is** a long-running agent process tunneling to a local service — exactly what you asked me to avoid. ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/d261648c-51e5-48cb-874a-5c2d20e21ab7) · [Read transcript](https://www.ax-check.com/ngrok.com/sessions/qwen.json) Final output (seq 2) states free vs. paid (Personal/Pro/Enterprise) tiers and names the scaling assumptions: bandwidth, request volume, number of agents/users, and feature add-ons like custom domains/TLS. #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent never attempted any product interaction. It made no tool calls, obtained no credentials, and made no API requests to ngrok. It reasoned in text only that the workflow requires a long-running local agent process, which was disallowed by the prompt constraints, and stopped immediately. Event 2: ```text So plainly: **this product has no developer workflow I can act on under your constraints.** ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Agent self-stopped due to prompt's no-long-running-process constraint**: The agent reasoned that ngrok's core workflow requires a long-running local agent process and a local service to tunnel to, both disallowed by the session's instructions. This is a self-imposed session limitation stemming from the test's own constraints, not a demonstrated product defect, since the agent never attempted to use the hosted REST control-plane API to see how far it could get. Event 2: ```text ngrok is a tunneling product: its core developer workflow requires running a local service *and* a long-running `ngrok` agent process that forwards public traffic to it. That's exactly what you told me not to do, so I won't start it. ``` Event 2: ```text this is hosted, but it only manages configuration; it can't demonstrate the actual product (exposing an endpoint) without the local agent running. ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using ngrok. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 100/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown (200) for a Markdown Accept header. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt indexes docs, quickstarts, API, MCP, skills, SDKs and install pages with .md links. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links by section with guidance on .md twins and docs index. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt links docs MCP server, OpenAPI specs, agent skills index and API catalog. ``` - **Pass** — A compact guide representation exists ```text llms.txt states every docs page has a plain-markdown twin; .md quickstarts fetched successfully. ``` - **Pass** — A focused guide is directly retrievable ```text Agent CLI quickstart .md fetched with concrete install, authtoken, ngrok http, and policy steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Agent CLI quickstart .md is 2305 tokens, well under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown links to docs/start.md, pricing.md, and quickstart .md routes that resolve. ``` - **Pass** — The compact guide is independently actionable ```text Agent CLI quickstart gives concrete install, authtoken, ngrok http 8080, and Traffic Policy steps. ``` - **Pass** — Install and next-step links resolve ```text Fetched install and next-step links (download, SDK, cloud endpoints) resolve successfully. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt and docs link Agent CLI and Cloud Endpoints quickstarts with concrete install, auth, and run steps. ``` - **Pass** — Installation commands are extractable ```text Quickstarts give extractable install commands: brew install ngrok, apt repo, winget, and ngrok help. ``` - **Pass** — Code examples are available without interaction ```text Quickstarts embed full code examples in tabs (Node, Go, Python, Rust) and YAML policies without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Prerequisites list account, authtoken, and API key with dashboard links; auth boundaries are explicit. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders plan cards and full feature tables as static HTML, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Free $0, Hobbyist $10/mo, Pay-as-you-go $20/mo, Enterprise contact-sales all stated. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: $0.02/endpoint-hour, $0.10/GB, $1 per 100k requests, rate limits per minute. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Final output gives a pricing table (Free/Hobbyist/Pay-as-you-go/Enterprise) sourced from ngrok.com/pricing and ties tiers to concrete assumptions (endpoint limits, transfer/request quotas, usage-based add-ons). Kimi K3: Final output gives a pricing table (Free/Hobbyist/Pay-as-you-go/Enterprise with $ figures) sourced from ngrok.com/pricing (seq 13) and names assumptions like usage-credit consumption, transfer limits, and no-rollover credit. Qwen 3.8 Max: Final output (seq 2) states free vs. paid (Personal/Pro/Enterprise) tiers and names the scaling assumptions: bandwidth, request volume, number of agents/users, and feature add-ons like custom domains/TLS. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text OpenAPI 3.0 specs for api.ngrok.com and AI Gateway are served at /openapi.yaml, /openapi.json, and /docs/openapi/ai-gateway.yaml. ``` - **Pass** — An MCP server is documented and well-formed ```text Docs MCP server documented with streamable-http endpoint, tools, and registry manifest at /server.json. ``` - **Pass** — A CLI install path is documented ```text Agent CLI install documented via brew, apt, winget/scoop, Docker, and direct download. ``` - **Pass** — SDK packages resolve on their registries ```text npm @ngrok/ngrok and PyPI ngrok-api registry lookups returned HTTP 200. ``` - **Pass** — Agent skills are published ```text Agent skills index and SKILL.md published, plus ngrok/skills repo with npx install. ``` [Full report data](https://www.ax-check.com/ngrok.com/report.json)