# AX Check: context.dev Checked 2026-10-19. Agents can sign up, install, and price context.dev with no friction All 23 checklist items passed: quickstart, SDK install, API reference, MCP, and CLI docs are all directly readable, and pricing is stated openly with credits and overage rates. ## Onboarding needs a login ## Coding sessions All three independent sessions completed the task. Two sessions reviewed pricing and both correctly read plan credits, concurrency limits, overage rates, and per-operation costs straight from the pricing page without needing to log in. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/94363a8e-83aa-426f-a734-e01e997b0d9e) · [Read transcript](https://www.ax-check.com/context.dev/sessions/deepseek.json) Pricing cited plan credits, concurrency, overage rates, and per-operation costs from the fetched docs. The example's 5-credit cost was explained as 1 base credit plus 4 for JSON extraction. (seq 80, seq 81, seq 95) #### End-to-end onboarding - **Onboarding needs a login**: The agent discovered a self-service agentic signup flow (auth.md) that would let it register, obtain a setup link/code, and poll for an API key without the user manually creating one. However, it never executed that flow. It only confirmed CONTEXT_DEV_API_KEY was unset locally and that an unauthenticated API call returned 401, then stopped to ask the user to either supply a key or give an email so it could start the self-service registration. No credentials were obtained and no authenticated product operation (e.g., a real scrape call) was ever made. Event 43: ```text CONTEXT_DEV_API_KEY is NOT set ``` Event 74: ```text unauthenticated scrape HTTP 401 ``` Event 95: ```text I could not make a **live** call: `CONTEXT_DEV_API_KEY` isn't set, and an unauthenticated request to the API returns `401` (confirmed). ``` Event 95: ```text Agent self-service signup**: give me the email to register, and I'll return a setup link + 6-digit code from https://www.context.dev/auth.md; you complete the browser claim and I'll poll for the key and run the example. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No API key obtained, so the live scrape call was never run**: The sandbox had no CONTEXT_DEV_API_KEY, and the agent confirmed via a direct curl that an unauthenticated request to the API fails with 401. This is a credential gap, not a product defect: Context.dev requires either a human to complete a browser claim step (per auth.md) or an existing key to be supplied. The agent correctly identified this and paused to ask the user for input instead of guessing or fabricating a key. Event 43: ```text CONTEXT_DEV_API_KEY is NOT set ``` Event 72: ```text unauthenticated scrape HTTP 401 ``` Event 95: ```text I could not make a **live** call: `CONTEXT_DEV_API_KEY` isn't set, and an unauthenticated request to the API returns `401` (confirmed). ``` - **Agent never initiated the self-service signup flow it found**: The agent read and understood the full auth.md agentic registration flow (discover, register, deliver setup link/code, poll for token) but did not attempt Step 2 (POST to /agent/identity) to start registration, even though it only needed a placeholder or the user's email to begin. This is an agent-behavior choice (stopping to ask) rather than a hard product block, since the flow explicitly supports agent-initiated registration once an email is available. Event 28: ```text You are an agent. This service supports **agentic registration** for Context.dev: discover → register → **deliver setup link & code to the user** → user completes claim in browser → poll for access_token → call API. ``` Event 95: ```text give me the email to register, and I'll return a setup link + 6-digit code from https://www.context.dev/auth.md; you complete the browser claim and I'll poll for the key and run the example ``` #### Suggested Changes - **Add a quickstart shortcut for cURL-only verification without a dashboard-created key**: The quickstart (docs.context.dev/quickstart) and auth.md both require either a human to visit a dashboard or complete a browser claim ceremony before any API call succeeds; there is no code path for an agent to self-verify end-to-end without a human click. Consider documenting a sandbox/test key or trial token path (clearly marked as low-limit) on the quickstart page, and verify it works by running the existing cURL example in the quickstart against it. Event 37: ```text [Create an account](https://context.dev/signup), then copy a secret key from the [API keys](https://context.dev/dashboard/api-keys) page. ``` Event 28: ```text The user completes Step 3 in **their own browser**. ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/91c10a51-55a3-4ac3-8b8f-1d8ff34a1316) · [Read transcript](https://www.ax-check.com/context.dev/sessions/kimi.json) Plan table from pricing page with credits, overage and limits; named the 1-credit-per-scrape base rate, Answers 10/100 credits, and yearly discount (seq 26, seq 57) #### End-to-end onboarding - **Onboarding needs a login**: Agent researched Context.dev's docs, pricing, and SDK thoroughly, scaffolded a working example script, and installed the official npm SDK locally. It never obtained a real API key: no credentials existed in the environment, and the documented signup flow requires a human to open a browser link, enter a 6-digit code, and verify email before an access token is issued. The agent correctly stopped and asked the user for an email to kick off that human claim ceremony rather than fabricating or bypassing it. No authenticated API call against api.context.dev was ever made. Event 29: ```text env | grep -iE 'context|ctxt' ; echo "exit=$?"; ls ~/.context* 2>/dev/null; echo done ``` Event 25: ```text The user completes Step 3 in **their own browser**. ``` Event 57: ```text I checked for an existing key (env vars, local config) — none found. The registration endpoint requires your email as `login_hint`, so I can't start the flow without it. ``` Event 51: ```text const client = new ContextDev({ apiKey: process.env.CONTEXT_DEV_API_KEY }); ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Signup requires human browser verification and an email address**: Context.dev's only supported registration path (service_auth) requires sending the user's email, having a human open a verification URL, enter a 6-digit code, and verify email in-browser before an API key is issued. This is normal product-mandated authentication, not a defect, but it stopped the agent from completing an end-to-end run in this session since no email or credential was supplied. Event 25: ```text Only `service_auth` is supported. Send the user's email as `login_hint`; the user completes signup, email verification, and claim ceremony on a service-owned page. ``` Event 57: ```text To unblock: tell me the email address you want on the account. ``` #### Suggested Changes - **Fix the context.dev npm package's exports map for package.json access**: Running `require('context.dev/package.json')` from Node fails with ERR_PACKAGE_PATH_NOT_EXPORTED because the package's exports field does not expose package.json. Verify by running `node -e "require('context.dev/package.json')"` after `npm install context.dev` — it should resolve instead of throwing, which would remove this minor rough edge for anyone scripting a version check. Event 47: ```text Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './package.json' is not defined by "exports" in /sandbox/repo/node_modules/context.dev/package.json ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/89aa3d71-3075-487d-bc61-3f2097edb702) · [Read transcript](https://www.ax-check.com/context.dev/sessions/qwen.json) #### End-to-end onboarding - **Onboarding needs a login**: Agent discovered the documented agentic signup flow (service_auth via POST /agent/identity with a claim link/code) but never invoked it or obtained a real API key. It explicitly told the user it would need either a human-supplied key or the user's email to start that flow, and the example script exits immediately with a missing-key error. No authenticated request to the real Context.dev API ever succeeded; an unauthenticated probe correctly returned 401. Event 136: ```text Missing CONTEXT_DEV_API_KEY. Export it before running. exit=1 === unauth probe === 401 ``` Event 146: ```text Blocked on one thing:** no `CONTEXT_DEV_API_KEY` in this environment, so I could not execute the example (the script exits cleanly with "Missing CONTEXT_DEV_API_KEY"; an unauthenticated probe to `api.context.dev` returns 401). ``` Event 23: ```text Hold `claim_token` in memory for polling; do not persist it. **Do not call `/oauth2/token` until you have completed Step 3** (deliver `claim.verification_uri` and `claim.user_code` to the user). ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No API key available, so the SDK example never ran live**: The sandbox had no CONTEXT_DEV_API_KEY, and the agent did not run the documented self-serve agentic signup (POST /agent/identity) because that flow requires delivering a claim link to a human and a real email address, which the agent correctly declined to invent. This is a session/credential limitation, not a product defect — the product does offer a self-service signup path the agent chose not to execute. Event 136: ```text Missing CONTEXT_DEV_API_KEY. Export it before running. exit=1 ``` Event 146: ```text give me an email address and I'll run the agentic signup flow (`POST /agent/identity`) and hand you the claim link + code to finish in a browser — I won't invent an email for you ``` - **SDK type definitions did not match the documented quickstart code**: TypeScript compilation against the installed context.dev SDK failed because Brand had no `name` property despite the SDK's own quickstart example using `domain`/`type` fields; the agent had to grep the .d.mts files to find the actual property was `title`/`domain`. This is an agent/documentation mismatch issue, not a blocking product failure — it was resolved within a couple of edits. Event 99: ```text example.mts(47,35): error TS2339: Property 'name' does not exist on type 'Brand'. ``` #### Suggested Changes - **Fix the brand lookup example field name in SDK docs**: The brand.retrieve JSDoc example shows domain/type:'by_domain' params but doesn't show reading the response, and the response type has no 'name' field (it's 'title'/'domain'), causing a type error. Add a one-line response read (brand.brand.title) to the JSDoc example in brand.d.mts and the docs.context.dev brand reference page; verify by re-running tsc --noEmit against the snippet. Event 99: ```text example.mts(47,35): error TS2339: Property 'name' does not exist on type 'Brand'. ``` Event 66: ```text const brand = await client.brand.retrieve({ * domain: 'stripe.com', * type: 'by_domain', * }); ``` ### Task given to each agent Help me build a simple example using Context.dev. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 100/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown (200) when requested with Accept: text/markdown. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links docs, API reference, OpenAPI, SDKs, MCP, CLI, quickstart and pricing. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under Get Started, Core API Endpoints, SDKs, and Use Cases. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions REST API, hosted MCP server, and agent skill. ``` - **Pass** — A compact guide representation exists ```text Homepage returns text/markdown; docs pages serve .md variants like quickstart.md and agent-quickstart.md. ``` - **Pass** — A focused guide is directly retrievable ```text agent-quickstart.md is a standalone, directly retrievable Markdown guide with concrete setup steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text agent-quickstart.md is 6441 tokens, under 8000, and far below the HTML extraction size. ``` - **Pass** — Product-docs links survive format changes ```text Markdown homepage retains docs, OpenAPI, auth.md, MCP and pricing links; docs routes resolve. ``` - **Pass** — The compact guide is independently actionable ```text Agent quickstart gives concrete setup paths: signup, API key, SDK/CLI/MCP/skill install, and verification steps. ``` - **Pass** — Install and next-step links resolve ```text Fetched quickstart, agent-quickstart, install-mcp, install-skill and API reference pages all returned 200. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Quickstart page gives concrete first steps: get API key, install SDK, scrape example.com. ``` - **Pass** — Installation commands are extractable ```text Install commands extractable: npm install context.dev, pip install context.dev, gem, go get, composer. ``` - **Pass** — Code examples are available without interaction ```text Full code examples in TypeScript, Python, Ruby, Go, PHP, cURL shown without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Auth explicit: API key via Authorization Bearer header, obtained from dashboard after signup. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders full plan table as static Markdown; no interaction needed to read prices. ``` - **Pass** — Prices are stated, not gated ```text All plan prices stated openly: Free $0, Developer $19, Pro $99, Growth $299, Scale $499. ``` - **Pass** — Pricing units and limits are explicit ```text Credits per plan, overage rates, concurrency and per-API credit costs are explicit. ``` - **Pass** — Agents identify pricing and its assumptions ```text 2 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Pricing cited plan credits, concurrency, overage rates, and per-operation costs from the fetched docs. The example's 5-credit cost was explained as 1 base credit plus 4 for JSON extraction. (seq 80, seq 81, seq 95) Kimi K3: Plan table from pricing page with credits, overage and limits; named the 1-credit-per-scrape base rate, Answers 10/100 credits, and yearly discount (seq 26, seq 57) This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text OpenAPI 3.1 spec at /openapi.json (36 paths) and per-endpoint API reference pages are reachable. ``` - **Pass** — An MCP server is documented and well-formed ```text Hosted MCP server at mcp.context.dev/mcp documented with OAuth, tools, and per-client setup. ``` - **Pass** — A CLI install path is documented ```text CLI install documented via release binary and go install, with first command and API key setup. ``` - **Pass** — SDK packages resolve on their registries ```text npm and PyPI registry lookups for context.dev returned HTTP 200. ``` - **Pass** — Agent skills are published ```text Agent skill published at docs.context.dev/skill.md, installable via npx skills add. ``` [Full report data](https://www.ax-check.com/context.dev/report.json)