# AX Check: bag.coffee Checked 2026-10-20. Coffee prices are visible instantly; API/CLI setup docs are thin. Pricing and per-100g units show right on the homepage. But quickstart, install commands and code examples weren't found, so setup steps are unclear. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed and correctly reported pricing: the site and Free API plan are free with stated rate limits, while Pro pricing is by arrangement and not published. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/ba662fc9-e235-41ae-b5e7-a4c3b139d08e) · [Read transcript](https://www.ax-check.com/bag.coffee/sessions/deepseek.json) Said the Free plan costs nothing and named its per-key limits (60 req/min, 60 list reads/hr, 2,000 coffees/day); said Pro is by arrangement with no published price. (seq 44, seq 97, seq 130) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent never obtained a bag.coffee API key (that requires interactive sign-in at bag.coffee/account, which it explicitly could not do self-service) and never exercised a key-gated authenticated operation. It built and ran a working example against the hosted API, but only against endpoints that are public/anonymous (/api/coffees, /api/counts, /api/filters). No credentials were acquired or used, so this does not meet the bar for verified or login_required (no human-login blocker stalled progress; the agent simply worked around it using public endpoints and completed its task). Event 114: ```text cd /sandbox/repo && timeout 45 node index.mjs ``` Event 130: ```text I couldn't get an API key because it requires interactive sign-in at `bag.coffee/account`, so the key-only endpoints are documented but not exercised. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **API key requires interactive sign-in, not available in session**: Getting a bag.coffee API key requires signing in at bag.coffee/account and creating a key through the UI. This is normal product behavior (a standard auth wall), not a product defect, but it meant the agent could not test or demonstrate any key-gated endpoint (roasters detail, coffee history, browse/*, and the MCP server) in this automated session. The agent worked around it by building the example entirely against publicly accessible endpoints. Event 44: ```text Get a key: sign in at https://bag.coffee/account, then API keys, Create key. Send it as `Authorization: Bearer bag_…` ``` Event 130: ```text I couldn't get an API key because it requires interactive sign-in at `bag.coffee/account`, so the key-only endpoints are documented but not exercised. ``` #### Suggested Changes - **Align OpenAPI security metadata with actual per-endpoint auth behavior**: The OpenAPI spec marks every /api/* path with a global BearerApiKey security requirement, but testing showed /api/coffees, /api/counts, and /api/filters respond 200 without any key while /api/roasters, /api/browse/*, coffee detail, and history return 401 without one. Update the OpenAPI document (api.bag.coffee/openapi.json) to mark the actually-public endpoints with an empty security override, so generated clients and docs correctly reflect which calls need a key. Verify by re-checking that /docs or the spec no longer lists auth as required on the endpoints that answer anonymously. Event 50: ```text GET /api/coffees security=GLOBAL ``` Event 51: ```text === no auth /coffees === HTTP 404 === body === {"error":"Not found"} ``` Event 56: ```text /api/coffees -> HTTP 200 : {"data":[{"id":"dcb2c968-e107-4fa2-ba0c-27a82ceeab33" ``` Event 56: ```text /api/roasters -> HTTP 401 : {"error":"Authentication required"} ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/b5f90697-78a2-4e3d-8216-a4c3c771f072) · [Read transcript](https://www.ax-check.com/bag.coffee/sessions/kimi.json) Stated product is free, with the basis split by user type: browsing, free accounts, free roaster listings, no commission, all sourced from the site's FAQ (seq 15, seq 31) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: Agent never attempted onboarding because it determined there is no public API, SDK, or developer credential workflow to use. No signup, API key, or authenticated call was ever attempted. Event 31: ```text Developer workflow: none that I can act on. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No public API or developer docs exist**: Product behavior, not agent error: probing common developer paths returned 404s and robots.txt explicitly disallows /api/, confirming no self-serve programmable surface for the site. Event 23: ```text api -> 404 api/v1 -> 404 developers -> 404 docs -> 404 openapi.json -> 403 ``` Event 22: ```text Disallow: /api/ Disallow: /go/ Disallow: /admin Disallow: /account Disallow: /my-list Disallow: /dev/ Disallow: /unsubscribe/ ``` - **No SDK published for the product**: Agent error-free check of npm registry found zero packages, confirming there is no client library to build an example against. Event 29: ```text No matches found for "bag.coffee" {"objects":[],"total":0,"time":"2026-10-20T00:43:37.421Z"} ``` - **Only integration path is manual outreach, not programmable**: Product behavior: the sole way to get content onto the site is a roaster emailing the company directly, which is a human process the agent cannot execute as a developer workflow. Event 31: ```text The only "integration" path is a roaster emailing `hello@bag.coffee` to get listed — a manual, human process, not a programmable one ``` #### Suggested Changes - **Publish a developer/API docs page if programmatic access is intended**: Currently hitting /developers and /docs returns 404 and /api/ is disallowed in robots.txt, so there's no discoverable path for third-party developers. If any API access is meant to exist, add a linked docs page from the homepage footer and verify it resolves with a 200 and lists real endpoints. Event 23: ```text developers -> 404 docs -> 404 ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/56e8a7af-4e64-46f5-9cc7-58c74b9d7c00) · [Read transcript](https://www.ax-check.com/bag.coffee/sessions/qwen.json) Pricing came from the API terms: the site and the Free API plan cost nothing, with stated per-key limits (60 req/min, 60 list reads/hr, 2,000 coffees/day). Pro is 'by arrangement' with no published price. (seq 38, seq 32, seq 102) #### End-to-end onboarding - **Onboarding needs a login**: The agent found bag.coffee's hosted read-only API (OpenAPI doc) and successfully called unauthenticated endpoints (/api/coffees, /api/filters, /api/counts) via a working CLI. Getting an actual API key requires account creation and sign-in at bag.coffee/account, which the agent could not self-serve (no credentials in environment). It confirmed all key-gated endpoints return 401 and explicitly flagged this as a blocker rather than claiming success. Event 48: ```text /api/roasters?limit=1 401 {"error":"Authentication required"} ``` Event 72: ```text 515 coffees in stock at 41 roasters (refreshed 2026-10-20T00:30:00.000Z) ``` Event 102: ```text I could not obtain an API key. Keys come from `https://bag.coffee/account` *after signing in*, and account creation is a human email-verification step I can't perform; there's no key in the environment. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service way to get an API key**: Session limitation: bag.coffee's key-gated endpoints (roasters, browse index, price-index, drop-calendar, weekly-drops, coffee history, by-slug lookups) all require a Bearer API key obtained only after creating an account and signing in at bag.coffee/account. The agent had no credentials and no way to self-register, so it could exercise only the three endpoints that work unauthenticated. This is a normal login wall, not a product defect. Event 48: ```text /api/roasters?limit=1 401 {"error":"Authentication required"} ``` Event 32: ```text **Authentication**: an API key, from https://bag.coffee/account once signed in, in every request: ``` - **Pro plan pricing is not published**: Product behavior: the Pro plan (price/availability history, past weeks' drops, higher rate limits) has no listed price. It is gated behind emailing hello@bag.coffee, so the agent could not fully answer the pricing question for that tier. Event 32: ```text The Pro plan adds price and availability history (every price change and sell-out we've recorded for a coffee) and past weeks' drops (what dropped in any week before the current one). A key on the free plan gets `403` for those, with `feature` and `plan` in the body. ``` Event 37: ```text Need more for an app? A Pro plan has higher limits, by arrangement: email hello@bag.coffee . ``` #### Suggested Changes - **Publish Pro plan pricing on the API terms page**: On https://bag.coffee/api-terms (or the OpenAPI info description), list an indicative price or price range for Pro instead of only 'by arrangement, email hello@bag.coffee'. Verify by checking a first-time developer can answer 'what does this cost at scale' without contacting support. Event 37: ```text Need more for an app? A Pro plan has higher limits, by arrangement: email hello@bag.coffee . ``` - **Add a developers/API landing page linked from site navigation**: /pricing, /docs, /developers, /api, and /sdk all 404 on the marketing site; the only path to API docs is knowing to hit api.bag.coffee/openapi.json directly. Add a nav or footer link to an API landing page that explains signup and links to the OpenAPI spec and account page. Verify a new visitor reaches API docs within one click from the homepage. Event 18: ```text === docs === 404 https://bag.coffee/docs === developers === 404 https://bag.coffee/developers === api === 404 https://bag.coffee/api === sdk === 404 https://bag.coffee/sdk ``` ### Task given to each agent Help me build a simple example using bag.coffee. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 84/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a text/markdown request; no Markdown representation offered. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists coffees, drops, browse, prices, Markdown, API, MCP and optional pages with URLs. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under Site, Markdown, API, MCP and Optional headings with URL patterns. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt documents the API, MCP server tools and registry entry; no skills offered. ``` - **Pass** — A compact guide representation exists ```text llms.txt documents .md versions of coffee, roaster, week and New pages. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt gives concrete .md URLs like /new.md and /drops/week/2026-W43.md. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 1008 tokens, well under the 8000-token guide budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be judged. ``` - **Pass** — The compact guide is independently actionable ```text llms.txt gives a compact, self-contained guide: site routes, .md pages, API key steps, MCP tools and install command. ``` - **Pass** — Install and next-step links resolve ```text llms.txt links resolve: OpenAPI 200, server.json 200, MCP endpoint responds 405 to GET with POST guidance. ``` ### Onboarding - **Skipped** — Docs lead to a relevant quickstart ```text llms.txt lists site pages and API/MCP surfaces but no developer quickstart page was fetched. ``` - **Skipped** — Installation commands are extractable ```text No installation or setup page was fetched; llms.txt only shows an MCP add command. ``` - **Skipped** — Code examples are available without interaction ```text No docs or API reference page content was fetched to confirm code examples. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text llms.txt states API needs a key, obtained at /account, sent as Bearer bag_… ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Homepage shows coffee prices like $22.00 · $8.80/100g directly in listings, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Prices are stated openly on the homepage listings and per-100g figures, not gated. ``` - **Pass** — Pricing units and limits are explicit ```text Prices shown with explicit units, e.g. $22.00 · $8.80/100g, and per-100g comparisons. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Said the Free plan costs nothing and named its per-key limits (60 req/min, 60 list reads/hr, 2,000 coffees/day); said Pro is by arrangement with no published price. (seq 44, seq 97, seq 130) Kimi K3: Stated product is free, with the basis split by user type: browsing, free accounts, free roaster listings, no commission, all sourced from the site's FAQ (seq 15, seq 31) Qwen 3.8 Max: Pricing came from the API terms: the site and the Free API plan cost nothing, with stated per-key limits (60 req/min, 60 list reads/hr, 2,000 coffees/day). Pro is 'by arrangement' with no published price. (seq 38, seq 32, seq 102) This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text OpenAPI 3.1.0 document and reference reachable at api.bag.coffee, with auth and rate limits. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP server documented with four tools, Streamable HTTP endpoint, bearer key, and registry server.json. ``` - **Skipped** — A CLI install path is documented ```text No CLI install path documented; only an MCP add command is shown. ``` - **Skipped** — SDK packages resolve on their registries ```text No SDK or packaged CLI registry lookup supplied for bag.coffee. ``` - **Skipped** — Agent skills are published ```text No agent skills are published or documented in the fetched evidence. ``` [Full report data](https://www.ax-check.com/bag.coffee/report.json)