# AX Check: trycaspianai.com Checked 2026-09-19. Setup docs are solid, but real pricing is nowhere to be found. Quickstart, SDK, CLI and MCP docs are all actionable, but no pricing page exists — 4 of 23 checklist items on pricing are unassessed. ## Onboarded with real credentials ## Coding sessions All three independent sessions found clear install and handler instructions. On pricing, one session did not complete, one inferred a prepaid credit model while flagging it as an assumption since no official pricing page exists, and one refused to state any pricing figure at all, calling it unverifiable. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/f92069ae-edbd-4518-ba95-38373304de55) · [Read transcript](https://www.ax-check.com/trycaspianai.com/sessions/deepseek.json) #### End-to-end onboarding - **Onboarded with real credentials**: Agent minted a no-signup sandbox API key from the hosted Caspian gateway (api.trycaspianai.com) and then used it for real authenticated operations: created a customer, agent, and email connection, sent a test email through the hosted mailbox, saw the resulting message.received event, and posted a reply that produced a message.sent event via /v1/conversations/{id}/messages — all confirmed by live JSON responses from the hosted API, not a local mock. Event 137: ```text {"project_id":"proj_d580c5e16b4bfbdb779d5d0d","api_key":"comm_sandbox_4eb05e1f8d6747b41cd8b565735870286441a96b"} ``` Event 152: ```text === Create email connection === {"id":"conn_2507fa411a7b12f09d2dc193","channel":"email","capabilities":["initiate","media","receive","reply","send"],"status":"provisioning" ``` Event 164: ```text "type": "message.received", "occurred_at": "2026-09-19T15:33:15.333021" ``` Event 171: ```text "type": "message.sent", "occurred_at": "2026-09-19T15:33:28.923095" ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Local demo script's sandbox call was Cloudflare-blocked without a browser User-Agent**: Product/environment interaction, not a real product defect: the sandbox script written to disk failed with a JSON decode error because urllib's default request had no User-Agent and hit Cloudflare's bot check (error code 1010, HTTP 403). Adding a browser-like User-Agent header immediately fixed it. This is a quirk of scripting against the WAF-protected endpoint with a bare urllib client, not a hosted-product failure — the same endpoint worked fine via curl (which sends a default UA) earlier in the session. Event 193: ```text json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ``` Event 205: ```text HTTPError 403 b'error code: 1010\n' ``` Event 210: ```text UA Mozilla/5.0 (X11; Linux x86_64) AppleWeb -> OK 201 b'{"project_id":"proj_812504192822480fed817352" ``` #### Suggested Changes - **Document the WAF/User-Agent requirement on the sandbox endpoint**: The POST /v1/projects/sandbox endpoint (and likely other API routes) sits behind a WAF that returns a non-JSON Cloudflare error page (error code 1010) to clients without a browser-style User-Agent, which breaks naive stdlib HTTP clients like Python's urllib with a cryptic JSON decode error instead of a clear message. Add a note in the SKILL.md quickstart or API docs at trycaspianai.com/docs stating a User-Agent header is required, and verify by re-running a bare urllib POST without a UA header and confirming it now returns JSON or a clear error. Event 193: ```text json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) ``` Event 205: ```text HTTPError 403 b'error code: 1010\n' ``` - **Publish pricing details directly on trycaspianai.com/pricing**: The dedicated pricing page returned an empty response (0 bytes) during the session, forcing reliance on the /v1/billing API response to infer that email is free and imessage/whatsapp/x are paid channels. Populate the /pricing page with the per-channel cost breakdown seen in the billing API (paid_channels list, credit/topup model) so developers do not have to reverse-engineer pricing from API responses. Verify by loading trycaspianai.com/pricing and confirming it renders non-empty content. Event 106: ```text 0 /tmp/pricing.html ``` Event 144: ```text "paid_channels": [ "imessage", "whatsapp", "x" ], ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/72275b25-9104-4255-b0d1-5834b624d96e) · [Read transcript](https://www.ax-check.com/trycaspianai.com/sessions/kimi.json) Final output states the pricing model (prepaid usage-based credit, $1-$5,000 top-ups via Stripe, autopay caps, 402-on-empty-balance) and explicitly flags the assumption basis — 'their site has no pricing page; this is from the live API spec' — and notes per-unit prices aren't published rather than inventing one. #### End-to-end onboarding - **Onboarding needs a login**: Agent found Caspian's device-auth flow, generated a device code and verification URL, and polled the token endpoint repeatedly, but device approval requires a human to open a browser and approve it. After ~45 seconds of polling the status stayed pending, so no API key was obtained and no authenticated call was ever made. The example code was written but never executed against the live gateway. This is a normal login wall, not a resolved recovery. Event 67: ```text "device_code": "hZRHJEGshyaYKEGTSgEJEIo_4fKTYjGwntq215OPLtw", "user_code": "LNXF-B5CH", "verification_uri": "https://api.trycaspianai.com/device" ``` Event 92: ```text {"status":"pending","interval":5} {"status":"pending","interval":5} {"status":"pending","interval":5} {"status":"pending","interval":5} ``` Event 94: ```text Sign-in requires a human in a browser — there's no anonymous key. Please open: **https://api.trycaspianai.com/device?code=LNXF-B5CH** ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Human browser sign-in required to obtain an API key**: Caspian's onboarding has no anonymous key; the only self-serve path (device-auth) still needs a human to visit a URL and approve. The agent correctly identified this, started the flow, and polled for ~45 seconds per the session's no-waiting rule before stopping. This is expected product behavior (a normal login requirement) rather than a defect, and it fully blocked any authenticated API call (channel creation, test email, billing) for the rest of the session. Event 34: ```text SHOW the developer `verification_uri_complete` and WAIT for them to sign in. Poll `POST https://api.trycaspianai.com/v1/auth/device/token` with `{"device_code": ...}` until `status` is `approved`. ``` Event 92: ```text {"status":"pending","interval":5} {"status":"pending","interval":5} {"status":"pending","interval":5} {"status":"pending","interval":5} ``` - **Pricing details not published on a normal pricing page**: The marketing site's /pricing path 404s, and the homepage is a JS-heavy landing page with no rate-card copy, so pricing had to be reverse-engineered from OpenAPI schema field names (topup_cents, monthly_cap_cents) rather than documented directly. This is a product documentation gap, not an agent error — the agent tried the obvious pricing URL first before falling back to the API spec. Event 41: ```text 404 0 ``` Event 60: ```text "### AutopayUpdate" { "properties": { "enabled": { "type": "boolean" ``` #### Suggested Changes - **Publish a pricing page with actual unit rates**: trycaspianai.com/pricing returns a 404, and the homepage has no pricing content, forcing reliance on OpenAPI field names like topup_cents and monthly_cap_cents to infer a prepaid-credit model with no visible per-message/action rates. Add a real /pricing route listing per-channel or per-message costs; verify by curling https://trycaspianai.com/pricing and confirming it returns rate information instead of 404. Event 38: ```text timeout 20 curl -s "https://trycaspianai.com/pricing" -A "Mozilla/5.0" -H "Accept: text/html" -o /tmp/pricing.html -w "%{http_code} %{size_download}\n" ``` Event 41: ```text 404 0 ``` - **Shorten or clarify device-auth polling expectations for unattended agents**: SKILL.md tells the integrating agent to poll the device/token endpoint until approved with no guidance on how long an automated session should wait before giving up, leading the agent to poll only a handful of times over under a minute before stopping. Document an expected approval-latency range in SKILL.md so agent-driven integrations know how long to wait before surfacing the blocker to the human. Event 34: ```text Poll `POST https://api.trycaspianai.com/v1/auth/device/token` with `{"device_code": ...}` until `status` is `approved`. ``` Event 92: ```text {"status":"pending","interval":5} {"status":"pending","interval":5} {"status":"pending","interval":5} {"status":"pending","interval":5} ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/1750c4d9-8009-4049-baed-610cca1861c7) · [Read transcript](https://www.ax-check.com/trycaspianai.com/sessions/qwen.json) Final output explicitly refuses to state any pricing ('I can't tell you how it works... anything I said would be fabricated') — no figure or assumptions given at all. #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: No product credentials were obtained or used. Environment scan only revealed the agent's own pi platform variables (PI_CODING_AGENT, PI_GATEWAY_API_KEY, etc.), and the agent determined 'Caspian' as a hosted product does not exist findable via PyPI/npm, so it never attempted any authenticated operation. Event 6: ```text GIT_CONFIG_KEY_0= PI_CODING_AGENT= PI_GATEWAY_API_KEY= PI_MODEL= PI_PROVIDER= PI_REASONING_LEVEL= PI_SESSION_FILE= PI_SESSION_ID= ``` Event 24: ```text No Caspian credentials or endpoints in the environment — only pi's own `PI_*` variables. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Caspian is not identifiable as a hosted product**: The agent searched the workspace, environment variables, PyPI, and npm for anything called Caspian. The only matches were an unrelated open-source PyPI library (Dask design patterns for data lakes) and an empty npm placeholder package. No vendor site, docs, API base URL, or credentials surfaced, so the agent could not identify a real hosted product to build against. This is a test-environment/task-definition issue (ambiguous or nonexistent product name), not a product defect or agent error. Event 21: ```text caspian 0.2.0 None None None # Caspian [![Documentation Status](https://readthedocs.org/projects/caspian/badge/?version=latest)]... Dask Design Patterns for Data Lakes ``` Event 16: ```text {"_id":"caspian","_rev":"1-793786102c052783870411f000d96f74","name":"caspian","dist-tags":{"latest":"0.0.1"} ``` Event 24: ```text nothing called "Caspian" resolves to a hosted product ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using Caspian. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 81/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a text/markdown request; no Markdown representation served. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists getting-started, usage, channels, platform, and SDK reference docs with descriptions. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under Getting started, Usage, Channels, Platform, and SDK reference headings. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt links REST API, CLI, and MCP docs; skills offered via SKILL.md. ``` - **Pass** — A compact guide representation exists ```text llms.txt is a compact Markdown guide covering quickstart, concepts, SDK and CLI. ``` - **Pass** — A focused guide is directly retrievable ```text Quickstart page gives install, handler code, self-host and hosted steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Quickstart is 1337 tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be judged. ``` - **Pass** — The compact guide is independently actionable ```text SKILL.md gives ordered steps: sign in, install caspian-sdk, one handler, verify via test email. ``` - **Pass** — Install and next-step links resolve ```text Quickstart install (pip/bun add caspian-sdk) and next-step links fetched successfully. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt and index link a Quickstart with concrete install, handler, and run steps. ``` - **Pass** — Installation commands are extractable ```text Quickstart shows `pip install caspian-sdk` and `bun add caspian-sdk` in code blocks. ``` - **Pass** — Code examples are available without interaction ```text Quickstart embeds full Python and TypeScript handler, bot, and hosted examples without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Quickstart states Python 3.10+, BotFather token, and hosted CASPIAN_API_KEY vs self-host no key. ``` ### Pricing - **Failed** — Agents identify pricing and its assumptions ```text 2 of 3 sessions were judged on pricing; 1 fell short. Kimi K3: Final output states the pricing model (prepaid usage-based credit, $1-$5,000 top-ups via Stripe, autopay caps, 402-on-empty-balance) and explicitly flags the assumption basis — 'their site has no pricing page; this is from the live API spec' — and notes per-unit prices aren't published rather than inventing one. Qwen 3.8 Max: Final output explicitly refuses to state any pricing ('I can't tell you how it works... anything I said would be fabricated') — no figure or assumptions given at all. This behavioural item does not affect the fast grade. ``` - **Skipped** — Pricing is readable without interaction ```text No Caspian pricing page was fetched; the only pricing pages belong to GitHub, a foreign product. ``` - **Skipped** — Prices are stated, not gated ```text No Caspian pricing page was fetched; GitHub's prices measure nothing about The Constructor. ``` - **Skipped** — Pricing units and limits are explicit ```text No Caspian pricing page was fetched, so units and limits for its plans are unknown. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text llms.txt links a REST reference: hosted /v1/* the SDK already speaks. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP docs give URL, config JSON, tool list, auth header, Streamable HTTP transport. ``` - **Pass** — A CLI install path is documented ```text CLI page documents caspian init/login, channels add, catalog, call, threads. ``` - **Pass** — SDK packages resolve on their registries ```text PyPI caspian-sdk and npm caspian-sdk registry lookups both returned HTTP 200. ``` - **Pass** — Agent skills are published ```text SKILL.md published at api.trycaspianai.com with per-framework agent guides. ``` [Full report data](https://www.ax-check.com/trycaspianai.com/report.json)