# AX Check: staging.impossiblecarrot.cc Checked 2026-09-19. Quickstart works, but the two AI agents priced it completely differently Docs, CLI and pricing pages are all readable without login. Yet one agent called it free open-source, the other found a paid EUR tier table. ## Onboarding needs a login ## Coding sessions Of three independent sessions, one did not complete. Of the two that did, one concluded the product is free and open-source with no paid tiers, while the other reported a tiered EUR pricing table with bring-your-own-key assumptions — a direct conflict on what this product costs. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/aab512df-50bd-4b34-be78-40f921444791) · [Read transcript](https://www.ax-check.com/staging.impossiblecarrot.cc/sessions/deepseek.json) #### End-to-end onboarding - **Onboarding needs a login**: The agent found real docs, endpoints, and a working registration page for the hosted product (heimdallapp.org / RejiCo LLC) but never created an account or obtained an API key. It only probed the Read API endpoint without credentials and got a 403. No self-service signup, key generation, or authenticated call was performed within the session. Event 125: ```text curl -sS -m 20 -X POST "https://read.heimdallapp.org/read/v1/api/process" -H "Content-Type: application/json" -d '{"text":"Hello Heimdall"}' -w "\nHTTP %{http_code}\n" ``` Event 131: ```text

403 Forbidden

``` Event 138: ```text Create Account Free to start. No data science team required. First name Last name Email address Username Password ``` #### Hallucinated URLs - **Guessed Heimdall marketing domains before finding the real one**: The agent tried several plausible top-level domains for Heimdall before locating the actual product site. These guesses returned gateway errors or connection failures rather than a real 404 on a documented page, indicating invented URLs rather than a broken link. Event 54: ```text === https://heimdall.dev => 502 === === https://www.heimdall.dev => 502 === === https://heimdall.io => 000 === === https://heimdall.app => 000 === === https://heimdallsec.com => 502 === === https://www.heimdallsec.com => 502 === ``` #### Blockers - **Product identity ambiguous across many similarly named projects**: Search results and GitHub searches surfaced dozens of unrelated open-source and commercial projects also named Heimdall (dashboard app, Go HTTP client, Polygon blockchain client, database proxy, etc.), forcing many exploratory fetches before the agent located the correct SaaS product (heimdallapp.org). This is a naming/discoverability issue rather than a defect in the actual product being tested. Event 78: ```text 9329 linuxserver/Heimdall An Application dashboard and launcher 3101 Benjamin-Dobell/Heimdall Heimdall is a cross-platform open-source tool suite used to flash firmware ``` Event 84: ```text Heimdall Data heimdalldata.com › home › pricing Pricing - Heimdall Data || https://www.heimdalldata.com/pricing/ ``` - **No account created, so no authenticated API call was possible**: The session ended after reading docs and probing an endpoint without a key (expected 403). Since the workflow requires creating an account via a web signup form (email/password) to obtain API keys, and the agent did not perform that signup, it could not exercise any authenticated operation. This is a normal login/signup requirement, not a product defect. Event 131: ```text

403 Forbidden

``` Event 138: ```text Create Account Free to start. No data science team required. First name Last name Email address Username Password ``` - **Search engines blocked automated queries**: DuckDuckGo returned an anomaly/bot-challenge page and Bing's HTML did not contain parsable organic result links, both stemming from anti-bot protections on third-party search engines, not from the Heimdall product itself. Event 49: ```text Unfortunately, bots use DuckDuckGo too. ``` Event 72: ```text href="#" href="/?FORM=Z9FD1" ``` #### Suggested Changes - **Add a dedicated pricing page instead of relying on the usage dashboard reference**: The llms-full.txt content notes 'No public per-seat enterprise pricing page; contact ... for team deployments' and www.heimdallapp.org/pricing returns a 404. Add a public pricing page at that URL (or update llms.txt/llms-full.txt to point to the correct location) so prospective developers do not have to sign up just to see usage-based cost details. Verify by curling https://www.heimdallapp.org/pricing and confirming it no longer returns the site's 404 page. Event 109: ```text ===================== https://www.heimdallapp.org/pricing bytes=42265 404: This page could not be found. ``` - **Disambiguate the Heimdall name earlier in top-level SEO/search presence**: Because at least a dozen unrelated projects share the Heimdall name (dashboard app, Go client, blockchain client, database proxy, etc.), first-time developers searching generically for 'Heimdall SDK API pricing' get buried results. Strengthen SEO/branding (e.g., more distinctive title tags, backlinks) so heimdallapp.org ranks above unrelated projects; check by repeating a generic web search for the product name and confirming the RejiCo product appears on the first page. Event 84: ```text Heimdalldata heimdalldata.com › home › pricing Pricing - Heimdall Data || https://www.heimdalldata.com/pricing/ ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/2c111a54-eb55-4a91-b848-1a2f6fcacbe2) · [Read transcript](https://www.ax-check.com/staging.impossiblecarrot.cc/sessions/kimi.json) Final output states pricing is 'Free, Apache 2.0, no paid tiers, no hosted/cloud offering' and names the assumption explicitly ('your only real costs would be the infrastructure you run it on and operational effort'), plus notes commercial support exists informally. #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: Heimdall is self-hosted, open-source infrastructure software (Apache 2.0) with no hosted SaaS product, no API keys, and no credentials to obtain. The agent correctly identified this via GitHub repo metadata and docs checks, then stopped without attempting any local service stack per the session's explicit constraints. No credential acquisition or authenticated product operation occurred or was attempted. Event 15: ```text Heimdall has no hosted product and no SDK/API I can act on remotely. It is entirely self-hosted infrastructure software. ``` Event 15: ```text That's exactly the "local service stack" scenario you told me not to start. Since there is no developer workflow I can act on within your constraints, I'm stopping here rather than spinning anything up ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No hosted Heimdall service or API exists to act on**: Heimdall is self-hosted proxy/access-control software distributed as a binary or Docker container, requiring local deployment and YAML configuration. This is a product characteristic, not an agent error or missing credential: there is no SaaS endpoint or SDK to call. The agent's own research (GitHub repo description, docs 404 check) supports this conclusion, and it correctly stopped instead of spinning up a local stack as instructed. Event 15: ```text Every quickstart path requires: - downloading and running the heimdall binary or Docker container locally, - wiring it in front of a backend service, - and writing YAML rule configuration. ``` #### Suggested Changes - **Add a direct link to the installation guide from the docs root**: The agent's request to /dev/docs/getting-started/installation/ returned a 404 while the docs root (https://dadrus.github.io/heimdall/) returned 200. Verify the installation/getting-started path is correctly linked from the main docs landing page so that automated or first-time visitors following a natural URL guess do not hit a dead end. Check by loading the docs homepage and confirming a working link to the getting-started/installation content. Event 13: ```text 404 ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/7c7ef3ee-2c0c-4161-b4c9-d95a9aeb5a9b) · [Read transcript](https://www.ax-check.com/staging.impossiblecarrot.cc/sessions/qwen.json) Final output gives a tiered pricing table (Free/Starter/Professional/Business in EUR) with stated assumptions: bring-your-own-provider-keys, tiered by monthly routed requests, plus trial terms for the alternate product ('10-day free trial, no credit card required'). #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained or used any Heimdall credentials. It spent the entire session doing open-web research to figure out which of four unrelated products named "Heimdall" the user meant (an LLM routing proxy, a content-strategy SaaS, an antivirus brand, and a database proxy), found no API key in the environment, and stopped, explicitly stating that signup requires an interactive web flow with email verification it can't perform. No authenticated call to any Heimdall product was attempted. Event 13: ```text PI_GATEWAY_API_KEY= ``` Event 57: ```text I'd be glad to build a minimal example — it's just an OpenAI-compatible base URL swap, and it would be very easy for me to manage (plain HTTP calls, key in an env var, no local services needed). But I can't run anything against the hosted product without an account and API key, and none are present here. Signing up requires an interactive web flow with email verification, which I can't and shouldn't do on your behalf. ``` Event 57: ```text Tell me **which Heimdall**, and if it's heimdall.run, drop an API key into the environment (e.g. `HEIMDALL_API_KEY`). I'll then write and actually execute a ~20-line example ``` #### Hallucinated URLs - **Guessed developer subdomain that does not exist**: The agent tried developers.useheimdall.app as a guess for a docs/developer portal without any prior evidence it existed; DNS resolution failed outright, indicating the URL was invented rather than discovered. Event 35: ```text "https://developers.useheimdall.app" ``` Event 39: ```text == https://developers.useheimdall.app == curl: (6) Could not resolve host: developers.useheimdall.app 000 ``` #### Blockers - **Ambiguous product identity: four unrelated companies share the "Heimdall" name**: Product behavior / naming collision, not agent error or missing environment setup. Web search returned at least four distinct products under the Heimdall name (LLM routing proxy at heimdall.run, content-strategy SaaS at useheimdall.app, antivirus at heimdalsecurity.com, and a database proxy from Heimdall Data), and the sandboxed repo contained no hint which one the task meant. This made it impossible to proceed confidently to any single hosted product's SDK. Event 24: ```text Pricing - Heimdall Data\nresult__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fuseheimdall.app%2Fpricing&...">Pricing · Heimdall\nresult__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fwww.heimdall%2Dob.com... ``` Event 38: ```text "name": "Heimdall",\n "alternateName": "Heimdall LLM Routing Proxy" ``` Event 57: ```text "Heimdall" is at least four different products ``` - **No credentials available for the hosted product, and signup is an interactive human flow**: Missing credentials / test environment limitation. The only secret present in the sandbox was a gateway API key for the agent's own LLM provider, unrelated to Heimdall. The agent checked the environment for anything Heimdall-related and found nothing, then correctly identified that creating an account on heimdall.run requires an interactive signup with email verification, which is outside what it can self-serve. Event 13: ```text PI_GATEWAY_API_KEY= ``` Event 57: ```text I can't run anything against the hosted product without an account and API key, and none are present here. Signing up requires an interactive web flow with email verification, which I can't and shouldn't do on your behalf. ``` - **useheimdall.app has no discoverable developer API/SDK**: Product behavior. For the content-strategy SaaS candidate, both /docs and /api returned 404, and robots.txt explicitly disallows crawling /api, /dashboard, and /onboarding, suggesting no self-serve public developer surface exists for that particular product. Event 39: ```text == https://useheimdall.app/docs == 404 ``` Event 39: ```text Disallow: /dashboard Disallow: /admin Disallow: /api Disallow: /onboarding ``` #### Suggested Changes - **Disambiguate the product name on the heimdall.run homepage or docs entry point**: Since heimdall.run shares its exact brand name with at least three unrelated products (a content-strategy SaaS, an antivirus vendor, and a database proxy vendor), add a clear tagline or subtitle distinguishing it (e.g. explicitly stating 'LLM routing proxy, not to be confused with...') near the top of https://heimdall.run/ and https://heimdall.run/docs. Verify by searching "Heimdall API SDK pricing" and confirming the correct product surfaces unambiguously in the first result snippet. Event 24: ```text result__a" href="//duckduckgo.com/l/?uddg=https%3A%2F%2Fuseheimdall.app%2Fpricing&...">Pricing · Heimdall ``` Event 57: ```text "Heimdall" is at least four different products ``` ### Task given to each agent Help me build a simple example using Heimdall. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 84/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a text/markdown request; no Markdown representation served. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists docs, API reference, guides and provider pages with one-line descriptions. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt has a 'Start here' section pointing to AGENTS.md and Documentation. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt links a full API reference and CLI reference; no MCP or skills offered. ``` - **Pass** — A compact guide representation exists ```text llms.txt and quickstart.md are site-published Markdown guides; quickstart.md is a compact standalone guide. ``` - **Pass** — A focused guide is directly retrievable ```text quickstart.md fetched directly as text/markdown with three concrete steps and code. ``` - **Pass** — Equivalent instructions fit a token budget ```text quickstart.md is 1245 tokens, well under 8000 and far below the 2337-token HTML version. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text Quickstart gives three concrete steps: create key, export it, send cURL/SDK request. ``` - **Pass** — Install and next-step links resolve ```text Fetched quickstart, login, docs and homepage links all returned 200. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt and docs link a Quickstart with three concrete steps: create key, export, send request. ``` - **Pass** — Installation commands are extractable ```text Quickstart shows export HEIMDALL_API_KEY and curl/TS/Python install-and-call snippets. ``` - **Pass** — Code examples are available without interaction ```text Quickstart and CLI reference show cURL, TypeScript, Python and JSON examples inline. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Quickstart requires sign-in and sk-hd-v1- key; login page explains key creation. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders full price tables and units as static Markdown text, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Exact per-unit prices published as decimal strings; listed price is billed price, no gating. ``` - **Pass** — Pricing units and limits are explicit ```text Billing units table maps each price key to token, second, image, character units. ``` - **Pass** — Agents identify pricing and its assumptions ```text 2 of 3 sessions were judged on pricing; 0 fell short. Kimi K3: Final output states pricing is 'Free, Apache 2.0, no paid tiers, no hosted/cloud offering' and names the assumption explicitly ('your only real costs would be the infrastructure you run it on and operational effort'), plus notes commercial support exists informally. Qwen 3.8 Max: Final output gives a tiered pricing table (Free/Starter/Professional/Business in EUR) with stated assumptions: bring-your-own-provider-keys, tiered by monthly routed requests, plus trial terms for the alternate product ('10-day free trial, no credit card required'). This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text llms.txt links a full API reference; chat.md documents POST /chat/completions request and response fields. ``` - **Skipped** — An MCP server is documented and well-formed ```text No MCP server is documented in llms.txt, quickstart, CLI reference or API docs. ``` - **Pass** — A CLI install path is documented ```text CLI reference documents the heimdall binary from @heimdall/cli with commands, flags and login. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup result for @heimdall/cli or any Heimdall SDK package was supplied. ``` - **Skipped** — Agent skills are published ```text No agent skills are published or referenced in the fetched Heimdall documents. ``` [Full report data](https://www.ax-check.com/staging.impossiblecarrot.cc/report.json)