# AX Check: oleander.dev Checked 2026-10-02. Agents can install, connect, and price oleander.dev with no login wall. 22 of 23 checklist items passed: docs, CLI install, MCP server, and pricing all render as plain text with no sign-in required to read them. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed their task and pulled matching pricing details straight from the pricing page, reporting Hobby, Pro, Pro++, and Enterprise tiers with usage-based compute and storage rates and explicit assumptions. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/55c9d047-f626-4576-9242-c8c1071a4027) · [Read transcript](https://www.ax-check.com/oleander.dev/sessions/deepseek.json) Pricing was scraped live from oleander.dev/pricing (seq 27-29) and the final summary lists per-tier assumptions (Hobby/Pro/Pro++/Enterprise limits, storage $/GB, per-second compute billing) rather than a bare number. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained an oleander API key. It searched the environment and repo for existing credentials, found none, and built an example app that reads OLEANDER_API_KEY from the environment. It confirmed the hosted endpoint is reachable by sending a dummy key and getting a 401, but no real credential was ever acquired or used, so no authenticated product operation occurred. The session ended with the agent telling the user it is blocked on signup/API-key creation, which requires human action on oleander.dev. Event 6: ```text PI_GATEWAY_API_KEY= GIT_CONFIG_KEY_0= ``` Event 75: ```text Hosted API reachable; rejected credentials -> 401 POST /api/v1/query ``` Event 81: ```text I'm blocked on one thing only: **no oleander API key** (`OLEANDER_API_KEY` is unset; this workspace has none). I did not sign up for an account. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service way to obtain an API key**: The sandbox had no oleander credentials, and the product's account/API-key creation flow requires a human to sign up and visit the settings page in a browser. This is a normal login/credential requirement, not a product defect — the agent correctly identified it, confirmed the hosted endpoint works (401 on a dummy key), and stopped rather than inventing a key or standing up a local workaround. Event 71: ```text No OLEANDER_API_KEY set. Create one at https://oleander.dev/app/settings, then: export OLEANDER_API_KEY=... && npm start ``` Event 81: ```text Create a key at `https://oleander.dev/app/settings`, then: ``` #### Suggested Changes - **Add a working /docs path alongside docs.oleander.dev**: Requests to https://oleander.dev/docs and https://oleander.dev/docs/quickstart both 404'd even though the documentation is live at the separate host https://docs.oleander.dev/quickstart. Add a redirect from oleander.dev/docs to the docs subdomain (or vice versa) so a developer who guesses the in-domain path lands on the quickstart instead of a 404. Verify by curling https://oleander.dev/docs and confirming a 200/redirect. Event 36: ```text === https://oleander.dev/docs === HTTP 404 final=https://oleander.dev/docs The All-in-One Data Stack for Agents & Builders | oleander 404: This page could not be found. ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/4b9af91f-f19e-49be-9771-0e67bd21a69c) · [Read transcript](https://www.ax-check.com/oleander.dev/sessions/kimi.json) Final output lists Hobby/Pro/Pro++/Enterprise plan prices plus usage-based compute/storage rates, sourced from oleander.dev/pricing with explicit tier/workload assumptions. #### End-to-end onboarding - **Onboarding needs a login**: Agent researched the oleander product (npm registry, pricing page) and wrote a working example script using @oleanderhq/sdk, but never obtained or used a real API key. No sign-up or key-generation step was attempted; the agent explicitly declined to do the interactive sign-up itself and stopped at the authentication wall. Event 54: ```text OLEANDER_API_KEY is not set. Create a key at https://oleander.dev/app/settings and export it. exit=1 ``` Event 56: ```text I can't execute a live query: Oleander requires an API key from https://oleander.dev/app/settings, and there's no unauthenticated demo tier or sandbox endpoint I can hit. Signing up is an interactive OAuth/email flow I shouldn't do on your behalf. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service API key, session stopped at login wall**: The oleander SDK requires an API key created via the hosted settings page before any real query can be executed. The agent correctly identified this as a normal auth requirement (not a product defect) and declined to attempt an interactive sign-up/OAuth flow on the user's behalf, so the example script was written and exits cleanly but was never run against the live API. Event 54: ```text OLEANDER_API_KEY is not set. Create a key at https://oleander.dev/app/settings and export it. exit=1 ``` Event 56: ```text Signing up is an interactive OAuth/email flow I shouldn't do on your behalf. ``` #### Suggested Changes - **Add a free/unauthenticated demo or sandbox query endpoint**: The agent noted there is no way to try a real query without first creating an account and API key at oleander.dev/app/settings. Adding a no-signup demo endpoint (even rate-limited) or a public sample dataset callable without auth would let prospective developers validate the workflow end-to-end before committing to sign-up. Verify by having a fresh, unauthenticated client call the demo endpoint and receive real query results. Event 56: ```text there's no unauthenticated demo tier or sandbox endpoint I can hit ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/3debb881-3502-4c6d-92c8-61a9fed069e8) · [Read transcript](https://www.ax-check.com/oleander.dev/sessions/qwen.json) README.md pricing table lists Hobby/Pro/Pro++/Enterprise tiers with explicit assumptions (plan, machine size, compute-per-second, card-on-file gating) sourced from oleander.dev/pricing. #### End-to-end onboarding - **Onboarding needs a login**: The agent built a complete SDK example and installed the real oleanderhq-sdk package, but never obtained an OLEANDER_API_KEY. Getting one requires creating an account and signing in via browser at oleander.dev/app/settings, which the agent could not do itself. Every live call against the hosted API (list_catalogs, query_run) was only exercised with no key or a bogus key, both of which failed with 401/auth errors — no authenticated operation against real product data was ever completed. Event 76: ```text OLEANDER_API_KEY is not set. Create one at https://oleander.dev/app/settings (Settings -> API keys) and export it. ``` Event 76: ```text SDK bug while formatting an API error: 'NoneType' object has no attribute 'message' Most likely an auth failure - check OLEANDER_API_KEY. ``` Event 98: ```text There is **no `OLEANDER_API_KEY`** in this environment, and getting one requires an account + browser sign-in at `oleander.dev/app/settings`. So I could not execute a live query. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service way to get an API key**: The product's query API and MCP org tokens require a Pro-or-above account, and the key itself is only issued after creating an account and signing in through a browser at oleander.dev/app/settings. This is a normal auth requirement (not a product defect) but it fully blocked the agent from running any authenticated query in this session; the example code was written and validated for syntax/local behavior only. Event 76: ```text OLEANDER_API_KEY is not set. Create one at https://oleander.dev/app/settings (Settings -> API keys) and export it. ``` Event 55: ```text query unauth: 401 ``` - **SDK crashes instead of raising a clean auth error**: Calling list_catalogs() with an invalid API key caused the installed oleanderhq-sdk (0.8.0) to throw an unhandled AttributeError inside its own error-parsing code rather than a typed OleanderHttpError, because the API's 401 body apparently has error: null. This is an agent-discovered library bug, not an agent mistake — it was reproduced directly and then worked around with an extra except clause. Event 76: ```text AttributeError: 'NoneType' object has no attribute 'message' ``` Event 82: ```text resolved_error = ( parsed.error if isinstance(parsed.error, str) else parsed.error.message ) ``` #### Suggested Changes - **Fix the null-error crash in oleanderhq-sdk's error parser**: In oleander_sdk/errors.py, parse_api_error_body assumes parsed.error is either a string or an object with .message, but a 401 response with {"error": null} causes an AttributeError instead of a proper OleanderHttpError. Patch it to fall back to a generic message when parsed.error is None, and verify by calling list_catalogs() with an invalid/missing API key and confirming an OleanderHttpError (not AttributeError) is raised. Event 76: ```text File "/opt/freestyle/python/lib/python3.12/site-packages/oleander_sdk/errors.py", line 138, in parse_api_error_body parsed.error if isinstance(parsed.error, str) else parsed.error.message ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'message' ``` ### Task given to each agent Help me build a simple example using oleander. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 100/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with Accept: text/markdown, confirming content negotiation. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links setup guide, docs, docs MCP, pricing, product, blog with Markdown twins. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under Setup, Product, Markdown, Optional with connection commands. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions MCP endpoint, docs MCP server, and skills via setup guide. ``` - **Pass** — A compact guide representation exists ```text Homepage content negotiation returns text/markdown, and llms.txt documents Markdown twins for key pages. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt links a one-shot agent setup guide at /install.md with per-client connection steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Homepage Markdown is 1248 tokens, well under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown retains docs, quickstart, MCP, pricing and product links. ``` - **Pass** — The compact guide is independently actionable ```text llms.txt and install.md give concrete MCP endpoint, per-client add commands, and first prompts. ``` - **Pass** — Install and next-step links resolve ```text Fetched quickstart, MCP setup pages, and skills repo all resolve with working install steps. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs quickstart page gives concrete steps: connect agent, upload data, ask first question. ``` - **Pass** — Installation commands are extractable ```text CLI install commands extractable: brew tap/install and APT keyring plus apt-get install. ``` - **Pass** — Code examples are available without interaction ```text Quickstart and CLI docs show inline code blocks for plugin, auth, and query commands. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Auth explicit: OAuth sign-in for MCP, API key from settings for CLI. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders plan tiers and rates as static Markdown text, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Hobby free, Pro $300/mo, Pro++ $1,800/mo, Enterprise contract all stated openly. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: per vCPU-second, per GB/month storage, query GiB caps, retention days. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Pricing was scraped live from oleander.dev/pricing (seq 27-29) and the final summary lists per-tier assumptions (Hobby/Pro/Pro++/Enterprise limits, storage $/GB, per-second compute billing) rather than a bare number. Kimi K3: Final output lists Hobby/Pro/Pro++/Enterprise plan prices plus usage-based compute/storage rates, sourced from oleander.dev/pricing with explicit tier/workload assumptions. Qwen 3.8 Max: README.md pricing table lists Hobby/Pro/Pro++/Enterprise tiers with explicit assumptions (plan, machine size, compute-per-second, card-on-file gating) sourced from oleander.dev/pricing. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text API Reference page documents REST endpoints, base URL, and Bearer auth. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP server documented: endpoint, OAuth, 50 tools, registry listing, annotations. ``` - **Pass** — A CLI install path is documented ```text CLI install via Homebrew and APT, plus configure and command reference. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup result for an oleander SDK or CLI package was supplied. ``` - **Pass** — Agent skills are published ```text Public skills pack OleanderHQ/skills with install command and skill list. ``` [Full report data](https://www.ax-check.com/oleander.dev/report.json)