# AX Check: useautumn.com Checked 2026-09-19. Autumn's quickstart link is broken, but pricing is upfront The homepage and llms.txt both point to a Quickstart URL that 404s. Pricing tiers (Free, Pro $375/mo, Scale custom) are stated in plain text with no login needed. ## Onboarded with real credentials ## Coding sessions Of three independent sessions, two completed and both correctly reported Autumn's pricing tiers straight from the pricing page, including usage overage rates and Stripe fees. The third session did not reach a completed result. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/e9322271-f646-4589-937c-94daf2275579) · [Read transcript](https://www.ax-check.com/useautumn.com/sessions/deepseek.json) Final output states Autumn's own pricing (Free $0/8K billing volume, Pro $375/mo/50K volume then $5 per 1M requests, Scale custom, plus Stripe's 2.9%+30¢ fees) sourced from useautumn.com/pricing, explicitly tying cost to plan tier and workload/volume assumptions. Session findings unavailable. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/c0e1facf-b270-43ab-aef3-9f17c26f92c6) · [Read transcript](https://www.ax-check.com/useautumn.com/sessions/kimi.json) Final output gives Autumn's own tiered pricing (Free $0, Pro $375/mo, Scale custom) plus explicit product-pricing example ($20/mo pro plan, 1,000 msgs + $0.01 overage) and notes Stripe fees on top as an assumption/add-on. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained an Autumn secret key. It confirmed the hosted API responds (401 no_secret_key) and verified SDK method signatures against local type definitions, but the example script only ran up to a hard stop requesting AUTUMN_SECRET_KEY, and no authenticated call to the real product (creating a feature, plan, or customer) ever succeeded. Getting a key requires manual sign-up at app.useautumn.com, which is a normal human login step outside the agent's reach in this session. Event 13: ```text 401 {"message":"Secret key not found in Authorization header","code":"no_secret_key","env":"sandbox"} ``` Event 75: ```text Set AUTUMN_SECRET_KEY (get one at https://app.useautumn.com) ``` Event 77: ```text I have **no `AUTUMN_SECRET_KEY`**, so I couldn't execute the flow against the live sandbox. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service way to obtain an Autumn secret key**: The hosted API requires a secret key issued via the Autumn dashboard (app.useautumn.com). There is no API-driven signup or key-issuance flow reachable from the sandbox, so the agent could confirm the API's shape (401 with a clear error) but never execute an authenticated call. This is a missing-credentials blocker rooted in normal product security (dashboard-gated key creation), not a bug. Event 13: ```text {"message":"Secret key not found in Authorization header","code":"no_secret_key","env":"sandbox"} ``` Event 77: ```text I have **no `AUTUMN_SECRET_KEY`**, so I couldn't execute the flow against the live sandbox. ``` #### Suggested Changes - **Add a CLI or API path to provision a sandbox secret key**: Currently the only way to get a secret key is the dashboard at app.useautumn.com. Adding a scriptable way (e.g., a signup/key-issuance CLI command or API endpoint) to obtain a sandbox key would let automated agents complete the quickstart end-to-end. Verify by checking whether a coding agent can go from zero credentials to an authenticated customers.getOrCreate() call without visiting a browser-based dashboard. Event 75: ```text Set AUTUMN_SECRET_KEY (get one at https://app.useautumn.com) ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/354c5348-a793-4896-b2b9-12cabcbd53f3) · [Read transcript](https://www.ax-check.com/useautumn.com/sessions/qwen.json) #### End-to-end onboarding - **Onboarded with real credentials**: The agent self-served a real Autumn sandbox with no human login: `npx atmn login --keyless` created a hosted sandbox org and wrote a live secret key to .env, unassisted. It then pushed a real pricing config (features + plans) to that hosted sandbox via `npx atmn push --yes`, and ran example.py using the autumn-sdk Python client authenticated with that key against the actual hosted API — creating a customer, checking/tracking usage, and attaching a paid plan, all returning live server-generated IDs and balances (not mocked). Event 85: ```text ✓ Created sandbox org repo (keyless). Wrote AUTUMN_SECRET_KEY to /sandbox/repo/.env ``` Event 137: ```text Applied. Wrote internalId into 4 fixtures. ``` Event 215: ```text 1. customer=user_1789794573 plans=['free'] 2. check(messages) -> allowed=True balance=5/5 3. check(advanced_models) -> allowed=False (expected False on Free) 4. track(messages) -> remaining balance=4/5 5. after 5 messages -> allowed=False balance=0/5 6. attach(pro) -> advanced_models=True messages balance=100/100 ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Wrong field names in SDK response objects (agent error, self-corrected)**: The agent initially guessed field names for the SDK's Customer, CheckResponse, and TrackResponse objects (e.g. s.plan.id, track.balances['messages']) without checking the actual installed package source. This caused AttributeError and TypeError crashes on the first two run attempts. The agent recovered by inspecting the installed autumn_sdk source directly and fixed the script; final run succeeded. This is agent error working from stale/generic doc examples rather than a product defect. Event 181: ```text AttributeError: 'NoneType' object has no attribute 'id' ``` Event 197: ```text TypeError: 'NoneType' object is not subscriptable ``` Event 215: ```text 1. customer=user_1789794573 plans=['free'] 2. check(messages) -> allowed=True balance=5/5 ``` - **Sandbox created without account ownership**: The keyless sandbox has no linked account/owner; the CLI itself flags this as needing follow-up action within days. This is expected product behavior for the keyless flow, not a defect, but it is a real limitation of the unattended onboarding path if left unresolved. Event 85: ```text This org has no owner yet. Link it within 3 days: atmn login --claim you@example.com ``` #### Suggested Changes - **Publish the Python SDK's real method/field names in the getting-started docs**: The setup.md quickstart only shows JS/CLI examples and a pip install line; it does not show actual Python response object shapes (e.g., CheckResponse.balance.remaining/granted, TrackResponse.balance singular, Customer.subscriptions[].plan_id). The agent had to reverse-engineer these by grepping the installed autumn_sdk package source after two runtime crashes. Add a Python code sample to docs.useautumn.com/documentation/getting-started/gating.md (which returned empty/no output when fetched) showing check() and track() response field access, and verify by running the documented snippet as-is. Event 144: ```text (no output) ``` Event 181: ```text AttributeError: 'NoneType' object has no attribute 'id' ``` Event 197: ```text TypeError: 'NoneType' object is not subscriptable ``` - **Fix the stale pip install command for the Python SDK**: The setup.md quickstart tells users to `pip install autumn-sdk`, and PyPI's `autumn` package name is actually an unrelated ORM, while the correct working package is `autumn-sdk` (module name autumn_sdk) — confirmed correct only after checking PyPI JSON metadata directly. Cross-check that docs.useautumn.com/documentation/getting-started/setup.md's pip line matches the real PyPI package name/version, and note the import is `from autumn_sdk import Autumn`, not `from autumn import Autumn` as shown in the unrelated py-sdk GitHub README. Event 21: ```text A minimal ORM http://autumn-orm.org UNKNOWN ``` Event 29: ```text from autumn import Autumn client = Autumn( secret_key=os.environ.get("AUTUMN_SECRET_KEY"), # This is the default and can be omitted ) ``` Event 145: ```text ImportError: cannot import name 'Autumn' from 'autumn' (unknown location) ``` ### Task given to each agent Help me build a simple example using Autumn. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 88/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — llms.txt mentions offered API, MCP, and skills ```text llms.txt lists docs and API reference but never mentions the offered MCP server. ``` - **Failed** — The compact guide is independently actionable ```text Quickstart link on homepage and llms.txt returns 404; no actionable first steps fetched. ``` - **Failed** — Install and next-step links resolve ```text Homepage/llms.txt Quickstart link 404s; other install links (sign-in, docs) resolve. ``` - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with 200 for a Markdown Accept header. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links docs, quickstart, API reference, comparisons and blog with descriptions. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt organizes content under Docs, Comparisons and Blog headings for navigation. ``` - **Pass** — A compact guide representation exists ```text Homepage returns text/markdown via content negotiation; llms.txt is a compact Markdown index. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt and homepage Markdown are directly retrievable focused guides. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 1811 tokens, well under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown preserves Docs, Quickstart, API reference and GitHub links. ``` ### Onboarding - **Failed** — Docs lead to a relevant quickstart ```text Quickstart link on homepage and llms.txt returns 404 at docs.useautumn.com/quickstart. ``` - **Pass** — Installation commands are extractable ```text CLI guide shows extractable install commands: bunx atmn init, npx atmn init, pnpm dlx atmn init. ``` - **Pass** — Code examples are available without interaction ```text CLI getting-started page shows code examples inline without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text CLI guide documents AUTUMN_SECRET_KEY env keys and login/claim flow for auth. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders all tiers and prices as static text without interaction. ``` - **Pass** — Prices are stated, not gated ```text Free $0, Pro $375/month, and Scale custom are all stated openly. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: 8K/50K billing volume, 10M/20M API requests, $5 per 1M. ``` - **Pass** — Agents identify pricing and its assumptions ```text 2 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Final output states Autumn's own pricing (Free $0/8K billing volume, Pro $375/mo/50K volume then $5 per 1M requests, Scale custom, plus Stripe's 2.9%+30¢ fees) sourced from useautumn.com/pricing, explicitly tying cost to plan tier and workload/volume assumptions. Kimi K3: Final output gives Autumn's own tiered pricing (Free $0, Pro $375/mo, Scale custom) plus explicit product-pricing example ($20/mo pro plan, 1,000 msgs + $0.01 overage) and notes Stripe fees on top as an assumption/add-on. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Attach API reference page includes a full OpenAPI 3.1 spec for the Autumn API. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP docs give server URL, auth, and per-client setup; endpoint returns 401 without key. ``` - **Pass** — A CLI install path is documented ```text CLI getting-started documents atmn install via bunx/npx/pnpm dlx and init flow. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup result for an Autumn SDK or CLI package was supplied. ``` - **Pass** — Agent skills are published ```text CLI init installs Autumn skills next to config so coding agents work with the CLI. ``` [Full report data](https://www.ax-check.com/useautumn.com/report.json)