# AX Check: puzzle.io Checked 2026-09-19. Puzzle.io's docs and pricing are clear and machine-readable. 17 of 23 checks passed: pricing is stated openly by tier, and a compact docs index with a working quickstart guide is directly retrievable. ## Onboarding needs a login ## Coding sessions All three independent agent sessions completed. Both sessions that reviewed pricing found a clear tiered table (Starter $30 up to Scale $360/mo) with explicit seat and credit assumptions; one noted API/embedded pricing wasn't published. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/9448138a-2772-4eef-85e3-a728439f5b44) · [Read transcript](https://www.ax-check.com/puzzle.io/sessions/deepseek.json) #### End-to-end onboarding - **Onboarding needs a login**: The agent found real hosted API endpoints (sandbox at staging.southparkdata.com, production at api.puzzle.io) and correctly identified that Puzzle requires OAuth2 with a human browser consent step to get an access token, and that even client credentials are not self-serve (must email api@puzzle.io). It verified the auth wall directly with real HTTP calls returning 401/invalid_client, then wrote a client script but never obtained a token or made any authenticated call. No credentials were obtained during the session, so no authenticated operation was possible. Event 98: ```text {"errors":[{"message":"Missing authentication credentials","code":"UNAUTHENTICATED"}]} ``` Event 104: ```text {"error":"invalid_client"} ``` Event 51: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. You will need to provide us with a list of valid redirect URIs and we will provide you with a client ID, a client secret, and a key. ``` Event 120: ```text Missing credentials: set PUZZLE_CLIENT_ID and PUZZLE_CLIENT_SECRET. Puzzle issues sandbox credentials manually — email api@puzzle.io (or your Puzzle contact) with your redirect URIs. exit=1 ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Sandbox credentials require manual, human-mediated provisioning**: Puzzle's getting-started docs state that client ID/secret/key are issued by emailing api@puzzle.io or a Puzzle contact with a list of redirect URIs — there is no self-serve signup or API key generation flow. This is a product/process characteristic, not an agent error or environment limitation, and it fully blocked any live API call. Event 51: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. You will need to provide us with a list of valid redirect URIs and we will provide you with a client ID, a client secret, and a key. ``` - **OAuth2 authorization requires a human browser consent step**: Even with credentials, Puzzle's only documented auth flow needs a user to open an authorize URL, log into an existing Puzzle account, and approve access before a code (and thus token) can be obtained. This is standard OAuth behavior, not a defect, but it is an unresolved blocker for unattended agent-driven onboarding. Event 51: ```text The user is redirected to Puzzle where they can log in to their account (or create a new account) and authorize the integration with your application for one of their companies. ``` - **Confirmed no anonymous access to hosted endpoints**: The agent tested the boundary with real requests: an unauthenticated GET to /rest/v0/me returned 401 on both sandbox and production, and a token exchange with placeholder client credentials returned invalid_client. This confirms the auth wall is real product behavior, not a broken test. Event 98: ```text {"errors":[{"message":"Missing authentication credentials","code":"UNAUTHENTICATED"}]} ``` Event 104: ```text {"error":"invalid_client"} ``` #### Suggested Changes - **Add a self-serve sandbox key option to the Getting Started page**: On puzzle-api.readme.io/docs/getting-started, the only path to a client ID/secret is emailing api@puzzle.io — a manual, unattended-unfriendly step. Adding a self-serve sandbox application/key generation flow (even rate-limited) would let developers and agents run the documented OAuth quickstart end-to-end without waiting on a human reply. Verify by confirming a script can obtain sandbox credentials programmatically and successfully call GET /rest/v0/me with a resulting token. Event 51: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/9230ed3c-5d11-41b3-b592-9c6f4ac82139) · [Read transcript](https://www.ax-check.com/puzzle.io/sessions/kimi.json) Final output lists tiered pricing ($30/$60/$100/$300) with explicit assumptions: annual vs monthly billing (~20% higher month-to-month), per-seat/feature scope per tier, and notes embedded API pricing is partnership-based/not public (seq 147). #### End-to-end onboarding - **Onboarding needs a login**: Agent found a real, public Puzzle Embedded Accounting API (OpenAPI spec, docs, live sandbox host) and built a working stdlib Python client covering the full OAuth2 flow. It verified the sandbox responds with documented error codes for bad/missing credentials, but could not obtain real client_id/client_secret itself: the docs state credentials are issued manually by Puzzle staff via email, and the OAuth consent step requires a human browser login. No authenticated call against real company data was ever made. Event 102: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. You will need to provide us with a list of valid redirect URIs and we will provide you with a client ID, a client secret, and a key. ``` Event 110: ```text {"errors":[{"message":"Missing authentication credentials","code":"UNAUTHENTICATED"}]} HTTP 401 ``` Event 134: ```text HTTP 401 from https://staging.southparkdata.com/oauth/token {"error":"invalid_client"} ``` Event 147: ```text Credentials aren't self-serve — you email **api@puzzle.io** with your redirect URIs and they manually issue sandbox `client_id`/`client_secret`. The OAuth consent step also needs a human with a Puzzle account in a browser. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **API credentials require manual human issuance via email**: Puzzle's getting-started docs state sandbox client_id/client_secret are obtained by emailing api@puzzle.io with redirect URIs, not through a self-service signup or console. This is documented product behavior (partner-gated API), not agent error or a test-environment artifact, and it stopped the agent from completing a live authenticated call. Event 102: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. ``` - **OAuth consent flow requires a human browser login**: Even with credentials, the authorization_code flow needs a real user to open the /oauth/authorize URL, log into Puzzle, and grant company access in a browser. This is a standard OAuth login requirement, not a product defect, but it is an unresolved need for human interaction within this session. Event 102: ```text The user is directed to the Puzzle authorize URL. 2. The user authorizes your application and grants access to one of their companies. ``` #### Suggested Changes - **Add a self-service sandbox credential option to the API getting-started page**: On https://puzzle-api.readme.io/docs/getting-started, replace or supplement the 'email api@puzzle.io' credential request step with an automated sandbox signup (e.g., a dashboard button issuing test client_id/secret instantly). Check success by having a new developer obtain sandbox credentials end-to-end without any email exchange or manual approval. Event 102: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. ``` - **Publish Embedded API pricing on the API page**: The /api page and pricing page show consumer subscription tiers but no pricing for the Embedded Accounting API itself; add a pricing section or link on https://puzzle.io/api so developers evaluating the API don't have to schedule a call to learn costs. Verify by confirming a developer can find API pricing without contacting sales. Event 53: ```text Request API Access Explore Documentation Become a Partner ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/588bd1bc-fa8a-437e-90cf-c6327ba8f186) · [Read transcript](https://www.ax-check.com/puzzle.io/sessions/qwen.json) Final output gives a full pricing table with plan/seat/AI-credit assumptions (Starter $30/mo 1 seat, Core $72/mo 5 seats, etc.) and separately flags that API/embedded pricing is unpublished, so no unfounded figure is given. #### End-to-end onboarding - **Onboarding needs a login**: Agent researched Puzzle's public marketing site and API docs but never obtained real credentials. Getting-started docs require emailing api@puzzle.io for a sandbox client ID/secret, and auth requires OAuth2 browser consent from a human. Agent confirmed empirically that both production and sandbox API hosts return redirects to login when hit unauthenticated, then stopped without any credential-acquisition or authenticated call. Event 54: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. You will need to provide us with a list of valid redirect URIs and we will provide you with a client ID, a client secret, and a key. ``` Event 60: ```text == https://staging.southparkdata.com/companies 307 == https://api.puzzle.io/companies 307 ``` Event 62: ```text Credentials are gated behind a human. Per `https://puzzle-api.readme.io/docs/getting-started.md`... No self-serve signup, no trial API key. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Sandbox credentials require emailing Puzzle support**: Puzzle's official getting-started guide states sandbox client ID/secret must be requested by email with a list of redirect URIs, meaning there is no self-serve API key generation. This is a product design choice (partner-gated access), not an agent error or test-environment artifact, and it stopped the agent from writing or running any working example. Event 54: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. You will need to provide us with a list of valid redirect URIs and we will provide you with a client ID, a client secret, and a key. ``` - **OAuth2 flow requires a human browser login**: Even with credentials, Puzzle's authentication flow requires a user to be redirected to a Puzzle-hosted authorize page to log in and approve access in a browser, which cannot be completed headlessly by an agent. This is normal OAuth product behavior, not a defect. Event 54: ```text The user is directed to the Puzzle authorize URL. 2. The user authorizes your application and grants access to one of their companies. 3. The user is redirected to your application with an authorization code. ``` - **Unauthenticated API calls redirect to login**: Direct calls to both the production and sandbox company-list endpoints without an access token returned 307 redirects toward login, confirming no anonymous or trial access path exists. This is expected API security behavior given the partner-gated model. Event 60: ```text == https://staging.southparkdata.com/companies 307 == https://api.puzzle.io/companies 307 ``` #### Suggested Changes - **Add a self-serve sandbox key option to the Getting Started page**: On https://puzzle-api.readme.io/docs/getting-started, replace or supplement the email-based credential request with an instant sandbox client ID/secret signup flow (similar to how Stripe or other API-first products issue test keys immediately). Verify the fix by confirming a new developer can obtain a working sandbox key without contacting api@puzzle.io. Event 54: ```text Request your sandbox credentials by emailing us at or contacting your point of contact at Puzzle directly. ``` - **Publish an SDK or code sample for the Accounting API**: No official SDK exists on npm or PyPI for Puzzle's Accounting API (only an unrelated @puzzlehq/* privacy/Aleo package appears in npm search results). Add a first-party SDK or copy-pasteable code sample near https://puzzle-api.readme.io/docs/getting-started so developers can validate the API surface without hand-rolling OAuth token exchange. Verify by confirming the SDK package installs and completes a token exchange against the sandbox server. Event 59: ```text xero-node | Xero NodeJS OAuth 2.0 client for xero-node accounting | number, money and currency formatting library @mergeapi/merge-sdk-typescript | NodeJS client for Merge API, Inc's unified API's. ``` ### Task given to each agent Help me build a simple example using Puzzle. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 84/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html even when text/markdown was requested; no Markdown representation served. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists core product, products, firm/company pages, comparisons, resources, help and developer docs. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under clear headings with descriptive summaries guiding navigation. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions API/MCP page and links developer documentation; skills not offered. ``` - **Pass** — A compact guide representation exists ```text llms.txt is a compact Markdown index of Puzzle docs, products, and API/MCP surfaces. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt and the API getting-started .md page are directly retrievable Markdown guides. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 1840 tokens; getting-started Markdown is 2597 tokens, both well under budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown is unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text llms.txt is a compact, organized index with descriptions and starting guidance for docs, API and MCP. ``` - **Pass** — Install and next-step links resolve ```text Homepage links to pricing, API/MCP, help center and signup all resolve with 200 responses. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt links Developer Documentation; Getting Started page gives concrete OAuth2 first steps and endpoints. ``` - **Skipped** — Installation commands are extractable ```text No CLI or SDK install commands offered; only API credentials and OAuth flow documented. ``` - **Pass** — Code examples are available without interaction ```text Getting Started shows inline HTTP, JSON, env and HTML examples without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Credentials via api@puzzle.io, OAuth2 flow, Bearer tokens and 24h expiry are explicit. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders plan tiers and prices as static text without interaction. ``` - **Pass** — Prices are stated, not gated ```text Starter $30, Core $72, Complete $120, Scale $360/mo stated openly. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: per month, seats, AI credits, expense tiers, add-on rates. ``` - **Pass** — Agents identify pricing and its assumptions ```text 2 of 3 sessions were judged on pricing; 0 fell short. Kimi K3: Final output lists tiered pricing ($30/$60/$100/$300) with explicit assumptions: annual vs monthly billing (~20% higher month-to-month), per-seat/feature scope per tier, and notes embedded API pricing is partnership-based/not public (seq 147). Qwen 3.8 Max: Final output gives a full pricing table with plan/seat/AI-credit assumptions (Starter $30/mo 1 seat, Core $72/mo 5 seats, etc.) and separately flags that API/embedded pricing is unpublished, so no unfounded figure is given. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Developer docs at puzzle-api.readme.io document OAuth2 auth, endpoints, and an OpenAPI-based API reference. ``` - **Pass** — An MCP server is documented and well-formed ```text API/MCP page documents read-only, user-permissioned MCP for Claude, ChatGPT, Cursor, Windsurf. ``` - **Skipped** — A CLI install path is documented ```text No CLI install path is documented in the fetched pages. ``` - **Skipped** — SDK packages resolve on their registries ```text No SDK package registry result was supplied for Puzzle's own packages. ``` - **Skipped** — Agent skills are published ```text No agent skills are published or documented in the fetched evidence. ``` [Full report data](https://www.ax-check.com/puzzle.io/report.json)