# AX Check: swarmia.com Checked 2026-11-11. Swarmia's quickstart and pricing are clear and easy to follow A 15-minute Markdown quickstart lays out concrete setup steps, and pricing ($45-$55 per developer/month, free under 10) is stated plainly with no login wall. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) read the public pricing page and correctly reported the same figures: free for under 10 developers, Standard at $45 and Enterprise at $55 per developer per month billed annually, plus add-on modules, all without hitting any login wall. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/bc80fb94-d663-4a17-b02a-16a2c533303b) · [Read transcript](https://www.ax-check.com/swarmia.com/sessions/deepseek.json) Quoted prices from the fetched pricing page and named the basis: per developer seat, annual billing, free up to 9 developers, plan tiers and add-on modules, and higher rates for monthly billing. (seq 24, seq 104, swarmia-example/README.md) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: No self-service credential acquisition occurred. The agent never obtained a real Swarmia API token; it only tested the live endpoints with no token and with a fabricated placeholder string, both of which correctly returned 401 Unauthorized. The agent explicitly states the token is a human-supplied prerequisite it does not have, so no authenticated operation against the hosted product was demonstrated. Event 99: ```text SWARMIA_API_TOKEN="fake-token-for-demo" python3 swarmia_report.py --days 30 ``` Event 101: ```text HTTP 401: {"error":true,"message":"Unauthorized"} token missing/invalid - create one at /settings/api-tokens ``` Event 104: ```text I could not pull real metrics because no API token is available in this environment — that's the only blocker, and it's a credential, not a product limitation. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No API token available to call real Swarmia data**: The hosted Swarmia API requires a bearer token created via Settings -> API tokens inside a real Swarmia account. The sandbox had no such credential and no way to self-provision one (Swarmia requires a signup/login flow, not an open registration endpoint). This is a missing-credentials limitation of the test environment, not a product defect: both unauthenticated and fake-token requests correctly returned structured 401 errors. Event 101: ```text HTTP 401: {"error":true,"message":"Unauthorized"} token missing/invalid - create one at /settings/api-tokens ``` Event 104: ```text I could not pull real metrics because no API token is available in this environment — that's the only blocker, and it's a credential, not a product limitation. ``` #### Suggested Changes - **Document the required token scope for v1 built-in reports endpoints**: The built-in-reports.md page and its OpenAPI schema describe bearerToken/queryToken security but never name which scope (e.g. Entity Query, as used for the legacy v0 equivalents) is required for /reports/metrics/code and /reports/metrics/dora. Add an explicit scope name next to each v1 endpoint in https://help.swarmia.com/settings/integrations/swarmia-apis/built-in-reports, and verify by asking the GitBook doc assistant the same scope question and confirming it returns a direct answer instead of 'the docs don't confirm that it applies.' Event 88: ```text The v1 documentation doesn’t specify the required token scope for `/reports/metrics/code` or `/reports/metrics/dora`. It says the token must have the endpoint’s required scope, but doesn’t name it in the [Built In Reports] ``` Event 87: ```text The `/pullRequests`, `/dora`, and `/investment` endpoints require an API token with the **Entity Query** scope. ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/db0e88b5-3c0c-41b1-8c1b-5c601055cade) · [Read transcript](https://www.ax-check.com/swarmia.com/sessions/kimi.json) Quoted prices from the pricing page with their basis: free up to 9 developers, Standard $45 and Enterprise $55 per developer per month billed annually, add-on module prices, and a 14-day trial. (seq 38, seq 75, seq 17, seq 99) #### End-to-end onboarding - **Onboarding needs a login**: The agent found a real hosted Swarmia API with a documented OpenAPI spec and confirmed the endpoint is live, but every endpoint returned 401 Unauthorized without a token. API tokens can only be created by a human inside an existing Swarmia org via the app UI after OAuth signup, so the agent could not self-serve credentials or perform any authenticated operation. It stopped and explicitly reported being blocked rather than faking success. Event 76: ```text {"error":true,"message":"Unauthorized"} HTTP 401 openapi: 200 ``` Event 99: ```text I cannot run the example end-to-end. Every endpoint requires an API token, and tokens can only be created by a human inside an existing Swarmia organization (Settings → API tokens), after signing up and connecting a GitHub/Azure DevOps org via OAuth. ``` Event 97: ```text urllib.error.HTTPError: HTTP Error 401: Unauthorized ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **API token requires human-driven OAuth signup**: The Export API and REST API both require a Bearer token, and tokens are only created from inside the Swarmia app's Settings page after a human has signed up and connected a GitHub/Azure DevOps org through OAuth. This is normal product authentication behavior, not a defect, but it is an unresolved credential gap for an unattended agent since no self-serve token or sandbox org exists. Event 99: ```text tokens can only be created by a human inside an existing Swarmia organization (Settings → API tokens), after signing up and connecting a GitHub/Azure DevOps org via OAuth. There's no sandbox, demo workspace, or unauthenticated endpoint I can act on ``` Event 76: ```text {"error":true,"message":"Unauthorized"} HTTP 401 ``` #### Suggested Changes - **Add a sandbox or demo-org token option to the API docs**: On the Swarmia APIs doc page (help.swarmia.com/settings/integrations/swarmia-apis), consider offering a way to obtain a scoped demo token or trial-org API key without the full OAuth git-provider signup flow, so developers can test the Export/REST API before committing an org. Verify by having a fresh, un-authenticated user attempt to call /reports/dora and succeed without first connecting a GitHub/Azure DevOps org. Event 99: ```text There's no sandbox, demo workspace, or unauthenticated endpoint I can act on, and I can't complete the OAuth org-connection flow myself. ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/7c3acf76-6945-4025-8b0a-6ce84fb37828) · [Read transcript](https://www.ax-check.com/swarmia.com/sessions/qwen.json) Quoted Free (≤9 devs), Standard $45, Enterprise $55 and à-la-carte plans, framed as per developer per month, billed annually, with a no-card trial. Also said the CLI runs free locally. Figures came from the fetched pricing page. (seq 23, seq 137, seq 218) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent never obtained a Swarmia account, workspace, or CLI token. It only installed the open-source @swarmia/cli npm package and fed it hand-crafted hook payloads (not a real coding-agent session) entirely offline. The 'report' output comes from a local JSONL log file and a local per-session JSON record under /sandbox/.local/state; no token was configured, so swarmia-cli send --dry-run always returned 0 sessions and nothing was ever transmitted to any hosted Swarmia endpoint. The agent explicitly acknowledges this: no account, no network calls, nothing sent without a token. Event 61: ```text token: (not set — nothing will be sent) ``` Event 127: ```text 0 session(s) would be sent to https://hook.swarmia.com/cli (see `swarmia-cli report --json`) ``` Event 218: ```text Two honest limits: without a token nothing is sent, and only **pushed** commits are sendable — this repo has no remote, so `send --dry-run` correctly reports `0 session(s)`. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No hosted API/SDK surface reachable without a Swarmia account**: The product's actual engineering-intelligence features (dashboards, metrics, integrations) only live behind the authenticated web app and require a workspace/token the agent was never issued. This is a product/credential limitation, not an agent error: the agent correctly identified the write-only CLI ingest endpoint as the only self-serve surface and chose to demonstrate it locally rather than claim a false hosted success. Event 218: ```text The hosted product: no, not really.** Everything else (GitHub/issue-tracker/AI-tool integrations, teams, surveys, capitalization, dashboards) is configured in the web app; there's no general public REST API, SDK, or Terraform provider I can drive ``` - **api.swarmia.com does not resolve from the sandbox**: An early curl to a guessed API host failed DNS resolution entirely (test-environment network restriction, not a broken product link), and the agent correctly pivoted to the CLI's actual documented endpoint (hook.swarmia.com) instead. Event 12: ```text curl: (6) Could not resolve host: api.swarmia.com ``` Event 24: ```text api.swarmia.com: curl: (6) Could not resolve host: api.swarmia.com 000 ``` - **Bash tool-call pairs silently produced no attributed edits without a shared tool_use_id**: Agent error / undocumented-at-a-glance requirement: the first version of the demo script omitted tool_use_id on Pre/PostToolUse Bash events, so the codegen script's lines were never snapshotted or attributed, even though the hook calls exited 0 with no visible error. The agent diagnosed this by reading the CLI source and fixed it by adding matching tool_use_id values. Event 75: ```text Session demo-179 claude-sonnet-4-5 2026-11-11 17:01 → 17:01 1 prompt(s), 0 response(s), 1 edit(s), 9 AI line(s) at session end ``` Event 99: ```text func snapshotShellCommand(store snapshot.Store, ev agent.Event, rec *events.Record) { if !ev.Tool.Shell || ev.Tool.UseID == "" { return } ``` #### Suggested Changes - **Document the tool_use_id requirement for Bash hook pairs in the CLI README**: In swarmia-cli's README 'Capture' section, add an explicit note that PreToolUse/PostToolUse payloads for shell commands must share the same tool_use_id or the snapshot silently records nothing (no error, no stderr). Verify by replaying a Bash pair without tool_use_id and confirming swarmia-cli report now shows a non-zero edit count for that file. Event 99: ```text if !ev.Tool.Shell || ev.Tool.UseID == "" { return } ``` Event 75: ```text 1 prompt(s), 0 response(s), 1 edit(s), 9 AI line(s) at session end ``` - **Clarify in docs that api.swarmia.com is not a valid host**: Since developers may reasonably guess a REST API lives at api.swarmia.com, add a line to the CLI README or help docs stating the only network endpoint used is hook.swarmia.com/cli, and that there is no general public REST API. Verify by checking the updated README renders this before the 'Sending sessions to Swarmia' section. Event 12: ```text curl: (6) Could not resolve host: api.swarmia.com ``` ### Task given to each agent Help me build a simple example using Swarmia. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 81/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a Markdown Accept header; no Markdown representation offered. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists product, solutions, learn, customers and company docs with descriptions. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under clear headings with descriptive labels for navigation. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions Swarmia AI and an MCP server for Claude and Cursor. ``` - **Pass** — A compact guide representation exists ```text help.swarmia.com serves a standalone Markdown quickstart at get-started-in-15-minutes.md. ``` - **Pass** — A focused guide is directly retrievable ```text The .md quickstart is directly retrievable with concrete setup steps and links. ``` - **Pass** — Equivalent instructions fit a token budget ```text Markdown guide is 1974 tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text 15-minute guide gives concrete steps: sign up, connect code host, issue tracker, AI tools, messaging, teams. ``` - **Pass** — Install and next-step links resolve ```text Signup, settings, and detailed integration links fetched successfully from the getting-started guide. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt links Help & docs; docs index leads to 'Get started in 15 minutes' quickstart with concrete steps. ``` - **Skipped** — Installation commands are extractable ```text No CLI or SDK install commands; onboarding is web-app integration setup, not package installation. ``` - **Pass** — Code examples are available without interaction ```text Quickstart shows inline numbered steps and commands without interaction; Markdown version available. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Quickstart states required admin permissions per integration and sign-up auth methods. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page lists Standard $45, Enterprise $55, and feature plans with per-developer monthly rates. ``` - **Pass** — Prices are stated, not gated ```text Public pricing page states concrete prices: $45, $55, $5, $9, $18, $23 per developer/month. ``` - **Pass** — Pricing units and limits are explicit ```text Prices are per developer per month, billed annually; free plan for under 10 developers. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Quoted prices from the fetched pricing page and named the basis: per developer seat, annual billing, free up to 9 developers, plan tiers and add-on modules, and higher rates for monthly billing. (seq 24, seq 104, swarmia-example/README.md) Kimi K3: Quoted prices from the pricing page with their basis: free up to 9 developers, Standard $45 and Enterprise $55 per developer per month billed annually, add-on module prices, and a 14-day trial. (seq 38, seq 75, seq 17, seq 99) Qwen 3.8 Max: Quoted Free (≤9 devs), Standard $45, Enterprise $55 and à-la-carte plans, framed as per developer per month, billed annually, with a no-card trial. Also said the CLI runs free locally. Figures came from the fetched pricing page. (seq 23, seq 137, seq 218) This behavioural item does not affect the fast grade. ``` ### Activation - **Skipped** — An API reference or OpenAPI spec is reachable ```text No API reference or OpenAPI spec fetched; llms.txt mentions a team API but no reference page. ``` - **Skipped** — An MCP server is documented and well-formed ```text MCP server is mentioned on the Swarmia AI page and llms.txt, but no setup docs were fetched. ``` - **Skipped** — A CLI install path is documented ```text No CLI install path documented in the fetched pages; only web app setup steps appear. ``` - **Skipped** — SDK packages resolve on their registries ```text No SDK or package registry lookup was supplied for any Swarmia developer package. ``` - **Skipped** — Agent skills are published ```text No agent skills are published or referenced in the fetched Swarmia documentation. ``` [Full report data](https://www.ax-check.com/swarmia.com/report.json)