# AX Check: loops.so Checked 2026-09-19. Loops.so gives agents everything needed to run and price it fast. All 23 checklist items passed: docs, API, MCP, CLI, and SDK are reachable, and pricing is stated openly with clear limits. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4 Pro, Kimi K3, Qwen 3.8 Max) completed the task and correctly reported free-tier limits (1,000 contacts, 4,000 sends/30 days) and paid plans starting around $49/mo scaling with contact count, citing the pricing page as the source. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/69a79a19-d7f6-4ec2-8467-550dd91610ed) · [Read transcript](https://www.ax-check.com/loops.so/sessions/deepseek.json) Pricing section names concrete assumptions (Free tier: 1,000 contacts/4,000 sends per 30 days; Starter: $49/mo for 5,000 contacts), sourced from live scraping of loops.so/pricing at seq 50-72. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained a real Loops API key. It only found PI_GATEWAY_API_KEY in the sandbox env (unrelated to Loops) and explicitly stated no LOOPS_API_KEY was available. It tested the live endpoint with a placeholder/dummy key and got a 401 Invalid API key, which is the expected credential-less outcome, not a successful authenticated operation. All actual product calls (testApiKey, listTransactionalEmails, findContact, updateContact, sendEvent) were left unexecuted in the example script pending a human-supplied key. Event 6: ```text PI_GATEWAY_API_KEY= GIT_CONFIG_KEY_0= ``` Event 93: ```text HTTP 401 {"success":false,"message":"Invalid API key","error":"Invalid API key"} ``` Event 130: ```text I don't have a valid `LOOPS_API_KEY` (none in the environment), so I can't make a *successful* live call — the API is reachable but rejects every key-guess with `401`. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Loops API key available to run authenticated calls**: The sandbox environment contained no Loops credentials. The agent correctly avoided inventing a key, tried a dummy key against the live endpoint to confirm the auth contract (clean 401), then built the demo script to require a human-supplied LOOPS_API_KEY before any real write/read call against contacts, events, or transactional emails could succeed. This is a credential/test-environment limitation, not a product defect — normal login/API-key generation is expected for any account-scoped SaaS API. Event 93: ```text HTTP 401 {"success":false,"message":"Invalid API key","error":"Invalid API key"} ``` Event 128: ```text === dummy key (expect handled 401) === Loops API error (HTTP 401): { success: false, message: 'Invalid API key', error: 'Invalid API key' } ``` Event 130: ```text I don't have a valid `LOOPS_API_KEY` (none in the environment), so I can't make a *successful* live call ``` #### Suggested Changes - **Consolidate pricing figures onto one crawlable pricing page section**: Scraping https://loops.so/pricing required multiple regex passes across raw HTML/JSON-LD blobs (schema.org price fields, FAQ text, plan descriptions scattered non-contiguously) to reconstruct just two numbers (Free tier limits and the $49/mo Starter plan). Check by re-running a single curl+grep against the pricing page and confirming the free-tier contact/send limits and paid-tier starting price appear in one readable, non-duplicated block near the top of the HTML. Event 52: ```text free" style="--range-progress:32.142857142857146%" value="9"> Free Per month free plan is for early products and smaller audiences. ``` Event 68: ```text 49.00 => Starter plan for up to 5,000 contacts ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/52486ad1-f0c6-44c4-a3dd-afe65c8b5a00) · [Read transcript](https://www.ax-check.com/loops.so/sessions/kimi.json) Final output states free tier (~1,000 contacts) and paid starting ~$49/mo scaling with contact count, and flags the caveat that the JS-rendered pricing page couldn't be verified live — names the basis (contact-count billing) and the uncertainty. #### End-to-end onboarding - **Onboarding needs a login**: No Loops API key existed in the sandbox environment and the agent did not obtain one; it only confirmed the environment lacked credentials (401 from the unauthenticated endpoint) and wrote example code that requires a human-supplied LOOPS_API_KEY to run. No authenticated call to the real Loops API was ever made. Event 7: ```text PI_GATEWAY_API_KEY= GIT_CONFIG_KEY_0= --- ``` Event 8: ```text 401 ``` Event 25: ```text Set LOOPS_API_KEY (get one at https://app.loops.so/settings/api) exit=1 ``` Event 27: ```text there's no `LOOPS_API_KEY` in this environment, so I couldn't make a single authenticated call — the API returns 401, as designed. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing Loops API key blocked any authenticated call**: The sandbox environment had no LOOPS_API_KEY set. This is a credentials/test-environment limitation, not a product defect — Loops requires a normal API key like any hosted SaaS, and the agent correctly did not fabricate one. The example script fails immediately with a clear instructional message when the key is absent. Event 8: ```text 401 ``` Event 25: ```text Set LOOPS_API_KEY (get one at https://app.loops.so/settings/api) exit=1 ``` - **Pricing page could not be scraped for live numbers**: The agent tried to curl loops.so/pricing and grep for dollar amounts, but got no output because the page is JS-rendered and curl only fetches raw HTML. This is a test-environment/tooling limitation (no browser rendering available), not a product defect. The agent flagged the uncertainty rather than presenting stale numbers as fact. Event 16: ```text (no output) ``` Event 19: ```text The pricing page is a JS-rendered app so I couldn't scrape numbers — I'll give pricing from known info with a caveat. ``` #### Suggested Changes - **Serve static/pre-rendered pricing content for non-browser clients**: A plain curl to https://loops.so/pricing returned no extractable price text because the page relies on client-side JS rendering. Product owner can verify by running the same curl command against the pricing URL and confirming dollar figures appear in the raw HTML response, which would let API consumers and simple scripts get accurate pricing without a full browser. Event 14: ```text curl -sS -m 10 https://loops.so/pricing | grep -oiE '\$[0-9,]+' | sort | uniq -c | sort -rn | head -15 ``` Event 16: ```text (no output) ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/dd505cb5-609f-45b5-bfa3-ef27ab2da360) · [Read transcript](https://www.ax-check.com/loops.so/sessions/qwen.json) README.md 'Pricing' section states free-tier limits (4,000 sends/30 days, 1,000 contacts, 10 emails/sec) and paid-tier basis (priced by subscribed contact count), explicitly naming the assumption that exact dollar figures depend on a contact-count slider rather than quoting an unbased number. #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: No Loops API key was ever obtained during the session. The agent checked environment variables early on and found none, then explicitly built a dry-run mode that activates whenever LOOPS_API_KEY is absent. Every SDK call (testApiKey, createContact, sendEvent, createTransactionalEmail, sendTransactionalEmail, etc.) was only exercised via the dry-run branch, which prints hypothetical payloads locally and never contacts the hosted Loops API. No authenticated operation against app.loops.so was demonstrated. Event 6: ```text PI_GATEWAY_API_KEY= ``` Event 107: ```text Dry run: no LOOPS_API_KEY set (or --dry-run passed). Printing the calls that would be made. ``` Event 121: ```text No `LOOPS_API_KEY` in this environment, so steps 0–6 have **not** run against a live account — only the dry run was executed. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing Loops API key blocked any live API call**: The sandbox had no LOOPS_API_KEY (or any Loops credential) in its environment, and no self-service signup flow was attempted to obtain one. This is a normal need-a-login/credential situation, not a product defect: the agent correctly detected the absence and fell back to a dry-run mode rather than faking success. All six SDK steps (testApiKey, contact upsert, sendEvent, transactional email create/draft/publish/send) were only ever exercised as printed dry-run payloads, never against the real app.loops.so API. Event 6: ```text PI_GATEWAY_API_KEY= ``` Event 107: ```text Dry run: no LOOPS_API_KEY set (or --dry-run passed). Printing the calls that would be made. ``` Event 121: ```text No `LOOPS_API_KEY` in this environment, so steps 0–6 have **not** run against a live account — only the dry run was executed. ``` #### Suggested Changes - **Publish exact dollar pricing tiers directly on the pricing page markup**: The agent scraped https://loops.so/pricing and found the paid-plan dollar amounts are rendered client-side via a subscriber-count slider, so no price figures appear in the static page/markup that curl retrieves (only the Free plan's $0 and feature bullets came through). Fix by including at least a representative price table (e.g. price at 1k/10k/100k contacts) in the server-rendered HTML or in the /docs pricing markdown. Verify by re-running a plain curl against the page and confirming a dollar figure appears without executing JavaScript. Event 111: ```text "49.00" ``` Event 121: ```text Exact dollar figures aren't in the page markup (the price is computed by a client-side subscriber-count slider), so I didn't quote a number — check the slider for your contact count. ``` ### Task given to each agent Help me build a simple example using Loops. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 100/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with 200 when requested with Accept: text/markdown. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links docs, API reference, OpenAPI spec, MCP, CLI, skills and guides. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt organizes links under clear sections like For AI agents, Resources, Examples, Tools. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions API reference, OpenAPI spec, MCP server, CLI and agent skills. ``` - **Pass** — A compact guide representation exists ```text Homepage returns text/markdown; docs pages serve .md and negotiated Markdown. ``` - **Pass** — A focused guide is directly retrievable ```text Quickstart.md gives concrete steps: domain records, contacts, first email, transactional send. ``` - **Pass** — Equivalent instructions fit a token budget ```text Quickstart.md is 5085 tokens, well under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown retains docs, API, SDK, and pricing links. ``` - **Pass** — The compact guide is independently actionable ```text Quickstart gives concrete steps: domain DNS, import contacts, build email, send transactional via API. ``` - **Pass** — Install and next-step links resolve ```text Fetched install/next-step pages (quickstart, API intro, skills, MCP) all returned 200. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs quickstart gives concrete steps: set up domain, import contacts, create email, send transactional. ``` - **Pass** — Installation commands are extractable ```text CLI install commands extractable: curl install script, brew install loops-so/tap/loops, npx skills add. ``` - **Pass** — Code examples are available without interaction ```text Code examples shown inline: cURL auth request, transactional JSON payload, CLI commands. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text API key required, obtained from Settings -> API, Bearer header; key must stay server-side. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders as static Markdown with plan details, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Free plan and contact-based pricing stated openly; no gated prices. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: subscribed contacts, 4,000 sends/30 days, 10 emails/sec. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Pricing section names concrete assumptions (Free tier: 1,000 contacts/4,000 sends per 30 days; Starter: $49/mo for 5,000 contacts), sourced from live scraping of loops.so/pricing at seq 50-72. Kimi K3: Final output states free tier (~1,000 contacts) and paid starting ~$49/mo scaling with contact count, and flags the caveat that the JS-rendered pricing page couldn't be verified live — names the basis (contact-count billing) and the uncertainty. Qwen 3.8 Max: README.md 'Pricing' section states free-tier limits (4,000 sends/30 days, 1,000 contacts, 10 emails/sec) and paid-tier basis (priced by subscribed contact count), explicitly naming the assumption that exact dollar figures depend on a contact-count slider rather than quoting an unbased number. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text API reference page documents endpoints and links OpenAPI YAML/JSON specs at app.loops.so. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP docs give remote URL mcp.loops.so, OAuth setup steps, and four tools. ``` - **Pass** — A CLI install path is documented ```text CLI page documents curl and Homebrew install plus auth login command. ``` - **Pass** — SDK packages resolve on their registries ```text npm registry lookup for the loops package returned HTTP 200. ``` - **Pass** — Agent skills are published ```text Skills page and GitHub repo document installable API, CLI, LMX, best-practices skills. ``` [Full report data](https://www.ax-check.com/loops.so/report.json)