# AX Check: pipeworx.io Checked 2026-10-12. Agents can install, connect, and price Pipeworx without a human 21 of 23 checklist items passed: docs, quickstart, CLI, SDK, and MCP setup all resolved cleanly, with a full rate card stated in plain prices. ## Onboarded with real credentials ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed and explained pricing with explicit assumptions across anonymous, free, and paid tiers; Kimi K3 noted some paid-tier figures couldn't be fully confirmed from the pricing page. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/da548a1f-9c32-4fb2-99d0-8c276deb8ab4) · [Read transcript](https://www.ax-check.com/pipeworx.io/sessions/deepseek.json) Pricing was explained with explicit assumptions — anonymous vs free vs paid tiers, credit-to-dollar conversion, per-category rate ranges, and how subscriptions offset usage. Evidence: Final output (seq 93) lists credit value, free-tier call limits, rate-card ranges by data category, and subscription-as-minimum-credit behavior, all tied to the real `_meta.cost` fields observed at seq 64 and 78. #### End-to-end onboarding - **Onboarded with real credentials**: Agent used the hosted Pipeworx gateway's anonymous tier (a legitimate self-service product tier, no human-supplied secrets) and performed real operations against the live service: calling get_weather and ask_pipeworx and getting back live structuredContent plus _meta billing/quota data, then running a script that executed successfully with real output and decrementing quota. This is a genuine authenticated-tier operation on the actual hosted product. Event 60: ```text structuredContent: {"location": "San Francisco", "country": "United States", "latitude": 37.77493, "longitude": -122.41942, "temperature_f": 63.7 ``` Event 70: ```text weather: 63.7 F, Clear sky answer: FRED MORTGAGE30US (30-Year Fixed Rate Mortgage Average in the United States): Percent, Weekly, Ending Thursday ``` Event 60: ```text _meta: {"tier": "anonymous", "rateLimit": {"limit": 50, "remaining": 48, "resetAt": "2026-10-13T00:00:00.000Z"} ``` #### Hallucinated URLs None identified in this transcript. #### Blockers None identified in this transcript. #### Suggested Changes - **Document when ask_pipeworx cost field is omitted vs populated**: Two similar ask_pipeworx calls returned different cost shapes: one had no cost data (script printed None), another returned a full cost object with 50 credits ($0.0050). Clarify in the pricing FAQ or raw-jsonrpc doc what determines cost field presence for routed answers, so developers do not need defensive fallback code when parsing _meta.cost. Verify by repeating similar ask_pipeworx calls and confirming cost presence is predictable from the docs. Event 70: ```text cost: None quota: {'limit': 50, 'remaining': 47, 'resetAt': '2026-10-13T00:00:00.000Z'} ``` Event 74: ```text "cost": { "total": 50, "components": [ { "name": "base", "credits": 50 } ], "usd": "$0.0050" }, ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/8d6ca625-268c-4d16-b3a6-38fc1448a448) · [Read transcript](https://www.ax-check.com/pipeworx.io/sessions/kimi.json) The agent stated the product is free for anonymous rate-limited use, with an optional free key for higher limits, and explicitly flagged that paid-tier pricing couldn't be confirmed because the pricing page was unreachable. Evidence: Final output and seq 54-55 state cost basis (anonymous free, IP rate-limited, free API key) sourced from the README, with explicit caveat that paid tiers are unconfirmed. #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: No account or API key was created during the session. All calls were anonymous (rate-limited by IP) against the Pipeworx hosted gateway via the npx CLI. The agent explicitly ran everything without a PIPEWORX_API_KEY and did not sign up for one, so there's no credential-acquisition step to verify — only anonymous usage of a public, keyless endpoint. Event 29: ```text "location": "Lisbon", "country": "Portugal", "latitude": 38.72509 ``` Event 55: ```text I ran everything above anonymously with no throttling. I **could not verify paid tiers** — pipeworx.io itself returned 502 through this sandbox's proxy ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Main website unreachable through sandbox proxy**: pipeworx.io returned a 502 through the sandbox's mitmproxy, which blocked verification of paid pricing tiers and the account signup page. This is a test-environment network restriction (proxy behavior), not a confirmed product defect, since the API itself (served separately from the marketing site) worked fine. Event 8: ```text HTTP/2 502 server: mitmproxy 12.2.3 content-type: text/html ``` - **Initial tool call errors from guessed arguments**: The agent initially called lookup_zipcode without the required country field and weather_get_forecast assuming a .forecast[] JSON shape that didn't match the actual .days[] response. Both were self-inflicted argument-shape mistakes (agent error), not product failures, and both were corrected within the same turn using the API's own structured error messages. Event 42: ```text "error": "invalid_arguments", "message": "lookup_zipcode received an invalid argument: country is required but was not provided. ``` Event 53: ```text -20 | jq -r '.forecast[] | "\(.date): \(.high_f)F / \(.low_f)F"' +20 | jq -r '.days[] | "\(.date): \(.high_f)F / \(.low_f)F — \(.conditions)"' ``` #### Suggested Changes - **Document the actual forecast response field name in the README**: The README's compose example uses `.forecast[]` for weather_get_forecast output, but the live API returns data under `.days[]`. Update the README/quickstart snippet at the 'It composes' section to match the real schema, and confirm by re-running the documented jq pipeline against a live call without needing a manual fix. Event 48: ```text "location": "Tokyo", "country": "Japan", "latitude": 35.6895, "longitude": 139.69171, "days": [ ``` Event 53: ```text +20 | jq -r '.days[] | "\(.date): \(.high_f)F / \(.low_f)F — \(.conditions)"' ``` - **Add country as a visible required parameter in lookup_zipcode's quickstart example**: The README shows `lookup_zipcode --json '{"zipcode":"90210"}'` without the required `country` field, which fails on first try. Update the README example to include country so a first-time developer copying it doesn't hit an avoidable error; verify by running the exact README snippet end-to-end. Event 42: ```text "required": [ "country", "zipcode" ] ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/a66d0054-537f-4a87-8604-85021e3993bd) · [Read transcript](https://www.ax-check.com/pipeworx.io/sessions/qwen.json) Pricing was broken down by tier (anonymous, free account, pay-as-you-go, subscription) with the specific assumptions (data category, request volume) driving each figure. Evidence: Final summary lays out credit unit, anonymous/free/paid tiers, per-category rate card, and subscription break-even math (seq 79/get_final_output). #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent never obtained its own Pipeworx account credentials (no signup flow, no API key generation). It only used the anonymous tier (no auth, 50 calls/day) to call the hosted gateway and successfully pulled live FRED data. This demonstrates a real authenticated-free (anonymous) operation against the hosted product, but since no credentials were acquired or used, self-service onboarding with real credentials was never exercised. Event 68: ```text "tier": "anonymous", "rateLimit": { "limit": 50, "remaining": 49, "resetAt": "2026-10-13T00:00:00.000Z" }, "source": "fred", ``` Event 77: ```text [rate limit] 49/50 calls left today 30-Year Fixed Rate Mortgage Average in the United States ``` #### Hallucinated URLs None identified in this transcript. #### Blockers None identified in this transcript. #### Suggested Changes - **Fix the bare-domain pricing/docs links to avoid redirect roundtrips**: Direct requests to https://pipeworx.io/pricing, /docs/getting-started/raw-jsonrpc, /packs, and /account all returned 308 redirects (to trailing-slash URLs) before resolving, per the curl checks at seq 48 and seq 54. Serving these canonical paths directly (or documenting the trailing slash) would save agents and scripts an extra hop. Verify by curling https://pipeworx.io/pricing and confirming a 200 instead of a 308. Event 48: ```text /pricing -> 308 (0 bytes) ``` Event 54: ```text 200 https://pipeworx.io/pricing/ ``` ### Task given to each agent Help me build a simple example using Pipeworx. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 84/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a text/markdown request; no Markdown representation offered. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links docs, agent guide, OpenAPI spec, catalog and pricing resources. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt gives connect instructions, per-client steps, vertical URLs and scope-by-topic guidance. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt documents the MCP gateway, OpenAPI spec, meta-tools, resources and prompts. ``` - **Pass** — A compact guide representation exists ```text llms.txt is a standalone Markdown guide with quickstart HTTP calls, connect config and pricing. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt and the AI-agents page give concrete first calls, endpoints and connect steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 8059 tokens, a compact standalone guide covering the essential workflow. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown is unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text llms.txt gives zero-setup HTTP calls, tool discovery, /ask, errors, and connect commands. ``` - **Pass** — Install and next-step links resolve ```text Fetched install, docs, SDK, and getting-started pages all returned 200. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Getting-started page offers a 10-minute guided Pipeworx 101 path with per-client quickstarts. ``` - **Pass** — Installation commands are extractable ```text Install page gives concrete commands: claude mcp add, gemini extensions install, npx pipeworx use. ``` - **Pass** — Code examples are available without interaction ```text Docs show curl POST examples, JSON configs, and SDK import snippets without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Anonymous 50/day tier and free-account key via signup are stated explicitly. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders a full rate card and plan table as static text, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Concrete prices stated: $0.005/$0.010/$0.020 per request, $49 and $99 monthly plans. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: per-request rates by source category, daily call limits, credit peg, included request counts. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Pricing was explained with explicit assumptions — anonymous vs free vs paid tiers, credit-to-dollar conversion, per-category rate ranges, and how subscriptions offset usage. Evidence: Final output (seq 93) lists credit value, free-tier call limits, rate-card ranges by data category, and subscription-as-minimum-credit behavior, all tied to the real `_meta.cost` fields observed at seq 64 and 78. Kimi K3: The agent stated the product is free for anonymous rate-limited use, with an optional free key for higher limits, and explicitly flagged that paid-tier pricing couldn't be confirmed because the pricing page was unreachable. Evidence: Final output and seq 54-55 state cost basis (anonymous free, IP rate-limited, free API key) sourced from the README, with explicit caveat that paid tiers are unconfirmed. Qwen 3.8 Max: Pricing was broken down by tier (anonymous, free account, pay-as-you-go, subscription) with the specific assumptions (data category, request volume) driving each figure. Evidence: Final summary lays out credit unit, anonymous/free/paid tiers, per-category rate card, and subscription break-even math (seq 79/get_final_output). This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text OpenAPI 3.1 spec at gateway.pipeworx.io/openapi.json covers /ask, /tools, /v1/tools/{name}. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP gateway documented with connect URLs, tools, resources, prompts, and per-client setup. ``` - **Pass** — A CLI install path is documented ```text CLI install path documented: npx pipeworx use ask-pipeworx, plus npm package. ``` - **Pass** — SDK packages resolve on their registries ```text npm registry lookup for @pipeworx/sdk returned HTTP 200. ``` - **Pass** — Agent skills are published ```text Gemini CLI extension ships a skills/pipeworx/SKILL.md agent skill. ``` [Full report data](https://www.ax-check.com/pipeworx.io/report.json)