# AX Check: summation.com
Checked 2026-09-18.

Docs and CLI setup are solid, but agents can't pin down pricing
Homepage, docs, and CLI install steps are all machine-readable, but pricing page details didn't translate into a clear cost answer in agent testing.

## Onboarding needs a login

## Coding sessions
Three independent sessions attempted the task; one did not complete. Of the two that finished, neither could report a concrete price or stated assumptions — one said pricing had "nothing to report," the other said no product existed to verify pricing against.

### DeepSeek V4 Pro
[View public run](https://agents.withgauge.com/p/runs/b4c956f5-a086-4f7b-964c-05bd9f9d1fd2) · [Read transcript](https://www.ax-check.com/summation.com/sessions/deepseek.json)

#### End-to-end onboarding
- **Onboarding needs a login**: Agent thoroughly researched Summation's API/CLI via docs and OpenAPI spec, confirmed sumcli installs and runs locally, and verified unauthenticated hosted calls return 401. Real access needs either browser device-login (human) or admin-issued M2M client_id/client_secret. No such credentials existed in the sandbox env. Nothing in the transcript shows the agent exchanging real credentials for a token and completing a live authenticated call; session stayed in discovery/docs mode.
  Event 7:

  ```text
  PI_GATEWAY_API_KEY=unused
  ```
  Event 81:

  ```text
  missing Authorization header
  ```
  Event 68:

  ```text
  Your Summation admin issues the `client_id` and `client_secret`. Ideal for CI and unattended jobs: M2M credentials on the profile are enough; no separate login step.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No machine credentials available for the hosted API**: Summation's authenticated endpoints require either human browser device-login approval or admin-issued M2M client_id/client_secret. Neither was present in the sandbox (env scan found nothing usable), and unauthenticated calls to the live API returned 401. This is a normal auth requirement, not a product defect, but it stopped the agent from completing any live authenticated operation within the session.
  Event 7:

  ```text
  PI_GATEWAY_API_KEY=unused
  ```
  Event 81:

  ```text
  missing Authorization header
  ```

#### Suggested Changes
- **Improve search visibility for docs.summation.com under the bare term "Summation"**: General web searches for "Summation API SDK hosted product" surfaced unrelated same-named npm/PyPI packages ahead of the real product docs. Verify by searching that phrase and checking docs.summation.com/api.summation.com rank above unrelated packages.
  Event 16:

  ```text
  {"total":142,"results":[{"package":{"name":"summation","scope":"unscoped","version":"2.0.0","description":"this will add and subtract two numbers"
  ```

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/136ec31d-71bb-423a-90dd-6c1fab14c4cc) · [Read transcript](https://www.ax-check.com/summation.com/sessions/kimi.json)
Final output explicitly states 'Pricing: nothing to report' with no plan/region/machine/workload assumptions given — no cost figure was stated at all.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: No credentials were ever obtained because the agent could not locate an actual hosted "Summation" product to sign up for. All discovered artifacts (a parked domain, an npm package, a PyPI package, a legacy e-discovery product) lacked any developer sign-up, API, or SDK, so no authenticated operation was ever attempted.
  Event 52:

  ```text
  "Summation" has no developer workflow I can act on — no hosted API, no real SDK, no sign-up path. Per your instructions, I'm saying so plainly and stopping rather than scaffolding an example against a toy math package or a dead endpoint.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No hosted "Summation" developer product exists to build against**: The agent searched the likely domain, npm, PyPI, and general web search but found only a parked GoDaddy landing page, a trivial npm arithmetic package, an unrelated PyPI arithmetic package, and a legacy on-prem e-discovery product with no public API. This is a product/naming reality issue (no such SaaS product with a developer workflow), not an agent error or missing credentials — the agent correctly recognized this and stopped per instructions rather than fabricating an example.
  Event 16:

  ```text
  HTTP/2 502 
  server: mitmproxy 12.2.3
  content-type: text/html
  ```
  Event 52:

  ```text
  summation.dev** — a parked GoDaddy site; `api.summation.dev` returns 502 (dead endpoint).
  ```
  Event 52:

  ```text
  npm `summation` (v2.0.0)** — a toy package whose own description is "this will add and subtract two numbers." Not a hosted product, no pricing, no API.
  ```

#### Suggested Changes
None identified in this transcript.

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/d1742137-6443-4574-8269-9d8f382182a3) · [Read transcript](https://www.ax-check.com/summation.com/sessions/qwen.json)
Final output (seq 14) states no pricing figure at all, only that pricing can't be verified since no product exists — no cost number or assumptions given.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: The agent never obtained or used any product credentials. It searched the local repo and public package registries for a product called Summation, found nothing but unrelated hobby packages, and stopped without attempting any authenticated operation.
  Event 14:

  ```text
  Since I have no documentation, no API key, and no identifiable hosted product, there's **no developer workflow I can execute** — I can't build an example, verify pricing, or assess manageability without inventing information.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No identifiable Summation product exists**: The agent could not find any product, SDK, API, docs, or credentials for 'Summation.' The local repo was empty aside from git metadata, no summation package was installed locally, and the only publicly registered packages named 'summation' on PyPI and npm were trivial unrelated hobby libraries (arithmetic add/subtract), not a commercial hosted product. This is a test environment/setup issue rather than a product defect or agent error, since the requested product appears not to exist or wasn't provided to the session.
  Event 6:

  ```text
  (no output)
  
  Command exited with code 1
  ```
  Event 12:

  ```text
  "description":"this will add and subtract two numbers"
  ```
  Event 14:

  ```text
  **There is no "Summation" product I can act on.**
  ```

#### Suggested Changes
- **Provide the correct product name or docs URL for Summation**: Since the agent found only unrelated hobby packages under the name 'summation' on PyPI and npm, the product owner should supply the actual vendor name, quickstart/docs URL, or a sample repo reference when running this test so the agent can locate the real hosted product. Verify the fix by rerunning the session and confirming the agent can find matching docs or an SDK on the first search.
  Event 14:

  ```text
  A URL to Summation's docs or quickstart
  ```

### Task given to each agent
Help me build a simple example using Summation. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: B · 83/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — llms.txt provides an actionable documentation index

  ```text
  /llms.txt returned 404; no documentation index is published at the standard path.
  ```

- **Pass** — Homepage answers Markdown requests

  ```text
  Homepage returned text/markdown (200) when requested with Accept: text/markdown.
  ```

- **Skipped** — llms.txt provides navigation guidance

  ```text
  No llms.txt body was fetched (404), so navigation guidance cannot be judged.
  ```

- **Skipped** — llms.txt mentions offered API, MCP, and skills

  ```text
  No llms.txt body was fetched (404), so API/MCP/skills mentions cannot be judged.
  ```

- **Pass** — A compact guide representation exists

  ```text
  Homepage returns text/markdown via content negotiation; docs pages also serve Markdown.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  Docs pages like /api/cli and /api/overview are directly retrievable as Markdown.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  Homepage Markdown is 1248 tokens, well under the 8000-token budget.
  ```

- **Pass** — Product-docs links survive format changes

  ```text
  Homepage Markdown retains Product, Deployment, Pricing, Security and Docs links.
  ```

- **Pass** — The compact guide is independently actionable

  ```text
  CLI page gives install commands, auth handling, and JSON output for agent workflows.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Signup and docs links resolve; app.summation.com/signup returns 200.
  ```


### Onboarding
- **Pass** — Docs lead to a relevant quickstart

  ```text
  Docs introduction links to sumcli CLI guide with install, sign-in, and workflow steps.
  ```

- **Pass** — Installation commands are extractable

  ```text
  CLI page gives extractable install commands: uv tool install summation-cli and curl installer.
  ```

- **Pass** — Code examples are available without interaction

  ```text
  CLI guide shows many inline code examples for install, auth, tables, reports without interaction.
  ```

- **Pass** — Prerequisites and auth boundaries are explicit

  ```text
  CLI guide states Python 3.11+/uv prerequisite and device-login or M2M client_id/secret auth.
  ```


### Pricing
- **Failed** — Agents identify pricing and its assumptions

  ```text
  2 of 3 sessions were judged on pricing; 2 fell short. Kimi K3: Final output explicitly states 'Pricing: nothing to report' with no plan/region/machine/workload assumptions given — no cost figure was stated at all. Qwen 3.8 Max: Final output (seq 14) states no pricing figure at all, only that pricing can't be verified since no product exists — no cost number or assumptions given. This behavioural item does not affect the fast grade.
  ```

- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders all tiers and prices directly in fetched Markdown, no interaction needed.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  Free $0, Pro $60/mo, Max $200/mo stated openly; Enterprise custom price.
  ```

- **Pass** — Pricing units and limits are explicit

  ```text
  Each tier lists explicit credit limits: 2k, 12k, 40k credits per month.
  ```


### Activation
- **Pass** — An API reference or OpenAPI spec is reachable

  ```text
  API overview names live OpenAPI document at api.summation.com/openapi.json as source of truth.
  ```

- **Pass** — An MCP server is documented and well-formed

  ```text
  MCP server documented with URL, OAuth, scopes; endpoint returns 401 challenge and OAuth metadata.
  ```

- **Pass** — A CLI install path is documented

  ```text
  CLI page documents uv tool install summation-cli plus curl/irm installers and verification.
  ```

- **Skipped** — SDK packages resolve on their registries

  ```text
  No registry lookup for summation-cli or any Summation SDK package was supplied.
  ```

- **Pass** — Agent skills are published

  ```text
  Skills page documents SKILL.md bundles, built-in skills, and authoring workflow.
  ```



[Full report data](https://www.ax-check.com/summation.com/report.json)