# AX Check: pilot.com
Checked 2026-09-19.

Pricing is on the page, but agents don't always find it
Pilot states real dollar figures (Essentials $99/mo, CFO Basic $1,750/mo) without login, yet 1 of 3 agent sessions refused to quote a price at all.

## End-to-end onboarding not demonstrated

## Coding sessions
All three sessions completed. Two produced concrete tiered pricing from the site, one citing exact Pilot figures and one flagging its numbers as possibly outdated general knowledge. The third refused to give any price, calling it quote-based only. No session encountered a login wall or local setup step.

### DeepSeek V4 Pro
[View public run](https://agents.withgauge.com/p/runs/18744859-cac6-45ce-86e9-bfb756e71649) · [Read transcript](https://www.ax-check.com/pilot.com/sessions/deepseek.json)
Final output gives concrete tiered pricing ($99/mo Essentials for ≤$100k/mo expenses, ~$1,000-2,450/yr tax filings, $250-400/hr CFO, 20% R&D contingency) with the plan/tier and volume assumptions each figure depends on.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: The agent never located a developer-facing product to onboard into. It spent the entire session searching the web to identify which real-world product is named "Pilot," landed on pilot.com (a bookkeeping/tax/CFO services firm), confirmed there is no public API, SDK, or docs, and stopped without ever obtaining or using any credentials.
  Event 143:

  ```text
  HTTP/2 401 
  ... {"message":"unauthorized"}
  ```
  Event 126:

  ```text
  /developers -> 404 https://pilot.com/developers
  /api -> 404 https://pilot.com/api
  /docs -> 404 https://pilot.com/docs
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No developer product exists for "Pilot" at pilot.com**: The agent identified pilot.com as the canonical "Pilot" product and found it is a bookkeeping/tax/CFO services company with no public API, SDK, or docs. Requests to /api, /docs, and /developers all returned 404, and the discovered api.pilot.com subdomain returned 401 unauthorized with CORS locked to the internal app.pilot.com origin only, confirming it is a private, non-public API. This is product behavior (no public developer offering), not agent error or missing credentials the agent could self-serve.
  Event 126:

  ```text
  /developers -> 404 https://pilot.com/developers
  /api -> 404 https://pilot.com/api
  /docs -> 404 https://pilot.com/docs
  /integrations -> 200 https://pilot.com/integrations
  ```
  Event 145:

  ```text
  HTTP/2 401 
  ... access-control-allow-origin: https://app.pilot.com
  ... {"message":"unauthorized"}
  ```

#### Suggested Changes
None identified in this transcript.

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/bee32bec-d589-4df2-a08d-0f563af2af46) · [Read transcript](https://www.ax-check.com/pilot.com/sessions/kimi.json)
Final output gives tiered pricing (Core ~$299/mo, Select ~$549/mo, Plus custom) explicitly labeled 'from general knowledge — may be outdated, verify at pilot.com/pricing', naming the assumption basis.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: The agent never obtained or used real Pilot credentials. It probed the public api.pilot.com endpoint unauthenticated and received 401 unauthorized on every route it tried, then concluded no developer workflow exists and stopped without attempting any signup or authenticated call.
  Event 18:

  ```text
  HTTP/2 401 
  date: Sat, 19 Sep 2026 07:41:38 GMT
  content-type: application/json
  server: TwistedWeb/26.4.0
  x-pilot-request-id: 81c2acee-570b-49e9-9fc7-b333dc252038
  ```
  Event 33:

  ```text
  No Pilot credentials in the environment (and no self-serve way to get any — signup is sales/web-form driven).
  ```

#### Hallucinated URLs
- **Guessed API and docs paths on api.pilot.com**: The agent guessed several endpoint paths (/v1, /docs, /openapi.json, /v1/pricing, /v1/health) on api.pilot.com without any documentation pointing to them. All returned 404 'not found on the server' rather than a real failure of a documented link, so this is a plausible-but-unconfirmed guess rather than a demonstrated hallucination causing a failed workflow step (the agent used the 404s correctly as evidence there is no public API, not as a broken action).
  Event 23:

  ```text
  {"id":"ff35c50b-b3b6-445c-b20a-e690e8b0099e","message":"The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again."}
  ```
  Event 31:

  ```text
  {"id":"be11457d-5b2f-4859-a5b0-24d821ea6e92","message":"The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again."}
  ```

#### Blockers
- **No public developer API or SDK for Pilot.com**: The only reachable host, api.pilot.com, rejects every request with 401 unauthorized and is CORS-restricted to the internal web app (app.pilot.com) using private session headers. This is product behavior: Pilot.com's backend appears to be a private API for its own dashboard, not a public developer product, so there was nothing for the agent to build against.
  Event 31:

  ```text
  access-control-allow-origin: https://app.pilot.com
  ```
  Event 31:

  ```text
  access-control-allow-headers: content-type, x-pilot-app-version, x-pilot-tab-id, x-pilot-tab-created, x-pilot-credstash-prefix, x-pilot-pytest-id, sentry-trace, baggage, traceparent
  ```
- **No credentials or signup path available in session**: The sandbox environment contained no Pilot API keys or tokens, and there was no self-serve signup flow discovered; the agent noted signup appeared to be sales/web-form driven, which it could not complete autonomously. This is a missing-credentials limitation of the test environment/product combination, not clearly a product defect.
  Event 33:

  ```text
  No Pilot credentials in the environment (and no self-serve way to get any — signup is sales/web-form driven).
  ```

#### Suggested Changes
- **Publish developer API documentation or an OpenAPI spec at a discoverable URL**: Requests to api.pilot.com/docs and api.pilot.com/openapi.json both returned 404, with no redirect to real developer documentation. If a public API is intended to exist, publish it at a documented, discoverable path (e.g., linked from pilot.com) and verify by confirming curl to that path returns valid docs/spec content instead of a 404.
  Event 23:

  ```text
  {"id":"0d8cebd0-41d7-4b9a-9139-1dcbcba85fe8","message":"The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again."}
  ```

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/ca59b545-0da5-4106-9a74-55165ac63172) · [Read transcript](https://www.ax-check.com/pilot.com/sessions/qwen.json)
Final output explicitly refuses to state a cost ('I can't tell you reliably, and I won't guess'), only describing pricing as 'quote-based' with no actual figure — no number is ever given, just a model description.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: Agent never obtained or used real Pilot credentials. It searched the local sandbox for a Pilot SDK/API, found nothing (no env vars, no package, no docs), and stopped without attempting any authenticated call against a hosted product.
  Event 14:

  ```text
  So I have no SDK, no API surface, and no credentials to build even a minimal example against. Writing one would mean inventing endpoints and auth — that's guessing, not building.
  ```
  Event 12:

  ```text
  ERROR: Could not find a version that satisfies the requirement pilot (from versions: none)
  ERROR: No matching distribution found for pilot
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No Pilot SDK, API, or credentials discoverable in the sandbox**: The agent searched the repo, environment variables, PyPI, npm, and local docs for anything related to a 'Pilot' product and found nothing usable: the repo was empty aside from an initial commit, only unrelated PI_* env vars were present, no PyPI package named pilot exists, and the npm 'pilot' package is an unrelated stub. This is a test-environment/setup gap rather than a demonstrated product defect, since no real Pilot credentials or SDK reference were ever provided to the agent.
  Event 5:

  ```text
  repo
  PI_CODING_AGENT=true
  PI_REASONING_LEVEL=medium
  ```
  Event 12:

  ```text
  ERROR: Could not find a version that satisfies the requirement pilot (from versions: none)
  ERROR: No matching distribution found for pilot
  latest: 0.1.1
  
  published over a year ago
  ```
  Event 14:

  ```text
  There's no Pilot developer workflow I can act on in this environment.
  ```

#### Suggested Changes
- **Provide a discoverable Pilot SDK package name or API base URL in onboarding materials**: The agent could not find any package named 'pilot' on PyPI or npm that matched the intended product, nor any documentation, environment variable, or repo scaffold pointing to it. Product owner should confirm the correct package/install name and surface it in a quickstart doc or environment variable so an agent (or new developer) starting from a blank repo can locate it without guessing. Verify by repeating the same blank-repo search (grep docs, pip/npm lookup) and confirming a hit.
  Event 12:

  ```text
  ERROR: No matching distribution found for pilot
  ```
  Event 14:

  ```text
  npm package `pilot` | Exists but is an unrelated 0.1.1 stub, not an official Pilot SDK
  ```

### Task given to each agent
Help me build a simple example using Pilot. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: B · 73/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — Homepage answers Markdown requests

  ```text
  Homepage returned text/html for a Markdown Accept header; no Markdown representation offered.
  ```

- **Pass** — llms.txt provides an actionable documentation index

  ```text
  llms.txt links service hubs, segments, definitions and integrations with key URLs.
  ```

- **Pass** — llms.txt provides navigation guidance

  ```text
  llms.txt is organized into numbered sections with key URLs and official site links.
  ```

- **Skipped** — llms.txt mentions offered API, MCP, and skills

  ```text
  llms.txt lists integrations but no API, MCP or skills surfaces are offered.
  ```

- **Pass** — A compact guide representation exists

  ```text
  llms.txt is a compact, organized guide covering services, segments, terminology and integrations.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  llms.txt fetched directly at /llms.txt with 200 and useful starting guidance.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  llms.txt is 1498 tokens, well under the 8000-token budget.
  ```

- **Skipped** — Product-docs links survive format changes

  ```text
  Homepage Markdown unsupported, so link preservation across formats cannot be measured.
  ```

- **Skipped** — The compact guide is independently actionable

  ```text
  No compact agent guide fetched; llms.txt is an index, not an actionable workflow.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Sampled install/next-step links (talk-to-expert, pricing) fetched successfully with HTTP 200.
  ```


### Onboarding
- **Skipped** — Docs lead to a relevant quickstart

  ```text
  No developer docs or quickstart fetched; llms.txt describes bookkeeping services, not a developer quickstart.
  ```

- **Skipped** — Installation commands are extractable

  ```text
  No installation or CLI/SDK setup instructions appear in the fetched pages.
  ```

- **Skipped** — Code examples are available without interaction

  ```text
  No code examples are present in the fetched llms.txt or talk-to-expert pages.
  ```

- **Skipped** — Prerequisites and auth boundaries are explicit

  ```text
  No API or developer auth boundary is described; pages cover service signup only.
  ```


### Pricing
- **Failed** — Agents identify pricing and its assumptions

  ```text
  3 of 3 sessions were judged on pricing; 1 fell short. DeepSeek V4 Pro: Final output gives concrete tiered pricing ($99/mo Essentials for ≤$100k/mo expenses, ~$1,000-2,450/yr tax filings, $250-400/hr CFO, 20% R&D contingency) with the plan/tier and volume assumptions each figure depends on. Kimi K3: Final output gives tiered pricing (Core ~$299/mo, Select ~$549/mo, Plus custom) explicitly labeled 'from general knowledge — may be outdated, verify at pilot.com/pricing', naming the assumption basis. Qwen 3.8 Max: Final output explicitly refuses to state a cost ('I can't tell you reliably, and I won't guess'), only describing pricing as 'quote-based' with no actual figure — no number is ever given, just a model description. This behavioural item does not affect the fast grade.
  ```

- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders plan names, dollar amounts and billing periods as static text without interaction.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  Prices stated: Essentials $99/month, CFO Basic $1,750/month, Tax from $1,000+/year.
  ```

- **Pass** — Pricing units and limits are explicit

  ```text
  Units and limits explicit: per month, per year, up to $100,000 monthly expenses, 10 vendor bills.
  ```


### Activation
- **Skipped** — An API reference or OpenAPI spec is reachable

  ```text
  No API reference or OpenAPI spec appears in the fetched Pilot service pages.
  ```

- **Skipped** — An MCP server is documented and well-formed

  ```text
  No MCP server is documented anywhere in the fetched Pilot pages.
  ```

- **Skipped** — A CLI install path is documented

  ```text
  No CLI install path is documented in the fetched Pilot pages.
  ```

- **Skipped** — SDK packages resolve on their registries

  ```text
  No SDK or developer package registry result was fetched for Pilot.
  ```

- **Skipped** — Agent skills are published

  ```text
  No agent skills are published in the fetched Pilot pages.
  ```



[Full report data](https://www.ax-check.com/pilot.com/report.json)