# AX Check: preview-pergola-website-dev.apps.pergola.cloud
Checked 2026-10-26.

Docs and pricing are clear, but agents disagree on cost.
18 of 23 checks pass: quickstart, CLI install and pricing page all read cleanly. Still, independent sessions reported conflicting pricing pictures.

## Onboarding needs a login

## Coding sessions
All three sessions completed. DeepSeek V4.1 Flash found and quoted the stated plan and usage rates. Kimi K3 found no live pricing source and quoted nothing. Qwen 3.8 Max concluded the product is free open-source software with no paid tier, since the hosted server it checked was unreachable.

### DeepSeek V4.1 Flash
[View public run](https://agents.withgauge.com/p/runs/7f45f52c-e528-4d3a-a843-951e232517be) · [Read transcript](https://www.ax-check.com/preview-pergola-website-dev.apps.pergola.cloud/sessions/deepseek.json)
Quoted base and usage rates per plan (Starter vs Power User), annual discount, 14-day trial, metered billing; no workload-level total (seq 51, seq 107)
#### End-to-end onboarding
- **Onboarding needs a login**: The agent never obtained real Pergola credentials. It downloaded the CLI, set a config endpoint, and ran `pergola validate manifest`, which triggered an interactive device-code browser login flow. The agent waited ~30s, the login timed out/was cancelled, and the agent gave up rather than supplying credentials, since no Pergola account or access key existed in the sandbox. No authenticated API call against the hosted product was ever made.
  Event 101:

  ```text
  ERROR   You are not logged in. Please login first.
  To log in, please open the following URL in your browser and make sure it displays the code KDQB-GLTR:
  
      https://auth.pergola.cloud/device?user_code=KDQB-GLTR
  ```
  Event 101:

  ```text
  ERROR   Login cancelled, please try again.
  ```
  Event 95:

  ```text
  ERROR   Cannot reach endpoint. Please provide a valid API endpoint URL
  ```
  Event 107:

  ```text
  No Pergola account/endpoint/credentials** exist in this sandbox.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **Manifest validation requires live login, not just local files**: The docs frame `pergola validate manifest` as a local, offline check you can run 'before even committing to Git,' but running it against a real pergola.yaml instead forced a device-code browser login flow that hung for ~30 seconds before timing out. This is a product/credential limitation (no human available to complete browser auth in this sandboxed session), not an agent mistake — the agent correctly killed the hanging interactive prompt per its instructions to avoid waiting on long-running commands.
  Event 99:

  ```text
  Validates a project manifest locally.
  ```
  Event 101:

  ```text
  ERROR   You are not logged in. Please login first.
  To log in, please open the following URL in your browser and make sure it displays the code KDQB-GLTR
  ```
  Event 101:

  ```text
  ERROR   Login cancelled, please try again.
  ```
- **No self-service credentials available in sandbox**: The sandbox environment had no Pergola account, API endpoint, or access key preconfigured, and the CLI's first-time login requires either an interactive browser flow or an access key provided by a Pergola administrator. Without either, the agent could not create a Project, Stage, or Release to run the example end-to-end. This is a missing-credentials/test-environment limitation rather than a product defect.
  Event 107:

  ```text
  I could **not run the example end-to-end**, and I stopped rather than wait:
  
  1. **No Pergola account/endpoint/credentials** exist in this sandbox.
  ```
  Event 91:

  ```text
  You are not logged in.
  ```

#### Suggested Changes
- **Label `pergola validate manifest` as requiring authentication**: The CLI reference page (docs.pergola.cloud/docs/cli/) describes the command as validating a manifest 'locally,' but in practice it calls out to the configured endpoint and demands login first. Update the docs/CLI help text to clarify that an authenticated session (or access key) is required, so developers don't expect an offline check. Verify by running `pergola validate manifest pergola.yaml` with no prior login and confirming the help text/docs match the observed endpoint-dependent behavior.
  Event 99:

  ```text
  Validates a project manifest locally.
  ```
  Event 95:

  ```text
  ERROR   Cannot reach endpoint. Please provide a valid API endpoint URL
  ```
- **Add a visible timeout or cancel hint to the device-login wait screen**: When `pergola login` (or any command that triggers login) is waiting on the device-code flow, the CLI prints an indefinite spinner ('I hold the fort for as long...') with no indication of how long it will wait or how to supply an access key instead. Add a one-line hint such as 'Press Ctrl+C and use --access-key for non-interactive environments' to the spinner output; verify by re-running a login-triggering command in a headless shell and checking the hint appears within the first few seconds.
  Event 101:

  ```text
  I hold the fort for as long... (28s)
  ```
  Event 101:

  ```text
  ERROR   Login cancelled, please try again.
  ```

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/6b4ddfdc-49de-48c2-be7b-04113772c3f9) · [Read transcript](https://www.ax-check.com/preview-pergola-website-dev.apps.pergola.cloud/sessions/kimi.json)
No pricing source existed: the domains were parked or returned errors. The agent said there was nothing to price and quoted no number, so the rubric can't be applied. (seq 19, seq 21)
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: No product credentials were obtained and no authenticated operation was attempted. The agent determined early that no hosted product named Pergola exists (only unrelated PyPI/npm packages and parked/redirecting domains), and stopped per the seed prompt's instruction to say so plainly rather than attempting any SDK/API workflow.
  Event 21:

  ```text
  The verdict: no actionable developer product exists under this name.
  ```
  Event 21:

  ```text
  There is no hosted service, SDK, or API for me to operate, so there's no developer workflow I can act on.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No hosted Pergola product discoverable**: The agent checked PyPI, npm, and several likely domains (pergola.ai, pergola.io, pergola.com, getpergola.com, pergola.dev) and found only unrelated open-source packages and parked/placeholder domains, not a developer product. This is a test-environment/discovery limitation (no web search available, only direct fetches) combined with the product possibly not existing under this name or at a discoverable URL, not a defect in an actual product.
  Event 19:

  ```text
  pergola.ai is for sale on Spaceship. Secure checkout and quick transfer.
  ```
  Event 19:

  ```text
  window.onload=function(){window.location.href="/lander"}
  ```
  Event 21:

  ```text
  I have no web search in this environment, only direct fetches, so if "Pergola" is a real product behind a less-obvious URL, I can't discover it on my own.
  ```

#### Suggested Changes
- **Publish a discoverable canonical URL for Pergola**: If Pergola is a real hosted developer product, its domain (e.g. pergola.ai or pergola.io) currently resolves to a domain-sale parking page or a redirect to a generic lander with no docs, signup, or API reference. Point the primary domain to the actual product site and verify by fetching the root URL and confirming it serves developer docs or a signup page rather than a parked-domain notice.
  Event 19:

  ```text
  Buy pergola.ai | Spaceship
  ```
  Event 19:

  ```text
  window.location.href="/lander"
  ```

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/d085b455-d29a-4110-aa66-75b50366ae4d) · [Read transcript](https://www.ax-check.com/preview-pergola-website-dev.apps.pergola.cloud/sessions/qwen.json)
The agent said it is free and gave the basis: the PyPI package is GPL v3 open source (the npm one is MIT), there are no accounts or metered tiers, and the hosted server is down, so there is no paid service. (seq 15, seq 35, seq 133)
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: Pergola is an open-source Python library (PyPI package) with no hosted API, no authentication, and no account system. The agent installed it locally via pip and ran it purely as a local library against a local TSV file and local BED output. There was no product credential to obtain and no hosted/authenticated operation to perform, so onboarding verification does not apply in the way the rubric intends.
  Event 106:

  ```text
  Time range: 0 -> 720
  Subjects: ['mouse_1', 'mouse_2']
  ```
  Event 31:

  ```text
  <h1>ERROR 404</h1>
  	<h2>Page not found</h2>
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **Hosted Pergola web server is down**: The only hosted component mentioned in the package's own README (pergola.crg.eu) returns a 404/maintenance page, confirming there is no live hosted API to call. This is a product/infrastructure issue (external project's hosted service being offline), not an agent error. The agent correctly fell back to the local PyPI library instead.
  Event 31:

  ```text
  <h1>ERROR 404</h1>
  	<h2>Page not found</h2>
  	</table>
      <div>
          <p>Our website is temporarily unavailable while we carry out maintenance. We will be back online as soon as possible.
  ```
- **Undeclared runtime dependency on pybedtools**: Importing the core pergola.intervals module fails immediately because pybedtools is not installed, despite being a hard import-time dependency of tracks.py. This is a packaging/product defect (missing dependency declaration) that the agent had to self-diagnose and fix by manually installing pybedtools.
  Event 66:

  ```text
  ModuleNotFoundError: No module named 'pybedtools'
  ```
- **Silent filename mangling in save_track**: Calling save_track with a path treated as a directory and a name_file parameter results in the library silently appending '.txt' to the requested filename, producing output.bed.txt instead of output.bed. This caused two failed runs before the agent adjusted expectations to match undocumented behavior.
  Event 115:

  ```text
  FileNotFoundError: [Errno 2] No such file or directory: 'output.bed'
  ```
  Event 124:

  ```text
  File output.bed.txt generated
  ```

#### Suggested Changes
- **Declare pybedtools as an install-time dependency**: Add pybedtools to the package's install_requires/setup metadata so `pip install pergola` pulls it automatically. Verify by running a fresh `pip install pergola` in a clean environment and confirming `from pergola.intervals import IntData` succeeds without a separate `pip install pybedtools` step.
  Event 66:

  ```text
  ModuleNotFoundError: No module named 'pybedtools'
  ```
- **Document that data_value column must be numeric**: Add a note in the README or docstring for IntData/map_dict usage stating that the field mapped to 'data_value' must contain numeric values, since _min_max calls float() on it internally. Verify by checking the error no longer surprises a first-time user following a documented example with numeric sample data.
  Event 97:

  ```text
  ValueError: could not convert string to float: 'feeding'
  ```
- **Fix or document the save_track filename behavior**: Either stop silently appending '.txt' to name_file in Track.save_track (in tracks.py), or document in the docstring that output files always get a '.txt' suffix regardless of the name_file value passed. Verify by calling save_track(path='.', name_file='output.bed') and confirming the resulting filename matches documented expectations.
  Event 115:

  ```text
  File output.bed.txt generated
  ```

### Task given to each agent
Help me build a simple example using Pergola. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: B · 84/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — Homepage answers Markdown requests

  ```text
  Homepage returned text/html for a text/markdown request; no Markdown representation offered.
  ```

- **Pass** — llms.txt provides an actionable documentation index

  ```text
  llms.txt links Online Documentation, Features, FAQ, Pricing and Console with organized sections.
  ```

- **Pass** — llms.txt provides navigation guidance

  ```text
  llms.txt groups links under More about Pergola, Pricing, Enterprise, Get Started and Optional.
  ```

- **Pass** — llms.txt mentions offered API, MCP, and skills

  ```text
  llms.txt notes agent skills in the docs and links the Online Documentation index.
  ```

- **Pass** — A compact guide representation exists

  ```text
  docs.pergola.cloud/docs/gettingstarted.md returns a standalone Markdown guide for Pergola.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  Getting-started Markdown fetched directly at docs.pergola.cloud/docs/gettingstarted.md, HTTP 200.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  Markdown guide is 863 tokens, well under 8000 and below the 1639-token HTML version.
  ```

- **Skipped** — Product-docs links survive format changes

  ```text
  Homepage Markdown unsupported, so link preservation across formats cannot be measured.
  ```

- **Pass** — The compact guide is independently actionable

  ```text
  Getting-started guide gives concrete steps: create Project, Stage, pergola.yaml manifest, build and deploy.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Fetched getting-started and pricing pages resolve; docs and console links reachable.
  ```


### Onboarding
- **Pass** — Docs lead to a relevant quickstart

  ```text
  Docs Getting Started page is a concrete 5-minute quickstart: create Project, Stage, manifest, build, deploy.
  ```

- **Pass** — Installation commands are extractable

  ```text
  CLI page gives install script curl command plus per-OS manual download links and pergola login setup.
  ```

- **Pass** — Code examples are available without interaction

  ```text
  Getting started shows pergola.yaml manifest and CLI reference shows many inline command examples.
  ```

- **Pass** — Prerequisites and auth boundaries are explicit

  ```text
  Preparation lists Dockerfile, Git repo, account prerequisites; CLI login and Access Keys cover auth.
  ```


### Pricing
- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders plan tiers and usage rates as static text without interaction.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  Starter €0.99/mo, Power User €29.99/mo, usage rates all stated openly.
  ```

- **Pass** — Pricing units and limits are explicit

  ```text
  Units explicit: per vCPU/hour, per GiB/hour, per Stage/month, per Build.
  ```

- **Pass** — Agents identify pricing and its assumptions

  ```text
  2 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Quoted base and usage rates per plan (Starter vs Power User), annual discount, 14-day trial, metered billing; no workload-level total (seq 51, seq 107) Qwen 3.8 Max: The agent said it is free and gave the basis: the PyPI package is GPL v3 open source (the npm one is MIT), there are no accounts or metered tiers, and the hosted server is down, so there is no paid service. (seq 15, seq 35, seq 133) This behavioural item does not affect the fast grade.
  ```


### Activation
- **Skipped** — An API reference or OpenAPI spec is reachable

  ```text
  No API reference or OpenAPI spec page was fetched; CLI docs describe commands, not a REST API reference.
  ```

- **Pass** — An MCP server is documented and well-formed

  ```text
  CLI docs document 'pergola mcp serve' over stdio with client config and access-key auth.
  ```

- **Pass** — A CLI install path is documented

  ```text
  CLI page gives install script, manual binaries, and 'pergola login' first-time setup.
  ```

- **Skipped** — SDK packages resolve on their registries

  ```text
  No registry lookup for a Pergola SDK or packaged CLI package was supplied.
  ```

- **Skipped** — Agent skills are published

  ```text
  CLI mentions Pergola Skills but no skills page was fetched to assess publication.
  ```



[Full report data](https://www.ax-check.com/preview-pergola-website-dev.apps.pergola.cloud/report.json)
