# AX Check: easyrunner.xyz
Checked 2026-10-08.

Quickstart and pricing are clear, but agents disagree on cost
Docs give a working CLI quickstart and plain-text pricing ($72/$144/$300 per year), but one session concluded the product is free open-source software.

## End-to-end onboarding not demonstrated

## Coding sessions
All three sessions completed. Two agents (DeepSeek V4.1 Flash, Kimi K3) read the pricing page and reported the same three-tier $72/$144/$300 yearly structure with matching assumptions about per-app licensing and separate compute costs. The third, Qwen 3.8 Max, instead concluded the product is free open-source software with no hosted plans, landing on a $0 cost estimate.

### DeepSeek V4.1 Flash
[View public run](https://agents.withgauge.com/p/runs/c997e145-3eff-4679-940d-695c3acbd1ad) · [Read transcript](https://www.ax-check.com/easyrunner.xyz/sessions/deepseek.json)
Final output gives a 3-tier pricing table ($72/$144/$300 per yr) with explicit assumptions: per-app perpetual license, annual billing, compute billed separately by VPS provider, no usage metering.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: No onboarding occurred. EasyRunner has no hosted API/SDK to sign up for; it is a locally-installed CLI (er) that manages the developer's own Ubuntu/Hetzner server over SSH. The agent never obtained product credentials (no license file, no Hetzner/Cloudflare tokens) and never ran an authenticated operation against a hosted product. The session was limited to public doc/website scraping.
  Event 93:

  ```text
  There is no hosted endpoint I can hit, and this sandbox has no `brew`/`er`, no server, and no license. Per your instruction ("if the quickstart requires either, say so plainly and stop"), **I'm stopping without executing anything.**
  ```
  Event 91:

  ```text
  /usr/bin/docker
  exit=0
  ```

#### Hallucinated URLs
- **Guessed EasyRunner domain variants that failed**: Agent probed several speculative TLD variants of easyrunner before finding the real site (easyrunner.xyz). The guessed domains returned no response or a blocked status, and were abandoned in favor of the real site found via GitHub search — this is exploratory guessing that failed, not a documented link treated as broken.
  Event 36:

  ```text
  easyrunner.com -> 403
  easyrunner.io -> 000
  easyrunner.dev -> 000
  easyrunner.ai -> 000
  easyrunner.app -> 000
  easyrunner.co -> 000
  ```
- **Guessed internal docs URLs that 404'd**: Agent constructed plausible doc URLs (e.g. /docs/quickstart/deploy-first-app/, /docs/install/install-cli/, /docs/reference/cli/) by pattern-guessing from the nav menu rather than the sitemap, all of which 404'd. It then found the real paths via /sitemap.xml (under /user-docs/ instead of /docs/), confirming the first set were invented guesses rather than broken documented links.
  Event 67:

  ```text
  urllib.error.HTTPError: HTTP Error 404: Not Found
  ```
  Event 71:

  ```text
  https://easyrunner.xyz/user-docs/setup/install/
  ```

#### Blockers
- **Product requires local CLI install, a server, and a paid license — no hosted API exists**: This is inherent product behavior, not an agent error or missing test credential: EasyRunner's entire workflow is a locally-installed CLI (er) that provisions and SSHes into a developer's own Ubuntu/Hetzner server. There is no SDK or REST API to call lightly. The agent correctly identified this from the docs and stopped per the session's own 'stay light' instruction rather than install Homebrew packages, provision a server, or run long-running SSH deploy commands.
  Event 93:

  ```text
  **But there is no hosted product, SDK, or REST API to call.** Control-plane state lives locally (`er setup --mode server`), and all cloud integrations (Hetzner, Cloudflare, R2 backup) link *your* accounts.
  ```
  Event 75:

  ```text
  er license install ~/Downloads/easyrunner-license.jwt # (4)!
  ```

#### Suggested Changes
- **Add a lightweight/API-only path note to the FAQ or quickstart for developers who can't install local tooling or provision a server**: The FAQ and quickstart both assume a local CLI install plus an owned Ubuntu/Hetzner server; there is no mention of any lighter-weight evaluation path (e.g., a hosted demo, sandbox trial, or API-only mode) for developers who want to assess the product before installing anything or paying for a license. Check by re-reading the FAQ at https://easyrunner.xyz/faq/ and the quickstart at https://easyrunner.xyz/user-docs/quickstart/first-app/ for a path that does not require brew install, a purchased license, and a provisioned server.
  Event 75:

  ```text
  You need 
   EasyRunner CLI installed 
   A license file 
   A GitHub repo for your app 
   A domain or subdomain you control 
   Either a Hetzner account or an existing Ubuntu 24.04 server
  ```

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/6fdc0778-cecf-4ba9-a7bd-e136fb28403b) · [Read transcript](https://www.ax-check.com/easyrunner.xyz/sessions/kimi.json)
Final output gives tiered pricing ($72/$144/$300 per year by app count) scraped from easyrunner.xyz/pricing and names assumptions: compute billed separately, perpetual annual license, pricing scales by app count not server count.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: The agent never attempted self-service signup or credential acquisition. It identified that EasyRunner requires a purchased license file, a VPS/Hetzner API token, GitHub repo, and domain -- all human-supplied prerequisites -- and correctly stopped without obtaining or using any real credentials. No authenticated operation against the hosted product occurred.
  Event 54:

  ```text
  Missing prerequisites I'd need from you anyway: a paid license file ($72+/yr), a Hetzner API token (or a VPS with root SSH), a GitHub repo, and a domain you control for DNS.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No hosted SDK/API -- product is a CLI that provisions real infrastructure**: EasyRunner has no lightweight hosted API or SDK to call. Its only interface is the 'er' CLI, and the documented quickstart requires buying a license, creating/initializing a VPS (Hetzner or existing Ubuntu server), and installing a full web-host stack on it -- exactly the long-running infrastructure setup the task instructed the agent to avoid. This is product behavior (the product's nature), not an agent error or missing test credential alone.
  Event 54:

  ```text
  The quickstart provisions infrastructure — `er server create` spins up a Hetzner VPS (or you supply an existing Ubuntu 24.04 server with root SSH), then `er server init` installs a full web-host stack on it. That's a long-running service-stack setup, which you explicitly ruled out.
  ```
  Event 52:

  ```text
  er link hetzner default --api-key <hetzner-api-token> # (1)! er server create my-first-server hetzner # (2)! er server init my-first-server --username root # (3)! er server doctor my-first-server # (4)!
  ```
- **Required paid license and external credentials unavailable in session**: Proceeding past the install step requires a purchased license JWT, a Hetzner API token or root SSH on an existing server, a GitHub repo, and a controlled domain -- none of which were available or obtainable by the agent itself. This is a missing-credentials/test-environment limitation rather than a product defect.
  Event 52:

  ```text
  You need EasyRunner CLI installed A license file A GitHub repo for your app A domain or subdomain you control Either a Hetzner account or an existing Ubuntu 24.04 server
  ```

#### Suggested Changes
None identified in this transcript.

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/bdb1c415-f4b2-4101-8dc2-c5ad26fdcfbf) · [Read transcript](https://www.ax-check.com/easyrunner.xyz/sessions/qwen.json)
Final output states the cost is $0 and grounds it in verified facts (MIT-licensed OSS package, no hosted service/plans/tiers/API keys, only cost is the user's own local/GPU compute) rather than quoting a bare figure.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: EasyRunner (as 'easyrunner') does not exist as a package or hosted product under that name. The agent found only an unrelated PyPI package 'easy-runner' (a local Python experiment-launching utility, MIT licensed, no API keys, no billing, no hosted service). No credentials were obtained or used, and no authenticated operation against a hosted product occurred. The agent correctly stopped rather than fabricate a workflow.
  Event 47:

  ```text
  There is no pricing. It is free MIT-licensed open source: no hosted service, no plans, tiers, credits, seats, usage metering, or billing, and no API keys.
  ```
  Event 47:

  ```text
  the entire interface is a local import: `from easy_runner import EasyRunner`.
  ```
  Event 47:

  ```text
  Not by anything technical — no missing credentials, no network failure, no broken repo (`/sandbox/repo` is just an empty git workspace with a `.gitignore`). I'm stopped by your own guardrails: no hosted API to use, and the quickstart is local + long-running.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **EasyRunner has no hosted SDK/API — only a local experiment-launcher package exists**: The agent searched PyPI, npm, and crates.io for 'easyrunner' and found nothing; the closest match, 'easy-runner', is a tiny local Python tool for running parallel subprocess experiments with no network API, pricing, or credentials. This is a product/naming mismatch rather than an agent or environment error — the requested product simply does not exist as a hosted offering under that name.
  Event 18:

  ```text
  ERROR: Could not find a version that satisfies the requirement easyrunner (from versions: none)
  ERROR: No matching distribution found for easyrunner
  ```
  Event 26:

  ```text
  == https://pypi.org/pypi/easyrunner/json
  404
  ```
  Event 31:

  ```text
  summary : A lightweight tool for managing and executing multiple parallel experiments.
  ```
- **Quickstart example requires long-running local processes and interactive input**: Per the task's explicit constraint to avoid local service stacks or long-running commands, the agent found that easy-runner's only usage pattern (runner.start with gpus and max_parallel) spawns blocking local subprocesses and an interactive foreground monitor requiring keyboard input, with no non-blocking or dry-run mode. This is a property of the (likely wrong) product found, not a defect in a real EasyRunner hosted product.
  Event 47:

  ```text
  The quickstart **requires exactly what you told me to avoid**: `runner.start(instructions, gpus=[0,1], max_parallel=2)` spawns long-running local processes and runs an interactive foreground monitor loop (it expects keyboard input to kill experiments).
  ```

#### Suggested Changes
- **Confirm the correct package or product name for 'EasyRunner' in onboarding materials**: Searches for 'easyrunner' returned no results on PyPI, npm, or crates.io, and the only near-match (easy-runner) is an unrelated local experiment tool. If a hosted EasyRunner product exists, publish its correct install name and a discoverable link (e.g., in the prompt or a docs site) so agents and new developers do not land on an unrelated same-named package. Verify by re-running a package search for the exact intended name and confirming it resolves to the real product.
  Event 18:

  ```text
  ERROR: Could not find a version that satisfies the requirement easyrunner (from versions: none)
  ERROR: No matching distribution found for easyrunner
  ```
  Event 31:

  ```text
  home_page : https://github.com/liuzuxin/easy-runner
  ```

### Task given to each agent
Help me build a simple example using EasyRunner. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: C · 63/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — Homepage answers Markdown requests

  ```text
  Homepage returned text/html for a text/markdown request; no Markdown representation offered.
  ```

- **Failed** — llms.txt provides an actionable documentation index

  ```text
  /llms.txt returns HTTP 404, so no documentation index is published.
  ```

- **Skipped** — llms.txt provides navigation guidance

  ```text
  No llms.txt body exists (404), so navigation guidance cannot be judged.
  ```

- **Skipped** — llms.txt mentions offered API, MCP, and skills

  ```text
  No llms.txt body exists (404), so API/MCP/skills mentions cannot be judged.
  ```

- **Skipped** — A compact guide representation exists

  ```text
  No site-published Markdown guide fetched; homepage Markdown unsupported, only HTML quickstarts seen.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  Quickstart 'Deploy Your First App' gives concrete steps from CLI install to HTTPS deploy.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  First-app guide is 3061 tokens, well under the 8000-token budget.
  ```

- **Skipped** — Product-docs links survive format changes

  ```text
  Homepage Markdown unsupported, so link preservation across formats cannot be measured.
  ```

- **Pass** — The compact guide is independently actionable

  ```text
  First-app quickstart gives concrete CLI steps from install through deploy and verification.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Fetched quickstart and Hetzner pages returned 200; install/next-step links resolve.
  ```


### Onboarding
- **Pass** — Docs lead to a relevant quickstart

  ```text
  Docs nav and install page link to quickstart 'Deploy Your First App' with concrete steps.
  ```

- **Pass** — Installation commands are extractable

  ```text
  Install page gives extractable Homebrew commands: brew tap janaka/easyrunner; brew install easyrunner-cli.
  ```

- **Pass** — Code examples are available without interaction

  ```text
  Quickstart shows inline code blocks for CLI commands and compose config without interaction.
  ```

- **Pass** — Prerequisites and auth boundaries are explicit

  ```text
  Quickstart lists prerequisites (license, GitHub repo, domain, Hetzner/Ubuntu) and auth steps.
  ```


### Pricing
- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders all three tiers with prices in plain text, no interaction needed.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  Prices stated openly: $72, $144, $300 yearly; no gating or contact-sales wall.
  ```

- **Pass** — Pricing units and limits are explicit

  ```text
  Tiers define app counts (5/20/50), yearly billing, per-app rates, and app definition.
  ```

- **Pass** — Agents identify pricing and its assumptions

  ```text
  3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Final output gives a 3-tier pricing table ($72/$144/$300 per yr) with explicit assumptions: per-app perpetual license, annual billing, compute billed separately by VPS provider, no usage metering. Kimi K3: Final output gives tiered pricing ($72/$144/$300 per year by app count) scraped from easyrunner.xyz/pricing and names assumptions: compute billed separately, perpetual annual license, pricing scales by app count not server count. Qwen 3.8 Max: Final output states the cost is $0 and grounds it in verified facts (MIT-licensed OSS package, no hosted service/plans/tiers/API keys, only cost is the user's own local/GPU compute) rather than quoting a bare figure. This behavioural item does not affect the fast grade.
  ```


### Activation
- **Skipped** — An API reference or OpenAPI spec is reachable

  ```text
  Fetched pages document a CLI only; no API reference or OpenAPI spec was offered or found.
  ```

- **Skipped** — An MCP server is documented and well-formed

  ```text
  No MCP server is documented anywhere in the fetched EasyRunner pages.
  ```

- **Pass** — A CLI install path is documented

  ```text
  Install page documents Homebrew tap and install of easyrunner-cli, entry point er.
  ```

- **Skipped** — SDK packages resolve on their registries

  ```text
  No registry lookup for an EasyRunner SDK or packaged CLI package was supplied.
  ```

- **Skipped** — Agent skills are published

  ```text
  No agent skills are published or mentioned in the fetched EasyRunner documentation.
  ```



[Full report data](https://www.ax-check.com/easyrunner.xyz/report.json)