# AX Check: leanctx.com
Checked 2026-09-19.

Docs, install and pricing pages all work without login
20 of 23 checks passed: quickstart, CLI/SDK install, API and MCP docs all render statically. Pro/Team pricing lacks a stated price or unit, only "confirmed before purchase."

## End-to-end onboarding not demonstrated

## Coding sessions
All three independent sessions completed and reviewed pricing. Each found the Free tier and SDK confirmed free, but had to state their own assumptions about Pro/Team costs since those figures aren't published, giving differing per-call or per-seat estimates instead of a single confirmed number.

### DeepSeek V4 Pro
[View public run](https://agents.withgauge.com/p/runs/8331b056-74f0-4cc6-883b-77c54b9c7617) · [Read transcript](https://www.ax-check.com/leanctx.com/sessions/deepseek.json)
Final output itemizes Free/$0, Pro/$9mo, Team/$18-seat, Enterprise/custom, Supporter/$5mo and explicitly flags the assumption that Pro/Team figures come from an application-only pilot with no GA plan, card, or confirmed terms.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: The agent never obtained real credentials or ran an authenticated operation against a hosted LeanCTX instance. It only researched docs, pricing, and SDK source via public web fetches, then wrote (but did not run) a local example script targeting http://127.0.0.1:8080. No lean-ctx serve process was started, no bearer token created, no API call made. Correctly self-identified as blocked: LeanCTX's free tier needs no account, but the only generally-available API surface is a local server the agent was instructed not to start.
  Event 92:

  ```text
  NOTE: not run in this session. The SDK is a thin HTTP client for LeanCTX's
  versioned /v1 contract, which is served by a *local* `lean-ctx serve` process
  ```
  Event 95:

  ```text
  There is **no hosted SDK/API I can act on right now**:
  
  1. Every documented API base URL is `http://127.0.0.1:8080` — served by the local `lean-ctx serve` process.
  ```
  Event 95:

  ```text
  The quickstart (`lean-ctx serve`, or `lean-ctx wrap <agent>`) requires installing the local binary and running a **long-running local service** — exactly what you told me not to do.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No hosted API reachable without running a local server**: Product behavior, not agent error: every documented /v1 API base URL in the docs and SDK source is http://127.0.0.1:8080, served only by locally running lean-ctx serve. The agent's instructions explicitly forbade starting local service stacks or long-running commands, so it could not stand up the only generally-available API surface and stopped instead of running one.
  Event 58:

  ```text
  Run the server Start the local server, it serves REST + SSE + MCP on one port and prints a loopback bearer token: $ lean-ctx serve # → http://127.0.0.1:8080
  ```
  Event 95:

  ```text
  The quickstart (`lean-ctx serve`, or `lean-ctx wrap <agent>`) requires installing the local binary and running a **long-running local service** — exactly what you told me not to do.
  ```
- **Hosted Pro/Team tiers are application-only, not self-serve**: Product behavior: the pricing page confirms Pro ($9/mo) and Team ($18/seat) are founding-pilot tiers requiring a sales-assisted application, with no card or account created automatically. No API key or hosted endpoint was obtainable during the session even in principle, independent of the local-server restriction.
  Event 29:

  ```text
  Availability and readiness confirmed before onboarding Scope, support and commercial terms agreed in writing Applying creates no subscription and requires no card
  ```
  Event 58:

  ```text
  Pricing: local use free forever (CI-enforced); Pro $9/mo (hosted personal cloud); Team $18/seat (shared, audited context plane)
  ```

#### Suggested Changes
- **Add a lightweight hosted sandbox or demo API endpoint**: On the API Reference page (leanctx.com/docs/api-reference/), the only base URL shown is the loopback address from lean-ctx serve. Add a public, rate-limited hosted /v1 endpoint or demo workspace so developers evaluating the SDK without running a local binary can make one authenticated call. Verify by confirming a fresh visitor can obtain a token and hit /v1/capabilities without installing the CLI.
  Event 58:

  ```text
  Two ways to reach the same /v1 contract Open Door (local): run lean-ctx serve and embed LeanCTX in any language via the Python, TypeScript or Rust SDK. Team Server: CI bots, dashboards and shared workspaces.
  ```
- **Fix or remove the broken docs.leanctx.com subdomain**: docs.leanctx.com returned a 502 Bad Gateway when fetched directly. Either redirect it to leanctx.com/docs/ or remove references to it, and confirm with a fresh curl that it returns 200 or a proper redirect.
  Event 17:

  ```text
  === https://docs.leanctx.com ===
  <title>502 Bad Gateway</title>
  ```

### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/9280c082-0a5f-41fe-a8ca-2d40584a7241) · [Read transcript](https://www.ax-check.com/leanctx.com/sessions/kimi.json)
Final output gives a pricing table (local free, self_llm ~$0.0001–$0.0016/call) with explicit assumptions (Gemini Flash vs Claude Haiku, per their benchmarks, MIT-licensed no vendor bill).
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: LeanCTX is an open-source pip/npm SDK, not a hosted product with an authenticated API surface. The agent installed the package locally, inspected its source, and ran a local Python script exercising the deterministic verbatim/dedup/purge-errors code path. No credentials were obtained or used, and no hosted endpoint or authenticated operation was ever invoked -- the agent explicitly avoided the API-key-requiring self_llm path and the model-download-requiring lingua path.
  Event 66:

  ```text
  messages: 8 -> 7
  tokens:   4264 -> 2092 (50.9% reduction)
  stats:    CompressionStats(input_tokens=2092, output_tokens=2092, ratio=1.0, method='verbatim', cost_usd=0.0)
  ```
  Event 68:

  ```text
  the full end-to-end quickstart (drop-in wrapper against a live Anthropic/OpenAI/Gemini API, or the LLMLingua-2 local model) would require either a provider API key or a multi-GB model download, so I stayed on the deterministic zero-dependency path per your constraints
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No API key available for the self_llm compression path**: Testing the self_llm compressor (which routes compression through a real Anthropic/OpenAI/Gemini call) requires a provider API key. None was set in the sandbox environment, so the agent could not exercise that path. This is an environment/credential limitation, not a product defect -- a normal API key requirement is expected for a paid LLM call.
  Event 21:

  ```text
  PI_GATEWAY_API_KEY=<set>
  ```
  Event 68:

  ```text
  self_llm mode` needs an Anthropic/OpenAI/Gemini API key (none set here)
  ```
- **Lingua local mode needs a multi-GB model download**: The local LLMLingua-2 compression mode requires downloading a large model, which conflicts with the session's instruction to stay light and avoid long-running downloads. The agent correctly identified this and avoided it rather than being blocked mid-task; this is inherent product behavior (a local ML model dependency), not an error.
  Event 68:

  ```text
  the lingua mode downloads the ~GB-scale LLMLingua-2 model
  ```

#### Suggested Changes
- **Clarify in the README that LeanCTX has no hosted product or dev workflow to onboard against**: The seed task assumed a hosted SDK/API with pricing and onboarding, but LeanCTX is a keyless, self-hosted, MIT-licensed pip/npm library with zero vendor billing. Add an explicit line near the top of the PyPI/GitHub README (e.g. under 'How it works') stating there is no hosted service or account needed, so evaluators and new users don't spend time hunting for API keys or a dashboard that doesn't exist.
  Event 68:

  ```text
  LeanCTX is not a hosted product — it's an open-source (MIT) pip/npm SDK, so there's no vendor bill at all
  ```
- **Add a zero-dependency quickstart snippet using Middleware directly**: The README's only runnable example uses the drop-in OpenAI/Anthropic wrapper, which needs a provider API key. Add a short snippet in the README's 'How it works' or 'Install' section showing the Middleware class with routing set to verbatim (no key, no model download), similar to what the agent had to reverse-engineer from source. Verify it works by running it fresh with `pip install leanctx` and no environment variables set.
  Event 47:

  ```text
  Lingua lazy-loads its model, so I can stay light by routing everything to verbatim and using the deterministic strategies.
  ```
  Event 60:

  ```text
  I have everything I need. Writing the example now:
  ```

### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/cb57630c-5c82-40e7-8c61-6af0311f2a6b) · [Read transcript](https://www.ax-check.com/leanctx.com/sessions/qwen.json)
Final output states the SDK itself is free/MIT and named assumptions (no metered hosted API, pay only provider token bills, ~10-40% savings claim, ~$78/1000 requests at Sonnet input pricing per their benchmark, and self-LLM mode adding ~$0.0003/call).
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: LeanCTX is a local, MIT-licensed Python SDK (pip install), not a hosted product. There were no credentials to obtain and no hosted API endpoint reachable — everything the agent ran (classify(), Verbatim().compress()) executed in-process with no network calls to a product backend. The agent explicitly stated it skipped live SDK usage because provider API keys (OPENAI_API_KEY etc.) were absent, so no authenticated operation against a real hosted service was ever attempted or demonstrated.
  Event 82:

  ```text
  == compression stats ==
    tokens before : 875
    tokens after  : 875
    stats         : CompressionStats(input_tokens=875, output_tokens=875, ratio=1.0, method='verbatim', cost_usd=0.0)
  ```
  Event 84:

  ```text
  Needs a provider API key, which isn't set in this environment.
  ```
  Event 84:

  ```text
  LeanCTX is **not a hosted product** — it's an MIT-licensed Python SDK (`pip install leanctx`) that runs compression **locally** and wraps the official OpenAI/Anthropic/Gemini clients. There's no hosted API or service to call.
  ```

#### Hallucinated URLs
None identified in this transcript.

#### Blockers
- **No provider API key available for live compression demo**: Product behavior combined with test environment limitation: the real compression path (Lingua or SelfLLM live modes) requires either a large local model download or a provider API key (OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY). Neither was present in the sandbox, so the agent could only exercise the passthrough/Verbatim compressor and had to comment out the live client example. This is a missing-credentials/environment limitation, not a product defect.
  Event 65:

  ```text
  selfllm-anthropic         SelfLLM compression via Anthropic (Haiku) live API.  [missing env: ANTHROPIC_API_KEY]
  ```
  Event 84:

  ```text
  Needs a provider API key, which isn't set in this environment.
  ```
- **LLMLingua-2 extra not installed, blocking real compression ratio test**: Agent error/product design: attempting to run the Lingua compressor failed because the 'llmlingua' package extra wasn't installed, and the quickstart's real 1.2GB model download was intentionally skipped per the 'stay light' instruction, leaving only the no-op Verbatim compressor demonstrated.
  Event 57:

  ```text
  ModuleNotFoundError: No module named 'llmlingua'
  ```
  Event 84:

  ```text
  Real compression needs `pip install 'leanctx[lingua]'`, and **first use downloads ~1.2 GB of model weights** — I deliberately skipped that per your "stay light" constraint
  ```
- **classify() call failed on raw string input**: Agent error: the first attempt called classify() with a plain string instead of a message dict, causing an AttributeError. The agent self-corrected on the next edit and the script then ran successfully, so this was a recovered mistake rather than an unresolved blocker.
  Event 74:

  ```text
  AttributeError: 'str' object has no attribute 'get'
  ```
  Event 82:

  ```text
  prose doc  -> prose   (high tolerance: compressible)
  ```

#### Suggested Changes
- **Clarify in the README that leanctx is a local SDK, not a hosted service**: On the GitHub README quickstart section, add an explicit line stating there is no hosted API/dashboard and all compute happens client-side, so first-time evaluators (like this session) don't need to search for API endpoints or credentials before realizing it's local-only. Verify by checking that a new reader can determine 'no hosted product' within the first screen of the quickstart.
  Event 26:

  ```text
  Open-source models, runs locally, MIT-licensed. Your prompts and user data never leave your infrastructure by default.
  ```
  Event 84:

  ```text
  LeanCTX is **not a hosted product** — it's an MIT-licensed Python SDK (`pip install leanctx`) that runs compression **locally**
  ```
- **Fix classify() to accept a plain string, not only a message dict**: In leanctx/classifier.py (get_text_content / classify), accept a bare string as valid input or raise a clearer TypeError explaining the expected message-dict shape, instead of throwing an unguarded AttributeError. Check by calling classify('some text') directly and confirming either a valid result or a descriptive error.
  Event 74:

  ```text
  AttributeError: 'str' object has no attribute 'get'
  ```
- **Document the llmlingua extra requirement earlier in the quickstart code sample**: The quickstart code block (README, 'Quickstart (60 seconds)') shows Lingua-based compression as if it works out of the box, but it actually needs the separate 'leanctx[lingua]' extra plus a ~1.2GB model download. Add an inline comment or callout right in the code sample (not just the paragraph below it) noting the extra is required, so developers testing lightly don't hit the ModuleNotFoundError mid-session. Verify by running the quickstart snippet exactly as shown after only 'pip install leanctx' and confirming the failure mode is anticipated by the docs.
  Event 57:

  ```text
  ModuleNotFoundError: No module named 'llmlingua'
  ```
  Event 33:

  ```text
  > **First Lingua call loads ~1.2 GB of model weights** to `~/.cache/huggingface/`. Subsequent calls reuse the cache. Add `pip install 'leanctx[lingua]'` to opt in; without it, leanctx falls back to passthrough.
  ```

### Task given to each agent
Help me build a simple example using LeanCTX. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.

No product credentials were supplied and no purchases were authorized.

## Score: B · 79/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.

### Clarity
- **Failed** — Homepage answers Markdown requests

  ```text
  Homepage returned text/html even when text/markdown was requested; no Markdown representation served.
  ```

- **Pass** — llms.txt provides an actionable documentation index

  ```text
  llms.txt lists docs, getting-started, CLI, tools, API reference and changelog with descriptions.
  ```

- **Pass** — llms.txt provides navigation guidance

  ```text
  llms.txt groups pages under Capabilities, Pages and Key Facts with starting guidance.
  ```

- **Pass** — llms.txt mentions offered API, MCP, and skills

  ```text
  llms.txt mentions the /v1 API with SDKs, 79 MCP tools and the MCP tool manifest link.
  ```

- **Pass** — A compact guide representation exists

  ```text
  llms.txt is a site-published Markdown guide covering install, capabilities, tools and pricing.
  ```

- **Pass** — A focused guide is directly retrievable

  ```text
  llms.txt fetched directly at /llms.txt as text/plain Markdown with full content.
  ```

- **Pass** — Equivalent instructions fit a token budget

  ```text
  llms.txt is 3126 tokens, well under the 8000-token budget.
  ```

- **Skipped** — Product-docs links survive format changes

  ```text
  Homepage Markdown unsupported, so link preservation across formats cannot be measured.
  ```

- **Pass** — The compact guide is independently actionable

  ```text
  Getting Started gives concrete install commands, wrap setup, and verify steps for LeanCTX.
  ```

- **Pass** — Install and next-step links resolve

  ```text
  Fetched install and next-step links (getting-started, tools, cli-reference) returned 200.
  ```


### Onboarding
- **Pass** — Docs lead to a relevant quickstart

  ```text
  Getting Started page is a concrete quickstart with 3 install/setup steps and verification.
  ```

- **Pass** — Installation commands are extractable

  ```text
  Install commands extractable: curl install.sh, brew, npm, cargo, pi, plus per-platform steps.
  ```

- **Pass** — Code examples are available without interaction

  ```text
  Numerous inline code examples for install, setup, MCP configs, and CLI usage without interaction.
  ```

- **Pass** — Prerequisites and auth boundaries are explicit

  ```text
  Prerequisites (Rust, Node, PATH) and auth boundaries (login/register, API keys) are explicit.
  ```


### Pricing
- **Failed** — Pricing units and limits are explicit

  ```text
  Pro and Team show no price, unit or limit — only 'Confirmed before purchase'.
  ```

- **Pass** — Pricing is readable without interaction

  ```text
  Pricing page renders plan cards, comparison table and FAQ as static HTML without interaction.
  ```

- **Pass** — Prices are stated, not gated

  ```text
  Free is $0 forever; Pro/Team are founding pilots with price confirmed before purchase; Enterprise contact sales.
  ```

- **Pass** — Agents identify pricing and its assumptions

  ```text
  3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Final output itemizes Free/$0, Pro/$9mo, Team/$18-seat, Enterprise/custom, Supporter/$5mo and explicitly flags the assumption that Pro/Team figures come from an application-only pilot with no GA plan, card, or confirmed terms. Kimi K3: Final output gives a pricing table (local free, self_llm ~$0.0001–$0.0016/call) with explicit assumptions (Gemini Flash vs Claude Haiku, per their benchmarks, MIT-licensed no vendor bill). Qwen 3.8 Max: Final output states the SDK itself is free/MIT and named assumptions (no metered hosted API, pay only provider token bills, ~10-40% savings claim, ~$78/1000 requests at Sonnet input pricing per their benchmark, and self-LLM mode adding ~$0.0003/call). This behavioural item does not affect the fast grade.
  ```


### Activation
- **Pass** — An API reference or OpenAPI spec is reachable

  ```text
  API Reference documents versioned /v1 HTTP API, REST endpoints, auth, and GET /v1/openapi.json OpenAPI 3.0 schema.
  ```

- **Pass** — An MCP server is documented and well-formed

  ```text
  MCP server documented with stdio transport, config JSON per client, and 81-tool manifest at /generated/mcp-tools.json.
  ```

- **Pass** — A CLI install path is documented

  ```text
  CLI install paths documented: curl install.sh, npm install -g lean-ctx-bin, cargo install, brew, plus lean-ctx setup.
  ```

- **Pass** — SDK packages resolve on their registries

  ```text
  Registry lookups returned HTTP 200 for pypi:lean-ctx-client and npm:lean-ctx-bin.
  ```

- **Pass** — Agent skills are published

  ```text
  Agent Skill installed to ~/.claude/skills/lean-ctx/ via lean-ctx init --agent claude; skills step in setup wizard.
  ```



[Full report data](https://www.ax-check.com/leanctx.com/report.json)