# AX Check: steel.dev Checked 2026-10-05. Steel.dev installs, runs, and prices cleanly for coding agents. 22 of 23 checklist items passed; only two secondary quickstart links 404'd. All 3 coding sessions found clear pricing tiers unprompted. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed the task and produced full pricing breakdowns with Launch, Scale, and Enterprise tiers, rates, and stated assumptions, each sourced directly from the docs pricing page with no login wall encountered. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/01867ef8-a627-4126-b223-22046bc242ca) · [Read transcript](https://www.ax-check.com/steel.dev/sessions/deepseek.json) Final output gives a full pricing table (Launch vs Scale tiers, browser hours, proxy, CAPTCHA) sourced from docs.steel.dev/overview/pricinglimits (fetched at seq 41-43) with explicit plan/usage assumptions and caveats (per-minute billing, concurrency caps, paid-balance requirement). #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained a real Steel API key. It searched the sandbox environment for credentials, found none, and confirmed the sandbox only had an unrelated gateway key. It built two example scripts and verified they correctly reach the hosted api.steel.dev endpoint (a dummy key returns a real 401 Unauthorized from the live API), but no authenticated operation was ever completed because no valid STEEL_API_KEY was available and self-service signup/key creation was not performed in-session. Event 5: ```text PI_GATEWAY_API_KEY= --- done ``` Event 102: ```text === dummy key (expect real API 401, proving code reaches hosted API) === linkToDocs: 'https://docs.steel.dev/overview/authentication' } } Node.js v22.23.2 exit=1 ``` Event 108: ```text I got blocked on credentials. There's no `STEEL_API_KEY` in this environment (only an unrelated gateway key), so I could not run a live request. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Steel API key available in sandbox**: The environment only exposed an unrelated gateway key (PI_GATEWAY_API_KEY); no STEEL_API_KEY existed, and Steel account/key creation requires a human to sign up via the dashboard. This is a normal credential/login requirement, not a product defect: the agent correctly identified it, built working example code, and validated the code path against the live API (confirmed via a real 401 from a dummy key) rather than faking success. Event 5: ```text PI_GATEWAY_API_KEY= --- done ``` Event 51: ```text {"error":"Unauthorized","message":"Missing API key. Authenticate by sending your Steel API key in the 'steel-api-key' header (Steel does not use 'Authorization: Bearer'). Create or rotate a key at https://app.steel.dev/settings/api-keys.","linkToDocs":"https://docs.steel.dev/overview/authentication"} HTTP 401 ``` Event 108: ```text I got blocked on credentials. There's no `STEEL_API_KEY` in this environment (only an unrelated gateway key), so I could not run a live request. ``` #### Suggested Changes - **Add a visible quickstart note on the non-standard auth header**: The API returns a helpful 401 message clarifying Steel uses 'steel-api-key' instead of 'Authorization: Bearer', but this is only discovered by hitting the API and reading the error, not from the Sessions API quickstart page itself. Add this header-name callout directly to docs.steel.dev/overview/sessions-api/quickstart near the 'Set Up Environment Variables' step so first-time developers do not need to trigger a 401 to learn it. Verify by checking the quickstart page shows the exact header name before the first code sample. Event 51: ```text Authenticate by sending your Steel API key in the 'steel-api-key' header (Steel does not use 'Authorization: Bearer') ``` Event 44: ```text import Steel from 'steel-sdk'; import dotenv from 'dotenv'; dotenv.config(); const client = new Steel({ steelAPIKey: process.env.STEEL_API_KEY, }); ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/fafda5a5-771b-4feb-a4ce-88779b39c35c) · [Read transcript](https://www.ax-check.com/steel.dev/sessions/kimi.json) Final output gives a pricing table (Launch/Scale/Enterprise rates) sourced from docs.steel.dev/overview/pricinglimits with explicit assumptions (plan tier, session length, credit validity, concurrency limits). #### End-to-end onboarding - **Onboarding needs a login**: No Steel API key was available in the sandbox environment. The agent checked env vars, found none, and when it ran its demo script against the live hosted API, the request failed with 401 Invalid Steel API key. The agent never obtained credentials itself (no self-service signup/key-creation flow was attempted or completed), so no authenticated product operation was demonstrated. Event 3: ```text env | grep -iE 'steel|api_key' | sed 's/=.*/=***/' ``` Event 47: ```text Error: scrape failed: 401 {"error":"Unauthorized","message":"Invalid Steel API key. The key sent in the 'steel-api-key' header does not match any account. Create or rotate a key at https://app.steel.dev/settings/api-keys." ``` Event 49: ```text I can't execute the demo against the live API: there's no `STEEL_API_KEY` in this environment, and the API correctly returns `401 Invalid Steel API key`. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing Steel API key blocks live execution**: The sandbox had no STEEL_API_KEY and the session had no self-service way to create one. The agent's demo script correctly called the hosted /v1/scrape endpoint but got a 401 Unauthorized, confirming the product's auth is working as intended — this is a missing-credentials limitation of the test environment, not a product defect. The agent correctly identified this, stopped, and reported it rather than faking success. Event 47: ```text Error: scrape failed: 401 {"error":"Unauthorized","message":"Invalid Steel API key. The key sent in the 'steel-api-key' header does not match any account. Create or rotate a key at https://app.steel.dev/settings/api-keys." ``` Event 49: ```text I can't execute the demo against the live API: there's no `STEEL_API_KEY` in this environment ``` - **Docs page path for pricing initially 404s**: The agent's first guess at the pricing docs URL (docs.steel.dev/overview/pricing) returned a 404 page-not-found. This is not proven to be an agent-invented URL since it closely mirrors the site's real nav structure, and the agent quickly recovered by following the llms.txt index to the correct path (pricinglimits), so it did not block progress. Event 21: ```text Page not found | Steel Docs ``` #### Suggested Changes - **Fix or redirect the guessed pricing docs path**: Requesting https://docs.steel.dev/overview/pricing returned a 404 'Page not found,' while the actual page lives at https://docs.steel.dev/overview/pricinglimits. Adding a redirect from the more intuitive /pricing path (or linking it directly from the main docs nav) would save a round trip; verify by curling docs.steel.dev/overview/pricing and confirming it no longer 404s. Event 21: ```text Page not found | Steel Docs ``` Event 36: ```text # Pricing/Limits URL: https://docs.steel.dev/overview/pricinglimits ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/3f1f77b6-25c0-4347-9363-39653f9af9eb) · [Read transcript](https://www.ax-check.com/steel.dev/sessions/qwen.json) Final output gives Launch/Scale/Enterprise tiers with explicit figures ($0/mo+usage, $250/mo, custom) and names the basis (per docs.steel.dev/overview/pricinglimits fetched at seq 57-59, dated June 2026). #### End-to-end onboarding - **Onboarding needs a login**: Agent never obtained a Steel API key. It installed the real steel-sdk (0.19.0), wrote working example code, and confirmed the code path was correct by hitting the real hosted api.steel.dev endpoint with a dummy key, getting back a clean 401 Unauthorized. No self-service signup flow was attempted or available in-session; the agent explicitly states the API key must come from a human visiting app.steel.dev. Event 91: ```text steel.AuthenticationError: Error code: 401 - {'error': 'Unauthorized', 'message': "Invalid Steel API key. The key sent in the 'steel-api-key' header does not match any account. Create or rotate a key at https://app.steel.dev/settings/api-keys." ``` Event 97: ```text I can't run it live: there's no `STEEL_API_KEY` in this environment, and creating one requires a human sign-up at app.steel.dev. ``` Event 60: ```text steel-sdk import OK version: 0.19.0 ``` #### Hallucinated URLs - **Guessed PyPI package name before finding the real one**: Agent tried installing a package called 'steel-client' on PyPI (guessed from the product name) before discovering the actual package is 'steel-sdk'. The guess failed outright rather than being a broken documented link, since no documentation pointed to 'steel-client'. Event 21: ```text pip3 install steel-client -q 2>&1 | tail -2; python3 -c "import steel; print('steel-client OK', steel.__file__)" ``` Event 24: ```text ERROR: Could not find a version that satisfies the requirement steel-client (from versions: none) ERROR: No matching distribution found for steel-client ``` #### Blockers - **No API key available to run the live example**: The sandboxed environment has no STEEL_API_KEY and obtaining one requires a human to sign up at app.steel.dev and generate a key in the dashboard. This is a normal authentication requirement, not a product defect, and the agent correctly identified it rather than faking success. Event 87: ```text STEEL_API_KEY is not set. Create a free account at https://app.steel.dev and a key at https://app.steel.dev/settings/api-keys, then: export STEEL_API_KEY='...' ``` Event 97: ```text there's no `STEEL_API_KEY` in this environment, and creating one requires a human sign-up at app.steel.dev ``` #### Suggested Changes - **Fix the Node SDK package name in llms.txt**: Steel's AI-agent doc feed (steel.dev/llms.txt) lists the Node SDK as '@anthropic-ai/steel', but that package does not exist on npm (confirmed via npm registry lookup returning 'Not found'). Update the SDKs section of llms.txt to reference the real package 'steel-sdk' (verified present on npm and PyPI) so agents following the doc do not waste a round-trip guessing package names. Event 42: ```text {"message": "Not Found"} ``` Event 41: ```text [Node SDK](https://docs.steel.dev/overview/steel-cli): `@anthropic-ai/steel` for JavaScript/TypeScript projects. ``` ### Task given to each agent Help me build a simple example using Steel. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: A · 98/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Install and next-step links resolve ```text llms.txt links Notte quickstart and Steel Browser Skill Package, both returning 404. ``` - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with 200 when requested with Accept: text/markdown. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links docs, quickstart, API reference, CLI, SDKs and skills with organized sections. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under headings like Core docs, Getting started, Integrations, Developer resources. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions API Reference, MCP cookbook, CLI and skills; API and skills offered. ``` - **Pass** — A compact guide representation exists ```text Homepage serves text/markdown (2586 tokens) and docs pages return Markdown representations. ``` - **Pass** — A focused guide is directly retrievable ```text Steel CLI guide and Sessions quickstart give concrete install, auth, and first-session steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text CLI guide is 2492 tokens and quickstart 875, both well under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Homepage Markdown retains docs, API reference, CLI, and SDK links. ``` - **Pass** — The compact guide is independently actionable ```text Steel CLI guide gives install, login, session start/open/snapshot/stop commands and skill setup. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt and docs link Sessions API quickstart with account, API key, install, first session steps. ``` - **Pass** — Installation commands are extractable ```text Quickstart shows npm install steel-sdk playwright and pip install steel-sdk playwright. ``` - **Pass** — Code examples are available without interaction ```text Quickstart and CLI docs include full TypeScript, Puppeteer, and shell examples inline. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Quickstart states sign up, get API key at Settings > API Keys, set STEEL_API_KEY. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Homepage pricing section and docs pricing page render tiers and rates as static text. ``` - **Pass** — Prices are stated, not gated ```text Launch $0, Scale $250, Enterprise custom; rates $0.10/hr, $10/GB, $3/1k stated. ``` - **Pass** — Pricing units and limits are explicit ```text Docs list per-unit rates, concurrency, session time, RPM and retention limits per tier. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Final output gives a full pricing table (Launch vs Scale tiers, browser hours, proxy, CAPTCHA) sourced from docs.steel.dev/overview/pricinglimits (fetched at seq 41-43) with explicit plan/usage assumptions and caveats (per-minute billing, concurrency caps, paid-balance requirement). Kimi K3: Final output gives a pricing table (Launch/Scale/Enterprise rates) sourced from docs.steel.dev/overview/pricinglimits with explicit assumptions (plan tier, session length, credit validity, concurrency limits). Qwen 3.8 Max: Final output gives Launch/Scale/Enterprise tiers with explicit figures ($0/mo+usage, $250/mo, custom) and names the basis (per docs.steel.dev/overview/pricinglimits fetched at seq 57-59, dated June 2026). This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text llms.txt links an API Reference at docs.steel.dev/api-reference; API docs describe endpoints. ``` - **Pass** — An MCP server is documented and well-formed ```text Cookbook documents MCP servers in TS, Python, Rust, Go with five tools and client config. ``` - **Pass** — A CLI install path is documented ```text Steel CLI install documented: curl -fsS https://setup.steel.dev | sh, plus npx skills add. ``` - **Pass** — SDK packages resolve on their registries ```text npm steel-sdk and PyPI steel-sdk registry lookups both returned HTTP 200. ``` - **Pass** — Agent skills are published ```text Available Skills page lists five installable skills with npx and plugin install commands. ``` [Full report data](https://www.ax-check.com/steel.dev/report.json)