# AX Check: langchain.com Checked 2026-10-22. LangChain's quickstart and pricing are clear, but llms.txt is missing Docs quickstart gives runnable install steps and pricing is fully public ($0 to $39/seat to custom), but the homepage has no Markdown version and llms.txt 404s. ## Onboarding needs a login ## Coding sessions All three independent sessions completed and reviewed pricing using the public pricing page, landing on the same figures: free MIT-licensed library, Developer tier at $0, Plus at $39/seat/month with trace limits, and usage-based add-ons, with two sessions noting the numbers as unverified or ballpark. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/e19507aa-077b-47b6-ad28-c5a2b499a77b) · [Read transcript](https://www.ax-check.com/langchain.com/sessions/deepseek.json) Cost tied to plan and limits from the fetched pricing page: free MIT library plus model-provider tokens; Developer $0 (1 seat, 5k traces); Plus $39/seat (10k traces); usage billed at $1/LSU with metered rates. (seq 27, seq 54, seq 55, seq 57) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent used an API key already present in the sandbox (PI_GATEWAY_API_KEY, from the pi gateway config, not a self-service signup) to call the Fireworks inference endpoint directly, bypassing LangChain/LangSmith's own product. No LangSmith account or hosted LangChain resource was created. The only operation shown was a raw chat completion via Fireworks, not an authenticated operation on LangChain/LangSmith itself. Event 21: ```text "apiKey": "$PI_GATEWAY_API_KEY" ``` Event 33: ```text "id":"chatcmpl-e1dad9e7" ``` Event 57: ```text Full LangSmith tracing/eval/deployment needs a LangSmith account and `LANGSMITH_API_KEY`; I didn't need it for the example. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **LangSmith account never obtained, so tracing/deployment untested**: The agent never acquired a LangSmith API key and explicitly skipped testing LangSmith's hosted tracing, evaluation, or deployment features, which are the paid parts of the product. This is a session limitation (no self-service credential acquisition occurred) rather than a demonstrated product defect, since the agent did not attempt to sign up. Event 57: ```text Full LangSmith tracing/eval/deployment needs a LangSmith account and `LANGSMITH_API_KEY`; I didn't need it for the example. ``` - **Fireworks /models endpoint returned 401 despite valid key**: A direct curl call to the Fireworks /models listing endpoint with the sandbox's gateway API key returned HTTP 401, even though the same key worked moments later for /chat/completions. This is a minor product/environment inconsistency rather than a blocker that stopped progress, since the agent worked around it immediately. Event 25: ```text models:401 ``` Event 33: ```text "id":"chatcmpl-e1dad9e7" ``` #### Suggested Changes - **Preinstall langchain-openai alongside the core langchain package**: The sandbox had langchain, langchain-core, and langsmith preinstalled but not langchain-openai, forcing an extra pip install step before any OpenAI-compatible example could run. Bundling langchain-openai in quickstart setup instructions would remove this friction; verify by checking that importing langchain_openai succeeds immediately after following quickstart install steps. Event 26: ```text ModuleNotFoundError: No module named 'langchain_openai' ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/31958dc7-1802-4fc2-9fc0-e2796435c47c) · [Read transcript](https://www.ax-check.com/langchain.com/sessions/kimi.json) Pricing came with its assumptions: the SDK is free under MIT, gpt-4o-mini runs about $0.15 per 1M input tokens, and LangSmith tiers depend on seats and trace volume. None of these figures were checked against a source during the session. (seq 47) #### End-to-end onboarding - **Onboarding needs a login**: The agent discovered a PI_GATEWAY_API_KEY in the sandbox environment and tried it against Anthropic, OpenAI, and Fireworks hosted endpoints, but all three rejected it as invalid/unauthorized. No valid credential was obtained during the session, so no authenticated hosted LLM call was ever made. The final script fell back to a local FakeListChatModel with zero network calls, which does not count as a real product operation. Event 7: ```text PI_GATEWAY_API_KEY= ``` Event 24: ```text {"type":"error","error":{"type":"authentication_error","message":"invalid x-api-key"} ``` Event 38: ```text {"error":{"message":"The API key you provided is invalid.","param":null,"code":"UNAUTHORIZED","type":"error"} ``` Event 47: ```text no live hosted-LLM call was possible. This sandbox has no valid provider credentials — the `PI_GATEWAY_API_KEY` present in the environment is rejected by Anthropic, OpenAI, and Fireworks endpoints. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No valid hosted LLM API key available in sandbox**: The only credential present (PI_GATEWAY_API_KEY) was rejected by all three hosted LLM providers tested (Anthropic, OpenAI, Fireworks), each returning an authentication/authorization error. This is a test-environment credential gap, not a LangChain product defect — LangChain itself is a client SDK and doesn't issue or manage these keys. The agent correctly identified this and stopped short of faking a live call, falling back to a local FakeListChatModel to prove the chain wiring only. Event 24: ```text invalid x-api-key ``` Event 38: ```text The API key you provided is invalid. ``` Event 47: ```text This sandbox has no valid provider credentials — the `PI_GATEWAY_API_KEY` present in the environment is rejected by Anthropic, OpenAI, and Fireworks endpoints. ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/f8f283b3-fb27-4486-9322-3024480a937e) · [Read transcript](https://www.ax-check.com/langchain.com/sessions/qwen.json) Said the library is free (MIT), inference is billed per token by the provider, and LangSmith has a free tier plus ~$39/seat/mo with usage-based traces; flagged the figures as ballpark (seq 47) #### End-to-end onboarding - **End-to-end onboarding not demonstrated**: The agent used a pre-provisioned Fireworks gateway API key (PI_GATEWAY_API_KEY) that was already present in the sandbox environment, not obtained through any self-service signup or onboarding flow during the session. It then used that key to successfully call the hosted Fireworks LLM endpoint via langchain-openai. This demonstrates a working developer workflow, but since the credential was pre-supplied by the test harness rather than obtained by the agent itself, it does not meet the bar for verified self-service onboarding. Event 7: ```text PI_GATEWAY_API_KEY= ``` Event 23: ```text "apiKey": "$PI_GATEWAY_API_KEY", "authHeader": true, ``` Event 36: ```text hello ``` Event 45: ```text == Structured output == sentiment='mixed' confidence=0.95 summary='Positive about battery life, negative about screen durability; overall mixed feelings.' ``` #### Hallucinated URLs None identified in this transcript. #### Blockers None identified in this transcript. #### Suggested Changes - **Update LangChain v1.x migration docs for create_agent import path**: The agent noted that langchain-openai was not preinstalled alongside langchain 1.3.15, and had to separately pip install it, plus flagged that v1.x reshuffled imports like create_agent. Clarifying in the quickstart/migration guide which packages (langchain vs langchain-openai vs langchain-core) are required for a basic ChatOpenAI + create_agent example would reduce setup friction. Verify by following the quickstart with only 'pip install langchain' and confirming whether langchain-openai needs explicit mention. Event 27: ```text ModuleNotFoundError: No module named 'langchain_openai' ``` Event 47: ```text LangChain's APIs evolve fast (v1.x recently reshuffled imports like `create_agent`), so pin versions ``` ### Task given to each agent Help me build a simple example using LangChain. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 65/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html even when Markdown was requested; no Markdown representation offered. ``` - **Failed** — llms.txt provides an actionable documentation index ```text https://langchain.com/llms.txt returns HTTP 404, so no documentation index exists. ``` - **Skipped** — llms.txt provides navigation guidance ```text llms.txt is 404, so its navigation guidance cannot be evaluated. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text llms.txt is 404, so mentions of API, MCP, or skills cannot be evaluated. ``` - **Pass** — A compact guide representation exists ```text Docs quickstart served as text/markdown at prompt-engineering-quickstart.md, a standalone Markdown representation. ``` - **Pass** — A focused guide is directly retrievable ```text Prompt engineering quickstart is directly retrievable as Markdown with concrete UI and SDK steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Markdown quickstart measures ~3.7k tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text Prompt engineering quickstart gives prerequisites, API keys, install commands, and runnable Python/TypeScript steps. ``` - **Pass** — Install and next-step links resolve ```text Quickstart's install and next-step links resolve; docs.langchain.com and reference.langchain.com fetches succeed. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs link to a prompt engineering quickstart with concrete create/test/iterate steps. ``` - **Pass** — Installation commands are extractable ```text Quickstart shows pip install langsmith openai langchain_core and npm install commands. ``` - **Pass** — Code examples are available without interaction ```text Python and TypeScript code examples appear inline in the quickstart without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Prerequisites list LangSmith account, LangSmith API key, and OpenAI API key. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders plan tiers, prices and feature tables as static text without interaction. ``` - **Pass** — Prices are stated, not gated ```text Developer $0, Plus $39/seat/month, Enterprise custom pricing all stated publicly. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: seats, 5k/10k base traces, LSU at $1.00, per-vCPU-hr rates. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Cost tied to plan and limits from the fetched pricing page: free MIT library plus model-provider tokens; Developer $0 (1 seat, 5k traces); Plus $39/seat (10k traces); usage billed at $1/LSU with metered rates. (seq 27, seq 54, seq 55, seq 57) Kimi K3: Pricing came with its assumptions: the SDK is free under MIT, gpt-4o-mini runs about $0.15 per 1M input tokens, and LangSmith tiers depend on seats and trace volume. None of these figures were checked against a source during the session. (seq 47) Qwen 3.8 Max: Said the library is free (MIT), inference is billed per token by the provider, and LangSmith has a free tier plus ~$39/seat/mo with usage-based traces; flagged the figures as ballpark (seq 47) This behavioural item does not affect the fast grade. ``` ### Activation - **Skipped** — An API reference or OpenAPI spec is reachable ```text No API reference or OpenAPI spec was fetched; docs quickstart only links to reference pages. ``` - **Skipped** — An MCP server is documented and well-formed ```text Docs mention connecting via MCP, but no MCP server documentation was fetched. ``` - **Skipped** — A CLI install path is documented ```text No CLI install path documented in fetched pages; only SDK pip/npm installs shown. ``` - **Pass** — SDK packages resolve on their registries ```text PyPI langsmith and npm langsmith registry lookups both returned HTTP 200. ``` - **Skipped** — Agent skills are published ```text No agent skills publication was found in the fetched evidence. ``` [Full report data](https://www.ax-check.com/langchain.com/report.json)