# AX Check: ai.google.dev Checked 2026-09-18. Gemini docs get agents running fast, but no llms.txt exists Quickstart has runnable code and clear auth steps, but the homepage skips Markdown and llms.txt returns a 404, so 3 of 23 checks flagged attention. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4 Pro, Kimi K3, Qwen 3.8 Max) completed successfully and reported Gemini's pricing clearly, citing free-tier limits and per-token rates by model, each noting rates can change over time. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/1e6405f6-d8be-4533-81cc-ead1f51fdb76) · [Read transcript](https://www.ax-check.com/ai.google.dev/sessions/deepseek.json) Final output gives per-token rates by model/context tier, names the free-tier trial limits, and frames cost against a one-shot 'hello world' workload assumption, flagging rates may change. #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained a Gemini API key on its own. It searched the environment, home directory, and repo for existing credentials, found none, and the example script failed at runtime with an explicit missing-key error. No authenticated call to the hosted Gemini API was ever made — only the SDK's local method signature was inspected (typeof check on a dummy key), which the task rules explicitly treat as insufficient. The agent stopped and reported the credential gap rather than resolving it. Event 11: ```text find /sandbox -maxdepth 3 -name '.env*' -o -maxdepth 3 -name '*.env' 2>/dev/null; grep -rilE 'api[_-]?key' /sandbox/repo 2>/dev/null | head; echo "---search home---" ``` Event 30: ```text Error: GEMINI_API_KEY is not set. Get one at https://aistudio.google.com/apikey and export it. ``` Event 35: ```text models.generateContent: function OK: SDK surface verified ``` Event 37: ```text There is **no `GEMINI_API_KEY` set in this environment**, and none in the repo/home files. So I could not make a live API call ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing Gemini API key prevented any live API call**: No GEMINI_API_KEY existed anywhere in the sandbox (env vars, repo, home dotfiles). This is a normal credential requirement, not a product defect — the agent correctly identified it, wrote a script that fails gracefully with a clear message, and stopped rather than fabricating a key or faking success. Session limitation, not a product bug. Event 6: ```text ---done--- ``` Event 30: ```text Error: GEMINI_API_KEY is not set. Get one at https://aistudio.google.com/apikey and export it. ``` #### Suggested Changes None identified in this transcript. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/07e647ae-e5fb-48a4-bcfc-d2d35c8ec571) · [Read transcript](https://www.ax-check.com/ai.google.dev/sessions/kimi.json) Final output gives free-tier limits (~10 req/min, ~250/day) and paid per-token rates (~$0.30/1M input, ~$2.50/1M output for Gemini 2.5 Flash) with explicit caveat that prices vary by model and change, linking the pricing doc. #### End-to-end onboarding - **Onboarding needs a login**: Agent found no Gemini API key in the environment and could not self-serve one — API keys for Google AI for Developers require a human to click through AI Studio's web console. The agent installed the SDK and wrote a working example script, but never executed it against the live API since it had no credentials. No authenticated operation against the hosted Gemini API occurred. Event 5: ```text PI_GATEWAY_API_KEY= Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'google.generativeai' ``` Event 16: ```text I'm blocked on credentials. There's no `GEMINI_API_KEY` in this environment (the `PI_GATEWAY_API_KEY` present is for the pi agent gateway, not Google). API keys can't be created programmatically — a human has to click through AI Studio. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Gemini API key available and none obtainable without human login**: The sandbox environment had no GEMINI_API_KEY, only an unrelated internal gateway key. Because Google requires interactive sign-in through AI Studio to mint an API key, the agent could not obtain credentials itself. This is a normal login/credential requirement, not a product defect — the agent correctly identified it and stopped rather than faking success. Event 16: ```text I'm blocked on credentials. There's no `GEMINI_API_KEY` in this environment (the `PI_GATEWAY_API_KEY` present is for the pi agent gateway, not Google). API keys can't be created programmatically — a human has to click through AI Studio. ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/2b80a732-8b8e-4434-84b9-c997d5fb12c1) · [Read transcript](https://www.ax-check.com/ai.google.dev/sessions/qwen.json) Final output gives pay-per-token rates for Flash-class models with named assumptions (free vs paid tier, model class, extras like thinking/image tokens, and notes rates shift by release). #### End-to-end onboarding - **Onboarding needs a login**: The agent installed the google-genai SDK and wrote a working quickstart script, but never obtained a real Gemini API key. The environment only had an internal gateway key (PI_GATEWAY_API_KEY) unrelated to Google's product. The script exited immediately with a message telling the human to manually get a key from AI Studio. No authenticated call to the hosted Gemini API was ever made, so no product operation was demonstrated. Event 3: ```text PI_GATEWAY_API_KEY= ``` Event 21: ```text GEMINI_API_KEY is not set. Get a free key at https://aistudio.google.com/apikey and export it. exit=1 ``` Event 23: ```text No `GEMINI_API_KEY` is set in this environment, so I couldn't run a live API call. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing Gemini API key blocked any live call**: The sandbox had no GEMINI_API_KEY (only an unrelated internal gateway key), so the agent could not authenticate to the hosted Gemini API. This is a missing-credential limitation of the test environment/session setup, not a product defect — API keys normally require a human to generate them via Google AI Studio. The agent correctly identified this and stopped rather than faking a result. Event 3: ```text PI_GATEWAY_API_KEY= ``` Event 21: ```text GEMINI_API_KEY is not set. Get a free key at https://aistudio.google.com/apikey and export it. exit=1 ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using Google AI for Developers. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: C · 62/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a Markdown Accept header; no Markdown representation offered. ``` - **Failed** — llms.txt provides an actionable documentation index ```text https://ai.google.dev/llms.txt returns HTTP 404, so no documentation index exists. ``` - **Failed** — A compact guide representation exists ```text No site-published Markdown guide; all fetched docs returned text/html despite Markdown Accept headers. ``` - **Skipped** — llms.txt provides navigation guidance ```text No llms.txt body was returned (404), so navigation guidance cannot be judged. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text No llms.txt body was returned (404), so API/MCP/skills mentions cannot be judged. ``` - **Pass** — A focused guide is directly retrievable ```text Managed agents quickstart gives concrete first steps with Python, JS, Java and REST examples. ``` - **Pass** — Equivalent instructions fit a token budget ```text Quickstart fits budget: 7431 tokens, well under 8000, and remains actionable. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text Managed agents quickstart gives concrete first steps: install SDK, create interaction, stream, save agent, with Python/JS/Java/REST code. ``` - **Pass** — Install and next-step links resolve ```text Fetched install/next-step links (get-started, ai-studio-quickstart, managed-agents-quickstart, api) all returned HTTP 200. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs sidebar and get-started page link quickstarts; managed-agents quickstart gives concrete first agent call steps. ``` - **Pass** — Installation commands are extractable ```text Quickstart shows installable SDK usage (from google import genai; @google/genai) and curl REST calls. ``` - **Pass** — Code examples are available without interaction ```text Quickstart embeds Python, JavaScript, Java and REST code samples inline without interaction. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Get API key link and x-goog-api-key/GEMINI_API_KEY usage make auth boundary explicit. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders full tier and per-model price tables as static HTML, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Free, Paid and Enterprise tiers plus per-model USD prices are stated openly on the page. ``` - **Pass** — Pricing units and limits are explicit ```text Prices are explicit per 1M tokens, per second, per image, with free-tier limits and dates. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Final output gives per-token rates by model/context tier, names the free-tier trial limits, and frames cost against a one-shot 'hello world' workload assumption, flagging rates may change. Kimi K3: Final output gives free-tier limits (~10 req/min, ~250/day) and paid per-token rates (~$0.30/1M input, ~$2.50/1M output for Gemini 2.5 Flash) with explicit caveat that prices vary by model and change, linking the pricing doc. Qwen 3.8 Max: Final output gives pay-per-token rates for Flash-class models with named assumptions (free vs paid tier, model class, extras like thinking/image tokens, and notes rates shift by release). This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Gemini API reference at /api documents REST endpoints, auth, and SDK references. ``` - **Skipped** — An MCP server is documented and well-formed ```text No MCP server documentation appears in the fetched Gemini API pages. ``` - **Pass** — A CLI install path is documented ```text Getting started documents npx skills add and pip/npm SDK install commands. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup result for a Gemini SDK package was supplied. ``` - **Pass** — Agent skills are published ```text Getting started publishes a Gemini API dev skill via npx skills add. ``` [Full report data](https://www.ax-check.com/ai.google.dev/report.json)