# AX Check: databricks.com Checked 2026-10-14. Docs guide agents well, but real prices stay out of reach The homepage lacks a markdown view and the pricing page skips actual per-SKU prices, even though it explains DBU/DSU billing units clearly. ## Onboarding needs a login ## Coding sessions All three independent sessions completed. Two were reviewed on pricing and both described per-DBU cost ranges tied to workload type and cloud provider rather than fixed prices, noting trial and free-tier limits but not a concrete cost for their example task. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/d134ee5d-c6fd-42b7-8fed-449a5fc94e90) · [Read transcript](https://www.ax-check.com/databricks.com/sessions/deepseek.json) #### End-to-end onboarding - **Onboarding needs a login**: Agent installed the databricks-sdk and wrote a runnable example, but never obtained real Databricks credentials. No DATABRICKS_HOST/DATABRICKS_TOKEN env vars, CLI config, or workspace existed in the sandbox, so WorkspaceClient() failed at the default-auth step. No authenticated operation against a hosted workspace was ever performed. Event 5: ```text no databricks env vars --- cli --- no databricks CLI ``` Event 30: ```text Failed: ValueError: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. ``` Event 37: ```text There are no `DATABRICKS_*` env vars, no `~/.databrickscfg`, and no CLI in this sandbox. So I can write and statically verify code, but I **cannot run it or observe real results** until you provide either: - a workspace host + token/service-principal credentials, or - a Databricks Free Edition workspace (still requires a signup I can't do for you). ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Databricks workspace credentials available**: The sandbox had no DATABRICKS_HOST/DATABRICKS_TOKEN, no CLI config file, and no CLI binary installed, so the SDK's WorkspaceClient() could not resolve any auth method. This is a missing-credentials limitation of the test environment, not a product defect — Databricks requires a real workspace and token/service-principal to operate, which is normal for a hosted enterprise product. Event 5: ```text no databricks env vars --- cli --- no databricks CLI --- config --- no ~/.databricks config ``` Event 30: ```text Failed: ValueError: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. ``` - **SDK type name guessed incorrectly from memory**: Agent first tried to import ExecuteStatementRequest from databricks.sdk.service.sql, which does not exist in installed SDK version 0.152.0, causing an ImportError. This was agent error (recalling an outdated or incorrect API shape) rather than a product issue, and was corrected a few steps later by inspecting the actual module contents. Event 14: ```text ImportError: cannot import name 'ExecuteStatementRequest' from 'databricks.sdk.service.sql' (/opt/freestyle/python/lib/python3.12/site-packages/databricks/sdk/service/sql.py) ``` - **Pricing page is JS-rendered, not scrapable via curl**: A curl fetch of the Databricks pricing page returned only sparse keyword hits (DBU mentions) because the page content is rendered client-side, preventing the agent from extracting concrete current pricing figures via simple HTTP fetch. This is a test-environment/tooling limitation (no browser rendering available), not necessarily a product defect. Event 35: ```text 5 DBU 1 DBu 1 Serverless 3 dbu ``` Event 37: ```text I could not scrape them (JS-rendered) ``` #### Suggested Changes None identified in this transcript. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/41624b3f-1460-4b23-8312-696b881d6542) · [Read transcript](https://www.ax-check.com/databricks.com/sessions/kimi.json) Pricing was given as per-DBU ranges that depend on workload type (serverless SQL, all-purpose, jobs), plus the underlying cloud VM cost. The 14-day trial and free Community Edition limits were also named. (seq 6) #### End-to-end onboarding - **Onboarding needs a login**: Agent checked the local environment for Databricks credentials (env vars, config file, CLI, SDK) and found none. No self-service path existed to provision a workspace or obtain a personal access token, so the agent stopped immediately without ever authenticating or performing any hosted operation. Event 3: ```text env | grep -iE 'databricks|DATABRICKS' ; ls ~/.databrickscfg 2>/dev/null; echo "---"; which databricks; pip list 2>/dev/null | grep -i databricks ``` Event 4: ```text Command exited with code 1 ``` Event 6: ```text I checked this environment: no credentials, no CLI, no SDK, and no way to create a workspace myself. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Databricks workspace or credentials available**: Databricks requires a provisioned cloud workspace plus a personal access token (DATABRICKS_HOST/DATABRICKS_TOKEN). This is a missing-credentials limitation of the test environment, not a product defect — the agent correctly identified there was no self-service way to create a workspace or token from inside the sandbox. Event 6: ```text It requires a provisioned workspace (on AWS/Azure/GCP) plus a personal access token (`DATABRICKS_HOST` + `DATABRICKS_TOKEN`). I checked this environment: no credentials, no CLI, no SDK, and no way to create a workspace myself. ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/7e3bd62b-16e4-4ca2-a55c-0021f6941781) · [Read transcript](https://www.ax-check.com/databricks.com/sessions/qwen.json) Gave a ~$0.07–0.55/DBU range tied to product tier, serverless vs classic, cloud provider and commitment, plus separate cloud infra costs. It didn't estimate a cost for the example itself (seq 23) #### End-to-end onboarding - **Onboarding needs a login**: The agent wrote a Databricks SDK example and attempted to run it, but no DATABRICKS_HOST or DATABRICKS_TOKEN were present in the environment and no self-service credential acquisition occurred. The script exited with a credentials-missing error, and the agent explicitly asked the human to supply a workspace URL and personal access token to proceed. No authenticated operation against a real Databricks workspace was ever demonstrated. Event 4: ```text env | grep -i -E 'databricks|dbx' || echo "no databricks env vars" ``` Event 21: ```text Missing credentials. Set DATABRICKS_HOST and DATABRICKS_TOKEN (workspace URL + personal access token from User Settings > Developer > Access Tokens). exit=1 ``` Event 23: ```text Blocked on credentials**: there's no `DATABRICKS_HOST` or `DATABRICKS_TOKEN` in this environment, so I can't execute against a real workspace. To unblock: give me a workspace URL and a personal access token ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No Databricks workspace credentials available in the sandbox**: The environment had no DATABRICKS_HOST or DATABRICKS_TOKEN set, and there is no self-service way for the agent to obtain a workspace or token (these require UI-based account/workspace creation and manual token issuance by a human). This is a normal credential/login requirement of the product, not a product defect, and it fully blocked any live execution against a real workspace. Event 4: ```text no databricks env vars ``` Event 23: ```text I can't do the UI-only steps like initial workspace creation or token issuance — you must provide those ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using Databricks. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 79/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html even when text/markdown was requested; no Markdown representation offered. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links Databricks Docs, Developer Hub, Community, Answers and full reference with routing guidance. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt gives source-selection guidance on which manifest to trust for each claim type. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt points to Developer Hub for CLI and coding-agent setup and Docs for APIs. ``` - **Pass** — A compact guide representation exists ```text llms.txt is a compact Markdown guide with routing index, products, and source guidance. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt fetched directly at /llms.txt as text/plain Markdown, 2999 tokens. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 2999 tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text llms.txt routes agents to docs, developer hub, CLI and coding-agent setup with source-selection guidance. ``` - **Pass** — Install and next-step links resolve ```text Docs, reference API and developer hub links fetched successfully from the llms.txt index. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs homepage links 'Get started tutorials' and free-trial signup for guided onboarding. ``` - **Skipped** — Installation commands are extractable ```text No installation or CLI setup commands appear in the fetched docs landing page. ``` - **Skipped** — Code examples are available without interaction ```text Fetched docs landing page contains no code examples, only navigation links. ``` - **Skipped** — Prerequisites and auth boundaries are explicit ```text No prerequisites or API-key/auth boundary details shown on fetched pages. ``` ### Pricing - **Failed** — Prices are stated, not gated ```text Pricing page states model but shows no actual per-SKU prices; product pages load prices dynamically. ``` - **Pass** — Pricing is readable without interaction ```text Pricing page renders pay-as-you-go, committed-use, and per-second billing text without interaction. ``` - **Pass** — Pricing units and limits are explicit ```text Pricing page defines DBU, DSU, GB, and per-second compute-based billing units explicitly. ``` - **Pass** — Agents identify pricing and its assumptions ```text 2 of 3 sessions were judged on pricing; 0 fell short. Kimi K3: Pricing was given as per-DBU ranges that depend on workload type (serverless SQL, all-purpose, jobs), plus the underlying cloud VM cost. The 14-day trial and free Community Edition limits were also named. (seq 6) Qwen 3.8 Max: Gave a ~$0.07–0.55/DBU range tied to product tier, serverless vs classic, cloud provider and commitment, plus separate cloud infra costs. It didn't estimate a cost for the example itself (seq 23) This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Reference page links Databricks REST API reference and many SDK/Python API references. ``` - **Skipped** — An MCP server is documented and well-formed ```text No MCP server documentation page was fetched; only a databricks-mcp Python API reference is listed. ``` - **Pass** — A CLI install path is documented ```text Reference lists Databricks CLI commands and CLI docs; llms.txt points to Developer Hub CLI setup. ``` - **Skipped** — SDK packages resolve on their registries ```text SDK references are linked but no registry lookup result for a Databricks package was supplied. ``` - **Skipped** — Agent skills are published ```text No agent skills publication was fetched or referenced in the supplied evidence. ``` [Full report data](https://www.ax-check.com/databricks.com/report.json)