# AX Check: newrelic.com Checked 2026-10-28. Pricing is clear up front, but install steps hide behind a login 10 of 23 checks passed, led by pricing clarity. Quickstart install commands and code examples sit behind a login-gated launcher, unassessed. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed and read pricing straight off the live page, citing the same figures: 100 GB/month free ingest, then $0.40 or $0.60 per GB depending on data option, plus per-user monthly rates by edition. None reported being blocked by a paywall for pricing itself; the gap found elsewhere was that guided install links led to a login page rather than a documented command. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/27497c3e-8bc5-461a-95aa-ab7aafa49f22) · [Read transcript](https://www.ax-check.com/newrelic.com/sessions/deepseek.json) Quoted live pricing-page figures, each tied to its basis: 100 GB/month free, then $0.40 or $0.60/GB by data option, +$0.05/GB for EU, and user prices by edition, user type and annual vs monthly billing (seq 36, seq 65) #### End-to-end onboarding - **Onboarding needs a login**: The agent found no New Relic credentials in the environment and no CLI/account for self-service signup. It checked for env vars, confirmed hosted endpoints exist but are credential-gated (403/401), and then stated signup requires a human. It built example scripts and validated them only with a dummy key (expected 403) or --dry-run (no network call), never obtaining or using real credentials for an authenticated operation. Event 8: ```text env | grep -iE 'new.?relic|NEW_RELIC|NR_' | sed 's/=.*/=/' ``` Event 52: ```text POST https://insights-collector.newrelic.com/v1/accounts/1234567/events -> HTTP 403 ``` Event 65: ```text I have no New Relic account ID or API keys in this environment, and I can't create them — that requires a human to sign up (and likely billing). ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service way to obtain New Relic credentials**: The sandbox had no New Relic account ID or API keys, and there's no CLI or API to create a new account without a human completing signup/billing. This is a normal login/credential requirement, not a product defect — the agent correctly identified it couldn't proceed past this without a human providing an account ID, ingest key, and user API key. Event 65: ```text I have no New Relic account ID or API keys in this environment, and I can't create them — that requires a human to sign up (and likely billing). ``` #### Suggested Changes None identified in this transcript. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/04d8b1b0-4d90-422f-80f8-7faab81cc254) · [Read transcript](https://www.ax-check.com/newrelic.com/sessions/kimi.json) Quoted live-page prices with their basis: 100 GB/month free, $0.40 or $0.60/GB by data option, user prices by type and edition, Compute add-ons. Said the one-event demo stays free. (seq 18, seq 92, README.md) #### End-to-end onboarding - **Onboarding needs a login**: The agent checked for New Relic credentials in the environment at the very start, found none, and never obtained any during the session. It built a hosted Event API/NerdGraph example but the script exited with a missing-credential error both before and after edits, and the final message explicitly states the run was blocked by missing credentials. No signup, token issuance, or authenticated call to New Relic ever occurred. Event 4: ```text env | grep -E '^(NEW_RELIC|NR|NEWRELIC)' | sed -E 's/(KEY|TOKEN|SECRET|PASSWORD)=.*/\1=[redacted]/' ``` Event 74: ```text Missing NEW_RELIC_LICENSE_KEY. Set it, then run `npm start` again. Command exited with code 2 ``` Event 92: ```text I’m blocked from performing the live New Relic request because this environment does not contain a New Relic license key or user API key. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No New Relic credentials available in sandbox**: The sandbox had no NEW_RELIC_* environment variables, and the session provides no mechanism for self-service signup or key issuance. This is a test-environment/missing-credential limitation rather than a product defect — New Relic normally requires account signup and key generation through its own UI, which a non-interactive coding session cannot complete on its own. Event 6: ```text (no output) ``` Event 74: ```text Missing NEW_RELIC_LICENSE_KEY. Set it, then run `npm start` again. Command exited with code 2 ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/cbfc62d2-0ab3-4b20-b069-d4ea5b21bc10) · [Read transcript](https://www.ax-check.com/newrelic.com/sessions/qwen.json) Quoted rates taken from the live pricing page ($0.40/GB, $49 per core user, etc.) and estimated $0, assuming the free tier, near-zero ingest and a single free full-platform user. (seq 35, seq 44) #### End-to-end onboarding - **Onboarding needs a login**: The agent searched the sandbox environment, home directory, and filesystem for New Relic credentials and found none. It confirmed New Relic's hosted API endpoints are live but require authentication (401 on both NerdGraph and REST v2), and it correctly stopped rather than proceeding without a key. No self-service signup or credential acquisition occurred, and no authenticated operation against the product was performed. Event 23: ```text --- any local config/creds anywhere --- (end search) --- pi dir --- /sandbox/.pi /sandbox/.pi/agent /sandbox/.pi/agent/auth.json ``` Event 24: ```text --- NerdGraph (GraphQL API) --- {"errors":[{"message":"authentication required"}]} HTTP:401 --- REST v2 --- {"errors":[{"message":"authentication required"}]} HTTP:401 ``` Event 44: ```text there are **no New Relic credentials anywhere** in this environment — I checked env vars, home, and the filesystem. Every hosted endpoint is auth-gated, and account signup is interactive (email/OAuth + ToS), which I can't and shouldn't do on your behalf. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No New Relic API credentials available in sandbox**: The environment has no User API key, license key, or ingest key for New Relic, and account creation requires interactive signup (email/OAuth, ToS) that the agent cannot perform on its own. This is a missing-credentials limitation of the test environment, not a product defect — New Relic's APIs correctly require authentication. The agent behaved correctly by stopping and asking for a key rather than fabricating a demo. Event 44: ```text But there are **no New Relic credentials anywhere** in this environment — I checked env vars, home, and the filesystem. Every hosted endpoint is auth-gated, and account signup is interactive (email/OAuth + ToS), which I can't and shouldn't do on your behalf. ``` #### Suggested Changes - **Clarify which API key type is needed for common first tasks**: On the API keys docs or NerdGraph getting-started page, map common first actions (query data, create a dashboard, send custom events) to the specific key type required (User API key vs. admin key vs. license/ingest key) and explain how to find the account ID. Agent quote: key taxonomy is confusing and most calls need an account ID looked up separately. Verify by having a new developer pick the right key for a basic NRQL query without extra lookups. Event 44: ```text The caveat: **key taxonomy is genuinely confusing** — user key vs. admin key vs. license key vs. ingest keys, and most calls also need an account ID you have to look up first. That's the main friction. ``` ### Task given to each agent Help me build a simple example using New Relic. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 78/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html even when text/markdown was requested; no Markdown representation served. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt links to docs, release notes, support, learning, GitHub and platform access. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt organizes links under clear sections like Platform, Pricing, Solutions, Integrations and Documentation. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text llms.txt lists docs, GitHub and platform but does not mention an API, MCP server or skills. ``` - **Pass** — A compact guide representation exists ```text llms.txt is a standalone Markdown guide covering platform, pricing, docs and quickstarts. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt fetched directly as text/plain Markdown with organized sections and links. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 9605 tokens, well under the 8000-token guide budget threshold. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Skipped** — The compact guide is independently actionable ```text No compact agent guide fetched; only llms.txt index and marketing pages, no standalone workflow. ``` - **Skipped** — Install and next-step links resolve ```text No install or next-step links were fetched to confirm resolution. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt links Instant Observability quickstarts and docs.newrelic.com technical documentation. ``` - **Skipped** — Installation commands are extractable ```text Quickstart catalog pages show only titles; install commands sit behind a login-gated launcher. ``` - **Skipped** — Code examples are available without interaction ```text Fetched pages show quickstart listings and nav only; no code examples rendered. ``` - **Skipped** — Prerequisites and auth boundaries are explicit ```text No fetched page states prerequisites or API-key/auth requirements for setup. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page lists editions, data rates, and user prices in readable tables without interaction. ``` - **Pass** — Prices are stated, not gated ```text Prices stated: $0.40/GB, $0.60/GB, $49/user, $10/$99/$349 per user. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: 100 GB free ingest, per-GB rates, per-user monthly, 5-user cap. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Quoted live pricing-page figures, each tied to its basis: 100 GB/month free, then $0.40 or $0.60/GB by data option, +$0.05/GB for EU, and user prices by edition, user type and annual vs monthly billing (seq 36, seq 65) Kimi K3: Quoted live-page prices with their basis: 100 GB/month free, $0.40 or $0.60/GB by data option, user prices by type and edition, Compute add-ons. Said the one-event demo stays free. (seq 18, seq 92, README.md) Qwen 3.8 Max: Quoted rates taken from the live pricing page ($0.40/GB, $49 per core user, etc.) and estimated $0, assuming the free tier, near-zero ingest and a single free full-platform user. (seq 35, seq 44) This behavioural item does not affect the fast grade. ``` ### Activation - **Skipped** — An API reference or OpenAPI spec is reachable ```text No API reference or OpenAPI spec fetched; llms.txt links only docs.newrelic.com generally. ``` - **Skipped** — An MCP server is documented and well-formed ```text No MCP server documentation found in the fetched New Relic pages. ``` - **Skipped** — A CLI install path is documented ```text Guided install link leads to a login page, not a documented CLI install path. ``` - **Skipped** — SDK packages resolve on their registries ```text No SDK package registry lookup result supplied for any New Relic package. ``` - **Skipped** — Agent skills are published ```text No agent skills published; event mentions Claude Skills only as a third-party tool. ``` [Full report data](https://www.ax-check.com/newrelic.com/report.json)