# AX Check: docs.contrastsecurity.com Checked 2026-10-06. Docs homepage loads, but no agent-ready guide or quickstart exists No Markdown or compact guide is served, and the llms.txt index returns a 404, leaving only a bare homepage and a working API docs link. ## Onboarding needs a login ## Coding sessions All three independent sessions (DeepSeek V4.1 Flash, Kimi K3, Qwen 3.8 Max) completed and reported the same concrete pricing tiers: Free, Pro at $750/month billed annually, and custom-quote Enterprise, each noting metering assumptions like app and service limits. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/ebddce66-7fef-4f79-810a-db7a19786441) · [Read transcript](https://www.ax-check.com/docs.contrastsecurity.com/sessions/deepseek.json) Final output states concrete tiers (Free $0, Pro $750/mo billed annually, Enterprise custom quote) with named assumptions — CVE Shield product line, Java-on-Linux runtime agent, app/service limits, and a caveat that free tier is agent-based not SDK/API. #### End-to-end onboarding - **Onboarding needs a login**: Agent built a working SDK example and CLI workflow against the real hosted Contrast API but never obtained real credentials. Non-interactive auth requires an API key, authorization header, org ID, and host that only exist inside a logged-in Contrast tenant, or an interactive browser OAuth flow the agent cannot complete unattended. Dummy credentials correctly returned HTTP 401 from the live API, proving the endpoint is real but access is refused without human-provisioned credentials. Event 124: ```text GET applications failed: HTTP 401\n{"timestamp":"2026-10-06T15:01:50.825Z","path":"/Contrast/api/ng/00000000-0000-0000-0000-000000000000/applications/filter","status":401,"error":"Unauthorized","requestId":"328a5cd1-261731"} ``` Event 76: ```text No parameters provided. Please run `contrast auth --api-key --authorization --host --organization-id ` ``` Event 46: ```text Get the Contrast API key, the authorization header, and organization ID by logging into the Contrast web interface and selecting user menu > User settings. ``` Event 38: ```text Authenticate Contrast using your Github or Google account OR include credentials if you are an existing licensed Contrast user. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-serve path to real Contrast credentials**: Both the SDK (ContrastSdk username/api_key/service_key/teamserver_url) and the CLI (contrast auth --api-key --authorization --host --organization-id) require four values that only exist after logging into the Contrast web UI under User settings, or completing an interactive GitHub/Google browser OAuth flow. This is a credentials/product-behavior limitation, not an agent error: the agent correctly identified the requirement, tested with dummy values against the live API, and got a clean 401 confirming the endpoint works but access is refused without a human-provisioned tenant. Event 76: ```text No parameters provided. Please run `contrast auth --api-key --authorization --host --organization-id ` ``` Event 124: ```text GET applications failed: HTTP 401\n{"timestamp":"2026-10-06T15:01:50.825Z","path":"/Contrast/api/ng/00000000-0000-0000-0000-000000000000/applications/filter","status":401,"error":"Unauthorized","requestId":"328a5cd1-261731"} ``` - **Free tier is runtime-agent only, not SDK/API accessible**: The published free tier (CVE Shield) is delivered by instrumenting a running Java application with the Contrast ADR agent, not via a lightweight SDK or API call. This conflicts with the session's constraint to stay light and avoid local service stacks, so the agent could not exercise the free offering within the test's restrictions. Event 60: ```text Does CVE Shield require code changes to install? No. CVE Shield uses the Contrast ADR agent, which is installed once. ``` Event 61: ```text Access is provided exclusively through the Northstar UI, and it does not include SIEM integration or support for multiple hosts beyond the defined application limits. ``` #### Suggested Changes - **Add a self-serve API key generation path for free-tier signups**: Obtaining the API key, authorization header, organization ID, and host needed for CLI/SDK use requires a human to log into the Contrast web UI's User settings page; there is no documented way to provision these programmatically. Add a signup-to-API-key flow and verify by confirming a fresh, unauthenticated session can reach contrast audit or the Python SDK's filter_applications call successfully end to end. Event 46: ```text Get the Contrast API key, the authorization header, and organization ID by logging into the Contrast web interface and selecting user menu > User settings. ``` - **Clarify on pricing page that CLI/SDK Assess/SCA products differ from CVE Shield's free tier**: The pricing and try-contrast-security pages only describe CVE Shield plans (Free/Pro/Enterprise), while the CLI and Python SDK tested belong to the classic Assess/SCA/Scan product line with no listed price. Add a note on the pricing-and-packaging page distinguishing CVE Shield pricing from Assess/SCA/Scan licensing so developers know which product the SDK/CLI credentials unlock. Event 53: ```text CVE Shield Your Mythos shield to detect, monitor and prevent exploitation of AI-accelerated attacks FREE $0 ``` Event 21: ```text contrastapi 1.24.0 Official Python SDK for ContrastAPI — security intelligence for developers and AI agents ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/ea43a8e6-5998-4f3a-9db8-571afc20b5fb) · [Read transcript](https://www.ax-check.com/docs.contrastsecurity.com/sessions/kimi.json) Final output gives concrete tiers (Free ~2 apps/12 services, Pro $750/mo billed annually ~8 apps/50 services, Enterprise custom) and names assumptions like metering unit being 'services', observation-mode-only limits, and 'available soon' status. #### End-to-end onboarding - **Onboarding needs a login**: Agent never obtained Contrast Security credentials. Signup requires a human-verified email account, and the product's core workflow requires attaching a language agent to a running, instrumented application and exercising it to generate data. The agent wrote an untested API scaffold script referencing placeholder env vars but never executed it or any authenticated call against the hosted API. No credential acquisition or authenticated operation occurred. Event 44: ```text That requires a live, instrumented service process plus credentials from a signup I can't complete (email-verified account). ``` Event 46: ```text Auth model (documented by Contrast): two headers on every request — # Authorization: base64(":") # API-Key: ``` Event 49: ```text Account provisioning is human-gated — signup needs an email-verified account (and sales contact for anything beyond free). ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Signup requires human email verification**: Contrast Security's account creation flow needs a verified email/business account, which the agent could not complete autonomously. This is a normal authentication gate, not a product defect, but it stopped any further hosted-product interaction. Event 44: ```text credentials from a signup I can't complete (email-verified account) ``` Event 49: ```text Account provisioning is human-gated — signup needs an email-verified account (and sales contact for anything beyond free). ``` - **Core workflow requires a running instrumented application**: Contrast's quickstart is agent-based: attach a language agent to a live running application and exercise its routes so the agent reports data to the hosted UI/API. This conflicts with the session's constraint to avoid starting local service stacks or long-running processes, so the agent correctly halted rather than spin up a local app. Event 44: ```text Attach the agent to your app (`javaagent` flag, or `@contrast/agent` for Node) and **run the application**. ``` Event 44: ```text no SDK/API-only developer workflow I can act on, and the quickstart requires a local running service. I'm stopping rather than faking it. ``` #### Suggested Changes - **Add a documented API-only quickstart path for existing data**: The docs site (docs.contrastsecurity.com/en/get-started.html and api.contrastsecurity.com) only describe the agent-instrumentation workflow; there is no path for a developer to interact with the hosted API without first running and instrumenting a live application. Add a lightweight path (e.g., sample/sandbox organization with pre-populated findings) so developers can exercise the REST API without standing up a service, and verify by confirming a new signup can call an endpoint like /Contrast/api/ng/{orgId}/applications and get non-empty results without installing an agent first. Event 44: ```text Attach the agent to your app (`javaagent` flag, or `@contrast/agent` for Node) and **run the application**. ``` - **Clarify API authentication requirements directly in the API docs page**: The api.contrastsecurity.com/en/api-overview.html page returned only a JavaScript app shell with no visible authentication details; the agent had to infer the Authorization/API-Key header scheme from indirect sources. Render the authentication section (API key, service key, base64 Authorization header) as static/server-rendered content so it is visible without executing JavaScript, and verify by fetching the page with a plain HTTP client and confirming the auth headers appear in the response body. Event 42: ```text ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/c22c7538-90c2-4ba0-b85d-1acca6d16ac2) · [Read transcript](https://www.ax-check.com/docs.contrastsecurity.com/sessions/qwen.json) README.md pricing table (fetched live per seq 43 web fetch of contrastsecurity.com/pricing) states Free/Pro/Enterprise figures with explicit assumptions: per-application/per-service metering, annual billing, app/user/history limits, and notes the broader platform is sales-quoted. #### End-to-end onboarding - **Onboarding needs a login**: No self-service credential acquisition occurred. The agent never obtained real Contrast Security account credentials; it only tested the live API with deliberately fake/dummy values (CONTRAST_USERNAME=x@example.com, CONTRAST_API_KEY=k, CONTRAST_SERVICE_KEY=c) and confirmed the API correctly returns 401 Unauthorized. The agent explicitly asked the user to supply real credentials at the end, which remains unresolved within the session. Event 84: ```text CONTRAST_USERNAME=x@example.com CONTRAST_API_KEY=k CONTRAST_SERVICE_KEY=c timeout 25 python contrast_inventory.py 2>&1 | tail -5; echo "exit=$?" ``` Event 97: ```text 401 {"timestamp":"2026-10-06T15:01:14.676Z","path":"/Contrast/api/ng/profile/organizations/default","status":401,"error":"Unauthorized","requestId":"968510dc-355119"} ``` Event 121: ```text Blocked on one thing: live calls need your `CONTRAST_USERNAME`, `CONTRAST_API_KEY`, `CONTRAST_SERVICE_KEY` (UI: Your Account → API Keys) and optionally `CONTRAST_ORG_UUID`. Give me those and I'll run the real inventory ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No real Contrast Security account available in session**: The sandbox environment had no Contrast credentials and no self-serve signup path was exercised. The agent could only validate its script's auth-failure handling against dummy credentials, confirming the hosted API correctly rejects bad auth with a 401, but could never reach a real org, application, or vulnerability (trace) data. This is a missing-credentials/test-environment limitation, not a product defect, since Contrast's platform does not offer a public sandbox and normally requires an existing account. Event 85: ```text No organization visible to this user; set CONTRAST_ORG_UUID explicitly. exit=0 ``` Event 97: ```text 401 {"timestamp":"2026-10-06T15:01:14.676Z","path":"/Contrast/api/ng/profile/organizations/default","status":401,"error":"Unauthorized" ``` - **SDK package ships empty __init__.py, breaking documented top-level imports**: The installed contrast-security 0.23 package has zero-byte __init__.py files at both the package root and filters/ subpackage, so the natural import style (from contrast_security import ContrastSdk) fails and the agent had to discover fully-qualified submodule paths by reading source. This is a product/package defect in the PyPI SDK, recovered by the agent through trial and error rather than documentation. Event 67: ```text 0 __init__.py 0 filters/__init__.py 0 total ``` Event 71: ```text # The SDK ships empty __init__.py files, so imports are fully qualified. + 28 from contrast_security.contrast_sdk import ContrastSdk ``` #### Suggested Changes - **Fix or populate __init__.py exports in the contrast-security PyPI package**: In the contrast-security package (filters/__init__.py and top-level __init__.py), add the expected re-exports (e.g. ContrastSdk, ApplicationFilter, ApplicationTraceFilter) so `from contrast_security import ContrastSdk` works as a typical new developer would expect. Verify by running `pip install contrast-security` fresh and executing `from contrast_security import ContrastSdk` without ImportError. Event 67: ```text 0 __init__.py 0 filters/__init__.py 0 total ``` - **Document that SDK calls return raw requests.Response objects, not parsed JSON**: Add a note in the SDK README/quickstart (github.com/Contrast-Security-OSS/contrast-sdk-python) clarifying that every API call returns a raw requests.Response requiring manual .json() parsing and status-code checks, since this was not evident from the installed package and the agent had to inspect api_support.py directly to learn it. Verify by confirming a new quickstart snippet shows response.json() usage. Event 93: ```text def _get(self, path, params=None): return requests.get(self.build_url(path), params=params, headers=self._headers) ``` ### Task given to each agent Help me build a simple example using Contrast Security. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: F · 23/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html even when text/markdown was requested; no Markdown representation served. ``` - **Failed** — llms.txt provides an actionable documentation index ```text /llms.txt returned the site's 404 page, so no documentation index exists. ``` - **Skipped** — llms.txt provides navigation guidance ```text No llms.txt body was returned, so navigation guidance cannot be judged. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text No llms.txt body was returned, so API/MCP/skills mentions cannot be judged. ``` - **Skipped** — A compact guide representation exists ```text No site-published Markdown guide representation was fetched; homepage Markdown is unsupported. ``` - **Skipped** — A focused guide is directly retrievable ```text No standalone or negotiated Markdown guide page was fetched for direct retrieval. ``` - **Skipped** — Equivalent instructions fit a token budget ```text No compact Markdown guide was fetched, so token budget cannot be measured. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown is unsupported, so link preservation across formats is unassessed. ``` - **Skipped** — The compact guide is independently actionable ```text No compact agent guide fetched; only homepage and a 404 page were retrieved. ``` - **Skipped** — Install and next-step links resolve ```text No install or next-step page was fetched to confirm its links resolve. ``` ### Onboarding - **Skipped** — Docs lead to a relevant quickstart ```text Only a 404 page was fetched; no quickstart or docs navigation content available. ``` - **Skipped** — Installation commands are extractable ```text No installation or setup page was fetched to extract commands from. ``` - **Skipped** — Code examples are available without interaction ```text No documentation page with code examples was fetched. ``` - **Skipped** — Prerequisites and auth boundaries are explicit ```text No auth or prerequisites content was fetched for assessment. ``` ### Pricing - **Skipped** — Pricing is readable without interaction ```text No pricing page was fetched; only the docs homepage is available. ``` - **Skipped** — Prices are stated, not gated ```text No pricing page was fetched; prices are not shown in the supplied evidence. ``` - **Skipped** — Pricing units and limits are explicit ```text No pricing page was fetched, so units and limits cannot be judged. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Final output states concrete tiers (Free $0, Pro $750/mo billed annually, Enterprise custom quote) with named assumptions — CVE Shield product line, Java-on-Linux runtime agent, app/service limits, and a caveat that free tier is agent-based not SDK/API. Kimi K3: Final output gives concrete tiers (Free ~2 apps/12 services, Pro $750/mo billed annually ~8 apps/50 services, Enterprise custom) and names assumptions like metering unit being 'services', observation-mode-only limits, and 'available soon' status. Qwen 3.8 Max: README.md pricing table (fetched live per seq 43 web fetch of contrastsecurity.com/pricing) states Free/Pro/Enterprise figures with explicit assumptions: per-application/per-service metering, annual billing, app/user/history limits, and notes the broader platform is sales-quoted. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Homepage links 'API docs' to api.contrastsecurity.com, which returns 'Contrast API Documentation'. ``` - **Skipped** — An MCP server is documented and well-formed ```text No MCP server documentation found in the fetched evidence. ``` - **Skipped** — A CLI install path is documented ```text No CLI install path documented in the fetched evidence. ``` - **Skipped** — SDK packages resolve on their registries ```text No SDK package registry results supplied in the fetched evidence. ``` - **Skipped** — Agent skills are published ```text No agent skills published in the fetched evidence. ``` [Full report data](https://www.ax-check.com/docs.contrastsecurity.com/report.json)