Secure your environment and eliminate secrets sprawl by removing hard-coded credentials from your source code, config files and CI/CD systems. Learn More
Custom Pricing
Request a Quote ``` Event 69: ```text "SecretManager": { "secretManagerMonthlyOverageCost": 20, "secretManagerUserCost": 2400 } ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/2ec9d421-7fb8-4630-a1e7-24f585defc51) · [Read transcript](https://www.ax-check.com/keepersecurity.com/sessions/qwen.json) Quoted ~$3.75–4.50/user/month for Business, annual billing, and said the figures were unverified because the page renders them in JS. The figures came from memory and Secrets Manager got no number (seq 123, seq 129, seq 136) #### End-to-end onboarding - **Onboarding needs a login**: Agent never obtained real Keeper Secrets Manager credentials. It installed the SDK and ran it against the SDK's own built-in mock (MockConfig/ResponseQueue), which generates fake tokens and fake encrypted responses entirely locally with no network calls to Keeper's hosted service. When run without a real KSM_CONFIG token, the actual example script exited immediately with a blocked message instructing the user to get a token from the Keeper Admin Console. No authenticated operation against the real hosted Keeper service was ever demonstrated. Event 118: ```text Application can access 2 secret(s): - uid=0ee32efab29c41c4b506ca title=Prod DB login: ['admin'] password: ['s3cret-prod'] ``` Event 118: ```text BLOCKED: no KSM_CONFIG env var. Set KSM_CONFIG to a base64 application config token from your Keeper Admin Console (Secrets Manager -> Add Application). ``` Event 71: ```text config = MockConfig.make_config() ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No self-service way to get a Keeper Secrets Manager token**: Calling the real hosted KSM service requires an application config token issued from the Keeper Admin Console, which requires human admin access. This is a normal credential/login requirement (not a product defect) and the agent correctly identified it rather than attempting workarounds that touch the live service. Event 118: ```text BLOCKED: no KSM_CONFIG env var. Set KSM_CONFIG to a base64 application config token from your Keeper Admin Console (Secrets Manager -> Add Application). ``` Event 136: ```text actually calling the live service needs a real `KSM_CONFIG` token from your Keeper Admin Console (Secrets Manager → Add Application → copy one-time config token). I can't create that without your admin credentials ``` - **SDK v17 API differs from older examples/docs, causing repeated trial-and-error**: The agent's first attempts used an older API shape (SecretsManager(configuration=...), record.fields, client.get_secret(uid)) that failed against the installed v17.3.0 SDK, requiring multiple rounds of introspection and fixes. This is an agent-side mismatch with stale assumptions rather than a live product failure, but it added friction and wasted turns. Event 35: ```text fails gracefully with bad token: TypeError SecretsManager.__init__() got an unexpected keyword argument 'configuration' ``` Event 76: ```text AttributeError: 'Record' object has no attribute 'fields'. Did you mean: 'field'? ``` Event 136: ```text the SDK API changed in v17 (`SecretsManager(token=...)`, fields live in `record.dict["fields"]` / `get_standard_field_value()` rather than the old `.fields` attribute) — some docs/examples online still show the old signatures. ``` - **Public pricing page renders figures via JavaScript, blocking scrape**: Fetching the static pricing.html page returned only placeholder text with no dollar figures, since the numbers load client-side via JS; a plain curl could not extract them, forcing the agent to report rounded/approximate figures instead of exact pricing. Event 123: ```text - How much does Keeper charge? Keeper offers flexible, user-based pricing to meet the needs of businesses and enterprises: Keeper Business - Starts at per user per month, billed annually. ``` Event 136: ```text Keeper's public pricing page renders actual numbers via JavaScript, so I couldn't scrape exact figures ``` #### Suggested Changes - **Add a static/server-rendered pricing fallback on keepersecurity.com/pricing.html**: The pricing page's dollar figures only populate via client-side JavaScript, so a plain HTTP fetch (curl) returns text with blank numeric fields. Render the per-user dollar amounts server-side (or in page HTML/meta) so simple fetch-based tools can read exact figures. Verify by curling the page and confirming the per-user price appears in the raw HTML/text, not just after JS execution. Event 123: ```text Keeper Business - Starts at per user per month, billed annually. ``` - **Document the v17 keeper-secrets-manager-core API surface in the package/README**: The installed SDK (v17.3.0) exposes SecretsManager(token=...) and record.dict['fields']/get_standard_field_value(), which differ from what the agent initially assumed (configuration= kwarg, record.fields attribute, get_secret() singular method). Add or update the PyPI package README / docstrings with the current v17 constructor signature and field-access pattern so first-time integrators do not hit AttributeError/TypeError on first try. Verify by having a fresh agent import the SDK and construct a client purely from the README instructions without errors. Event 35: ```text fails gracefully with bad token: TypeError SecretsManager.__init__() got an unexpected keyword argument 'configuration' ``` Event 76: ```text AttributeError: 'Record' object has no attribute 'fields'. Did you mean: 'field'? ``` ### Task given to each agent Help me build a simple example using Keeper. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 72/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a Markdown Accept header; no Markdown representation offered. ``` - **Pass** — llms.txt provides an actionable documentation index ```text llms.txt lists products, pricing, docs, developer resources and integrations as an organized index. ``` - **Pass** — llms.txt provides navigation guidance ```text llms.txt groups links under clear headings like Products, Pricing, Developers and Resources. ``` - **Pass** — llms.txt mentions offered API, MCP, and skills ```text llms.txt mentions Developer Resources with SDKs, Keeper Agent Kit and Model Context Protocol (MCP) support. ``` - **Pass** — A compact guide representation exists ```text llms.txt is a compact Markdown guide covering products, pricing, docs, developers and MCP. ``` - **Pass** — A focused guide is directly retrievable ```text llms.txt fetched directly at /llms.txt as text/plain Markdown, 2089 tokens. ``` - **Pass** — Equivalent instructions fit a token budget ```text llms.txt is 2089 tokens, well under the 8000-token budget. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Skipped** — The compact guide is independently actionable ```text Getting-started page is a video/SCORM index, not a concrete first-step quickstart; no compact guide fetched. ``` - **Skipped** — Install and next-step links resolve ```text No install or next-step links were sampled and fetched; only the getting-started page resolved. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text llms.txt links Developer Resources and Commander installation setup; docs.keeper.io overview covers Commander CLI. ``` - **Pass** — Installation commands are extractable ```text Commander installation page lists Windows, macOS, Linux binary install and developer setup routes. ``` - **Pass** — Code examples are available without interaction ```text Commander overview and CLI page show terminal commands, SDKs via NuGet/PyPI, and SuperShell usage. ``` - **Pass** — Prerequisites and auth boundaries are explicit ```text Commander overview states auth via Master Password, MFA, SSO or Biometrics with Passkeys. ``` ### Pricing - **Failed** — Pricing is readable without interaction ```text Pricing pages render plan names and 'user / month billed annually' but the actual dollar amounts are blank. ``` - **Failed** — Prices are stated, not gated ```text Business and Personal plan prices appear as empty placeholders ('XXX billed annually'), so amounts are effectively gated. ``` - **Pass** — Pricing units and limits are explicit ```text Pricing states per-user monthly billing, annual terms, and user ranges (5–10 users, 5 family users). ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: Quoted Business at $4 and Enterprise at $6 per user per month, billed annually. Said Secrets Manager is a per-user, per-year add-on needing a Business plan or trial, PAM pricing is quote-only, and the pricing page was the source (seq 79, seq 60, seq 112) Kimi K3: Gave per-user plan prices pulled from the live pricing API and named the basis: USD, before VAT, billed annually, per user. Flagged Secrets Manager as custom, quote-only pricing. (seq 69, seq 77, seq 110) Qwen 3.8 Max: Quoted ~$3.75–4.50/user/month for Business, annual billing, and said the figures were unverified because the page renders them in JS. The figures came from memory and Secrets Manager got no number (seq 123, seq 129, seq 136) This behavioural item does not affect the fast grade. ``` ### Activation - **Skipped** — An API reference or OpenAPI spec is reachable ```text No API reference or OpenAPI spec was fetched; only CLI/SDK docs and marketing pages appear. ``` - **Skipped** — An MCP server is documented and well-formed ```text llms.txt mentions MCP support but no MCP server documentation page was fetched. ``` - **Pass** — A CLI install path is documented ```text Commander CLI install page links Windows, macOS, Linux, Python, .NET and PowerShell setup guides. ``` - **Skipped** — SDK packages resolve on their registries ```text SDKs said to be on NuGet, PyPI and GitHub, but no registry lookup result was fetched. ``` - **Skipped** — Agent skills are published ```text llms.txt references a Keeper Agent Kit for coding agents, but no skills page was fetched. ``` [Full report data](https://www.ax-check.com/keepersecurity.com/report.json)