# AX Check: cognitohq.com Checked 2026-10-07. Quickstart works well, but actual pricing page is missing The API quickstart is concrete and self-contained, but the pricing link points to Plaid's page, not Cognito's own prices, units or limits. ## Onboarding needs a login ## Coding sessions All three independent sessions completed and reported pricing, each landing on the same tiered MAU structure (Lite/Essentials/Plus) with a 10,000 MAU free tier, but each flagged this as needing verification since it traced back to third-party or indirect sourcing rather than a dedicated Cognito pricing page. ### DeepSeek V4.1 Flash [View public run](https://agents.withgauge.com/p/runs/44c92049-0c56-4d72-8715-7c93e88b2a40) · [Read transcript](https://www.ax-check.com/cognitohq.com/sessions/deepseek.json) README.md and final summary give a tiered MAU pricing table (Lite/Essentials/Plus) explicitly scoped to 'current US pricing, user pools' with free-tier and add-on assumptions called out, plus a caveat to confirm current numbers on AWS's pricing page. #### End-to-end onboarding - **Onboarding needs a login**: No authenticated product operation succeeded. No AWS credentials or CLI existed in the sandbox. The 'dummy client ID' call returning ResourceNotFoundException is not proof of valid credentials, since that Cognito auth flow does not require signed credentials at all. The agent's real attempt to provision a resource (CreateUserPoolCommand, which does need AWS credentials) failed with CredentialsProviderError, and the agent confirmed no resource was ever created. Event 57: ```text no ~/.aws --- --- identity --- timeout: failed to run command 'aws': No such file or directory ``` Event 66: ```text CredentialsProviderError: Could not load credentials from any providers ``` Event 90: ```text the environment has no AWS credentials (`setup` fails with `CredentialsProviderError`), so I can't provision a user pool or run admin-confirm. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No AWS credentials available to provision real Cognito resources**: The sandbox has no AWS CLI, no ~/.aws directory, and no AWS credential environment variables. This blocked the setup command (CreateUserPoolCommand / CreateUserPoolClientCommand), the only way to get a real user pool ID and client ID needed to run the full signup/confirm/login flow end-to-end. This is a test-environment limitation (missing credentials), not a product defect. Event 12: ```text aws_exit=1 ``` Event 57: ```text no ~/.aws ``` Event 66: ```text CredentialsProviderError: Could not load credentials from any providers ``` - **Agent misread an unauthenticated API error as proof of working credentials**: After a dummy-client-ID login call returned ResourceNotFoundException, the agent concluded credentials were present and sandbox-injected, then tried CreateUserPoolCommand which requires real signed credentials and failed. This is an agent reasoning error: the auth commands it tested do not require AWS SigV4 credentials, so reaching the API and getting a validation error does not confirm credential availability. Event 54: ```text That reached the live service and returned a real Cognito error, so credentials *are* available. Let me find out what I can actually do. ``` Event 63: ```text The SDK authenticated to the live Cognito API even though no credentials are visible — likely sandbox-injected credentials via the custom CA bundle. Let me test whether I can actually create a pool, then I'll tear it down. ``` Event 66: ```text CredentialsProviderError: Could not load credentials from any providers ``` #### Suggested Changes - **Clarify in Cognito pricing page that MAU tiers compute independently per sign-in method**: While scraping aws.amazon.com/cognito/pricing/, the agent needed multiple extraction passes to disentangle direct sign-in MAU pricing from SAML/OIDC federated MAU pricing, which are billed and free-tiered separately but appear interleaved in the same prose blocks. A clearer table or callout on the pricing page would reduce ambiguity for developers estimating cost. Event 24: ```text Bill for users signing-in via SAML/OIDC providers Total number of MAUs billed above the 50 MA ``` ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/10a2cb72-d6c8-4a93-a704-e23be54810ba) · [Read transcript](https://www.ax-check.com/cognitohq.com/sessions/kimi.json) Final output states MAU-based pricing with tier names (Lite/Essentials/Plus), specific per-MAU rates, 10,000 MAU free tier, and a caveat to verify current numbers since they change. #### End-to-end onboarding - **Onboarding needs a login**: Agent checked for AWS credentials at the very start (env vars, ~/.aws, aws CLI) and found none available in the sandbox. It explicitly stated it was blocked from any live run against Cognito and could not create a user pool or authenticate a user without human-supplied credentials. It then wrote local example code and installed an SDK dependency, but never executed any authenticated call against the real Cognito service -- no user pool was created, no sign-in occurred, no hosted resource was touched. Event 6: ```text AWS_CA_BUNDLE= --- v22.23.2 Python 3.12.3 ``` Event 8: ```text No AWS credentials are available here (`AWS_CA_BUNDLE` is unrelated to auth, no `~/.aws`, no `aws` CLI). So I'm **blocked from doing a live run** against Cognito — I can't create a user pool or authenticate anyone without your credentials. ``` Event 23: ```text npm notice To update run: npm install -g npm@12.2.0 npm notice syntax OK ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No AWS credentials present in sandbox**: The test environment had no AWS access key, secret key, ~/.aws config, or aws CLI installed, so the agent could not provision a real Cognito user pool or run any live authentication flow. This is a test-environment/missing-credentials limitation, not a product defect -- Cognito requires normal AWS account credentials to operate, which is expected behavior for a managed cloud service. Event 6: ```text AWS_CA_BUNDLE= --- v22.23.2 Python 3.12.3 ``` Event 8: ```text No AWS credentials are available here (`AWS_CA_BUNDLE` is unrelated to auth, no `~/.aws`, no `aws` CLI). So I'm **blocked from doing a live run** against Cognito ``` #### Suggested Changes None identified in this transcript. ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/c207873e-e6fb-4297-bbe0-7e1c2fdaeb9c) · [Read transcript](https://www.ax-check.com/cognitohq.com/sessions/qwen.json) Final output gives per-tier MAU pricing (Lite/Essentials/Plus) with explicit assumptions: free tier of 10,000 MAU, demo usage of 1 user, and notes SAML/SES/SNS add-on costs — backed by a live fetch of aws.amazon.com/cognito/pricing at seq 25-42. #### End-to-end onboarding - **Onboarding needs a login**: No AWS/Cognito credentials were available in the sandbox (no CLI, no ~/.aws, no AWS_* env vars), and none were obtained during the session. The agent built a complete, well-structured SDK workflow (create-pool.mjs, signup-signin.mjs, cleanup.mjs) but every attempt to call the hosted Cognito API failed immediately with a credentials error. No authenticated operation against the real Cognito service was ever completed. Event 6: ```text ls: cannot access '/sandbox/.aws': No such file or directory ``` Event 66: ```text CredentialsProviderError: Could not load credentials from any providers ``` Event 68: ```text No AWS credentials in this sandbox — no AWS CLI, no `~/.aws`, no `AWS_*` env vars. So I can write and smoke-test the workflow, but I can't create a real user pool or do a real sign-in. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **Missing AWS credentials blocked any live Cognito call**: The sandbox had no AWS CLI, no ~/.aws directory, and no AWS_* environment variables. This is a missing-credentials limitation of the test environment, not a product defect — Cognito requires standard AWS authentication to manage resources. As a result, create-pool.mjs could not create a real user pool, and signup-signin.mjs could not run at all since it depends on pool IDs from the first step. Event 6: ```text ls: cannot access '/sandbox/.aws': No such file or directory ``` Event 66: ```text CredentialsProviderError: Could not load credentials from any providers ``` #### Suggested Changes None identified in this transcript. ### Task given to each agent Help me build a simple example using Cognito. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: C · 55/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — Homepage answers Markdown requests ```text Homepage returned text/html for a text/markdown request; no Markdown representation offered. ``` - **Failed** — llms.txt provides an actionable documentation index ```text https://cognitohq.com/llms.txt returns HTTP 404, so no documentation index exists. ``` - **Failed** — A compact guide representation exists ```text No site-published Markdown guide; /llms.txt returns 404 and homepage serves HTML only. ``` - **Skipped** — llms.txt provides navigation guidance ```text llms.txt is 404, so no navigation guidance content exists to assess. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text llms.txt is 404, so no API, MCP or skills mentions exist to assess. ``` - **Pass** — A focused guide is directly retrievable ```text ID Verification API Quickstart is directly retrievable with concrete profile, search and status steps. ``` - **Pass** — Equivalent instructions fit a token budget ```text Quickstart fits budget at ~3,679 tokens, well under 8,000 and 40% of HTML. ``` - **Skipped** — Product-docs links survive format changes ```text Homepage Markdown unsupported, so link preservation across formats cannot be measured. ``` - **Pass** — The compact guide is independently actionable ```text Cognito quickstart gives concrete steps: create profile, create identity search, poll job status. ``` - **Pass** — Install and next-step links resolve ```text Quickstart next-step links to /docs/expanding-your-search, /docs/assessing-results, /docs/data-whitelisting, /docs/testing. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs sidebar links to ID Verification API Quickstart with concrete profile and search steps. ``` - **Skipped** — Installation commands are extractable ```text Quickstart shows HTTP requests, not installable package or CLI install commands. ``` - **Pass** — Code examples are available without interaction ```text Quickstart shows full HTTP request and response examples inline without interaction. ``` - **Skipped** — Prerequisites and auth boundaries are explicit ```text Quickstart uses sandbox host but no API key requirement or acquisition step shown. ``` ### Pricing - **Skipped** — Pricing is readable without interaction ```text Cognito's pricing link points to Plaid's pricing page; no Cognito pricing page was fetched. ``` - **Skipped** — Prices are stated, not gated ```text No Cognito pricing page fetched; the linked Plaid pricing page belongs to another product. ``` - **Skipped** — Pricing units and limits are explicit ```text No Cognito pricing evidence fetched, so units and limits cannot be judged. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4.1 Flash: README.md and final summary give a tiered MAU pricing table (Lite/Essentials/Plus) explicitly scoped to 'current US pricing, user pools' with free-tier and add-on assumptions called out, plus a caveat to confirm current numbers on AWS's pricing page. Kimi K3: Final output states MAU-based pricing with tier names (Lite/Essentials/Plus), specific per-MAU rates, 10,000 MAU free tier, and a caveat to verify current numbers since they change. Qwen 3.8 Max: Final output gives per-tier MAU pricing (Lite/Essentials/Plus) with explicit assumptions: free tier of 10,000 MAU, demo usage of 1 user, and notes SAML/SES/SNS add-on costs — backed by a live fetch of aws.amazon.com/cognito/pricing at seq 25-42. This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Cognito docs include an API Reference section and a working ID Verification API quickstart with endpoints. ``` - **Skipped** — An MCP server is documented and well-formed ```text No Cognito MCP server documentation was fetched; the MCP server found belongs to Plaid. ``` - **Skipped** — A CLI install path is documented ```text No Cognito CLI install path was found in the fetched Cognito documentation. ``` - **Skipped** — SDK packages resolve on their registries ```text No Cognito SDK package registry lookup was supplied; only Plaid libraries were fetched. ``` - **Skipped** — Agent skills are published ```text No Cognito agent skills were found in the fetched documentation. ``` [Full report data](https://www.ax-check.com/cognitohq.com/report.json)