# AX Check: gitlab.com Checked 2026-09-19. GitLab's pricing and quickstart docs are clear and machine-readable. Pricing tiers, units, and limits are all stated openly with no login wall. The docs index (llms.txt) is blocked by a bot challenge, though. ## Onboarding needs a login ## Coding sessions All three independent agent sessions completed successfully and found GitLab's pricing page directly, reporting matching Free/$0, Premium/$29 per user per month, and Ultimate tier details pulled live from the public pricing page, with billing cadence and compute-minute assumptions clearly stated. ### DeepSeek V4 Pro [View public run](https://agents.withgauge.com/p/runs/6eac498e-e870-4c42-9b00-ceeba5d05d3f) · [Read transcript](https://www.ax-check.com/gitlab.com/sessions/deepseek.json) Final output lists Free/Premium ($29/user/month billed annually)/Ultimate pricing pulled live from about.gitlab.com/pricing (seq 40-46), with named assumptions (per-user, annual billing, compute-minute tiers, contact-sales for Ultimate). #### End-to-end onboarding - **Onboarding needs a login**: The agent never obtained real GitLab credentials. All API calls used no token and hit gitlab.com's public unauthenticated endpoints, returning 401 for authenticated endpoints (version, user, metadata) and 200 only for genuinely public read-only data (public project listings, gitlab-org/gitlab metadata, commits, README). The agent explicitly wrote a second script (gitlab_full_demo.sh) requiring a human-supplied Personal Access Token to create a project, commit a CI file, and check a pipeline, but never had a token to run it, so the authenticated operation was never executed in this session. Event 8: ```text 401 --- {"message":"401 Unauthorized"} ``` Event 63: ```text === 2. Project metadata: gitlab-org/gitlab === name=GitLab | stars=6139 | forks=12427 | default_branch=master ``` Event 75: ```text The environment has **no GitLab token, SSH key, or stored credentials**, so I can do **read-only public API calls now** but cannot create projects, push commits, or trigger CI myself — those all require authentication. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No GitLab Personal Access Token available in sandbox**: The sandbox had no GitLab token, SSH key, or netrc credentials, so every authenticated GitLab API call (version, user, metadata) returned 401 Unauthorized. This is a missing-credentials limitation of the test environment, not a GitLab product defect — the agent correctly identified it needs a human-supplied Personal Access Token to create projects, commit files, or trigger pipelines, and stopped short of claiming false success. Event 8: ```text 401 --- {"message":"401 Unauthorized"} ``` Event 26: ```text == who am I (public) == {"message":"401 Unauthorized"} == metadata/version == {"message":"401 Unauthorized"} ``` Event 75: ```text I'm blocked only by *missing credentials in this sandbox*. No local service stack is needed; this is purely an "I need a token" gap, not "you have to run servers." ``` #### Suggested Changes None identified in this transcript. ### Kimi K3 [View public run](https://agents.withgauge.com/p/runs/40dc4349-18f8-40c5-8f11-98277391cc4e) · [Read transcript](https://www.ax-check.com/gitlab.com/sessions/kimi.json) Pricing table states Free/Premium ($29)/Ultimate ($99) tiers explicitly framed as 'per user, per month, billed annually' for hosted SaaS, plus named CI-minute and Duo add-on assumptions. #### End-to-end onboarding - **Onboarding needs a login**: Agent checked the local environment for a GitLab token or CLI credentials, found none, and explicitly stated it could not sign up for GitLab.com autonomously due to interactive email verification. No API calls were ever made against the hosted product; the entire example was left as unexecuted sample code pending a human-supplied Personal Access Token. Event 4: ```text env | grep -iE 'gitlab|glab' | sed 's/=.*/=/'; which glab; ls ~/.config/glab-cli 2>/dev/null; echo "---done---" ``` Event 7: ```text There's no GitLab token, `glab` CLI, or stored credentials in this environment, and I can't sign up for GitLab.com autonomously (interactive email verification). So I can't make real API calls. ``` Event 7: ```text Give me a `GITLAB_TOKEN` (Personal Access Token with `api` scope) and I can run the example above against your GitLab.com account right now. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No GitLab credentials available in the sandbox**: The environment had no GitLab personal access token, glab CLI, or stored credentials, and GitLab.com signup requires interactive email verification that the agent cannot complete on its own. This is a missing-credentials/session limitation, not a product defect, and it halted all real execution — the agent produced only an unexecuted code sample instead of a live run. Event 4: ```text env | grep -iE 'gitlab|glab' | sed 's/=.*/=/'; which glab; ls ~/.config/glab-cli 2>/dev/null; echo "---done---" ``` Event 7: ```text There's no GitLab token, `glab` CLI, or stored credentials in this environment, and I can't sign up for GitLab.com autonomously (interactive email verification). So I can't make real API calls. ``` #### Suggested Changes - **Offer a self-service token creation path that avoids email verification**: Since the agent's only path forward was a human-supplied Personal Access Token due to GitLab.com's interactive email verification during signup, consider documenting or exposing an automatable account/token provisioning flow so agent-driven onboarding tests can obtain credentials without human intervention. Check this by re-running the same prompt and confirming the agent can generate a working GITLAB_TOKEN and complete an authenticated call (e.g., gl.auth() or project creation) without a human providing a token first. Event 7: ```text There's no GitLab token, `glab` CLI, or stored credentials in this environment, and I can't sign up for GitLab.com autonomously (interactive email verification). So I can't make real API calls. ``` ### Qwen 3.8 Max [View public run](https://agents.withgauge.com/p/runs/81bcac74-ad8b-4c49-917a-dfe61e76244e) · [Read transcript](https://www.ax-check.com/gitlab.com/sessions/qwen.json) Final output gives Free/$0, Premium/$29 per user/month billed annually, Ultimate custom pricing, naming plan tier and billing cadence as the basis, sourced from a live curl of about.gitlab.com/pricing/ (seq 16-35). #### End-to-end onboarding - **Onboarding needs a login**: Agent searched the sandbox for GitLab credentials and found none. It confirmed the public GitLab.com API works unauthenticated (project reads, README fetch), but explicitly could not obtain a GITLAB_TOKEN on its own, so the authenticated write path (create project, commit, open MR) was written in code but never executed against the hosted product. Event 4: ```text env | grep -iE "gitlab|glab|gl_|token|ci_" | sed 's/=.\{6,\}/=/'; echo "--- which"; which glab git python3 pip3 node npm 2>/dev/null ``` Event 47: ```text == Public demo against https://gitlab.com == Project: gitlab-org/gitlab-foss (id=13083) ``` Event 70: ```text Partially, one thing: **no `GITLAB_TOKEN` exists in this environment**, so I verified only the public read-only path. The write workflow (create project → commit → MR) is implemented and ready, but I can't run it without a token. ``` #### Hallucinated URLs None identified in this transcript. #### Blockers - **No GitLab access token available for write operations**: The sandbox environment had no GITLAB_TOKEN or equivalent credential. This is a test-environment/credential limitation, not a product defect — GitLab reasonably requires a personal access token to create projects, commit files, or open merge requests. The agent correctly scoped its demo to public-mode reads and left the authenticated write path implemented but unexecuted. Event 6: ```text Command exited with code 1 ``` Event 70: ```text no `GITLAB_TOKEN` exists in this environment ``` - **python-gitlab SDK attribute mismatch on Project object**: Agent error: the script called project.open_issues_count, which does not exist on the python-gitlab Project object, causing a runtime AttributeError. This was self-inflicted (incorrect assumption about the SDK's API surface) and was fixed within the same turn by removing the bad attribute reference. Event 47: ```text AttributeError: 'Project' object has no attribute 'open_issues_count' ``` #### Suggested Changes - **Document that the /version endpoint requires authentication on gitlab.com**: The agent expected the public /api/v4/version endpoint to work anonymously (a common assumption per GitLab's own docs) but got a 401. Clarify in the API reference/quickstart that this endpoint needs auth even for the hosted SaaS instance, so SDK users don't have to discover this via a failed unauthenticated call. Verify by re-running the same unauthenticated GET against /api/v4/version and confirming the docs reflect the actual response code. Event 12: ```text 401 ``` ### Task given to each agent Help me build a simple example using GitLab. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop. No product credentials were supplied and no purchases were authorized. ## Score: B · 83/100 (provisional) Grades come from completed site checks. Coding sessions and skipped checks do not affect the score. ### Clarity - **Failed** — llms.txt provides an actionable documentation index ```text /llms.txt returned 403 (Cloudflare challenge), so no actionable documentation index was served. ``` - **Pass** — Homepage answers Markdown requests ```text Homepage returned text/markdown with 200 for the Markdown Accept header. ``` - **Skipped** — llms.txt provides navigation guidance ```text No llms.txt body was retrieved, so navigation guidance cannot be judged. ``` - **Skipped** — llms.txt mentions offered API, MCP, and skills ```text No llms.txt body was retrieved, so API/MCP/skills mentions cannot be judged. ``` - **Pass** — A compact guide representation exists ```text Homepage served text/markdown on request; get-started page also returned a Markdown representation. ``` - **Pass** — A focused guide is directly retrievable ```text Get-started page fetched directly as Markdown with concrete guides and docs links. ``` - **Pass** — Equivalent instructions fit a token budget ```text Markdown get-started guide measured 1596 tokens, well under the 8000-token budget. ``` - **Pass** — Product-docs links survive format changes ```text Markdown get-started retains docs.gitlab.com links for CI/CD, security, migration and install. ``` - **Pass** — The compact guide is independently actionable ```text Get-started page gives concrete first steps: free trial, CI/CD, security, migration guides. ``` - **Pass** — Install and next-step links resolve ```text Fetched get-started, docs, and API pages all returned HTTP 200. ``` ### Onboarding - **Pass** — Docs lead to a relevant quickstart ```text Docs 'Get started' page links to Git, CI/CD, Runner and extending quickstarts. ``` - **Skipped** — Installation commands are extractable ```text No install page fetched; get-started links only, no extractable install commands. ``` - **Skipped** — Code examples are available without interaction ```text Fetched get-started pages are link indexes; no code examples shown without interaction. ``` - **Skipped** — Prerequisites and auth boundaries are explicit ```text No quickstart with prerequisites or auth boundaries was fetched in this sample. ``` ### Pricing - **Pass** — Pricing is readable without interaction ```text Pricing page renders plan tiers and prices directly in fetched Markdown, no interaction needed. ``` - **Pass** — Prices are stated, not gated ```text Free $0, Premium $29/user/month, credits $1, compute $10/1000 min all stated openly. ``` - **Pass** — Pricing units and limits are explicit ```text Units explicit: per user/month, 400/10,000/50,000 compute minutes, 10/500 GiB storage. ``` - **Pass** — Agents identify pricing and its assumptions ```text 3 of 3 sessions were judged on pricing; 0 fell short. DeepSeek V4 Pro: Final output lists Free/Premium ($29/user/month billed annually)/Ultimate pricing pulled live from about.gitlab.com/pricing (seq 40-46), with named assumptions (per-user, annual billing, compute-minute tiers, contact-sales for Ultimate). Kimi K3: Pricing table states Free/Premium ($29)/Ultimate ($99) tiers explicitly framed as 'per user, per month, billed annually' for hosted SaaS, plus named CI-minute and Duo add-on assumptions. Qwen 3.8 Max: Final output gives Free/$0, Premium/$29 per user/month billed annually, Ultimate custom pricing, naming plan tier and billing cadence as the basis, sourced from a live curl of about.gitlab.com/pricing/ (seq 16-35). This behavioural item does not affect the fast grade. ``` ### Activation - **Pass** — An API reference or OpenAPI spec is reachable ```text Docs 'Extend with GitLab' page links REST API and GraphQL API references. ``` - **Pass** — An MCP server is documented and well-formed ```text MCP page documents GitLab MCP server and clients, with tier and status. ``` - **Pass** — A CLI install path is documented ```text Extending guide links GitLab CLI (glab) installation instructions. ``` - **Skipped** — SDK packages resolve on their registries ```text No registry lookup for a GitLab SDK or CLI package was supplied. ``` - **Skipped** — Agent skills are published ```text No agent skills documentation was fetched or offered in evidence. ``` [Full report data](https://www.ax-check.com/gitlab.com/report.json)