# AX Check: folkweather.com
Checked 2026-09-19.
Agents can hit the weather API, but docs are locked behind a login
The API reference works and links resolve, but the documentation index returns an authorization error, leaving quickstart and setup details unreachable.
## Onboarded with real credentials
## Coding sessions
All three independent sessions concluded FolkWeather is free, basing this on the homepage tagline, working unauthenticated endpoints, and a blocked pricing page rather than any stated price list.
### DeepSeek V4 Pro
[View public run](https://agents.withgauge.com/p/runs/05f23b4d-9514-4fb4-98f0-713e70b379d9) · [Read transcript](https://www.ax-check.com/folkweather.com/sessions/deepseek.json)
Final output states pricing is free/open and grounds that in evidence: unauthenticated successful requests, site tagline, and the 401 on /pricing meaning no visible paid tier — a stated basis, not a bare number.
#### End-to-end onboarding
- **Onboarded with real credentials**: No credentials of any kind were needed. The agent discovered FolkWeather's public, unauthenticated OGC EDR API, wrote a Python stdlib script hitting the hosted https://folkweather.com/edr endpoints directly, hit one 403 (default urllib user-agent blocked), fixed it by setting a custom User-Agent header, and then successfully executed three real authenticated-free operations against the live hosted service: listing 90 collections, a GFS temperature forecast query, a live METAR observation at KDEN, and a live NDBC buoy reading — all returning real, current data (e.g. timestamps of 2026-09-19).
Event 156:
```text
FolkWeather collections available: 90
GFS 2m temperature forecast (Denver, CO):
39.740, -104.990 -> 16.3 C (289.45 K)
Latest METAR observation (KDEN):
Denver Intl, CO, US @ 2026-09-19T12:53:00+00:00
```
Event 147:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
Event 152:
```text
"User-Agent": "folkweather-example/1.0 (simple demo)",
```
#### Hallucinated URLs
None identified in this transcript.
#### Blockers
- **Default HTTP client user-agent gets a 403 from the hosted API**: Python's urllib default request (no custom User-Agent) was rejected with HTTP 403 Forbidden on the /edr/collections endpoint. This is product/server-side behavior (likely a WAF or bot-filtering rule on folkweather.com), not a credentials or agent-logic issue. It was fully resolved in one edit by adding a User-Agent header, after which all subsequent calls succeeded.
Event 147:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
Event 156:
```text
FolkWeather collections available: 90
```
- **OpenAPI spec advertises a stale localhost server URL**: The published OpenAPI definition at /edr/api lists its servers block as a local dev address rather than the public hostname, forcing the agent to infer the real base URL itself instead of copying it from docs. This is a product documentation issue, not an environment or credential limitation.
Event 109:
```text
servers:
- url: http://localhost:8083/edr
description: Local development server
```
- **/pricing and /docs pages are access-gated**: Requests to https://folkweather.com/pricing and /docs both returned nginx 401 Authorization Required, blocking any confirmation of a paid tier or documentation site. This appears to be an intentional admin/auth gate (consistent with robots.txt disallowing /admin) rather than a missing developer credential, since all data-serving endpoints worked without any authentication.
Event 95:
```text
=== /pricing body ===
401 Authorization Required
```
Event 83:
```text
=== /docs ===
401 text/html
=== /pricing ===
401 text/html
```
#### Suggested Changes
- **Fix the OpenAPI servers field to point at the production host**: Update the servers entry in the OpenAPI document served at https://folkweather.com/edr/api so it lists https://folkweather.com/edr instead of http://localhost:8083/edr. Verify by fetching /edr/api and confirming the servers.url matches the public domain, and that OpenAPI-generated client tooling (e.g. Swagger/Redoc codegen) targets the live host by default.
Event 109:
```text
servers:
- url: http://localhost:8083/edr
description: Local development server
```
- **Document the User-Agent requirement or relax the 403 filter for plain HTTP clients**: Either note in the quickstart/API docs that requests without a recognizable User-Agent header are rejected with 403, or loosen the filter so default client libraries (like Python's urllib) work out of the box. Check by making a GET to /edr/collections with no custom headers and confirming it no longer 403s, or that docs clearly instruct adding a User-Agent.
Event 147:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
- **Make a public pricing/docs page reachable without login**: The /pricing and /docs paths both return 401 Authorization Required, so prospective developers cannot see pricing terms or narrative documentation without hitting an auth wall. Publish a public-facing pricing/docs page (or link one from the homepage) and verify by curling /pricing unauthenticated and confirming a 200 response with actual content.
Event 95:
```text
=== /pricing body ===
401 Authorization Required
```
### Kimi K3
[View public run](https://agents.withgauge.com/p/runs/9051fae8-a911-4dcc-89ed-9c315d849f92) · [Read transcript](https://www.ax-check.com/folkweather.com/sessions/kimi.json)
Final output states pricing is 'Free' and names its basis: homepage text 'Free, open weather data services,' every tested endpoint working with no API key/signup/auth, and no published rate limits found.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: FolkWeather's tested endpoints (EDR data queries, WMS capabilities) required no API key or signup at all — the agent called them directly over HTTPS and got real data back. The only authenticated area found (/pricing, /docs, /developers, /signup) returned 401 with a Basic-Auth admin realm, which the agent correctly identified as an internal admin wall, not a customer signup flow, and did not attempt to obtain credentials for. Since no credential acquisition was needed or attempted, and the working operations were unauthenticated, this does not meet the bar for verified self-service onboarding.
Event 29:
```text
== pricing: 401
== docs: 401
== api: 301
== developers: 401
== signup: 401
```
Event 74:
```text
NDBC buoy 46042 (Monterey Bay) — latest observation
time: 2026-09-19T13:10:00+00:00
air temp: 14.8 °C
water temp: 16.0 °C
```
Event 44:
```text
www-authenticate: Basic realm="Weather WMS Admin"
```
#### Hallucinated URLs
None identified in this transcript.
#### Blockers
- **Cloudflare blocks default Python User-Agent**: A plain urllib request to the EDR endpoint returned 403 Forbidden because Cloudflare rejects the default library User-Agent string. This is test-environment/tooling friction rather than a product defect — the agent resolved it in one edit by setting an explicit User-Agent header, and the request succeeded afterward.
Event 65:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
Event 74:
```text
NDBC buoy 46042 (Monterey Bay) — latest observation
time: 2026-09-19T13:10:00+00:00
```
- **Unencoded space in query string broke the HTTP client**: Agent error: the coordinate query parameter POINT(-122.4 37.8) contained a literal space, which Python's http.client rejects as a control character. Fixed immediately by URL-encoding the space as %20; not a product-side issue since the working WMS/EDR example links on the homepage already used %20 encoding.
Event 74:
```text
http.client.InvalidURL: URL can't contain control characters. '/edr/collections/hrrr-surface/position?coords=POINT(-122.4 37.8)¶meter-name=GUST,VIS' (found at least ' ')
```
- **Wave forecast collection returned no data**: The gfswave-surface-latest collection referenced in the homepage's own example link returned a 404 'no available data' error, likely a stale model run on the product side. The agent worked around it by switching to the buoy and HRRR collections for the demo, so this did not block the overall task.
Event 45:
```text
{"type":"http://www.opengis.net/def/exceptions/ogcapi-edr-1/1.0/not-found","title":"Not Found","status":404,"detail":"Collection gfswave-surface-latest has no available data"}
```
#### Suggested Changes
- **Document the Cloudflare User-Agent requirement in the API docs**: Add a note to /edr/api.html (or the homepage quickstart) stating that requests need a non-default User-Agent header, since default urllib/curl agents can get 403'd by Cloudflare. Verify by making a fresh urllib.request.urlopen call with no custom headers against an EDR endpoint and confirming it no longer 403s, or confirming the doc note is visible before that step trips anyone up.
Event 65:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
- **Fix or remove the stale gfswave-surface-latest example link**: The homepage lists a live example query against /edr/collections/gfswave-surface-latest/position that currently 404s with 'no available data'. Either refresh the underlying wave model ingestion or swap the homepage example to a collection known to have current data, then confirm by re-running the same GET request shown on the homepage.
Event 45:
```text
"detail":"Collection gfswave-surface-latest has no available data"
```
### Qwen 3.8 Max
[View public run](https://agents.withgauge.com/p/runs/357faf6b-f771-4785-a723-c42a3739ba5b) · [Read transcript](https://www.ax-check.com/folkweather.com/sessions/qwen.json)
Final output states pricing is Free with stated basis: site's own tagline, absence of a pricing page (401 at /pricing, seq 38), no keys/account needed for the tested endpoints, plus a caveat that it's a no-SLA community service.
#### End-to-end onboarding
- **End-to-end onboarding not demonstrated**: FolkWeather's EDR/WMS endpoints are fully open (no API key, no signup, no auth header) as confirmed by the site itself and the 401 only appearing on a nonexistent /pricing path. The agent successfully queried live hosted data (GFS forecast, NDBC buoy, WMS map render) using plain HTTP, but this is not an onboarding flow in the credentialed sense — there was no account creation, API key issuance, or authenticated request. Since the product requires no credentials at all, verified/login_required do not apply; the correct outcome is not_verified.
Event 86:
```text
GUST = 8.58 m/s
APCP = 0.00 kg/m^2
VIS = 24,134.88 m
2) Latest NDBC buoy observation (station 44013)
time : 2026-09-19T13:20:00+00:00
```
Event 38:
```text
=== pricing ===
401
```
#### Hallucinated URLs
- **Guessed WMS layer name before checking capabilities**: The agent initially used a made-up layer name 'gfs-t2m' for the WMS GetMap request without first checking GetCapabilities. This did not cause an HTTP failure by itself (the code raised no error and the request path was corrected before being run), so it does not rise to a demonstrated failed hallucinated URL — the agent caught and fixed it proactively via GetCapabilities before ever executing the request with that layer name.
Event 55:
```text
"LAYERS": "gfs-t2m", # GFS 2m temperature
```
#### Blockers
- **Default urllib User-Agent blocked with 403**: The Python script's first run failed because the server rejected the default Python-urllib User-Agent header with a 403 Forbidden. This is product/server behavior (edge or WAF-level UA filtering) rather than a credential or agent-logic issue. The agent diagnosed it correctly and resolved it by sending a custom User-Agent, so the run succeeded afterward.
Event 77:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
Event 79:
```text
The 403 is likely due to the default `Python-urllib` user agent being blocked. Adding a User-Agent header:
```
- **Unescaped space in EDR coordinate query caused invalid URL**: The agent's first attempt to build the EDR position query embedded a literal space in the POINT(lon lat) parameter, which Python's http.client rejected as a control character before any request reached the server. This was an agent coding error (not URL-encoding a query parameter), quickly fixed by URL-encoding the coordinate string.
Event 69:
```text
http.client.InvalidURL: URL can't contain control characters. '/edr/collections/gfs-surface-latest/position?coords=POINT(-73.9857 40.7484)&f=GeoJSON' (found at least ' ')
```
#### Suggested Changes
- **Document the required User-Agent header in EDR API docs**: The hosted EDR endpoint returns HTTP 403 for requests using default HTTP client user agents (e.g., Python's urllib), which cost the agent an extra debug-and-fix cycle. Add a note to the API documentation page (folkweather.com/edr/api.html) stating that a non-default User-Agent header is required, and verify by re-running a plain urllib request without a custom User-Agent to confirm it now returns 200 or that the documented requirement matches actual behavior.
Event 77:
```text
urllib.error.HTTPError: HTTP Error 403: Forbidden
```
- **Add a working coordinate query example to the EDR quickstart**: The EDR collections/API discovery pages do not show a copy-pasteable example of a POINT coordinate query with correct URL encoding, leading the agent to first try an unencoded space that immediately failed client-side. Add a full example request (e.g., 'GET /edr/collections/gfs-surface-latest/position?coords=POINT(-73.9857%2040.7484)&f=GeoJSON') to the collections or API documentation and confirm a fresh copy-paste of that example succeeds without modification.
Event 69:
```text
http.client.InvalidURL: URL can't contain control characters. '/edr/collections/gfs-surface-latest/position?coords=POINT(-73.9857 40.7484)&f=GeoJSON' (found at least ' ')
```
### Task given to each agent
Help me build a simple example using FolkWeather. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.
No product credentials were supplied and no purchases were authorized.
## Score: D · 29/100 (provisional)
Grades come from completed site checks. Coding sessions and skipped checks do not affect the score.
### Clarity
- **Failed** — Homepage answers Markdown requests
```text
Homepage returned text/html for a Markdown Accept header, so no Markdown representation is served.
```
- **Failed** — llms.txt provides an actionable documentation index
```text
/llms.txt returned HTTP 401 Authorization Required, so no documentation index is available.
```
- **Skipped** — llms.txt provides navigation guidance
```text
/llms.txt returned 401 with no body, so navigation guidance cannot be evaluated.
```
- **Skipped** — llms.txt mentions offered API, MCP, and skills
```text
/llms.txt returned 401 with no body, so API/MCP/skills mentions cannot be evaluated.
```
- **Skipped** — A compact guide representation exists
```text
No compact guide or agent-specific Markdown representation was fetched for FolkWeather.
```
- **Skipped** — A focused guide is directly retrievable
```text
No focused guide page was fetched; only homepage and API docs were retrieved.
```
- **Skipped** — Equivalent instructions fit a token budget
```text
No compact guide was fetched, so token budget cannot be measured.
```
- **Skipped** — Product-docs links survive format changes
```text
Homepage Markdown is unsupported, so link preservation across formats is unassessed.
```
- **Skipped** — The compact guide is independently actionable
```text
No compact agent guide or standalone Markdown page was fetched; only the HTML homepage and API docs.
```
- **Pass** — Install and next-step links resolve
```text
Homepage install/next-step links (API docs, EDR collections, WMS/WMTS capabilities) fetched successfully.
```
### Onboarding
- **Skipped** — Docs lead to a relevant quickstart
```text
Only fetched document is a 401 on /llms.txt; no docs or quickstart content available.
```
- **Skipped** — Installation commands are extractable
```text
No installation or CLI documentation was fetched; only a 401 response.
```
- **Skipped** — Code examples are available without interaction
```text
No code examples were fetched; only a 401 response from /llms.txt.
```
- **Skipped** — Prerequisites and auth boundaries are explicit
```text
No docs fetched to establish prerequisites or auth boundaries.
```
### Pricing
- **Skipped** — Pricing is readable without interaction
```text
Not applicable: the relevant product is free. FolkWeather is free; no pricing page or paid plans were fetched.
```
- **Skipped** — Prices are stated, not gated
```text
Not applicable: the relevant product is free. No pricing page fetched; homepage states free, open weather data services.
```
- **Skipped** — Pricing units and limits are explicit
```text
Not applicable: the relevant product is free. No pricing units or limits exist; product is free with no paid tiers.
```
- **Skipped** — Agents identify pricing and its assumptions
```text
Not applicable: the relevant product is free. FolkWeather is free; no pricing page or paid plans were fetched.
```
### Activation
- **Pass** — An API reference or OpenAPI spec is reachable
```text
Homepage 'API docs' link resolves to a Weather WMS EDR API documentation page.
```
- **Skipped** — An MCP server is documented and well-formed
```text
No MCP server documentation was fetched or offered in the evidence.
```
- **Skipped** — A CLI install path is documented
```text
No CLI install path was documented in the fetched pages.
```
- **Skipped** — SDK packages resolve on their registries
```text
No SDK package registry lookup was supplied for FolkWeather.
```
- **Skipped** — Agent skills are published
```text
No agent skills were published or referenced in the fetched evidence.
```
[Full report data](https://www.ax-check.com/folkweather.com/report.json)