Skip to content
Gauge

ax-check.com

Langfuse

langfuse.com

ax-check.com

Checked 18 September 2026

A

100/100

Agents need your login

Langfuse's docs let an agent get tracing running fast.

All 23 checklist items passed: quickstart, install commands, and code samples are all readable and runnable without login. Pricing tiers are stated openly (Hobby free to Enterprise $2,499).

1 fix to start with

Concrete changes, backed by what we found.

Onboarding

Give agents a clear login handoff and a way to resume setup.

Agents cannot progress through onboarding without human help. Provide an agent-accessible path through authentication and setup.

Recorded coding sessions

Same task, 3 models, no langfuse.com account. Recorded 18 September 2026 on Pi.

3 recorded · 2m 10s

2m 10s · recorded

Session findings

Select a finding to see it in context.

piv0.84.2
modelKimi K3cwd/sandbox/repo
Help me build a simple example using Langfuse. Tell me how pricing works, and briefly tell me whether this product will be easy for you to manage. Let me know if you get blocked. If this product has no developer workflow you can act on, say so plainly and stop. Stay light: use the hosted product through its SDK or API. Do not start local service stacks or wait for long-running commands; if the quickstart requires either, say so plainly and stop.
Thinking…

The user wants help building a simple example using Langfuse, wants pricing info, and whether it'll be easy for me (the agent) to manage. Stay light: hosted product via SDK/API, no local stacks.

Let me check the environment, see what's available. Langfuse is an LLM observability platform. It has a Python/JS SDK. There's a cloud-hosted version (cloud.langfuse.com) with free tier. Pricing: Hobby free, Core ~$59/mo, Pro ~$199/mo, Enterprise custom (as of my knowledge).

I should check if there are API keys available in env. Likely not — without keys I can't actually send traces to Langfuse Cloud. But I can still build the example code. Let me check the repo and env.

Let me look around first.

$ ls -la /sandbox/repo && env | grep -i -E "langfuse|openai|anthropic" | sed 's/=.*/=<set>/'{"content":[{"text":"total 16\ndrwxr-xr-x 3 root root 4096 Sep 18 08:00 .\ndrwxr-xr-x 7 root root 4096 Sep 18 21:10 ..\ndrwxr-xr-x 8 root root 4096 Sep 18 08:00…ctrl+o
{"content":[{"text":"total 16\ndrwxr-xr-x 3 root root 4096 Sep 18 08:00 .\ndrwxr-xr-x 7 root root 4096 Sep 18 21:10 ..\ndrwxr-xr-x 8 root root 4096 Sep 18 08:00 .git\n-rw-r--r-- 1 root root  408 Sep 18 08:00 .gitignore\n","type":"text"}]}
$ python3 --version; pip show langfuse 2>/dev/null | head -3; node --version 2>/dev/null{"content":[{"text":"Python 3.12.3\nv22.23.2\n","type":"text"}]}
Thinking…

Empty repo, no Langfuse keys set, langfuse not installed. I can install the SDK and write the example. Without API keys, I can't actually send data to Langfuse Cloud — the user would need to sign up and provide keys. Let me check network access quickly, then install langfuse and write the example.

$ pip install -q langfuse 2>&1 | tail -2; python3 -c "import langfuse; print('langfuse', langfuse.__version__)"{"content":[{"text":"langfuse 4.15.4\n","type":"text"}]}
Thinking…

SDK installed. Now write a simple example. Langfuse Python SDK v3/v4 style:

from langfuse import Langfuse, observe

v3+ uses OpenTelemetry under the hood. Simple example:

from langfuse import get_client, observe

langfuse = get_client()  # reads LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, LANGFUSE_HOST

@observe()
def greet(name):
    return f"Hello, {name}!"

greet("world")
langfuse.flush()

Also a manual trace example. And optionally an LLM call via the OpenAI drop-in wrapper from langfuse.openai import openai — but that needs an OpenAI key. Keep it simple: the example works standalone (just a function trace), and optionally shows an LLM generation if OPENAI_API_KEY is set.

Write example file + brief README. Then verify the code at least parses / runs in a degraded way (it will warn about missing keys but shouldn't crash? Actually get_client() with no keys raises or warns — in v3+ it prints a warning about missing credentials and traces become no-ops... Let me check. In langfuse v3, if keys are missing it logs a warning "Langfuse client is disabled" or raises AuthenticationError on flush? I think it warns and disables. Let me just test quickly with a short timeout to avoid network hangs. Set LANGFUSE_HOST to something and see. Careful about network — flush may try to connect to cloud.langfuse.com; the sandbox may or may not have network. pip worked so network exists.

I'll write the example with graceful handling: reads env keys, if not set prints setup instructions. Then run it without keys to verify it doesn't crash.

SDK installed (langfuse 4.15.4). Now let me write a minimal example and verify it runs.

write /sandbox/repo/langfuse_example.py{"content":[{"text":"Successfully wrote 1962 bytes to /sandbox/repo/langfuse_example.py","type":"text"}]}
$ cd /sandbox/repo && python3 langfuse_example.py; echo "exit: $?"{"content":[{"text":"Missing Langfuse keys. Create a free project at https://cloud.langfuse.com,\nthen:\n export LANGFUSE_PUBLIC_KEY=\"pk-lf-...\"\n export LA…ctrl+o
{"content":[{"text":"Missing Langfuse keys. Create a free project at https://cloud.langfuse.com,\nthen:\n  export LANGFUSE_PUBLIC_KEY=\"pk-lf-...\"\n  export LANGFUSE_SECRET_KEY=\"sk-lf-...\"\n  export LANGFUSE_HOST=\"https://cloud.langfuse.com\"\n\nexit: 1\n","type":"text"}]}
Thinking…

The example runs and degrades gracefully without keys. There's no way for me to create a Langfuse Cloud account myself (requires signup/login with email or OAuth), so I'm blocked at actually sending a trace end-to-end. The user said "Let me know if you get blocked" — I should state this plainly.

The code path itself: I verified the SDK imports and the script behaves correctly. The @observe decorators and get_client() API are correct for langfuse v3+/v4 (OTel-based). Let me double check the v4 API quickly — get_client, observe, update_current_trace are all valid in v3/v4. Yes.

Also add a tiny README? Keep light. Maybe a short README is nice but user said stay light. I'll skip a README and just explain inline. Actually a minimal README with setup + pricing pointer could help, but keep response concise — I'll mention pricing in chat.

Pricing (as of my knowledge):

  • Hobby: free, 50k observations/month (formerly 50k units), 30-day... actually Hobby: free, 50k observations/mo, 1 user... Let me recall: Langfuse Cloud pricing: Hobby $0 (50k observations, 30 days data access, 1 project?), Core $59/mo (100k observations included then graduated, 90 days... ), Pro $199/mo, Enterprise custom. Details may have changed; I should hedge with "as of my knowledge cutoff, check pricing page."

Self-hosted: MIT open source, free, but that requires running a stack — out of scope per user's constraint.

Ease of management: easy — pure SDK against hosted SaaS, env-var config, no infra. Blocked only on needing an account + API keys which requires interactive signup.

Done. Here's the summary:

What I built

/sandbox/repo/langfuse_example.py — a minimal trace example against Langfuse Cloud (hosted, no local stack) using the official Python SDK v4 (installed, langfuse 4.15.4):

  • @observe decorators turn function calls into a nested trace (parent demo-trace → child greet span)
  • Optionally wraps an OpenAI call via langfuse.openai if OPENAI_API_KEY is set
  • Attaches user_id/tags metadata, then flush()es to the cloud
  • Verified it runs and fails gracefully with setup instructions when keys are absent

To run it for real:

export LANGFUSE_PUBLIC_KEY="pk-lf-..." LANGFUSE_SECRET_KEY="sk-lf-..." LANGFUSE_HOST="https://cloud.langfuse.com"
python3 langfuse_example.py

Blocked on one thing

I can't create a Langfuse Cloud account myself — signup requires email/OAuth login. The code is ready and the SDK is installed; I just need you to create a free project at cloud.langfuse.com and hand me the public/secret keys to complete an end-to-end trace.

Pricing (as of my knowledge; confirm at langfuse.com/pricing)

  • Hobby (free): ~50k observations/month, limited data retention, 1 project — plenty for this example
  • Core (~$59/mo): more included volume + pay-as-you-go overage, longer retention, more users
  • Pro (~$199/mo): higher limits, compliance features (SOC2/HIPAA options)
  • Enterprise: custom; self-hosting is also MIT open-source and free, but that's a local service stack — out of scope per your constraint

Is this easy for me to manage?

Yes. It's a pure SDK-to-SaaS workflow: config is three env vars, everything else is ordinary Python I can write, run, and verify in seconds. No infrastructure, no long-running services. The only human-in-the-loop step is the initial account signup — after that, tracing, prompt management, datasets, and scoring are all scriptable via the SDK/API.

AX Fundamentals

23 checks. Open an area for details.

This check is a quick preview. Test more tasks and models with Gauge Agents.

Test your product with Gauge Agents

AX Check by Gauge