docs: upgrade docstrings as a first-class documentation surface for AI agents (#1760)

* docs: upgrade docstrings as a first-class documentation surface for AI agents

Coding agents treat the installed package as primary documentation
(help(), inspect, grepping site-packages), so this improves the
docstrings they hit most, verified against code and tests:

- propagate_attributes: fix broken examples that called it as a client
  method (it is a top-level import) and used the nonexistent
  start_generation; document wrap-the-root-span ordering; add See also
- Langfuse.api/async_api: new docstrings covering list-vs-get semantics
  (list endpoints return observation/score IDs as strings), asynchronous
  ingestion with a bounded retry snippet, and scores read methods
  (get_many/get_by_id; there is no scores.get)
- flush(): note that server-side ingestion lags flush by a few seconds
- @observe: document capture_input/capture_output/transform_to_string,
  as_type="generation" with update_current_generation example
- run_experiment/Evaluation: add RAG LLM-as-a-judge example
  (faithfulness, answer relevance) and CI regression testing via
  langfuse/experiment-action; docstring for RunnerContext.run_experiment
- package docstring/README/pyproject: lead with full capability map,
  current env var names, llms.txt pointer; fix stale v3 docs link and
  broken langfuse.otel import in the Langfuse class example
- deprecation hygiene: emit DeprecationWarning when host= or
  LANGFUSE_HOST is the effective base URL source (behavior change);
  deprecation messages now name the exact replacement import

Generated code under langfuse/api/ is untouched; list-vs-get and
ingestion notes for it live on the non-generated api property and
should also be mirrored into the fern spec in langfuse/langfuse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: remove runtime DeprecationWarning for host/LANGFUSE_HOST

Keep the change docs-only; the deprecation remains documented in the
host parameter docstring and the LANGFUSE_HOST env var description.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: align docstrings with langfuse.com docs wording and links

Cross-checked against the langfuse-docs repo:

- api/async_api/flush: align ingestion-lag wording with query-via-sdk
  docs (typically 15-30s, not "a few seconds"), replace retry snippet
  with the documented deadline-based pattern (capped backoff), note that
  a successful trace.get does not imply observations/scores are complete,
  recommend observations.get_many(trace_id=...) for row-level reads and
  the Metrics API for aggregation; add anchored doc links
- run_experiment RAG example: use item.input (DatasetItem attribute)
  instead of item["input"], matching the documented dataset task shape
- RegressionError: correct action input name to should_fail_on_regression
  (was should_fail_on_error; verified against experiment-action action.yml)
- add See-also links: observation-types (observe/as_type) and
  environments (propagate_attributes)

All 10 docstring URLs verified to map to content pages in langfuse-docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: highlight v2 data APIs as recommended, correct score read guidance

- api property: call out api.observations and api.metrics as the
  recommended high-performance v2 endpoints (SDK v4 defaults) and note
  their v1 equivalents under api.legacy.* are less performant, not
  recommended for new workflows, and will be deprecated
- scores: point reads at api.scores_v3.get_many_v3; the v2
  api.scores.get_many/get_by_id are deprecated per the API spec and no
  longer available on Langfuse v4+ servers

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: address PR review — runnable propagation example, observations-first examples

- propagate_attributes example: start_observation() returns a plain
  observation (no __enter__), so both `with` usages crashed at runtime;
  switch to start_as_current_observation (smoke-ran the example)
- api/async_api/flush docstrings: use the v2 observations endpoint
  (api.observations.get_many with fields selection) as the primary
  example per review, matching the query-via-sdk docs page examples

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: clarify v2 score reads are removed in future Langfuse v4 servers

Current servers are v3.x; the spec's removal note is forward-looking.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* edits

* docs: revert deprecation message changes to keep PR strictly docs-only

Restore the original @deprecated decorator strings on set_trace_io and
set_current_trace_io so no runtime-visible string changes remain; the
corrected propagate_attributes import guidance stays in the docstrings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* push

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: jannikmaierhoefer

README.md

@@ -9,6 +9,8 @@
 [![Discord](https://img.shields.io/discord/1111061815649124414?style=flat-square&logo=Discord&logoColor=white&label=Discord&color=%23434EE4)](https://discord.gg/7NXusRtqYU)
 [![YC W23](https://img.shields.io/badge/Y%20Combinator-W23-orange?style=flat-square)](https://www.ycombinator.com/companies/langfuse)
 
+The [Langfuse](https://langfuse.com) Python SDK covers the full platform: **observability/tracing** (OpenTelemetry-based, with OpenAI and LangChain integrations), **datasets & experiments** (offline evaluation and regression testing of prompt/model changes, including [CI via GitHub Actions](https://github.com/langfuse/experiment-action)), **LLM-as-a-judge and custom evaluations/scores**, **prompt management**, and a **full REST API client**.
+
 ## Installation
 
 > [!IMPORTANT]
@@ -18,6 +20,35 @@
 pip install langfuse
 ```
 
+## Quickstart
+
+```python
+# env: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL
+
+from langfuse import get_client
+
+langfuse = get_client()
+
+# Create a span using a context manager
+with langfuse.start_as_current_observation(as_type="span", name="process-request") as span:
+    # Your processing logic here
+    span.update(output="Processing complete")
+
+    # Create a nested generation for an LLM call
+    with langfuse.start_as_current_observation(as_type="generation", name="llm-response", model="gpt-5.6") as generation:
+        # Your LLM call logic here
+        generation.update(output="Generated response")
+
+# All spans are automatically closed when exiting their context blocks
+
+
+# Flush events in short-lived applications
+langfuse.flush()
+```
+
 ## Docs
 
-Please [see our docs](https://langfuse.com/docs/sdk/python/sdk-v3) for detailed information on this SDK.
+- SDK guide: https://langfuse.com/docs/observability/sdk/overview
+- Full documentation: https://langfuse.com/docs
+- Machine-readable docs index (for AI agents): https://langfuse.com/llms.txt
+- API reference of this package: https://python.reference.langfuse.com

langfuse/__init__.py

@@ -1,4 +1,53 @@
-""".. include:: ../README.md"""
+"""Langfuse Python SDK — observability, evaluation, and prompt management for LLM applications.
+
+Capabilities:
+
+- **Tracing / observability**: `@observe` decorator, `Langfuse.start_observation` /
+  `start_as_current_observation` context managers, OpenTelemetry-based; integrations
+  for OpenAI (`langfuse.openai`) and LangChain (`langfuse.langchain.CallbackHandler`).
+- **Trace attributes**: `propagate_attributes` (top-level function) sets user_id,
+  session_id, tags, and metadata on all spans in a context.
+- **Datasets & experiments**: `Langfuse.get_dataset`, `Langfuse.run_experiment` for
+  offline evaluation and regression testing of prompt/model changes (CI support via
+  https://github.com/langfuse/experiment-action and `RegressionError`).
+- **Evaluation / LLM-as-a-judge**: `Evaluation` results from custom or model-based
+  evaluators; scores via `Langfuse.create_score` / `span.score`.
+- **Prompt management**: `Langfuse.get_prompt`, `Langfuse.create_prompt` with
+  client-side caching and version/label control.
+- **Full REST API**: `Langfuse.api` (sync) / `Langfuse.async_api` (async) clients.
+
+Quickstart:
+
+```python
+# env: LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL
+from langfuse import get_client
+
+langfuse = get_client()
+
+# Create a span using a context manager
+with langfuse.start_as_current_observation(as_type="span", name="process-request") as span:
+    # Your processing logic here
+    span.update(output="Processing complete")
+
+    # Create a nested generation for an LLM call
+    with langfuse.start_as_current_observation(as_type="generation", name="llm-response", model="gpt-3.5-turbo") as generation:
+        # Your LLM call logic here
+        generation.update(output="Generated response")
+
+# All spans are automatically closed when exiting their context blocks
+
+# Flush events in short-lived applications
+langfuse.flush()
+```
+
+Configuration is via constructor args or environment variables: `LANGFUSE_PUBLIC_KEY`,
+`LANGFUSE_SECRET_KEY`, `LANGFUSE_BASE_URL` (defaults to https://cloud.langfuse.com). See `langfuse._client.environment_variables`
+for the full list.
+
+Docs: https://langfuse.com/docs — machine-readable index: https://langfuse.com/llms.txt
+
+.. include:: ../README.md
+"""
 
 from langfuse.batch_evaluation import (
     BatchEvaluationResult,

langfuse/_client/client.py

@@ -248,13 +248,13 @@ def mask_otel_spans(
 
     Example:
         ```python
-        from langfuse.otel import Langfuse
+        from langfuse import Langfuse
 
         # Initialize the client (reads from env vars if not provided)
         langfuse = Langfuse(
             public_key="your-public-key",
             secret_key="your-secret-key",
-            host="https://cloud.langfuse.com",  # Optional, default shown
+            base_url="https://cloud.langfuse.com",  # Optional, default shown
         )
 
         # Create a trace span
@@ -421,6 +421,43 @@ def __init__(
 
     @property
     def api(self) -> LangfuseAPI:
+        """Synchronous client for the full Langfuse REST API (traces, observations, scores, datasets, prompts, ...).
+
+        Use this to read or manage data on the Langfuse server; use the tracing methods
+        (`start_observation`, `@observe`) to create traces. Use `async_api` for the
+        asyncio variant.
+
+        Semantics that are easy to miss:
+
+        - **Ingestion is asynchronous.** `langfuse.flush()` only guarantees delivery to
+          the API, not read visibility: reads such as `api.trace.get(trace_id)` may
+          raise `langfuse.api.NotFoundError` until processing completes (typically
+          within 15-30 seconds; longer under load). The same applies to scores and
+          dataset run reads. Instead of a fixed sleep, retry with a deadline:
+
+        - **List endpoints return lightweight views.** `api.trace.list(...)` returns
+          `TraceWithDetails`, where `observations` and `scores` are lists of ID strings.
+          Fetch the full objects with `api.trace.get(trace_id)` (`TraceWithFullDetails`),
+          or prefer `api.observations.get_many(trace_id=...)` for row-level observation
+          queries. The same list-view vs. get-detail pattern applies to other resources.
+
+        - **Prefer the v2 data APIs — they are the defaults since SDK v4.**
+          `api.observations` and `api.metrics` map to the high-performance
+          `/api/public/v2/...` endpoints and are the recommended read path. Their v1
+          equivalents remain available under `api.legacy.observations_v1` /
+          `api.legacy.metrics_v1` but are less performant at scale, not recommended
+          for new workflows, and will be deprecated.
+
+        - For large-scale aggregation (usage/cost by model, user, etc.), prefer the
+        v2 Metrics API (`api.metrics.metrics(...)`) over paginating row-level data.
+
+
+        See also: `async_api`,
+        https://langfuse.com/docs/api-and-data-platform/features/query-via-sdk
+        (ingestion lag: #ingestion-lag, list vs. get: #traces-list-vs-get),
+        https://langfuse.com/docs/api-and-data-platform/features/observations-api,
+        https://langfuse.com/docs/metrics/features/metrics-api
+        """
         if self._resources is None:
             raise AttributeError("Langfuse client is not initialized")
 
@@ -1519,7 +1556,7 @@ def set_current_trace_io(
             evaluators). It will be removed in a future major version.
 
             For setting other trace attributes (user_id, session_id, metadata, tags, version),
-            use :meth:`propagate_attributes` instead.
+            use :func:`langfuse.propagate_attributes` (top-level import) instead.
 
         Args:
             input: Input data to associate with the trace.
@@ -2239,6 +2276,16 @@ def flush(self) -> None:
 
             # Continue with other work
             ```
+
+        Note:
+            `flush()` guarantees data was *delivered* to the API, not that it is
+            *readable* yet: server-side ingestion is asynchronous, so flushed data
+            may not be queryable for 15-30 seconds —
+            `api.observations.get_many(trace_id=...)` may return empty results and
+            `api.trace.get()` may raise `langfuse.api.NotFoundError` right after a
+            successful flush. See the `api` property docs for a bounded retry
+            pattern, or
+            https://langfuse.com/docs/api-and-data-platform/features/query-via-sdk#ingestion-lag
         """
         if self._resources is not None:
             self._resources.flush()

langfuse/_client/observe.py

@@ -109,8 +109,18 @@ def observe(
             as_type (Optional[Literal]): Set the observation type. Supported values:
                     "generation", "span", "agent", "tool", "chain", "retriever", "embedding", "evaluator", "guardrail".
                     Observation types are highlighted in the Langfuse UI for filtering and visualization.
-                    The types "generation" and "embedding" create a span on which additional attributes such as model metrics
-                    can be set.
+                    The types "generation" and "embedding" create a span on which additional attributes such as model,
+                    usage_details, and cost_details can be set — use `as_type="generation"` for LLM calls and update the
+                    observation via `langfuse.update_current_generation(...)` inside the function.
+            capture_input (Optional[bool]): Whether to capture the function's arguments as the observation's input.
+                    Defaults to the LANGFUSE_OBSERVE_DECORATOR_IO_CAPTURE_ENABLED environment variable (True if unset).
+                    Set to False for sensitive or very large inputs, then set input explicitly via
+                    `langfuse.update_current_span(input=...)` if needed.
+            capture_output (Optional[bool]): Whether to capture the function's return value as the observation's output.
+                    Same default and override mechanism as capture_input.
+            transform_to_string (Optional[Callable[[Iterable], str]]): For functions returning generators, joins the
+                    yielded chunks into the string stored as output. Without it, chunks are concatenated if all are
+                    strings, otherwise stored as a list.
 
         Returns:
             Callable: A wrapped version of the original function that automatically creates and manages Langfuse spans.
@@ -126,16 +136,25 @@ def process_user_request(user_id, query):
 
             For language model generation tracking:
             ```python
+            from langfuse import get_client, observe
+
             @observe(name="answer-generation", as_type="generation")
             async def generate_answer(query):
-                # Creates a generation-type span with extended LLM metrics
+                # Creates a generation-type observation with extended LLM metrics
                 response = await openai.chat.completions.create(
                     model="gpt-4",
                     messages=[{"role": "user", "content": query}]
                 )
                 return response.choices[0].message.content
             ```
 
+            Disabling input/output capture (e.g. for sensitive or large payloads):
+            ```python
+            @observe(capture_input=False, capture_output=False)
+            def handle_pii(user_record):
+                return process(user_record)
+            ```
+
             For trace context propagation between functions:
             ```python
             @observe()

langfuse/_client/propagation.py

@@ -131,9 +131,13 @@ def propagate_attributes(
     environment, and metadata dimensions that should be consistently applied across
     all observations in a trace.
 
-    **IMPORTANT**: Call this as early as possible within your trace/workflow. Only the
-    currently active span and spans created after entering this context will have these
-    attributes. Pre-existing spans will NOT be retroactively updated.
+    This is a module-level function, not a method on the Langfuse client:
+    import it with `from langfuse import propagate_attributes`.
+
+    **IMPORTANT**: Call this as early as possible within your trace/workflow —
+    ideally wrapping the creation of your root span, or immediately inside it. Only
+    the currently active span and spans created after entering this context will have
+    these attributes. Pre-existing spans will NOT be retroactively updated.
 
     **Why this matters**: Langfuse aggregation queries (e.g., total cost by user_id,
     filtering by session_id) only include observations that have the attribute set.
@@ -188,40 +192,43 @@ def propagate_attributes(
         Context manager that propagates attributes to all child spans.
 
     Example:
-        Basic usage with user and session tracking:
+        Basic usage with user and session tracking (note: `propagate_attributes` is a
+        top-level import, not a client method):
 
         ```python
-        from langfuse import Langfuse
+        from langfuse import Langfuse, propagate_attributes
 
         langfuse = Langfuse()
 
-        # Set attributes early in the trace
+        # Set attributes early: wrap everything inside the root span
         with langfuse.start_as_current_observation(name="user_workflow") as span:
-            with langfuse.propagate_attributes(
+            with propagate_attributes(
                 user_id="user_123",
                 session_id="session_abc",
                 environment="production",
                 metadata={"experiment": "variant_a"}
             ):
                 # All spans created here will have user_id, session_id, environment, and metadata
-                with langfuse.start_observation(name="llm_call") as llm_span:
+                with langfuse.start_as_current_observation(name="llm_call") as llm_span:
                     # This span inherits user_id, session_id, environment, and experiment metadata
                     ...
 
-                with langfuse.start_generation(name="completion") as gen:
+                with langfuse.start_as_current_observation(
+                    name="completion", as_type="generation"
+                ) as gen:
                     # This span also inherits all attributes
                     ...
         ```
 
         Prompt linking with auto-instrumented libraries:
 
         ```python
-        from langfuse import Langfuse
+        from langfuse import Langfuse, propagate_attributes
 
         langfuse = Langfuse()
         prompt = langfuse.get_prompt("my-prompt")
 
-        with langfuse.propagate_attributes(prompt=prompt):
+        with propagate_attributes(prompt=prompt):
             # Generations emitted by auto-instrumentation (LiteLLM langfuse_otel,
             # OpenAI Agents SDK, OpenInference, ...) within this context are
             # linked to the prompt version.
@@ -240,7 +247,7 @@ def propagate_attributes(
             early_span.end()
 
             # Set attributes in the middle
-            with langfuse.propagate_attributes(user_id="user_123"):
+            with propagate_attributes(user_id="user_123"):
                 # Only spans created AFTER this point will have user_id
                 late_span = langfuse.start_observation(name="late_work")
                 late_span.end()
@@ -253,7 +260,7 @@ def propagate_attributes(
         ```python
         # Service A - originating service
         with langfuse.start_as_current_observation(name="api_request"):
-            with langfuse.propagate_attributes(
+            with propagate_attributes(
                 user_id="user_123",
                 session_id="session_abc",
                 environment="staging",
@@ -282,6 +289,12 @@ def propagate_attributes(
 
     Raises:
         No exceptions are raised. Invalid values are logged as warnings and dropped.
+
+    See also:
+        `Langfuse.start_as_current_observation` (create the root span this wraps),
+        https://langfuse.com/docs/observability/features/sessions,
+        https://langfuse.com/docs/observability/features/users,
+        https://langfuse.com/docs/observability/features/environments
     """
     return _propagate_attributes(
         user_id=user_id,

langfuse/_client/span.py

@@ -245,7 +245,7 @@ def set_trace_io(
             evaluators). It will be removed in a future major version.
 
             For setting other trace attributes (user_id, session_id, metadata, tags, version),
-            use :meth:`Langfuse.propagate_attributes` instead.
+            use :func:`langfuse.propagate_attributes` (top-level import) instead.
 
         Args:
             input: Input data to associate with the trace.

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "langfuse"
 version = "4.14.0"
-description = "A client library for accessing langfuse"
+description = "Langfuse Python SDK - LLM observability/tracing, datasets, experiments, LLM-as-a-judge evaluation, and prompt management"
 readme = "README.md"
 authors = [{ name = "langfuse", email = "developers@langfuse.com" }]
 license = "MIT"