docs: start doc for distributed tracing and logs guidance

docs/README.md

@@ -0,0 +1,44 @@
+# OpenTelemetry Rust Guidance
+
+Guidance documents for using OpenTelemetry in Rust applications.
+
+- [logs.md](logs.md) — Logs and Events
+- [traces.md](traces.md) — Distributed Traces
+- [metrics.md](metrics.md) — Metrics
+
+## Why this guidance exists
+
+Rust had a mature observability ecosystem before OpenTelemetry Rust matured.
+The [`tracing`] crate was created by the Tokio project for structured logging
+and in-process context propagation in async Rust, with deep integration into
+the async runtime. It was never designed around the OpenTelemetry data model.
+
+OpenTelemetry came later with a different scope: a vendor-neutral standard
+for **distributed** tracing — spans that cross process boundaries — adopting
+[W3C Trace Context] for propagation, with first-class concepts like span
+kind, links, and remote parents. The OpenTelemetry Tracing API in this repo
+is built around that data model.
+
+The third-party [`tracing-opentelemetry`] crate bridges `tracing` spans into
+OpenTelemetry spans. It predates parts of OpenTelemetry's evolution, and
+because `tracing` itself has no first-class notion of OpenTelemetry-specific
+span concepts, the bridge cannot fully express the OpenTelemetry model on
+its own.
+
+These docs reflect that history: we recommend `tracing` for logs and events
+(where it excels), and the OpenTelemetry Tracing API for spans (where it is
+the spec-aligned choice). For users invested in `tracing::span!`, the
+bridge remains a viable option, with the caveats noted in
+[traces.md](traces.md).
+
+## A note on guidance
+
+This is guidance, not policy. Rust users have a strong, established
+ecosystem and the freedom to combine these libraries in ways that fit their
+applications. Where we make a firm recommendation, it reflects what we
+believe gives the best alignment with the OpenTelemetry specification and
+the broadest compatibility — but the choice remains yours.
+
+[`tracing`]: https://crates.io/crates/tracing
+[`tracing-opentelemetry`]: https://crates.io/crates/tracing-opentelemetry
+[W3C Trace Context]: https://www.w3.org/TR/trace-context/

docs/logs.md

@@ -0,0 +1,110 @@
+# OpenTelemetry Rust Logs
+
+Status: **Stable**
+
+## Introduction
+
+This document provides guidance on leveraging OpenTelemetry logs in Rust
+applications.
+
+In short: for application logging, use [`tracing`] with the
+[`opentelemetry-appender-tracing`] appender. Existing `tracing` or `log`
+instrumentation can continue to work as-is; adopting OpenTelemetry for logs
+is primarily a setup change, not a code rewrite. For span guidance, see
+[traces.md](traces.md).
+
+[`tracing`]: https://crates.io/crates/tracing
+[`opentelemetry-appender-tracing`]: ../opentelemetry-appender-tracing/README.md
+
+## OpenTelemetry Log Bridge API
+
+The OpenTelemetry Log Bridge API (part of the `opentelemetry` crate) is public
+and technically usable directly, but OpenTelemetry Rust deliberately does not
+advertise it as an end-user logging API. Instead, we point users at `tracing`
+and its existing ecosystem, with the Log Bridge API reserved for authoring
+appenders. Bridges for the
+[`tracing`](https://docs.rs/opentelemetry-appender-tracing/) and
+[`log`](https://docs.rs/opentelemetry-appender-log/) crates are already
+available.
+
+## Instrumentation Guidance
+
+1. **Use the `tracing` crate**: We strongly recommend using the
+   [`tracing`](https://crates.io/crates/tracing) crate for structured logging in
+   Rust applications.
+
+2. **Lean on the `tracing` ecosystem**: OpenTelemetry doesn't replace what
+   `tracing` already offers. The appender is a standard `tracing-subscriber`
+   `Layer`, so it composes with `fmt::Layer`, `EnvFilter`, and any other
+   existing layer — for example, sending logs to stdout via `tracing`'s
+   `fmt::Layer` while exporting the same logs to an OTLP endpoint via
+   OpenTelemetry, or filtering what reaches the OpenTelemetry pipeline. Use
+   `tracing`'s ecosystem directly; OpenTelemetry just plugs into it.
+
+3. **Explicitly provide `name` and `target` fields**: These map to OpenTelemetry's
+   EventName and Instrumentation Scope respectively. Without them, `tracing`
+   synthesizes a `name` from the source location (e.g. `event src/foo.rs:42`)
+   and uses the module path as `target`, neither of which is meaningful as an
+   EventName or Instrumentation Scope.
+
+4. **Trace correlation is automatic**: When a log is emitted inside an active
+   OpenTelemetry span, the appender attaches the current `TraceId` and `SpanId`
+   to the resulting `LogRecord`. No extra wiring is required.
+
+5. **In-proc contextual enrichment via `tracing::span!`**: Use `tracing::span!`
+   to attach contextual attributes (e.g. `session.id`, `request.id`) that
+   should apply to every log inside that scope. The appender supports copying
+   these span attributes onto each emitted `LogRecord` via the experimental
+   `experimental_span_attributes` cargo feature; see the
+   [appender README](../opentelemetry-appender-tracing/README.md) for usage.
+
+### Example
+
+```rust
+use tracing::error;
+error!(
+    name: "db.client.connection.failed",
+    target: "myapp.db",
+    db.system.name = "postgresql",
+    db.namespace = "orders",
+    error.type = "connection_timeout",
+    retry_count = 3,
+    message = "Failed to connect to database after retries"
+);
+```
+
+## Terminology
+
+OpenTelemetry defines Events as Logs with an EventName. When you follow the guidance
+above and explicitly set the `name` field on every `tracing` log, each log maps to
+an OpenTelemetry Event. (Without an explicit `name`, the synthesized source-location
+string is technically present but is not a meaningful EventName.)
+
+**Note**: These are **not** mapped to Span Events. If you specifically need
+Span Events today, record them with the OpenTelemetry tracing API
+(`Span::add_event` / `add_event_with_timestamp`). Otherwise, prefer Events
+(Logs with an EventName) as described above. The OpenTelemetry specification
+is deprecating the *Span Events API* (see [OTEP-4430]) in favor of Events.
+
+[OTEP-4430]: https://github.com/open-telemetry/opentelemetry-specification/blob/main/oteps/4430-span-event-api-deprecation-plan.md
+
+## See Also
+
+- [Main README](../README.md) — setup guidance for logging libraries and appenders
+- [OpenTelemetry Logs
+  Specification](https://opentelemetry.io/docs/specs/otel/logs/)
+- [`tracing` Documentation](https://docs.rs/tracing/)
+- [`opentelemetry-appender-tracing`
+  Documentation](https://docs.rs/opentelemetry-appender-tracing/)
+
+## TODO
+
+This document is intentionally high-level. Areas to expand over time, similar
+to the depth in [metrics.md](metrics.md):
+
+- Best practices, with links to runnable examples
+- `LoggerProvider` lifecycle and shutdown
+- Performance considerations (allocation, attribute cost)
+- Attribute modelling and semantic conventions
+- Common pitfalls (lost logs, missing correlation, mis-set `target`)
+- Batching, exporter configuration, and back-pressure

docs/traces.md

@@ -0,0 +1,84 @@
+# OpenTelemetry Rust Traces
+
+Status: **Work-In-Progress**
+
+## Introduction
+
+This document provides guidance on leveraging OpenTelemetry traces in Rust
+applications.
+
+In short: prefer instrumentation libraries that use the OpenTelemetry Tracing
+API for framework-level spans, and use the OpenTelemetry Tracing API
+directly to create your own custom spans. [`tracing`] is for logs and events,
+not spans. See [logs.md](logs.md) for log guidance.
+
+## Instrumentation Guidance
+
+1. **Prefer instrumentation libraries.** For framework-level spans (HTTP
+   servers/clients, database drivers, messaging), prefer instrumentation
+   libraries that use the OpenTelemetry Tracing API directly. Most
+   applications start by plugging these in to capture the
+   request/response/downstream shape, then add custom spans for in-process
+   work as needed. No stable instrumentation libraries exist yet in the
+   OpenTelemetry Rust ecosystem, but in-progress ones for Tower and
+   Actix-Web exist in the [opentelemetry-rust-contrib] repository:
+   [`opentelemetry-instrumentation-tower`] and
+   [`opentelemetry-instrumentation-actix-web`].
+
+2. **Use the OpenTelemetry Tracing API to create custom spans.** The
+   `opentelemetry::trace` API is designed around the OpenTelemetry
+   specification, with first-class support for span kind
+   (server/client/producer/consumer/internal), links, remote parents, and
+   context propagation across process boundaries.
+
+3. **[`tracing`] is designed to collect structured, event-based diagnostic
+   information, and is not a complete substitute for the OpenTelemetry
+   Tracing API, which focuses on distributed tracing.** The `tracing` crate
+   does not have a first-class notion of an OpenTelemetry Span. It cannot,
+   on its own, set span kind, attach links, or set a remote parent —
+   concepts central to the OpenTelemetry specification, particularly for
+   *edge* spans (see #4 below for nuance). Use it primarily for logs and
+   events (see [logs.md](logs.md)).
+
+4. **Bridging from `tracing::span!` to OpenTelemetry spans.** If you are
+   already using `tracing::span!` and want those spans surfaced as
+   OpenTelemetry spans, the third-party [`tracing-opentelemetry`] crate
+   provides a bridge. It is maintained outside the OpenTelemetry project
+   and is not part of this repo; we mention it here for completeness.
+
+   For *internal* spans (spans that represent in-process work and never cross
+   a process boundary), `tracing::span!` through this bridge produces a
+   result nearly identical to using the OpenTelemetry Tracing API directly —
+   span kind, links, and remote parent are not relevant for internal spans.
+   The `tracing` limitations matter primarily for *edge* spans (e.g.,
+   incoming/outgoing HTTP, messaging), where span kind, links, and remote
+   parents are central to the OpenTelemetry data model. The bridge offers
+   extension APIs to express these concepts.
+
+## See Also
+
+- [OpenTelemetry Traces Specification](https://opentelemetry.io/docs/specs/otel/trace/)
+- [Main README](../README.md)
+- [logs.md](logs.md) — guidance for logs/events
+- [examples/tracing-http-propagator](../examples/tracing-http-propagator/) — end-to-end span creation and W3C context propagation
+- [examples/tracing-grpc](../examples/tracing-grpc/) — span creation and propagation over gRPC
+
+## TODO
+
+This document is intentionally high-level. Areas to expand over time, similar
+to the depth in [metrics.md](metrics.md):
+
+- Best practices, with links to runnable examples
+- `TracerProvider` lifecycle and shutdown
+- Sampling strategies and configuration
+- Context propagation: W3C Trace Context, Baggage, custom propagators
+- Span attribute modelling and semantic conventions
+- Performance considerations (allocation, attribute cost, span overhead)
+- Common pitfalls (broken context, missed parents, mis-set span kind)
+- Batching, exporter configuration, and back-pressure
+
+[`tracing`]: https://crates.io/crates/tracing
+[`tracing-opentelemetry`]: https://crates.io/crates/tracing-opentelemetry
+[opentelemetry-rust-contrib]: https://github.com/open-telemetry/opentelemetry-rust-contrib
+[`opentelemetry-instrumentation-tower`]: https://github.com/open-telemetry/opentelemetry-rust-contrib/tree/main/opentelemetry-instrumentation-tower
+[`opentelemetry-instrumentation-actix-web`]: https://github.com/open-telemetry/opentelemetry-rust-contrib/tree/main/opentelemetry-instrumentation-actix-web