Merge branch 'opensearch-project:main' into main

kylehounslow · web-flow · commit b726303ac729 · 2026-03-10T20:37:21.000-07:00
diff --git a/README.md b/README.md
@@ -4,6 +4,8 @@
 
 Observability Stack is an open-source stack designed for modern distributed systems. Built on OpenTelemetry, OpenSearch, and Prometheus, Observability Stack provides a complete, pre-configured infrastructure for monitoring microservices, web applications, and AI agents—with first-class support for agent observability through [OpenTelemetry Gen-AI Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/).
 
+https://github.com/opensearch-project/observability-stack/releases/download/v3.6.0-alpha.1/observability-stack-overview.mp4
+
 ![OpenSearch Observability Stack Architecture - docker-compose](./docs/observability-stack-arch-compose.excalidraw.png)
 
 ## Components
diff --git a/docs/starlight-docs/astro.config.mjs b/docs/starlight-docs/astro.config.mjs
@@ -7,6 +7,9 @@ import mermaid from 'astro-mermaid';
 export default defineConfig({
 	site: 'https://observability.opensearch.org',
 	base: '/docs',
+	redirects: {
+		'/get-started': '/get-started/installation/',
+	},
 	integrations: [
 		mermaid({
 			autoTheme: true,
@@ -35,7 +38,18 @@ export default defineConfig({
 				{
 					label: 'Get Started',
 					collapsed: true,
-					autogenerate: { directory: 'get-started' },
+					items: [
+						{ label: 'Installation', link: '/get-started/installation/' },
+						{ label: 'Platform Overview', link: '/get-started/overview/' },
+						{ label: 'Core Concepts', link: '/get-started/core-concepts/' },
+						{
+							label: 'Quickstart',
+							items: [
+								{ label: 'Ingest Your First Traces', link: '/get-started/quickstart/first-traces/' },
+								{ label: 'Create Your First Dashboard', link: '/get-started/quickstart/first-dashboard/' },
+							],
+						},
+					],
 				},
 				{
 					label: 'Send Data',
diff --git a/docs/starlight-docs/src/content/docs/get-started/core-concepts.md b/docs/starlight-docs/src/content/docs/get-started/core-concepts.md
@@ -40,4 +40,4 @@ PromQL is Prometheus's query language for time-series metrics. The Observability
 OpenSearch stores observability data in indices following naming conventions:
 - `otel-v1-apm-span-*` — trace spans
 - `otel-v2-apm-service-map` — service map data
-- `ss4o_logs-*` — log data
+- `logs-otel-v1*` — log data
diff --git a/docs/starlight-docs/src/content/docs/get-started/index.md b/docs/starlight-docs/src/content/docs/get-started/index.md
diff --git a/docs/starlight-docs/src/content/docs/get-started/installation.mdx b/docs/starlight-docs/src/content/docs/get-started/installation.mdx
@@ -0,0 +1,106 @@
+---
+title: Installation
+description: Install and run the OpenSearch Observability Stack
+---
+
+The OpenSearch Observability Stack is an open-source, OpenTelemetry-native observability platform. It runs locally via Docker Compose and provides traces, logs, metrics, dashboards, service maps, and AI agent tracing out of the box.
+
+<video src="https://github.com/opensearch-project/observability-stack/releases/download/v3.6.0-alpha.1/observability-stack-installation.mp4" controls></video>
+
+## Prerequisites
+
+- **Docker** and **Docker Compose** (v2.20+)
+- At least 8 GB of available RAM
+- macOS, Linux, or WSL2 on Windows
+
+## Installation
+
+Installation takes 8–15 minutes depending on your internet connection.
+
+import { Tabs, TabItem } from '@astrojs/starlight/components';
+
+<Tabs>
+  <TabItem label="Interactive Installer">
+    ```bash
+    curl -fsSL https://raw.githubusercontent.com/opensearch-project/observability-stack/main/install.sh | bash
+    ```
+    The installer TUI guides you through component selection and configuration. Installation takes 8–15 minutes. Once complete, the stack starts automatically.  
+    Access OpenSearch Dashboards at http://localhost:5601
+  </TabItem>
+  <TabItem label="Docker Compose">
+    ```bash
+    git clone https://github.com/opensearch-project/observability-stack.git
+    cd observability-stack
+    docker compose up -d
+    ```
+    Access OpenSearch Dashboards at [http://localhost:5601](http://localhost:5601)
+  </TabItem>
+</Tabs>
+
+Credentials are stored in the `.env` file at the installation root:
+
+```bash
+cd observability-stack
+grep -E '^OPENSEARCH_(USER|PASSWORD)=' .env
+```
+
+## What gets installed
+
+| Component | Purpose |
+|-----------|---------|
+| OpenTelemetry Collector | Receives traces, logs, and metrics via OTLP |
+| Data Prepper | Processes traces, generates service maps and RED metrics |
+| OpenSearch | Stores and indexes all observability data |
+| Prometheus | Scrapes and stores time-series metrics |
+| OpenSearch Dashboards | Visualizes data, agent traces, and service maps |
+| [Example services](#example-services) | Multi-agent travel planner, weather, and events agents that generate agent traces with GenAI semantic conventions |
+
+### Example services
+
+The stack ships with example AI agent services that generate agent traces automatically — a multi-agent travel planner, weather agent, and events agent. These are enabled by default.
+
+To disable them, comment out this line in `.env`:
+
+```env
+# INCLUDE_COMPOSE_EXAMPLES=docker-compose.examples.yml
+```
+
+#### OpenTelemetry Demo
+
+You can also run the [OpenTelemetry Demo](https://opentelemetry.io/docs/demo/), a full microservices e-commerce app that generates realistic telemetry. Uncomment in `.env`:
+
+```env
+INCLUDE_COMPOSE_OTEL_DEMO=docker-compose.otel-demo.yml
+```
+
+This adds ~2 GB memory usage. See [Resource Requirements](https://github.com/opensearch-project/observability-stack#resource-requirements) for details.
+
+After any `.env` change: `docker compose down && docker compose up -d`
+
+## Updating
+
+Component images are updated on new OpenSearch releases. To pull the latest:
+
+```bash
+cd observability-stack
+git pull
+docker compose down
+docker compose up -d --pull always
+```
+
+## Uninstall
+
+The stack runs via Docker Compose from the cloned `observability-stack` repository. To stop all services and remove data:
+
+```bash
+cd observability-stack
+docker compose down -v --remove-orphans
+```
+
+To fully remove, delete the repository directory.
+
+## Next steps
+
+- [Platform Overview](/docs/get-started/overview/) — architecture and data flow
+- [Core Concepts](/docs/get-started/core-concepts/) — key terms and ideas
+- [Quickstart](/docs/get-started/quickstart/) — send your first traces
diff --git a/docs/starlight-docs/src/content/docs/get-started/quickstart/first-traces.md b/docs/starlight-docs/src/content/docs/get-started/quickstart/first-traces.md
@@ -3,35 +3,27 @@ title: Ingest Your First Traces
 description: Instrument your application to send traces to the Observability Stack
 ---
 
-This guide shows how to instrument a Python application with OpenTelemetry and send traces to the Observability Stack.
-
-## Install dependencies
+## 1. Install dependencies
 
 ```bash
 pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp
 ```
 
-## Configure the exporter
+## 2. Send traces
 
 ```python
 from opentelemetry import trace
 from opentelemetry.sdk.trace import TracerProvider
 from opentelemetry.sdk.trace.export import BatchSpanProcessor
 from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
 
-# Point to the OTel Collector
 exporter = OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)
-
 provider = TracerProvider()
 provider.add_span_processor(BatchSpanProcessor(exporter))
 trace.set_tracer_provider(provider)
 
 tracer = trace.get_tracer("my-app")
-```
 
-## Create spans
-
-```python
 with tracer.start_as_current_span("handle-request") as span:
     span.set_attribute("http.method", "GET")
     span.set_attribute("http.url", "/api/users")
@@ -41,14 +33,17 @@ with tracer.start_as_current_span("handle-request") as span:
         # your database query here
 ```
 
-## Verify in Dashboards
+For other languages, see [Send Data](/docs/send-data/).
+
+## 3. View traces in OpenSearch Dashboards
 
-1. Open `http://localhost:5601`.
-2. Go to **Observability** > **Traces**.
-3. Search for your trace by name or filter by time range.
-4. Click a trace to see the span tree, timing, and attributes.
+1. Open [OpenSearch Dashboards](http://localhost:5601)
+2. Navigate to **Observability** → **Traces**
+3. Search for your trace by name or filter by time range
+4. Click a trace to see the span tree, timing, and attributes
 
 ## Next steps
 
-- [Create Your First Dashboard](/docs/get-started/quickstart/first-dashboard/)
+- [Create Your First Dashboard](/docs/get-started/quickstart/first-dashboard/) — build custom visualizations
+- [Agent Tracing](/docs/ai-observability/agent-tracing/) — trace AI agent workflows with GenAI semantic conventions
 - [Send Data](/docs/send-data/) — more instrumentation options
diff --git a/docs/starlight-docs/src/content/docs/get-started/quickstart/index.md b/docs/starlight-docs/src/content/docs/get-started/quickstart/index.md
diff --git a/docs/starlight-docs/src/content/docs/index.mdx b/docs/starlight-docs/src/content/docs/index.mdx
@@ -9,33 +9,35 @@ import IconCardGrid from '../../components/IconCardGrid.astro';
 
 OpenSearch Observability Stack is an **open-source, OpenTelemetry-native observability platform** ([GitHub](https://github.com/opensearch-project/observability-stack)) for monitoring services, infrastructure, and AI agents. Install locally via Docker Compose — traces, logs, Prometheus metrics, service maps, and agent tracing out of the box.
 
-## What You Can Do
-
 <IconCardGrid>
   <IconCard title="Traces, Logs & Metrics" icon="/docs/icons/app_visualize.svg" items={["Collect via OTLP", "Log-trace correlation", "Search with PPL and PromQL"]} href="/docs/investigate/discover-logs/" />
   <IconCard title="Application Monitoring" icon="/docs/icons/app_apm.svg" items={["Auto-generated service maps", "RED metrics from traces", "Latency and error tracking"]} href="/docs/apm/" />
   <IconCard title="AI Agent Observability" icon="/docs/icons/app_ml.svg" items={["Trace LLM calls, token usage, and tool use", "Agent execution graphs", "Python and JavaScript SDKs"]} href="/docs/ai-observability/" />
   <IconCard title="Dashboards & Alerting" icon="/docs/icons/app_dashboard.svg" items={["Custom dashboards", "Anomaly detection", "Slack, email, webhook alerts"]} href="/docs/dashboards/" />
 </IconCardGrid>
 
-## Why Observability Stack?
+### See it in action
 
-- **Open source**: Fully open source, no vendor lock-in, self-host everything
-- **OpenTelemetry-native**: All data ingestion uses OTel protocols and [semantic conventions](https://opentelemetry.io/docs/specs/semconv/)
-- **GenAI-first**: Purpose-built views for AI agent tracing using standard `gen_ai.*` attributes
-- **Local-first**: Runs entirely on your machine via Docker Compose — no cloud account required
-- **Production path**: Same components (OpenSearch, Prometheus, OTel Collector) scale to production
+<video src="https://github.com/opensearch-project/observability-stack/releases/download/v3.6.0-alpha.1/observability-stack-overview.mp4" controls></video>
 
 ## Quickstarts
 
-<LinkCard title="Install & Explore" href="/docs/get-started/" description="Install the stack and explore traces already flowing from the built-in example services." />
-<LinkCard title="Send Your First Traces" href="/docs/get-started/quickstart/" description="Instrument an app with the Python SDK and see traces in dashboards." />
+<LinkCard title="Install & Explore" href="/docs/get-started/installation/" description="Install the stack and explore traces already flowing from the built-in example services." />
+<LinkCard title="Send Your First Traces" href="/docs/get-started/quickstart/" description="Instrument an app with OpenTelemetry and see traces in dashboards." />
 <LinkCard title="Trace an AI Agent" href="/docs/ai-observability/agent-tracing/" description="Instrument an AI agent with GenAI semantic conventions and visualize the execution graph." />
 
 <Aside type="tip">
-The stack optionally ships with example services — a multi-agent travel planner, weather agent, and events agent that generate agent traces automatically. You can also enable the [OpenTelemetry Demo](https://opentelemetry.io/docs/demo/architecture/), a full microservices e-commerce app for realistic telemetry. Install, open Dashboards, and start exploring — no instrumentation needed.
+The stack optionally ships with [example services](https://github.com/opensearch-project/observability-stack/tree/main/examples) — a multi-agent travel planner, weather agent, and events agent that generate agent traces automatically. You can also enable the [OpenTelemetry Demo](https://opentelemetry.io/docs/demo/architecture/), a full microservices e-commerce app for realistic telemetry. Install, open Dashboards, and start exploring — no instrumentation needed.
 </Aside>
 
+## Why Observability Stack?
+
+- **Open source**: Fully open source, no vendor lock-in, self-host everything
+- **OpenTelemetry-native**: All data ingestion uses OTel protocols and [semantic conventions](https://opentelemetry.io/docs/specs/semconv/)
+- **GenAI-first**: Purpose-built views for AI agent tracing using standard `gen_ai.*` attributes
+- **Local-first**: Runs entirely on your machine via Docker Compose — no cloud account required
+- **Production path**: Same components (OpenSearch, Prometheus, OTel Collector) scale to production
+
 ## Community
 
 - [GitHub](https://github.com/opensearch-project/observability-stack) — issues, PRs, and discussions
diff --git a/examples/README.md b/examples/README.md
@@ -0,0 +1,37 @@
+# Examples
+
+Instrumented agent examples for the Observability Stack. Each demonstrates OpenTelemetry tracing with [Gen-AI Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). Follow the README in each folder to try the example agents, which are fully instrumented with OpenTelemetry so you can explore agent traces and telemetry data in OpenSearch Dashboards.
+
+## Examples
+
+| Example | Framework | Description |
+|---------|-----------|-------------|
+| [plain-agents/weather-agent](./plain-agents/weather-agent/) | OpenTelemetry SDK | Standalone weather assistant with fault injection, OTLP traces/metrics/logs |
+| [plain-agents/multi-agent-planner](./plain-agents/multi-agent-planner/) | OpenTelemetry SDK | Distributed travel planner with trace context propagation across sub-agents |
+| [strands/code-assistant](./strands/code-assistant/) | Strands SDK | AI coding assistant with auto-instrumented GenAI spans |
+| [langchain/bedrock-financial-assistant](./langchain/bedrock-financial-assistant/) | LangChain | Financial assistant using Bedrock Claude with automatic LangChain tracing |
+
+## Prerequisites
+
+- Python 3.9+, [uv](https://docs.astral.sh/uv/)
+- Observability Stack running (`docker compose up -d` from repo root)
+
+## Quick Start
+
+The `plain-agents` examples run automatically with the stack. For standalone examples:
+
+```bash
+cd <example-dir>
+uv run python main.py
+```
+
+View telemetry at:
+- **OpenSearch Dashboards**: http://localhost:5601
+- **Prometheus**: http://localhost:9090
+
+## OTLP Endpoints
+
+| Port | Protocol | Use With |
+|------|----------|----------|
+| 4317 | gRPC | OpenTelemetry SDK (`OTLPSpanExporter`) |
+| 4318 | HTTP/protobuf | Strands SDK (`setup_otlp_exporter()`) |
