Skip to content

Improve docs SEO: sitemap cleanup, noindex for unreleased pages, and add meta tags #761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
oshankkkk wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Improve docs SEO: sitemap cleanup, noindex for unreleased pages, and add meta tags #761

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2e3e3c7
Add sitemap config to exclude non-canonical and preview routes from s…
oshankkkk Apr 17, 2026
9f616cf
Add noindex to unreleased pages
oshankkkk Apr 19, 2026
ca09f87
Add SEO meta titles and descriptions for docs pages
oshankkkk Apr 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions documentation/docs/components/amp-instrumentation.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
title: WSO2 Agent Manager Instrumentation | OpenTelemetry for Python Agents
description: Enable zero-code OpenTelemetry tracing for Python agents using amp-instrumentation in WSO2 Agent Manager. Capture LLM calls, MCP requests, and more.
---

# WSO2 Agent Manager Instrumentation Package

Zero-code OpenTelemetry instrumentation package for externally-hosted Python agents using the Traceloop SDK, with trace visibility in the WSO2 Agent Manager.
Expand Down
3 changes: 2 additions & 1 deletion documentation/docs/concepts/evaluation.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 2
title: AI Agent Evaluation in WSO2 Agent Manager
description: Learn how WSO2 Agent Manager evaluates AI agent quality with trace-based scoring, built-in evaluators, and continuous monitoring.
---

import Tabs from '@theme/Tabs';
Expand Down Expand Up @@ -380,4 +382,3 @@ To use LLM-as-Judge evaluators, you need to provide an API key for at least one
| **Mistral AI** | `MISTRAL_API_KEY` |

Credentials are stored securely with the monitor and used only when the evaluation job runs. You only need to add each provider once per monitor. All evaluators using that provider share the same credentials.

2 changes: 2 additions & 0 deletions documentation/docs/concepts/observability.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 1
title: Observability for AI Agents in WSO2 Agent Manager
description: Understand how WSO2 Agent Manager captures traces, metrics, and logs for deployed and externally hosted AI agents.
---

# Observability
Expand Down
7 changes: 6 additions & 1 deletion documentation/docs/contributing/contributing.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
---
title: WSO2 Agent Manager Contributing Guidelines
description: Guidelines for using GitHub Discussions and Issues to collaborate on WSO2 Agent Manager.
---

# Contributing Guidelines

This document establishes guidelines for using GitHub Discussions and Issues for technical conversations about the Agent Manager.
Expand Down Expand Up @@ -84,4 +89,4 @@ Once a design proposal is approved:
1. Create GitHub Issues for implementation tasks
2. Link issues back to the design proposal discussion
3. Assign issues to appropriate milestones
4. Track progress through milestone completion
4. Track progress through milestone completion
2 changes: 2 additions & 0 deletions documentation/docs/getting-started/on-k3d.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 2
title: Install the Agent Manager on a local Kubernetes cluster
description: Install WSO2 Agent Manager on a local single-node k3d Kubernetes cluster for development and exploration.
---

import AmpInstallation from './_partials/_amp-installation.mdx';
Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/getting-started/on-your-environment.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 3
title: Install the Agent Manager on an existing Kubernetes cluster
description: Install WSO2 Agent Manager on an existing Kubernetes cluster such as EKS, GKE, AKS, or any distribution with LoadBalancer support.
---

import AmpInstallation from './_partials/_amp-installation.mdx';
Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/getting-started/quick-start.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 1
title: WSO2 Agent Manager Quick Start Guide
description: Install WSO2 Agent Manager locally with a single command using the dev container quick start.
---
# Quick Start Guide

Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/overview/what-is-amp.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 1
title: WSO2 Agent Manager
description: WSO2 Agent Manager is an open control plane for deploying, managing, observing, and evaluating AI agents at scale.
---
# WSO2 Agent Manager

Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/tutorials/configure-agent-llm-configuration.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 7
title: Configure LLM Providers for an Agent in WSO2 Agent Manager
description: Attach organization-level LLM providers to platform-hosted or external agents and use the generated connection details in code.
---

# Configure LLM Providers for an Agent
Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/tutorials/custom-evaluators.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 3
title: Custom Evaluators in WSO2 Agent Manager
description: Create custom code-based and LLM-judge evaluators in the AMP Console for domain-specific quality checks.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Standardize “LLM-Judge” capitalization in the description.

Use the same casing as the rest of the page to keep terminology consistent.

✏️ Suggested wording tweak 
-description: Create custom code-based and LLM-judge evaluators in the AMP Console for domain-specific quality checks.
+description: Create custom code-based and LLM-Judge evaluators in the AMP Console for domain-specific quality checks.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
description: Create custom code-based and LLM-judge evaluators in the AMP Console for domain-specific quality checks.
description: Create custom code-based and LLM-Judge evaluators in the AMP Console for domain-specific quality checks.
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@documentation/docs/tutorials/custom-evaluators.mdx` at line 4, Summary:
Normalize the capitalization of "LLM-Judge" in the description. Replace the
lowercase "LLM-judge" token in the description string ("description: Create
custom code-based and LLM-judge evaluators in the AMP Console for
domain-specific quality checks.") with the standardized casing "LLM-Judge" to
match the rest of the page and ensure consistent terminology across the
document.

---

import Tabs from '@theme/Tabs';
Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/tutorials/evaluation-monitors.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 2
title: Evaluation Monitors in WSO2 Agent Manager
description: Create, configure, and manage evaluation monitors in the AMP Console to score agent traces over time.
---

# Evaluation Monitors
Expand Down
2 changes: 2 additions & 0 deletions documentation/docs/tutorials/observe-first-agent.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 1
title: Observe Your First Agent in WSO2 Agent Manager
description: Register an externally hosted agent, enable zero-code instrumentation, and view traces in the AMP Console.
---

import Tabs from '@theme/Tabs';
Expand Down
4 changes: 3 additions & 1 deletion documentation/docs/tutorials/register-ai-gateway.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 4
title: Register an AI Gateway in WSO2 Agent Manager
description: Register and configure a WSO2 AI Gateway in WSO2 Agent Manager to route LLM traffic through a controlled proxy.
---

# Register an AI Gateway
Expand Down Expand Up @@ -132,4 +134,4 @@ Once running, the gateway appears in the **AI Gateways** list with status **Acti
- A **Default AI Gateway** is pre-provisioned in new organizations.
- Each gateway can serve multiple environments (e.g., `Default`, `Production`).
- The registration token (generated via **Reconfigure**) is environment-specific and must be kept secret.
- Marking a gateway as **Critical production gateway** helps signal its importance for operational monitoring.
- Marking a gateway as **Critical production gateway** helps signal its importance for operational monitoring.
2 changes: 2 additions & 0 deletions documentation/docs/tutorials/register-llm-service-provider.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
---
sidebar_position: 5
title: Register an LLM Service Provider in WSO2 Agent Manager
description: Register upstream LLM providers such as OpenAI, Anthropic, and Bedrock and expose them through an AI Gateway.
---

# Register an LLM Service Provider
Expand Down
31 changes: 23 additions & 8 deletions documentation/docusaurus.config.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ const config: Config = {
current: {
label: 'Next',
banner: 'unreleased',
noIndex: true,
},
cloud: {
label: 'Cloud',
Expand All @@ -105,24 +106,38 @@ const config: Config = {
// Please change this to your repo.
// Remove this to remove the "edit this page" links.
editUrl:
'https://github.com/wso2/agent-manager/edit/main/documentation/',
'https://github.com/wso2/agent-manager/edit/main/documentation/',
},
sitemap: {
lastmod: 'date',
changefreq: 'weekly',
priority: 0.5,
ignorePatterns: [
...versions
.filter((version) => version !== latestVersion)
.map((version) => `/agent-manager/docs/${version}/**`),
'/agent-manager/docs/next/**',
'/agent-manager/docs/',
'/agent-manager/search/',
'/agent-manager/docs/getting-started/quick-start/',
],
Comment on lines +115 to +123
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 27, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Confirm whether quick-start is intended to remain indexable.
rg -n 'quick-start' documentation/docs documentation/docusaurus.config.ts documentation/sidebars.ts

Repository: wso2/agent-manager

Length of output: 2075

Remove /agent-manager/docs/getting-started/quick-start/ from the sitemap ignore patterns.

The quick-start guide is a primary onboarding resource (referenced in sidebars navigation, tutorials, and overview pages) and excluding it from the sitemap will harm search engine discoverability. This appears to be an unintended exclusion—the actual served path structure differs from the ignore pattern, and a redirect rule on line 178 explicitly directs to the quick-start for the latest version.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@documentation/docusaurus.config.ts` around lines 98 - 106, Remove the
unintended exclusion from the sitemap by deleting the
'/agent-manager/docs/getting-started/quick-start/' entry inside the
ignorePatterns array (the array built from versions.filter((version) => version
!== latestVersion).map(...) and the hardcoded strings); update the
ignorePatterns so it no longer contains that quick-start path, ensuring the
quick-start remains discoverable and consistent with the redirect targeting the
latest version.

Copy link
Copy Markdown
Author

@oshankkkk oshankkkk Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/agent-manager/docs/getting-started/quick-start/ is a redirect page, not actual content. The canonical content lives under versioned paths like /agent-manager/docs/{version}/getting-started/quick-start/.

Including the redirect in the sitemap could lead to duplicate/low-value indexing, so it’s excluded on purpose.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oshankkkk, that makes complete sense — excluding the unversioned redirect path from the sitemap is the right call. Serving the canonical versioned URL (e.g., /agent-manager/docs/{version}/getting-started/quick-start/) avoids duplicate content signals, while the redirect itself adds no indexable value. Thanks for the clarification!

✏️ Learnings added 
Learnt from: oshankkkk
Repo: wso2/agent-manager PR: 761
File: documentation/docusaurus.config.ts:98-106
Timestamp: 2026-04-28T10:06:12.055Z
Learning: In the wso2/agent-manager docs site (documentation/docusaurus.config.ts), the path `/agent-manager/docs/getting-started/quick-start/` is a redirect page, not actual content. The canonical content lives under versioned paths like `/agent-manager/docs/{version}/getting-started/quick-start/`. It is intentionally excluded from the sitemap's `ignorePatterns` to prevent duplicate/low-value indexing.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

🧠 Learnings used 
Learnt from: AnoshanJ
Repo: wso2/agent-manager PR: 609
File: documentation/docusaurus.config.ts:118-120
Timestamp: 2026-03-20T09:02:05.237Z
Learning: In Docusaurus projects that use `docusaurus/preset-classic`, ensure Algolia DocSearch is configured via `themeConfig.algolia` and does not require installing a separate Algolia theme plugin. If you want an offline/local search fallback, you may add `docusaurus-lunr-search` alongside Algolia; both integrations can be configured together in the same `docusaurus.config.ts`.

},
blog: false, // Disable blog until we have content
theme: {
customCss: './src/css/custom.css',
customCss: './src/css/custom.css',
},
} satisfies Preset.Options,
],
],

themeConfig: {

// Replace with your project's social card
// image: 'img/amp-social-card.png',
announcementBar: {
id: `release_${quickStartDockerTag.replace(/\./g, '_')}`,
content:
`🎉 WSO2 Agent Manager <a target="_blank" rel="noopener noreferrer" href="https://github.com/wso2/agent-manager/releases/tag/amp%2F${quickStartDockerTag}">${quickStartDockerTag}</a> has been released! Explore what's new. 🎉`,
// Replace with your project's social card
// image: 'img/amp-social-card.png',
announcementBar: {
id: `release_${quickStartDockerTag.replace(/\./g, '_')}`,
content:
`🎉 WSO2 Agent Manager <a target="_blank" rel="noopener noreferrer" href="https://github.com/wso2/agent-manager/releases/tag/amp%2F${quickStartDockerTag}">${quickStartDockerTag}</a> has been released! Explore what's new. 🎉`,
isCloseable: true,
},

Expand Down