Improve docs SEO: sitemap cleanup, noindex for unreleased pages, and add meta tags#761
Improve docs SEO: sitemap cleanup, noindex for unreleased pages, and add meta tags#761oshankkkk wants to merge 3 commits into
Conversation
|
|
📝 WalkthroughWalkthroughThis pull request adds YAML frontmatter with title and description metadata to multiple documentation pages across components, concepts, contributing, getting-started, overview, and tutorials directories. Additionally, it updates the Docusaurus configuration to add sitemap settings, adjust noIndex behavior for the current/Next version, and refine ignored URL patterns. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
documentation/docs/tutorials/configure-agent-llm-configuration.mdx (1)
58-83:⚠️ Potential issue | 🟡 MinorFix the environment variable table to match the actual generated variable name.
The table documents
<NAME>_BASE_URLbut the platform generates<NAME>_URL(confirmed in agent_configuration_service.go). Update line 63 from:| `<NAME>_BASE_URL` | Base URL of the LLM Provider API endpoint |to:
| `<NAME>_URL` | Base URL of the LLM Provider API endpoint |The code snippet is correct and reads the properly named variable
OPENAI_GPT5_URL. The OpenAI SDK initialization pattern with emptyapi_keyand custom headers is also valid.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@documentation/docs/tutorials/configure-agent-llm-configuration.mdx` around lines 58 - 83, The table entry for the environment variable is incorrect: replace the documented `<NAME>_BASE_URL` with `<NAME>_URL` so the docs match the actual generated variable name (the code in agent_configuration_service.go produces `<NAME>_URL`), and ensure any surrounding descriptive text also references `<NAME>_URL` to match the Python example that uses `OPENAI_GPT5_URL`.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@documentation/docs/tutorials/custom-evaluators.mdx`:
- Line 4: Summary: Normalize the capitalization of "LLM-Judge" in the
description. Replace the lowercase "LLM-judge" token in the description string
("description: Create custom code-based and LLM-judge evaluators in the AMP
Console for domain-specific quality checks.") with the standardized casing
"LLM-Judge" to match the rest of the page and ensure consistent terminology
across the document.
In `@documentation/docusaurus.config.ts`:
- Around line 98-106: Remove the unintended exclusion from the sitemap by
deleting the '/agent-manager/docs/getting-started/quick-start/' entry inside the
ignorePatterns array (the array built from versions.filter((version) => version
!== latestVersion).map(...) and the hardcoded strings); update the
ignorePatterns so it no longer contains that quick-start path, ensuring the
quick-start remains discoverable and consistent with the redirect targeting the
latest version.
---
Outside diff comments:
In `@documentation/docs/tutorials/configure-agent-llm-configuration.mdx`:
- Around line 58-83: The table entry for the environment variable is incorrect:
replace the documented `<NAME>_BASE_URL` with `<NAME>_URL` so the docs match the
actual generated variable name (the code in agent_configuration_service.go
produces `<NAME>_URL`), and ensure any surrounding descriptive text also
references `<NAME>_URL` to match the Python example that uses `OPENAI_GPT5_URL`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 5a830fa9-f6c9-4d8f-baa7-92c710315fcb
📒 Files selected for processing (15)
documentation/docs/components/amp-instrumentation.mdxdocumentation/docs/concepts/evaluation.mdxdocumentation/docs/concepts/observability.mdxdocumentation/docs/contributing/contributing.mdxdocumentation/docs/getting-started/on-k3d.mdxdocumentation/docs/getting-started/on-your-environment.mdxdocumentation/docs/getting-started/quick-start.mdxdocumentation/docs/overview/what-is-amp.mdxdocumentation/docs/tutorials/configure-agent-llm-configuration.mdxdocumentation/docs/tutorials/custom-evaluators.mdxdocumentation/docs/tutorials/evaluation-monitors.mdxdocumentation/docs/tutorials/observe-first-agent.mdxdocumentation/docs/tutorials/register-ai-gateway.mdxdocumentation/docs/tutorials/register-llm-service-provider.mdxdocumentation/docusaurus.config.ts
| --- | ||
| sidebar_position: 3 | ||
| title: Custom Evaluators in WSO2 Agent Manager | ||
| description: Create custom code-based and LLM-judge evaluators in the AMP Console for domain-specific quality checks. |
There was a problem hiding this comment.
Standardize “LLM-Judge” capitalization in the description.
Use the same casing as the rest of the page to keep terminology consistent.
✏️ Suggested wording tweak
-description: Create custom code-based and LLM-judge evaluators in the AMP Console for domain-specific quality checks.
+description: Create custom code-based and LLM-Judge evaluators in the AMP Console for domain-specific quality checks.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| description: Create custom code-based and LLM-judge evaluators in the AMP Console for domain-specific quality checks. | |
| description: Create custom code-based and LLM-Judge evaluators in the AMP Console for domain-specific quality checks. |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@documentation/docs/tutorials/custom-evaluators.mdx` at line 4, Summary:
Normalize the capitalization of "LLM-Judge" in the description. Replace the
lowercase "LLM-judge" token in the description string ("description: Create
custom code-based and LLM-judge evaluators in the AMP Console for
domain-specific quality checks.") with the standardized casing "LLM-Judge" to
match the rest of the page and ensure consistent terminology across the
document.
| ignorePatterns: [ | ||
| ...versions | ||
| .filter((version) => version !== latestVersion) | ||
| .map((version) => `/agent-manager/docs/${version}/**`), | ||
| '/agent-manager/docs/next/**', | ||
| '/agent-manager/docs/', | ||
| '/agent-manager/search/', | ||
| '/agent-manager/docs/getting-started/quick-start/', | ||
| ], |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Confirm whether quick-start is intended to remain indexable.
rg -n 'quick-start' documentation/docs documentation/docusaurus.config.ts documentation/sidebars.tsRepository: wso2/agent-manager
Length of output: 2075
Remove /agent-manager/docs/getting-started/quick-start/ from the sitemap ignore patterns.
The quick-start guide is a primary onboarding resource (referenced in sidebars navigation, tutorials, and overview pages) and excluding it from the sitemap will harm search engine discoverability. This appears to be an unintended exclusion—the actual served path structure differs from the ignore pattern, and a redirect rule on line 178 explicitly directs to the quick-start for the latest version.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@documentation/docusaurus.config.ts` around lines 98 - 106, Remove the
unintended exclusion from the sitemap by deleting the
'/agent-manager/docs/getting-started/quick-start/' entry inside the
ignorePatterns array (the array built from versions.filter((version) => version
!== latestVersion).map(...) and the hardcoded strings); update the
ignorePatterns so it no longer contains that quick-start path, ensuring the
quick-start remains discoverable and consistent with the redirect targeting the
latest version.
There was a problem hiding this comment.
/agent-manager/docs/getting-started/quick-start/ is a redirect page, not actual content. The canonical content lives under versioned paths like /agent-manager/docs/{version}/getting-started/quick-start/.
Including the redirect in the sitemap could lead to duplicate/low-value indexing, so it’s excluded on purpose.
There was a problem hiding this comment.
@oshankkkk, that makes complete sense — excluding the unversioned redirect path from the sitemap is the right call. Serving the canonical versioned URL (e.g., /agent-manager/docs/{version}/getting-started/quick-start/) avoids duplicate content signals, while the redirect itself adds no indexable value. Thanks for the clarification!
✏️ Learnings added
Learnt from: oshankkkk
Repo: wso2/agent-manager PR: 761
File: documentation/docusaurus.config.ts:98-106
Timestamp: 2026-04-28T10:06:12.055Z
Learning: In the wso2/agent-manager docs site (documentation/docusaurus.config.ts), the path `/agent-manager/docs/getting-started/quick-start/` is a redirect page, not actual content. The canonical content lives under versioned paths like `/agent-manager/docs/{version}/getting-started/quick-start/`. It is intentionally excluded from the sitemap's `ignorePatterns` to prevent duplicate/low-value indexing.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: AnoshanJ
Repo: wso2/agent-manager PR: 609
File: documentation/docusaurus.config.ts:118-120
Timestamp: 2026-03-20T09:02:05.237Z
Learning: In Docusaurus projects that use `docusaurus/preset-classic`, ensure Algolia DocSearch is configured via `themeConfig.algolia` and does not require installing a separate Algolia theme plugin. If you want an offline/local search fallback, you may add `docusaurus-lunr-search` alongside Algolia; both integrations can be configured together in the same `docusaurus.config.ts`.
|
@oshankkkk could you resolve the merge conflicts |
oshankkkk
force-pushed
the
feat/docs-seo-improve
branch
from
248fee8 to
ca09f87
Compare
|
@oshankkkk We should not modify the titles of the current documentation. Earlier version titles should be retained |
4 participants
Purpose
Changes
noindexto unreleased (current) docs versiontitleanddescriptiontags to docs pagesWhy
Summary by CodeRabbit
Documentation
Chores