Add fix-geo-links.sh server script for GEO link URLs#692
Open
fredericsimard wants to merge 2 commits into
Open
Add fix-geo-links.sh server script for GEO link URLs#692fredericsimard wants to merge 2 commits into
fredericsimard wants to merge 2 commits into
Conversation
Every built page ships a <link rel="alternate" type="text/markdown"> tag pointing AI bots at the page's raw Markdown source, built from each config/<lang>/mkdocs.yml's repo_url. That's still set to the old google/transit repo (kept for the "Edit this page" link and GitHub icon), but docs/<lang>/ now lives in MobilityData/gtfs.org, so the generated raw-content URL 404s. This script rewrites that URL across the built HTML so bots can actually fetch the source. Corrected the header comment/description, which was copy-pasted from remove-subscribe-banner.sh and described the wrong script entirely, dropped an unused leftover variable from that same copy-paste, and added an OS check so the sed in-place edit works on both the Ubuntu server (GNU sed) and macOS (BSD sed) without changing the Ubuntu command at all. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a server-side post-processing script to rewrite the built pages’ link[rel="alternate"][type="text/markdown"] raw-Markdown URL base from google/transit to MobilityData/gtfs.org, preserving existing MkDocs repo_url usage for “Edit this page” while ensuring GEO crawlers don’t hit 404s.
Changes:
- Added
scripts/fix-geo-links.shto replace the old raw GitHub base URL across generated.htmlfiles. - Included a macOS vs Linux
sed -icompatibility branch for local testing.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
OLD_BASE was used unescaped as the sed search pattern, so the literal "." in github.com was interpreted as "match any character" instead of a literal dot, loosening the match more than intended. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/fix-geo-links.sh, a server-side script that rewrites the<link rel="alternate" type="text/markdown">GEO (Generative Engine Optimization) tag on every built page — this tag points AI bots/crawlers at the page's raw Markdown source, built from eachconfig/<lang>/mkdocs.yml'srepo_url. That's still set to the oldgoogle/transitrepo (kept intentionally for the "Edit this page" link and GitHub icon), butdocs/<lang>/content now actually lives inMobilityData/gtfs.org, so the generated raw-content URL 404s for bots. This script rewrites just that URL across the built HTML.sed -iworks on both the Ubuntu server (GNU sed) and macOS (BSD sed) for local testing, without changing the command that actually runs on Ubuntu at all.Why
AI/LLM crawlers following the GEO markdown-alternate link were hitting a 404 against the stale
google/transitraw URL instead of reaching the real page source inMobilityData/gtfs.org.Test plan
text/markdownalternate link