Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/Build-Test-And-Deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,49 @@ jobs:
git tag -f "deployed/prod/${{ github.sha }}"
git push origin "deployed/prod/${{ github.sha }}" --force

- name: Notify IndexNow of Sitemap Update
continue-on-error: true
env:
INDEXNOW_API_KEY: ${{ secrets.INDEXNOW_API_KEY }}
run: |
# IndexNow Protocol: Notifies search engines (Bing, Yandex, Naver) of content updates.
# See: https://www.indexnow.org/documentation
#
# Domain ownership is verified via a key file hosted at:
# https://essentialcsharp.com/{key}.txt (served from wwwroot/{key}.txt)
# The file must contain exactly the key value. Without it, submissions return HTTP 403.
#
# We fetch the live sitemap (app is confirmed up after smoke test) and extract all <loc>
# URLs to submit as actual content URLs per the IndexNow spec.

# Fetch the live sitemap and extract all <loc> URLs into a newline-delimited list
SITEMAP=$(curl -sf "https://essentialcsharp.com/sitemap.xml")
if [ -z "$SITEMAP" ]; then
echo "::warning::Could not fetch sitemap; skipping IndexNow submission."
exit 0
fi

# Build a JSON array from <loc>...</loc> entries
URL_ARRAY=$(echo "$SITEMAP" \
| grep -oP '(?<=<loc>)[^<]+' \
| jq -Rn '[inputs]')

URL_COUNT=$(echo "$URL_ARRAY" | jq 'length')
echo "Submitting $URL_COUNT URLs to IndexNow..."

RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" -X POST https://api.indexnow.org/indexnow \
-H "Content-Type: application/json; charset=utf-8" \
-d "{
\"host\": \"essentialcsharp.com\",
\"key\": \"$INDEXNOW_API_KEY\",
\"urlList\": $URL_ARRAY
}")

echo "IndexNow response: HTTP $RESPONSE"
if [ "$RESPONSE" != "200" ] && [ "$RESPONSE" != "202" ]; then
echo "::warning::IndexNow submission returned HTTP $RESPONSE (200/202 = success, 202 = key validation pending)."
fi

- name: Logout of Azure CLI
if: always()
uses: azure/CLI@v3
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
6b397fbb613bfbacb059e4972cf3ec15
34 changes: 34 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,37 @@ For any bugs, questions, or anything else with specifically the code found insid
For information on setting up your local development environment, please see the [Getting Started Guide](docs/getting-started.md).

Please use issues or discussions to report issues found.

## IndexNow Setup (Production Deployment)

The deployment workflow notifies search engines (Bing, Yandex, Naver) of content updates via the [IndexNow protocol](https://www.indexnow.org/). After each successful production deploy the workflow fetches the live `sitemap.xml`, extracts all content URLs, and submits them in a single batch POST to IndexNow.

### How It Works

1. After the smoke test confirms the app is live, the workflow fetches `https://essentialcsharp.com/sitemap.xml`.
2. All `<loc>` URLs are extracted and POSTed to `https://api.indexnow.org/indexnow`.
3. IndexNow distributes those URLs to all participating search engines (Bing, Yandex, Naver, etc.).

### Setup Requirements

#### 1. Key Verification File (already committed)
IndexNow requires domain ownership proof via a publicly accessible `.txt` file at the root of the domain:
- The key file lives at `EssentialCSharp.Web/wwwroot/{key}.txt`
- The filename and file content are the same value (the key itself)
- ASP.NET Core's static file middleware serves it at `https://essentialcsharp.com/{key}.txt`
- IndexNow crawls that URL to verify domain ownership before accepting submissions
- Without this file, all submissions return HTTP 403

The key file is intentionally public — that is by design. The "security" is that only the domain owner can host a file at that path.

#### 2. GitHub Secret
Store the key value in GitHub repository secrets as `INDEXNOW_API_KEY`. The workflow reads it from there — the secret value must match the key filename/content committed to the repo.

Go to: **GitHub repo → Settings → Secrets and variables → Actions → New repository secret**
- Name: `INDEXNOW_API_KEY`
- Value: the key string (same value as the `.txt` filename in `wwwroot/`)

### Important Notes
- The IndexNow step uses `continue-on-error: true` — a submission failure will never block a deployment
- Submitting all sitemap URLs on every deploy is intentional; IndexNow has no hard rate limit for batch submissions and the full URL list ensures nothing is missed
- To rotate the key: generate a new hex string, add a new `wwwroot/{newkey}.txt`, update the GitHub Secret, and remove the old `.txt` file
Loading