feat(prerender): scope domain-wide suggestion and top-page filtering to site subpath

[dhavkuma]

Problem

For subpath sites (e.g. nba.com/kings registered as its own site object), the prerender audit had two issues:

1. Critical — wrong CDN rule scope: allowedRegexPatterns and pathPattern in the domain-wide suggestion were hardcoded to '/*'. When a customer deploys the domain-wide suggestion for nba.com/kings, the CDN prerender rule would apply to all of nba.com, not just /kings/*.

2. Safety — organic URL bleed: SiteTopPage.allBySiteIdAndSourceAndGeo is scoped by siteId, but GSC can return domain-wide data. Top-page URLs not belonging to the subpath would get submitted for scraping under the wrong site.

Changes

src/prerender/handler.js

getSitePathPattern(baseUrl) — new helper that derives the correct glob pattern from the site's base URL pathname:

• https://nba.com → '/*'
• https://nba.com/kings → '/kings/*'

isUrlUnderSiteBase(url, baseUrl) — new helper that returns true when a URL falls within the site's registered path; no-op for root sites.

prepareDomainWideAggregateSuggestion — uses getSitePathPattern(baseUrl) for both allowedRegexPatterns and pathPattern instead of hardcoded '/*'.

getDomainWideSuggestionUrl — labels subpath sites as 'All Subpath URLs' instead of 'All Domain URLs'.

getTopOrganicUrlsFromSeo — filters top-page URLs through isUrlUnderSiteBase before returning.

What was already working

• Agentic URLs from Athena: buildSiteFilters in cdn-utils.js already generates a REGEXP_LIKE SQL clause scoped to the site's pathname — no change needed.
• All DB queries (Opportunity, Suggestion, PageCitability): already scoped by siteId, which is unique per subpath site.
• Guidance handler Mystique URL matching: toPathname keying works correctly with subpath pathnames.
• S3 status.json key: uses siteId, naturally isolated per site.

Testing

• Unit tests for getSitePathPattern and isUrlUnderSiteBase
• Unit test for prepareDomainWideAggregateSuggestion with a subpath baseUrl
• Verify domain-wide suggestion for nba.com/kings has pathPattern: '/kings/*'
• Verify root site nba.com still gets pathPattern: '/*'

🤖 Generated with Claude Code

[ssilare-adobe]

Duplicated PR.

Original PR : #2500

[MysticatBot-Dev]

Hey @dhavkuma,

Strengths

• src/prerender/handler.js:45-55 - isUrlUnderSiteBase has clean path-prefix matching. The startsWith(${normalized}/) check with the trailing slash correctly prevents false positives like /kingston matching /kings. The early return for root sites makes the no-op case explicit.
• src/prerender/handler.js:27-35 - getSitePathPattern handles trailing-slash normalization correctly, preventing double-slash patterns like /kings//*.
• src/prerender/handler.js:341-346 - Filter placement before .slice(0, limit) is correct. Subpath sites get the top N URLs that belong to this site, rather than top N with out-of-scope ones removed afterward (which could silently return fewer results).
• The approach of deriving scope from the existing baseUrl data model (rather than introducing new config or site properties) is architecturally sound - zero new state, zero migration needed.
• PR description is excellent - clearly articulates both bugs, explains what was already working, and provides a concrete test plan.

Issues

Important (Should Fix)

1. Tests absent for new production logic

The PR adds three new helper functions (getSitePathPattern, isUrlUnderSiteBase, rewritten getDomainWideSuggestionUrl) with non-trivial URL parsing and path-matching logic, plus integration changes in getTopOrganicUrlsFromSeo and prepareDomainWideAggregateSuggestion. No test files are modified or added. The PR description itself lists four unchecked test items, confirming these are needed but not yet written.

This is a bug-fix PR addressing a critical production issue (wrong CDN rule scope). Shipping without tests means the fix is unverified and could regress silently.

How to fix: Add unit tests covering at minimum:

• getSitePathPattern: root URL, subpath URL, trailing-slash subpath, invalid URL
• isUrlUnderSiteBase: URL matching subpath, URL outside subpath, root site (no-op), URL exactly equal to basePath, invalid URLs
• prepareDomainWideAggregateSuggestion with a subpath baseUrl, asserting pathPattern and allowedRegexPatterns reflect the subpath

2. //todo:dhavkuma debug markers committed on unchanged lines

src/prerender/handler.js:741, src/prerender/handler.js:1110, src/prerender/handler.js:1510, src/prerender/handler.js:1724 - Four personal TODO comments are appended to lines with no functional change. They communicate nothing actionable to future readers and make the PR look unfinished. If these mark pending work, capture it in a follow-up ticket. If they're leftover investigation markers, remove them.

Minor (Nice to Have)

3. Silent catch blocks in new helpers

src/prerender/handler.js:15-17, src/prerender/handler.js:32-34, src/prerender/handler.js:52-54 - All three helpers catch URL parsing failures and return safe defaults without logging. The fallback choices are defensively correct (fail-open), but if site.getBaseURL() ever returns an invalid URL, you'll have no signal that a site record is corrupted until someone notices the CDN rule is too broad. The helpers don't have access to log so this is a design tradeoff, not a blocker.

Recommendations

• Coordinate with PR #2500 (flagged as a duplicate by ssilare-adobe, still open). Whichever lands second will have merge conflicts in the same functions. Resolve which PR moves forward before either merges.
• Validate that the downstream CDN consumer interprets allowedRegexPatterns values (e.g., /kings/*) as globs rather than raw regex. Characters valid in URL pathnames (parentheses, dots) could produce overly-permissive patterns if the field is interpreted as regex.

Assessment

Ready to merge? No

The core logic is correct and well-implemented, but the PR ships zero tests for critical bug-fix code, CI is failing, and the //todo:dhavkuma markers indicate unfinished state. Address the Important items and get CI green.

Note: CI checks are currently failing - this should be resolved before merge.

Next Steps

1. Remove all four //todo:dhavkuma comments.
2. Add unit tests for the new helpers and the updated integration points.
3. Resolve the duplicate-PR situation with PR #2500.

---

_{Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 0m 59s | Cost: $2.41 | Commit: 91ddad8c02109744619cddec5b19432ee3d2c783
If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[github-actions]

This PR will trigger a minor release when merged.

[dhavkuma]

changes scoped with prerender moved to #2605