Skip to content

Additional website sources — hiero.org/blog + Support & Care (config-only)#30

Merged
hendrikebbers merged 3 commits into
mainfrom
feat/015-additional-web-sources
Jul 13, 2026
Merged

Additional website sources — hiero.org/blog + Support & Care (config-only)#30
hendrikebbers merged 3 commits into
mainfrom
feat/015-additional-web-sources

Conversation

@herbie-bot

Copy link
Copy Markdown
Collaborator

Summary

Adds hiero.org/blog and Open Elements Support & Care as configuration-only sources — no code changes, proving the ContentSource abstraction. Open questions resolved against the live sites.

Findings

  • hiero.org: real /sitemap.xml (flat urlset, 62 /blog/<post> entries) → /sitemap.xml + /blog/**, no fallback needed. robots.txt is a Next.js HTML catch-all (allow-all). Posts use <main><article>; no OpenGraph meta (title falls back to <title>/<h1>).
  • Support & Care: 2 pages under /en/ (/en/support-care, /en/support-care-maven) in /en/sitemap.xml/en/support-care*, selector body + content-exclude. OE robots is Allow: /.

Changes

  • application.yaml: hiero and support-and-care sources
  • ConfiguredSourcesTest: verifies all three sources bind from YAML with the expected type/URLs/selectors

Test coverage

  • Binding is unit-tested (config-only spec). Live crawl/extraction is verified manually against a dev index — CI does not hit external sites. Source-filter isolation is covered by spec 011. 145 tests total; both reviews clean.

Closes #29
🤖 Generated with Claude Code

…nly)

Resolved open questions against the live sites:
- hiero.org has /sitemap.xml (flat urlset, 62 /blog/<post> entries); robots
  is a Next.js HTML catch-all (allow-all); posts use <main><article>, no OG
  meta. -> hiero source: /sitemap.xml, /blog/**, selector "main article".
- Support & Care is 2 pages under /en/ (support-care, support-care-maven) in
  /en/sitemap.xml. -> support-and-care source: /en/support-care*, selector
  "body" + content-exclude.

No code changes — both added to application.yaml. ConfiguredSourcesTest
verifies binding. 145 tests green. steps.md records the findings; CLAUDE.md updated.

Refs #29
@hendrikebbers hendrikebbers merged commit f652113 into main Jul 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Additional website sources — hiero.org/blog + Support & Care (config-only)

2 participants