Skip to content

Commit 2ff9c21

Browse files
authored
chore: added EOT 2024 via hive partitioning (#3)
* chore: added EOT 2024 via hive partitioning * chore: rename host-index hive partition eot=<year> to crawl=<year> * fix: replaced internal with public S3 bucket prefix * docs: restore full url-index walkthrough, adapt to crawl partition The hive-partition rename commit had condensed the url-index section down to two examples, dropping the original step-by-step investigations. Restore the full congress.gov 4xx walkthrough and the irs.gov LOTE walkthrough (pinned to crawl = 'EOT-2020', view renamed eot2020_url -> eot_url) and keep the EOT-2024 cross-crawl comparisons as additive extensions. * fix: host-index crawl partition values are EOT-YYYY, not bare year The hive partition is crawl=EOT-2020 / crawl=EOT-2024 (matching the url index), not crawl=2020 / crawl=2024. The crawl column is therefore VARCHAR, and host-index filters use crawl = 'EOT-2020'. Update select.py partition list, the README paths, schema box, filters, prose, and the regenerated result boxes.
1 parent 145ff86 commit 2ff9c21

5 files changed

Lines changed: 340 additions & 192 deletions

File tree

0 commit comments

Comments
 (0)