Skip to content

extract: scan way nodes once across all extracts in complete_ways pass 1#314

Open
yuiseki wants to merge 1 commit intoosmcode:masterfrom
yuiseki:feat/extract-complete-ways-eway-all
Open

extract: scan way nodes once across all extracts in complete_ways pass 1#314
yuiseki wants to merge 1 commit intoosmcode:masterfrom
yuiseki:feat/extract-complete-ways-eway-all

Conversation

@yuiseki
Copy link
Copy Markdown

@yuiseki yuiseki commented Apr 29, 2026

See #312.

In Pass 1 of --strategy=complete_ways, eway() is called independently for
each extract on every way. This means way.nodes() is scanned up to twice per
extract, giving up to 2N scans total for N extracts.

This adds an eway_all() override in strategy_complete_ways that scans
way.nodes() at most twice regardless of N. A uint64_t bitmask tracks which
extracts have claimed the way in the first pass; the second pass then records
all node refs into extra_node_ids for matched extracts only.

When there are more than 64 extracts the method falls back to the original
per-extract eway() loop.

Benchmark

japan-260423.osm.pbf (~1 GB), 8-tile extraction:

Version Elapsed vs baseline
upstream 1m 29s
this PR 1m 25s -4%

planet-260413.osm.pbf (~86 GB), 16-tile extraction:

Version Elapsed vs baseline
upstream 51m 36s
this PR 49m 27s -4%

Output verified to be identical to the upstream result for all tiles.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant