Skip to content

Replace RowDictWithCache with on-demand RowFromId to fix memory exhaustion#237

Draft
limakzi with Copilot wants to merge 4 commits into
feature/change-encodingfrom
copilot/sub-pr-235-again
Draft

Replace RowDictWithCache with on-demand RowFromId to fix memory exhaustion#237
limakzi with Copilot wants to merge 4 commits into
feature/change-encodingfrom
copilot/sub-pr-235-again

Conversation

Copilot AI commented Mar 6, 2026

Copy link
Copy Markdown
Contributor

RowDictWithCache eagerly materialised all n^n possible rows at once — for n=8 that's 16,777,216 entries — making it unusable for even moderately large orders.

Changes

  • lib/helper.gd

    • Replaced RowDictWithCache with RowFromId(row_id, n): computes a single row on demand using the existing PowersOfNWithCache helper, no pre-allocation.
    • MultiplicationTableDecode: calls RowFromId per row instead of indexing into a precomputed dictionary.
    • MultiplicationTableGetEntry: decodes the column entry directly without constructing any row intermediate, preserving O(1) access.
  • tst/test_helper_multiplication_table_encoding.tst: replaced RowDictWithCache tests with RowFromId tests covering row ID 0 (all-ones), max row ID n^n−1 (all-n), and intermediate IDs across multiple values of n.

# Before: builds and caches all n^n rows upfront
row_dict := __SmallAntimagmaHelper.RowDictWithCache(n);  # 16M entries for n=8
return row_dict[row_id + 1];

# After: decodes exactly the one row needed
return __SmallAntimagmaHelper.RowFromId(row_id, n);

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…y exhaustion

Co-authored-by: limakzi <50334623+limakzi@users.noreply.github.com>
Copilot AI changed the title [WIP] Update implementation of Elias-Fano encoding based on feedback Replace RowDictWithCache with on-demand RowFromId to fix memory exhaustion Mar 6, 2026
@limakzi
limakzi force-pushed the feature/change-encoding branch 3 times, most recently from 59de591 to 333d8a5 Compare July 15, 2026 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants