cap oversized str/bytes and sequence concatenation during inference#3129
cap oversized str/bytes and sequence concatenation during inference#3129kali834x wants to merge 2 commits into
Conversation
Merging this PR will not alter performance
Comparing 
Footnotes
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3129 +/- ##
=======================================
Coverage 93.61% 93.61%
=======================================
Files 92 92
Lines 11381 11387 +6
=======================================
+ Hits 10654 10660 +6
Misses 727 727
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
DanielNoord
left a comment
DanielNoord
left a comment
There was a problem hiding this comment.
Please shorten the comments, they read like AI generated text and are too detailed
| * Bound str/bytes and list/tuple concatenation (``a + b``) during inference. | ||
| The repetition operators (``*``, ``<<``, ``**``) are size-capped, but | ||
| concatenation was not, so two operands each just under the cap combine past | ||
| it and chaining ``+`` grows the result without bound. ``const_infer_binary_op`` | ||
| and ``tl_infer_binary_op`` now infer ``Uninferable`` once the combined length | ||
| exceeds ``1e8``, matching the repetition guards. The list/tuple path also | ||
| inferred every element of the concatenated sequence before the fix. |
There was a problem hiding this comment.
Agree with Daniel this is way too verbose.
| * Bound str/bytes and list/tuple concatenation (``a + b``) during inference. | |
| The repetition operators (``*``, ``<<``, ``**``) are size-capped, but | |
| concatenation was not, so two operands each just under the cap combine past | |
| it and chaining ``+`` grows the result without bound. ``const_infer_binary_op`` | |
| and ``tl_infer_binary_op`` now infer ``Uninferable`` once the combined length | |
| exceeds ``1e8``, matching the repetition guards. The list/tuple path also | |
| inferred every element of the concatenated sequence before the fix. | |
| * Bound str/bytes and list/tuple concatenation (``a + b``) during inference. |
There was a problem hiding this comment.
Done, cut it down to the single line.
|
Trimmed the code comments and test docstrings down to one line each, and the changelog entry is now just the single line Pierre suggested. |
3 participants
Type of Changes
Description
the repetition operators are size-capped in
const_infer_binary_op(*,<<) and_multiply_seq_by_int(*), but concatenation with+is not, and each operand can sit just under the cap.("a" * 90000000) + ("b" * 90000000)is 34 source characters yet materializes a 180 MBConststring during inference, and chaining more+terms grows it without bound. the list/tuple path intl_infer_binary_opis worse:_filter_uninferable_nodesinfers every element of the concatenated sequence, so([0] * 90000000) + ([0] * 90000000)walks ~1.8e8 elements. both+branches now yieldUninferableonce the combined length passes1e8, the additive analog of the existing repetition guards. small concatenations keep inferring their exact value.