`-Xmint`/`-Xmaxt` and `-Xminf`/`-Xmaxf` modification for balanced GC

[cedrichansen]

Issue or pull request number:
depends on eclipse-openj9/openj9#12043

Overview:
Since the changes for -Xmint/-Xmaxt and -Xminf/-Xmaxf are semantically very similar, I have decided to include all notes/requested changes in this single issue. All of these changes only apply for the Balanced GC policy.

Changes for both sets of options

• If the -Xmint/-Xmaxt and/or -Xminf/-Xmaxf criteria are not being met and this results in a heap resize, then the heap resize that occurs only happens on non-eden heap (similar to how these options apply to tenure for gencon)
• Heap (again, non-eden heap here) resizing occurs at the end of a GMP cycle, or global collection (rather than simply global collection as stated in the docs). At this point heap resizing decision is made by observing both -Xmint/-Xmaxt and -Xminf/-Xmaxf compared to the appropriate proportion of time spent in GC, and free heap respectively.
• If either -Xmint/-Xmaxt and/or -Xminf/-Xmaxf criteria are not being met, there is no guarantee that a heap resize occurs. The heap sizing logic is (in essence) looking at 2 things:
  1. if % of time in GC pauses is between -Xmint/-Xmaxt. If it's too high, expand, if it's too low, contract
  2. if % of free heap is between -Xminf/-Xmaxf. If it's too high(too much free), contract, if too low, expand.
  Since these 2 criteria may be providing completely opposite recommendations (ex, lots of free memory, but high % of time in GC) causing oscillations in heap size, the balanced GC heap sizing logic is finding a balance between these 2 criteria

Changes specific to -Xmint/-Xmaxt

• -Xmint and -Xmaxt defaults for balanced GC are now 2% and 5% respectively. For this option, the values (-Xmint/-Xmaxt) are compared with % of time that GMP and global GC are active, and DOES NOT include PGC pauses (local GC operations, which are collecting eden regions of the heap).

Changes specific to Xminf/-Xmaxf

• these values only apply to non-eden portion of the heap (very similar to gencon's tenure). Free memory in eden space is not considered for Xminf/-Xmaxf purposes.

Release target:
Eclipse OpenJ9 0.30.0

Applies to the following JDK versions:
all

Applies to the following platforms:
all

For new command line options:
existing -Xmint/-Xmaxt and -Xminf/-Xmaxf options

[cedrichansen]

@amicic @dmitripivkine FYI

[cedrichansen]

The doc changes for this issue could be fairly verbose. I'd love to get some ideas as to how to communicate these ideas both succinctly, and completely. There is lots to unpack here, and making sure users reading the docs understand what is happening (especially why heap resize might not occur if one criteria is not being respected) with a quick read, would certainly be the goal

[doveye]

I guess doc changes have not happened yet for this issue? I see there's a blog post at https://blog.openj9.org/2021/09/24/balanced-gc-performance-improvements-eden-heap-sizing-improvements/, which would be good background/reference/something to point doc readers at.

[pshipton]

There were release notes added in #891 but I don't see the -Xmint/-Xmaxt doc itself modified. @amicic is anything further required?

[pshipton]

@amicic @dmitripivkine can this be closed?

[amicic]

I created #934 to cover missing docs (yet to be populated).
We can close this issue now.

[doveye]

The ID team will take a look to see what further doc updates are required.

[Sreekala-Gopakumar]

#835 (comment)
@amicic, I have following doubts:

• Under Changes for both sets of options, 2nd point, "... (rather than simply global collection as stated in the docs)", in which topic exactly has it been stated thus? I tried searching for the same. Any pointers will be helpful to make the changes in the correct place.
• Under the same section, 3rd point, the logic which has been explained, is it something that user needs to know? I was trying to fit in this information in the Heap Allocation topic (https://eclipse-openj9.github.io/openj9-docs-staging/942/allocation/#expansion-and-contraction) or in the Xmint/Xmaxt topic but I am not sure whether that is the right place.
• Is the following understanding correct that the main change in GC logic with xmin/maxf/t is that before, the logic looked at xmin/maxf, THEN it looked at xmin/maxt, whereas now it looks at both at the same time. Also, does the gencon policy behave in the same way?
• The existing Xminf topic says: "If the free space is above or below these limits, the OpenJ9 VM attempts to adjust the heap size so that: -Xminf ≤ free space ≤ -Xmaxf. With the mentioned change, it seems like with the balanced policy, the VM now might attempt to adjust the heap size, but also might not according to -Xmin/maxt. Is that correct and does the gencon policy also behave in the same way?

Thanks.

[pshipton]

Cedric is no longer working on this project, you need to ask @amicic

[Sreekala-Gopakumar]

#835 (comment)
@amicic, I have following doubts:

Under Changes for both sets of options, 2nd point, "... (rather than simply global collection as stated in the docs)", in which topic exactly has it been stated thus? I tried searching for the same. Any pointers will be helpful to make the changes in the correct place.
Under the same section, 3rd point, the logic which has been explained, is it something that user needs to know? I was trying to fit in this information in the Heap Allocation topic (https://eclipse-openj9.github.io/openj9-docs-staging/942/allocation/#expansion-and-contraction) or in the Xmint/Xmaxt topic but I am not sure whether that is the right place.
Is the following understanding correct that the main change in GC logic with xmin/maxf/t is that before, the logic looked at xmin/maxf, THEN it looked at xmin/maxt, whereas now it looks at both at the same time. Also, does the gencon policy behave in the same way?
The existing Xminf topic says: "If the free space is above or below these limits, the OpenJ9 VM attempts to adjust the heap size so that: -Xminf ≤ free space ≤ -Xmaxf. With the mentioned change, it seems like with the balanced policy, the VM now might attempt to adjust the heap size, but also might not according to -Xmin/maxt. Is that correct and does the gencon policy also behave in the same way?
Thanks.

[Sreekala-Gopakumar]

@amicic - Basic changes done in the related topics. Content pending clarification (as stated in #835 (comment)) not updated.

What's new in version 0.30.0 topic tidied up (does not require a review since no content was changed).

[amicic]

I'm not completely sure what docs this was referring to when talking about global GC points, but could be from here, when talking about gencon:

https://www.eclipse.org/openj9/docs/xminf/

For the gencon GC policy, the values apply only to the tenure part of the heap and only at global GC points.

Yes, your understanding of the core change was correct. Both criteria were active before but there was a conditional dependency. See this:
eclipse-openj9/openj9#12043

If free memory was within the acceptable boundary, the heap sizing logic would then look at gc cpu% to decide whether to expand/contract.

Now both criteria are combined/blended with equal weight/contribution to the final decision (at the same time - if you want to say so, it's ok, too)
There were some other changes, too. What is worth mentioning is that default values for Xmint/maxt are different: 5 and 13 for gencon/optavgpause/optthruput, but only 2 and 5 for balanced.

All GC policies now blend those 2 criteria. Although how Balanced does it vs others is different, since for others, there is still more weight/priority on occupancy criteria - but is not important to mention.

Expansion and contraction section needs to have full explanation.

Descriptions of 2 set of command line options have to be updated too, since they don't seem to mention that those options affect Balanced at all, but the exact relationship between the 2 sets of options does not to be repeated there. It just needs to be stated that relationship exists, that final decision about resizing is a combination and then refer to more detail documentation in Expansion and contraction section.

Not sure I understand your last concern/question, but I'll say this: meeting only one of the conditions (only minf/maxf or only mint/maxt) may not necessarily lead to heap resizing. True for all GC policies now, but again, exact way how they are combined is probably out of scope of this documentation.