Compile the framework statically

[matrei]

These changes have been tested on grails-core functional tests:
https://develocity.apache.org/s/fs5tgfkrlla7o/tests/overview

[cbmarcum]

Hi @matrei , Thanks for all the work on this!
These changes look good to me. All checks and tests pass for me also.

[jonnybot0]

Hey, @matrei ! Thanks for including the Gradle build scan. I'm curious, is there a comparison scan for how the tests execute without this change? Grails' tests working with it is a good proof that it should generally work, but I'd like to see a comparison build to demonstrate that it makes some meaningful difference.

One tricky thing about a sweeping change like this is that it's hard to discern how it might impact downstream consumers. CompileStatic pulls out the meta-object protocol, for example. That's good for a lot of cases, but it's hard to know who's using it. Given that, it seems reasonable to me to assume that this would be a breaking change and should probably go in a major version release. The Grails build indicates pretty broad backward compatibility, but again, hard to be sure.

Any thoughts? Feel free to call me out if I'm wrong about the risk.

[matrei]

Hi @jonnybot0, here are fresh build scans from grails-core with:

• (A) geb-spock:8.0.1: https://develocity.apache.org/s/2qxlt7n4azbwq
• (B) geb-spock:8.0.99-SNAPSHOT (PR changes loaded from mavenLocal): https://develocity.apache.org/s/lmjyrrnzz33sm

and a comparison: https://develocity.apache.org/c/2qxlt7n4azbwq/lmjyrrnzz33sm/task-inputs

I filtered to only run the Geb tests instead of running all integration tests to get more meaningful results.

On this particular machine and these particular runs:
Compiling tests: 3m 35.889s vs 3m 20.419s which is a 15.470s or 7.2% performance improvement.
Running tests: 15m 47.155s vs 15m 42.096s which is a 5.059s or 0.53% performance improvement.

I totally understand that you stress the importance of not introducing breaking changes. I targeted 8.0.x as there should be no API differences, and both Geb and Grails test suites came up clean.

I added a DynamicDispatchCompatibilitySpec, to test some scenarios. Maybe you know of other ways this PR could break existing code?

[jonnybot0]

So, what's the goal of statically compiling, then?

Looking at the performance tab, it looks like the build with 8.0.1 ran about 14 seconds slower, but it used about 500 MB less memory at the peak:

Performance reports

8.0.1 - https://develocity.apache.org/s/2qxlt7n4azbwq/performance/build
Compile Static Build - https://develocity.apache.org/s/lmjyrrnzz33sm/performance/build

Test Reports

8.0.1 - https://develocity.apache.org/s/2qxlt7n4azbwq/tests/overview
Compile Static Build - https://develocity.apache.org/s/lmjyrrnzz33sm/tests/overview

For the actual task execution time (not total "serial" time), it was only about 5 seconds different. That could be down to environmental variations.

The static compiler is great for when you need it, but what does using it here add?

Or, perhaps the more useful question, what's the felt need you're dealing with? Is it a performance need? To be clear, I don't want to block us moving forward, but I do want to make sure we're meeting the heart of the felt need, and I don't think I understand it from context.

[matrei]

The goal was primarily performance. We have seen radically decreased performance for dynamic Groovy in Grails on Groovy 4 and l wanted to see if there were some low hanging fruits for shaving off some time on our test suites.

I agree that there is not much gained in this for the current Grails tests, but now we at least know from testing it.

Also, on another tangent, I do think Groovy framework code should optimally be compiled statically if possible and reaching for the dynamic nature of Groovy only for scripts and DSLs.

If you think it is a bad idea, I have no problem targeting another version or scrapping it all together.

[jonnybot0]

@matrei - I do think considering performance more deeply is a good idea. I'd also agree that @CompileStatic can have a big impact on that, and it may be worth doing some deep comparisons with Groovy 5 & 6 with & without this @CompileStatic thing. It doesn't appear to have made a difference for the Grails suite, but I'm not sure I would say "scrap it" right away. Still, to merge anything on performance grounds, we'd need some kind of demonstration of the performance benefits.

Spock has had a Groovy 5 supporting release since December, so it should be pretty doable to work up a version of Geb that can run on Groovy 5. I wouldn't expect to see any performance gains until Groovy 6 (currently in alpha), but again, you never know until you measure. :)

That said, I recognize you're volunteering the project & I want to be respectful of your time. If you're keen to have a go at performance, here are things I can think of that might be useful:

1. Do some performance profiles in reproducible infrastructure, like GitHub Actions. The Java Flight Recorder would probably be the best choice for that. If you could find some "hot paths" or obvious memory leaks, we could try and address those directly, using the profiler to verify them.
2. Do some kind of benchmarking or load testing. I'm a little less certain what the best choice would be, though. I've used jmh before, and know that Cedric Champeau made a nice Gradle plugin for it which should make it reasonably easy to use. At the same time, Browser testing is such a high level thing that I'm not confident micro-benchmarking is the right fit. Maybe gatling would be better? The main idea would be something that could exercise the code really heavily & repeatedly to see if there were changes over a large number of iterations.

Those are both fairly deep dives, though. If you weren't up for that, no worries! We can close this and keep it as a record for someone who wants to chase that down on some future date.