Fix: Gradle Wrapper native updated run for every dependency#13501
Conversation
050954e to
d6d9df5
Compare
yeikel
left a comment
There was a problem hiding this comment.
Thanks for the follow up fix. Could you please
- Update the pull request title with what's being fixed
- Add a test/Integration test that demonstrates the issue and proves the fix?
🤦 just didn't noted. Let me work on the tests then
Can you elaborate what "empty dependencies" mean? The test I had in mind was to somehow (need to research) mock the native runner, then duplicate any of the existing tests (not related to wrapper dependencies), enable the experiment, and assert the native runner is not called |
I jjust meant |
It is ok to mock the native runner to return proper result (how normally native runner wil return) into spec related file and get proper output which was causing the issue before and use that to reproduce the new expected output properly in the and show in the spec that passes successfully then we are good. |
cc603c3 to
cf41f55
Compare
|
Ok, just pushed the changes with the test scenarios. First, I've enabled both experiments (the wrapper and lockfile) in the tests, to make sure native runners logic is allowed to run. And also, an By the way, Finally, in the wrapper test, I've relaxed the You can verify the issue by running the test after commenting the line: Any feedback is welcome, not 100% it's a "good pattern" the one I applied. |
Thank you @gmazzo for the effort and @kbukum1 for reviewing it 🥇 @kbukum1 Did you update the service API to allow these new files? From what I re-tested the error still exists
|
|
Same here. I can confirm the fix was effective, since we run the native updated only once (when bumping the wrapper dependency): But the PR is still failing with the Two other PRs were successfully created (#1, #2): |
Thank you for checking. Yes, I think I missed that. I am going to update API to allow the files. |
|
I am allowing additionally to existing update followings files. After deploying the change, I will let you know. |
|
@kbukum1 I think that the following list should be enough |
|
"subproject" is generic or literal? The logic (if I remember correctly) will pick up wrapper updates at root but also any included build. It would be nice if you white list those files at any directory, since we can't know in advance the structure of the project. For those with that kind of complexity, the PR will still fail |
|
Here’s an example I was able to find: https://github.com/FunkyMuse/KAHelpers/tree/main/build-logic @gmazzo, would the current logic actually scan dependabot-core/gradle/lib/dependabot/gradle/file_fetcher.rb Lines 27 to 32 in c883730 I’ve forked the project here: https://github.com/yeikel/KAHelpers so we can run some tests as needed To be fair, I haven’t seen this pattern often. I struggled to find more examples here in github.com We might be fine shipping it, cover roughly 90% of use cases, and gather feedback from there if any. Gradle supports tons of customization features and unless we parse the build (#1164), we may not be able to cover every single use case/customization |
Currently, only the property files are being updated. Are you OK rolling that initially? The following items are open:
|
Thanks @yeikel for outlining the outstanding items. I agree that it’s best to hold off for now, since rolling out the feature flag as-is could impact existing Gradle users, especially those relying on dependency updates. Once we make the Gradle wrapper update optional, users can opt in when they’re ready. Additionally, rather than adding another choice, have we considered treating Would love to hear thoughts from @yeikel, @gmazzo, @ryanbrandenburg, and @honeyankit. |
I previously suggested this approach in my original review to help with isolating the components. @gmazzo and I had a valuable discussion in that PR, which is worth revisiting, as each method has its advantages and drawbacks. The decision we make will also impact the Maven wrapper, since I plan to propose a similar feature once we finalize the behavior for the Gradle wrapper. |
Can we wait a bit? I 'd like to work on @yeikel 's outstanding items first
Actually this was proposed by @yeikel in the first PR. I was advocating for the opposite, and at the end, we were waiting feedback from you on that. I'd like to understand the rationale of this. Why would it be treated differently? The way I see it's just another piece of the same system. The only reason why it was but there before was because it's implementation was different than Maven/Gradle regular dependencies right? By the way, if some wants to opt-out, they can always disable the But if we want an opt-in behavior otherwise, we should move it to a dedicated manager, or, if possible, set the feature flag in the configs itself. That's is going to be a bigger effort. Are you accepting new managers right now? I remember someone tagging the original PR because another blocked PR. |
I think that my last two paragraphs here summarize the reason to separate them. It is a subjective opinion. And there is no right answer. What separating would allow is to collect feedback without special feature flags
In most beta releases, the team tries to ask users to opt-in, and as far as I know, it is not possible to opt-in for feature flags A different ecosystem would allow us to create a "beta ecosystem" and collect feedback separately
Yes, the team is accepting new ecosystems now. For example #13091 Before we take a stance on that, let's please wait for @ryanbrandenburg, and @honeyankit to comment as it is a large change as you mentioned |
|
By the way, regarding this, I think both are the same and it's just a defect. I managed to reproduce it locally just now:
So the thing is, if you run I'm going to work on addressing the pending stuff assuming that's the issue, it would be nice if someone else can confirm it. Also, in the logic, we are first updating the requirements, and later running the native commands. So at the time we run it, the |
At first I tought that it should be another ecosystem, because gradle wrapper is not distributed via Maven Central. But on the other hand why I should put something more, when in my config i enable Similar solution should be implemented for Maven. And again I would like to vote to have possibliby for creating PRs with Gradle bump even if they are failing - this will give me an information that new Gradle version is out and I have to do something to bump it. |
That's true. And this is why Gradle in upgrade instructions is putting these two commands: ./gradlew wrapper --gradle-version=9.2.1 && ./gradlew wrapper |
The wrapper updater was only updating gradle-wrapper.properties instead of all 4 wrapper files (gradlew, gradlew.bat, gradle-wrapper.jar, and gradle-wrapper.properties). Changes: - Use ./gradlew instead of system gradle command - Run wrapper task twice (download distribution, then regenerate binaries) - Add binary file handling (binread/binwrite with Base64) - Set executable permissions (chmod 0o755) for gradlew scripts - Raise DependencyFileNotResolvable on wrapper task failure - Remove directory restriction from wrapper file selection - Return all dependency_files with wrapper files replaced Fixes: #13501
|
I just used copilot to fix some issues. Can you check the following Draft PR, if that is going to fix some of issues we have regarding updating wrapper? |
I think #13578 has some flaws. I did a quick review, and for instance, is picking Actually, just opened #13579 "fixing" the issues. I'll probably need to iterate it a bit more, but it seems it's working now. I plan to test it against a real repos too. |
|
@kbukum1 @gmazzo Thank you for putting the effort to move this forward Unfortunately, I am not sure where to send feedback now because we have two "competing changes" and it may be better if we can consolidate. Can we merge both into a single pull request? @kbukum1 Are we proceeding with an initial release where dependabot will fail if the upgrade is incomplete or should we wait for the gradle team first?
This should not be needed. See my comment |
@gmazzo, @yeikel – I'm good with proceeding using @gmazzo's PR. My draft was mainly to surface and explore some of the issues. For the OS selection, I agree—it's not quite handled properly, just a quick approach for demonstration. A couple things to consider:
Open to merging ideas if needed, but happy to consolidate on your PR if it looks solid! |
From what I tested, it seems to cache in |
|
For what it’s worth, I tested the a few of the main open-source tools used to keep the Gradle wrapper up to date.
For what is worth, in Renovate, it is a separate ecosystem This isn’t necessarily a reason for Dependabot to behave in any specific way, but it’s something worth keeping in mind. Personally, I prefer an approach like Renovate, where a pull request is created even if the generation fails, but only if a clear warning is included for the user within the pull request. I do not believe we should implement a solution that does not inform the user that Dependabot tried(and failed) to update the wrapper files as that will cause confusion and invalid defects Something like
|
Yes, it's cached at user's home. As @yeikel pointed out
It would be nice if someone can point me out if there is an API already to collaborate from a dependency update into the final PR body that I can use for this.
I'd wait for this PR first, to have a fully "working" approach
Great, less complexity then 💪 |
@gmazzo To clarify, this is my opinion and not a final design alignment. Let's wait for confirmation from the Dependabot team before discussing design or how to tackle this. @kbukum1 The sooner we can reach an agreement on the desired behavior, the better, so we can begin rolling this out. Any chance you can connect with @honeyankit /team this week? |
I think we can discuss & (hopefully implement that seperatelly). We may need to use our existing common notices to be able to add warning. I am saying that because PR creation is a common approach and we normally do not customize things directly from ecosystem to PR creation. However, I remember we did create notice approach so that any ecosystem can pass warnings into PR descriptions. Here is a PR that may help for that |
Thank you for the reference. Should we implement an initial version where it just fails if the wrapper update fails? We can create a follow up feature to allow partial updates + a warning |
|
I think that one more think is not implemented. https://docs.gradle.org/current/userguide/gradle_wrapper.html |
This is implemented already in this PR https://github.com/dependabot/dependabot-core/pull/13579/files#diff-25c6c1fa1ac7e0ccd7742c8143ec3bfead5f231bc84581ba1b01921c7ed212aaR106 |
I logged #13597 as a summary of the design decisions and next steps we can consider |
That's sounds better approach |
What are you trying to accomplish?
Fix a couples of issues introduced after #12891:
WrapperUpdaterfor every dependency bump, not sure the Gradle Distributions ones, and with the wrong Gradle version (as the random dependency's version is picked). Examples: #1, #2LockfileUpdaterandWrapperUpdaterupdaters: in the case we are bumping Gradle version, and the project does have.lockfiles, we want first to bump Gradle to pickup any possible change in the build logic or the content of the.lockfile, introduced by recent versions of Gradle.Anything you want to highlight for special attention from reviewers?
The first issue is critical for a correct implementation. The second, desirable, since
.lockfilefeature is stable and are not tied to the Gradle version itself.How will you know you've accomplished your goal?
Native updaters are not covered by unit tests. Since we have an open PR for adding a smoke tests for the Gradle Wrappers, we may consider this case in there as well by either:
gradle-wrapper-lockfilescenario (keepinggradle-lockfileandgradle-wrapperin isolation).lockfiless togradle-wrappergradle/gradle-wrapper.propertiestogradle-lockfileLet me know which one is better, in terms of isolation but also redundancy.
Checklist