Skip to content

Audit fix fallback#14589

Merged
robaiken merged 13 commits into
mainfrom
robaiken/audit-fix-fallback
Apr 27, 2026
Merged

Audit fix fallback#14589
robaiken merged 13 commits into
mainfrom
robaiken/audit-fix-fallback

Conversation

@robaiken
Copy link
Copy Markdown
Contributor

@robaiken robaiken commented Mar 31, 2026

What are you trying to accomplish?

What: Add an audit fix fallback for npm, yarn (berry), and pnpm when the standard subdependency update command is a no-op. When the normal update command (npm update, yarn up -R, pnpm update) doesn't change the lockfile — common for transitive dependencies not listed in any package.json in workspace repos — Dependabot now falls back to npm audit fix, yarn npm audit --fix, or pnpm audit --fix respectively.

Why: Dependabot currently fails to update vulnerable transitive dependencies in npm workspace repositories because the standard update commands only target packages listed in package.json. This leaves workspace repos unable to receive Dependabot PRs for subdependency security updates (e.g., picomatch). The audit fix commands can update these transitive dependencies directly in the lockfile. I believe that is more preferable to attempt to deliver these prs rather than erroring out like we currently do.

Anything you want to highlight for special attention from reviewers?

  1. Removes the ToolFeatureNotSupported error previously raised for pnpm transitive dependency updates, unblocking that path entirely.
  2. Appends "(via audit fix)" to PR titles when the fallback was used, so maintainers know the update mechanism.
  3. Gated behind the enable_audit_fix_fallback feature flag for safe rollout.

How will you know you've accomplished your goal?

With the feature flag enabled, workspace repos with vulnerable transitive dependencies will receive Dependabot PRs that update the lockfile via audit fix.

Checklist

  • I have run the complete test suite to ensure all tests and linters pass.
  • I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
  • I have written clear and descriptive commit messages.
  • I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
  • I have ensured that the code is well-documented and easy to understand.

@robaiken robaiken requested a review from a team as a code owner March 31, 2026 14:26
Copilot AI review requested due to automatic review settings March 31, 2026 14:26
@robaiken robaiken force-pushed the robaiken/audit-fix-fallback branch from b473d72 to 31c19b0 Compare March 31, 2026 14:29
@robaiken robaiken marked this pull request as draft March 31, 2026 14:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds fallback behavior for npm/yarn/pnpm subdependency updates where the “normal” update command is a no-op (especially in workspace repos), by running the relevant * audit * --fix command and improving diagnostic context when no files change.

Changes:

  • Add npm audit fix / yarn npm audit --fix / pnpm audit --fix fallback paths when subdependency update commands appear to make no lockfile changes.
  • Add new workspace fixtures (npm8, Yarn Berry, pnpm) and specs covering the fallback behavior.
  • Improve NoChangeError messages and Sentry context with a detected package manager value.

Reviewed changes

Copilot reviewed 15 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
npm_and_yarn/lib/dependabot/npm_and_yarn/update_checker/subdependency_version_resolver.rb Adds audit-fix fallbacks and “no-op” detection for yarn berry, pnpm, and npm8 subdependency resolution.
npm_and_yarn/lib/dependabot/npm_and_yarn/native_helpers.rb Adds helper wrappers to run npm audit fix, pnpm audit --fix, and yarn npm audit --fix.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/yarn_lockfile_updater.rb Adds Yarn Berry subdependency fallback to yarn npm audit --fix when the update appears to be a no-op.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/npm_lockfile_updater.rb Adds npm8 subdependency fallback to npm audit fix when the update appears to be a no-op.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater.rb Improves NoChangeError message/context with detected package manager; adjusts pnpm no-change handling.
npm_and_yarn/spec/dependabot/npm_and_yarn/update_checker/subdependency_version_resolver_spec.rb Adds specs for Yarn Berry and pnpm workspace subdependency fallback behavior.
npm_and_yarn/spec/dependabot/npm_and_yarn/file_updater/yarn_lockfile_updater_spec.rb Adds spec asserting Yarn Berry audit-fix fallback when update is a no-op.
npm_and_yarn/spec/dependabot/npm_and_yarn/file_updater/npm_lockfile_updater_spec.rb Adds spec asserting npm audit-fix fallback for workspace subdependency no-op updates.
npm_and_yarn/spec/dependabot/npm_and_yarn/file_updater_spec.rb Updates NoChangeError expectation and adds tests for package manager detection.
npm_and_yarn/spec/fixtures/projects/yarn_berry/workspace_subdependency_update/** New Yarn Berry workspace fixture (package.json + yarn.lock).
npm_and_yarn/spec/fixtures/projects/pnpm/workspace_subdependency_update/** New pnpm workspace fixture (package.json + pnpm-workspace.yaml + pnpm-lock.yaml).
npm_and_yarn/spec/fixtures/projects/npm8/workspace_subdependency_update/** New npm8 workspace fixture (package.json + package-lock.json).
Files not reviewed (2)
  • npm_and_yarn/spec/fixtures/projects/npm8/workspace_subdependency_update/package-lock.json: Language not supported
  • npm_and_yarn/spec/fixtures/projects/pnpm/workspace_subdependency_update/pnpm-lock.yaml: Language not supported

@robaiken robaiken marked this pull request as ready for review March 31, 2026 17:47
@thavaahariharangit
Copy link
Copy Markdown
Contributor

@robaiken

I ran the dependabot cli with this changes against the workflow where ToolFeatureNotSupported error is recreated (package manager: pnpm)

And I got below error:

updater | 2026/03/31 18:04:50 ERROR Error processing tar-fs (Dependabot::NpmAndYarn::FileUpdater::NoChangeError)
updater | 2026/03/31 18:04:50 ERROR No files were updated! Package manager: pnpm
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater.rb:52:in 'Dependabot::NpmAndYarn::FileUpdater#updated_dependency_files'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'UnboundMethod#bind_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'T::Private::Methods::CallValidation.validate_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/_methods.rb:259:in 'block in Dependabot::NpmAndYarn::FileUpdater#_on_method_added'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/lib/dependabot/dependency_change_builder.rb:155:in 'Dependabot::DependencyChangeBuilder#generate_dependency_files'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'UnboundMethod#bind_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'T::Private::Methods::CallValidation.validate_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/_methods.rb:259:in 'block in Dependabot::DependencyChangeBuilder#_on_method_added'
...

And summary says:

updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +--------------------------------------------+
updater | |       Dependencies failed to update        |
updater | +------------+---------------+---------------+
updater | | Dependency | Error Type    | Error Details |
updater | +------------+---------------+---------------+
updater | | tar-fs     | unknown_error | null          |
updater | +------------+---------------+---------------+
  proxy | 2026/03/31 18:04:51 2/85 calls cached (2%)

kbukum1
kbukum1 previously approved these changes Mar 31, 2026
@thavaahariharangit
Copy link
Copy Markdown
Contributor

@robaiken

I ran the dependabot cli with this changes against the workflow where ToolFeatureNotSupported error is recreated (package manager: pnpm)

And I got below error:

updater | 2026/03/31 18:04:50 ERROR Error processing tar-fs (Dependabot::NpmAndYarn::FileUpdater::NoChangeError)
updater | 2026/03/31 18:04:50 ERROR No files were updated! Package manager: pnpm
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater.rb:52:in 'Dependabot::NpmAndYarn::FileUpdater#updated_dependency_files'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'UnboundMethod#bind_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'T::Private::Methods::CallValidation.validate_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/_methods.rb:259:in 'block in Dependabot::NpmAndYarn::FileUpdater#_on_method_added'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/lib/dependabot/dependency_change_builder.rb:155:in 'Dependabot::DependencyChangeBuilder#generate_dependency_files'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'UnboundMethod#bind_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/call_validation.rb:282:in 'T::Private::Methods::CallValidation.validate_call'
updater | 2026/03/31 18:04:50 ERROR /home/dependabot/dependabot-updater/vendor/ruby/3.4.0/gems/sorbet-runtime-0.6.12977/lib/types/private/methods/_methods.rb:259:in 'block in Dependabot::DependencyChangeBuilder#_on_method_added'
...

And summary says:

updater | Dependabot encountered '1' error(s) during execution, please check the logs for more details.
updater | +--------------------------------------------+
updater | |       Dependencies failed to update        |
updater | +------------+---------------+---------------+
updater | | Dependency | Error Type    | Error Details |
updater | +------------+---------------+---------------+
updater | | tar-fs     | unknown_error | null          |
updater | +------------+---------------+---------------+
  proxy | 2026/03/31 18:04:51 2/85 calls cached (2%)

@robaiken provided the fix and I have tested the same and it's working as expected

pdater | 2026/04/01 10:41:27 INFO Finished job processing
updater | 2026/04/01 10:41:27 INFO Results:
updater | +------------------------------------------+
updater | |   Changes to Dependabot Pull Requests    |
updater | +---------+--------------------------------+
updater | | created | tar-fs ( from 3.1.0 to 3.1.2 ) |
updater | +---------+--------------------------------+
  proxy | 2026/04/01 10:41:27 26/118 calls cached (22%)

@thavaahariharangit
Copy link
Copy Markdown
Contributor

The standard package manager update commands (npm update, pnpm update, yarn up -R) often do nothing for transitive dependencies that aren't listed in any package.json, leaving the lockfile unchanged.

Core mechanism
The pattern is applied uniformly across all three package managers (npm, pnpm, yarn) in both the file updater and the subdependency version resolver:

  1. Snapshot the lockfile content before running the normal update command.
  2. Run the update npm update, pnpm update, yarn up -R).
  3. Compare the lockfile after — if it's unchanged and the enable_audit_fix_fallback feature flag is enabled, fall back to an audit fix command:
  • npm: npm audit fix --package-lock-only --ignore-scripts
  • pnpm: pnpm audit --fix (followed by pnpm install --lockfile-only)
  • yarn: yarn npm audit --fix --mode update-lockfile
  1. Gracefully handle failures — audit commands often exit non-zero even when they partially fix issues, so errors are rescued and logged.
  2. Tag the dependency with metadata[:audit_fix_used] = true so downstream code knows audit fix was used.

New helper methods
Three new methods in native_helpers.rb:

  • run_npm_audit_fix_command
  • run_pnpm_audit_fix_command
  • run_yarn_audit_fix_command

PR naming
In message_builder.rb:217, PRs created via audit fix get " (via audit fix)" appended to the title for transparency.

Pending:
RSpec failures needs to be addressed.
And I am happy with this approach

Copy link
Copy Markdown
Contributor

@thavaahariharangit thavaahariharangit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice approach! The audit-fix fallback is a solid way to handle those stubborn transitive deps in workspace repos. Left a few suggestions below — mostly around reducing the duplication and a couple of edge cases worth thinking about.

Comment thread npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/pnpm_lockfile_updater.rb Outdated
Comment thread common/lib/dependabot/pull_request_creator/message_builder.rb
Comment thread npm_and_yarn/lib/dependabot/npm_and_yarn/native_helpers.rb Outdated
@MattIPv4
Copy link
Copy Markdown

MattIPv4 commented Apr 1, 2026

  1. Removes the ToolFeatureNotSupported error previously raised for pnpm transitive dependency updates, unblocking that path entirely.

Will this allow Dependabot to properly utilise pnpm update <package> to update vulnerable transitive dependencies? This fallback is great, but I'm not sure it should be needed as the "solution" to pnpm, as pnpm update <package> should be capable by itself?

@karlhorky
Copy link
Copy Markdown

karlhorky commented Apr 1, 2026

@robaiken should this be marked as "Closes #13177" ?

@MattIPv4: as pnpm update <package> should be capable by itself?

I think you need the --depth flag. And maybe --recursive would be good for monorepos.

@robaiken robaiken force-pushed the robaiken/audit-fix-fallback branch from 5ff89e3 to 450210a Compare April 7, 2026 18:51
Comment thread npm_and_yarn/lib/dependabot/npm_and_yarn/native_helpers.rb Outdated
NativeHelpers.run_npm_audit_fix_command
sub_dependencies.each { |dep| dep.metadata[:audit_fix_used] = true }
rescue SharedHelpers::HelperSubprocessFailed
Dependabot.logger.info("npm audit fix failed or partially fixed — continuing with any changes made")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does audit fix raise error even when doing some updates. In that case is it make sense to have partial fix (if it raises errror)? I am just a little confused on this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal is to trigger security fixes no matter what, sometimes they are only partially fixed but this is still better than just failing

rescue SharedHelpers::HelperSubprocessFailed
Dependabot.logger.info(
"pnpm audit --fix failed or partially fixed — continuing with any changes made"
)
Copy link
Copy Markdown
Contributor

@kbukum1 kbukum1 Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question similar to the npm audit fix one asked above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is running pnpm audit fix, pnpm have their own version

rescue SharedHelpers::HelperSubprocessFailed
Dependabot.logger.info(
"yarn npm audit --fix failed or partially fixed — continuing with any changes made"
)
Copy link
Copy Markdown
Contributor

@kbukum1 kbukum1 Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question similar to the npm audit fix one asked above.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

@karlhorky correct — those --depth variants are not currently used. The pnpm commands in this PR / existing code are:

Without --depth Infinity, pnpm update won't traverse transitive deps, which is why the no-op happens and we drop into the audit-fix fallback. Trying pnpm update --depth Infinity <dep> (and pnpm -r --include-workspace-root update --depth Infinity <dep> for workspaces) before audit --fix would be a more targeted approach and would also sidestep the overrides write to package.json that audit --fix does. Happy to add that as a first-tier fallback ahead of audit --fix.

@MattIPv4
Copy link
Copy Markdown

Would be great to see that added, thank you! Assuming this lands while pnpm 10.x is around, I imagine the audit --fix fallback is going to annoying a lot of folks as it'll start trying to add overrides everywhere (I believe pnpm 11.x fixes this and switches the audit --fix behaviour to be essentially the same as update --depth Infinit -r <dep>).

robaiken and others added 13 commits April 27, 2026 17:24
…ckfile_updater_spec.rb

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
pnpm audit --fix adds overrides to package.json. Since run_pnpm_update
and run_pnpm_updater only return lockfile content, a manifest change
would produce inconsistent output. Snapshot package.json files before
the fallback and revert both manifest(s) and lockfile if any change is
detected.
Adds a first-tier fallback that runs pnpm update --depth Infinity <dep>
(with -r --include-workspace-root for workspaces) when the regular
update is a no-op. This updates transitive dependencies in the lockfile
without modifying any package.json (unlike pnpm audit --fix).

If --depth Infinity is also a no-op we fall through to the existing
audit --fix path.
@robaiken robaiken force-pushed the robaiken/audit-fix-fallback branch from 4b777e5 to 2f5ac87 Compare April 27, 2026 16:24
@robaiken robaiken merged commit 51f0bdc into main Apr 27, 2026
205 of 211 checks passed
@robaiken robaiken deleted the robaiken/audit-fix-fallback branch April 27, 2026 17:18
@karlhorky
Copy link
Copy Markdown

@robaiken @markhallen amazing that this got merged!

What is the release / rollout procedure for Dependabot changes?

I guess first of all it will show up as a new release on Releases? And then after some amount of time be available in regular Dependabot usage on https://github.com repos?

Looking forward to giving this a shot in a lot of different repos! And then after verification, closing my issue:

@robaiken
Copy link
Copy Markdown
Contributor Author

@karlhorky we are having some delays on rolling out this feature, I will keep you posted when we roll this out

@robaiken
Copy link
Copy Markdown
Contributor Author

robaiken commented May 6, 2026

@karlhorky The feature flag as been enabled

@karlhorky
Copy link
Copy Markdown

karlhorky commented May 6, 2026

@robaiken Great!

I'm not sure whether that means that this should be available for all repos on github.com now, but after using the comment @dependabot recreate on the broken PR karlhorky/electron-app-patcher#199, it did not create a commit with the new transitive dep picomatch@2.3.2:

I didn't dig into the logs on the workflow run yet.

@MattIPv4
Copy link
Copy Markdown

MattIPv4 commented May 6, 2026

👀 I just gave one of our open Dependabot vulns a re-run to see if things were better, but trying to update brace-expansion from 1.1.12 to 1.1.13 or beyond failed still: https://github.com/alveusgg/alveusgg/actions/runs/25448259396/job/74658207250

Running pnpm update -r brace-expansion locally updates it to 1.1.14 just fine 🤔

@robaiken
Copy link
Copy Markdown
Contributor Author

The feature flag has been rolled out to everyone now!

@MattIPv4
Copy link
Copy Markdown

https://github.com/alveusgg/alveusgg/actions/runs/25863233146/job/75998346176
https://github.com/alveusgg/alveusgg/pull/2091/changes

https://github.com/alveusgg/extension/actions/runs/25863257306/job/75998430099
https://github.com/alveusgg/extension/pull/500/changes

It works 🎉 I am incredibly grateful for you getting this out, Dependabot finally works with pnpm properly!!

@MattIPv4
Copy link
Copy Markdown

MattIPv4 commented May 14, 2026

@karlhorky
Copy link
Copy Markdown

karlhorky commented May 14, 2026

@robaiken thanks!

I tried it out by using @dependabot recreate in the same broken PR from my comment above and Dependabot decided to change the PR to upgrade multiple major versions instead, also without a useful diff 🤔

See the renaming of the PR here:

@dependabot changed the title from 'Bump picomatch from 2.3.1 to 2.3.2' to 'Bump picomatch from 2.3.1 to 4.0.3'

Maybe new picomatch vulnerabilities appeared in the meantime and Dependabot decided to pull those in too.

@karlhorky
Copy link
Copy Markdown

karlhorky commented May 14, 2026

On the other hand, this other Dependabot PR seems to do the right thing 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants