Skip to content

Add no-change diagnostics for npm/yarn/pnpm lockfile updates#15068

Open
robaiken wants to merge 6 commits into
mainfrom
robaiken/metrics-for-nochange
Open

Add no-change diagnostics for npm/yarn/pnpm lockfile updates#15068
robaiken wants to merge 6 commits into
mainfrom
robaiken/metrics-for-nochange

Conversation

@robaiken
Copy link
Copy Markdown
Contributor

@robaiken robaiken commented May 19, 2026

What are you trying to accomplish?

Make NoChangeError debuggable for npm/yarn/pnpm. Today it surfaces as unknown_error with no detail on which native command ran, whether the fallback was tried, or what the package manager printed.

This PR adds:

  • A new CommandTrace that wraps every native npm/yarn/pnpm invocation and records timing, success, and truncated stdout/stderr.
  • Per-command traces on NoChangeError#error_context, plus reason, commands_succeeded, fallback_attempted, fallback_succeeded.
  • A new updater.no_change metric (tagged by package manager + the four fields above) and a structured no_change_error error-type sent to the backend.
  • A one-line summary per command in the job log (truncated stdout/stderr at debug level).

Anything you want to highlight for special attention from reviewers?

  • The NoChangeError → no_change_error mapping lives in ErrorHandler#no_change_error_details, not in common/, so common/ stays ecosystem-agnostic. It matches by class-name suffix, which also picks up the existing Dependabot::Bun::FileUpdater::NoChangeError.
  • Stdout/stderr are capped at 4 KiB and error messages at 2 KiB to bound the payload.
  • Metric tags are all low-cardinality strings (no dep names, versions, or repo info).
  • NoChangeError's raise behavior and sentry_context are unchanged.

How will you know you've accomplished your goal?

  • New + existing specs pass (command_trace_spec, error_handler_spec, npm/yarn lockfile no-op specs), srb tc and rubocop clean.
  • Post-deploy: updater.no_change metric appears in dashboards broken down by reason/fallback, and no-change jobs get a No-change diagnostics: block in their logs.

Checklist

  • I have run the complete test suite to ensure all tests and linters pass.
  • I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
  • I have written clear and descriptive commit messages.
  • I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
  • I have ensured that the code is well-documented and easy to understand.

@robaiken robaiken self-assigned this May 20, 2026
robaiken added 4 commits May 20, 2026 13:13
Captures per-command traces (timing, success, truncated stdout/stderr,
content-changed flag) across every native npm/yarn/pnpm invocation in the
three lockfile updaters and surfaces them via NoChangeError#error_context.

- New CommandTrace class wraps native commands and records diagnostics.
- npm/yarn/pnpm lockfile updaters expose @command_traces; file_updater
  aggregates them into NoChangeError#error_context with reason,
  commands_succeeded, fallback_attempted, fallback_succeeded.
- Dependabot.updater_error_details maps NoChangeError (by class name) to
  error-type 'no_change_error' with the structured detail hash.
- ErrorHandler emits updater.no_change metric with low-cardinality tags
  and logs a structured summary line per command trace (debug-level dump
  of truncated stdout/stderr).

Raise behavior of NoChangeError is unchanged.
Keeps the common layer ecosystem-agnostic. The updater already loads
every ecosystem gem, so the structured 'no_change_error' payload is now
produced in ErrorHandler#no_change_error_details via a class-name match
(works for both Dependabot::NpmAndYarn::FileUpdater::NoChangeError and
Dependabot::Bun::FileUpdater::NoChangeError, which already exists).
The metric/log emission paths are unchanged.
…andTrace.record

Sorbet's block: sig requires the &block param to be named 'block', and
the convention in this repo is to invoke it via block.call with an
inline disable (matches bun/npm_and_yarn constraint_helper.rb).
…aces

Promotes the npm/yarn/pnpm-only Dependabot::NpmAndYarn::FileUpdater::CommandTrace
to Dependabot::SharedHelpers::CommandTrace. The class only depends on
SharedHelpers::HelperSubprocessFailed (already in common), so any
ecosystem can now record per-command diagnostics with the same wrapper.

A lightweight Dependabot::NpmAndYarn::FileUpdater::CommandTrace = ...
alias keeps existing references compiling without churn.
@robaiken robaiken force-pushed the robaiken/metrics-for-nochange branch from dc1519c to 40a5f6a Compare May 20, 2026 12:13
@robaiken robaiken marked this pull request as ready for review May 20, 2026 13:48
@robaiken robaiken requested a review from a team as a code owner May 20, 2026 13:48
Copilot AI review requested due to automatic review settings May 20, 2026 13:48
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds structured diagnostics for npm/yarn/pnpm “no change” lockfile update failures so they’re observable (backend error details + metric + logs) instead of surfacing as an unhelpful unknown_error.

Changes:

  • Introduces Dependabot::SharedHelpers::CommandTrace and wraps native helper invocations to record timing/success and truncated output.
  • Enhances npm_and_yarn NoChangeError#error_context with command traces and fallback metadata.
  • Updates the updater ErrorHandler to map NoChangeError to no_change_error, emit a new updater.no_change metric, and log a diagnostics block.
Show a summary per file
File Description
updater/spec/dependabot/updater/error_handler_spec.rb Adds coverage for NoChangeErrorno_change_error mapping, metric emission, and logging.
updater/lib/dependabot/updater/error_handler.rb Maps ecosystem NoChangeError to structured payload, emits updater.no_change, logs trace summaries.
npm_and_yarn/spec/dependabot/npm_and_yarn/file_updater/yarn_lockfile_updater_spec.rb Asserts yarn updater collects traces across primary + fallback flows.
npm_and_yarn/spec/dependabot/npm_and_yarn/file_updater/npm_lockfile_updater_spec.rb Asserts npm updater collects traces and records content_changed_after for fallback.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/yarn_lockfile_updater.rb Wraps yarn commands with CommandTrace.record and annotates content_changed_after.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/pnpm_lockfile_updater.rb Wraps pnpm commands/fallbacks with CommandTrace.record and annotates content_changed_after.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/npm_lockfile_updater.rb Wraps npm install/update/audit commands with CommandTrace.record and annotates content_changed_after.
npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater.rb Adds CommandTrace alias and enriches NoChangeError context with traces + fallback state.
common/spec/dependabot/shared_helpers/command_trace_spec.rb Adds unit tests for trace recording, truncation, and summaries.
common/lib/dependabot/shared_helpers/command_trace.rb New shared helper class implementing trace capture/truncation and serialization.

Copilot's findings

Comments suppressed due to low confidence (1)

npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/pnpm_lockfile_updater.rb:8

  • PnpmLockfileUpdater reopens Dependabot::NpmAndYarn::FileUpdater but doesn’t require "dependabot/npm_and_yarn/file_updater" first (unlike the npm/yarn lockfile updaters). This can create load-order issues and can lead to superclass-mismatch/partial-class problems if this file is required directly. Align with the established helper pattern by requiring the main file first (and rely on it to provide the CommandTrace alias).
# typed: strict
# frozen_string_literal: true

require "dependabot/shared_helpers/command_trace"
require "dependabot/npm_and_yarn/helpers"
require "dependabot/npm_and_yarn/package/registry_finder"
require "dependabot/npm_and_yarn/registry_parser"
require "dependabot/shared_helpers"
  • Files reviewed: 10/10 changed files
  • Comments generated: 5

Comment thread npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater.rb Outdated
Comment thread updater/lib/dependabot/updater/error_handler.rb Outdated
Comment thread updater/lib/dependabot/updater/error_handler.rb
Comment thread npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/pnpm_lockfile_updater.rb Outdated
markhallen
markhallen previously approved these changes May 20, 2026
Copy link
Copy Markdown
Contributor

@markhallen markhallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really cool. Nice work

Comment thread npm_and_yarn/lib/dependabot/npm_and_yarn/file_updater/pnpm_lockfile_updater.rb Outdated
- bun: expose error_context on NoChangeError so observability picks it up

- updater: emit "unknown" instead of empty-string for nil boolean metric tags via bool_tag helper

- npm_and_yarn: detect fallback traces by stable fingerprint to avoid false positives from dep names containing "audit"

- yarn lockfile updater: extract yarn berry audit-fix fallback into helper methods to drop Metrics/AbcSize + Metrics/MethodLength rubocop disables

- pnpm lockfile updater: extract fallback logic into maybe_run_pnpm_fallback to drop Metrics/AbcSize rubocop disable

- pnpm lockfile updater: add load-order require for dependabot/npm_and_yarn/file_updater
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants