Skip to content

Commit 4231fc8

Browse files
casey-robertson-paypalJamieMageethavaahariharangit
authored
go_modules: Add go.work workspace support (#14909)
* Add go.work workspace support to Go modules updater Dependabot can now detect and process go.work files, enabling dependency updates across multi-module Go workspaces. - FileFetcher: detect go.work, fetch all workspace module go.mod/go.sum - FileParser: parse dependencies from all workspace modules; DependencySet handles deduplication across modules automatically - FileUpdater: dispatch to workspace vs. single-module update paths - GoModUpdater: run `go get dep@version` in each module directory so every go.mod that requires the dep gets updated; follow with `go work sync` then per-module `go mod tidy -e` - GoWorkParser: shared utility for parsing `use` directives from go.work content, including inline comment stripping and `use .` (root module) - PackageDetailsFetcher: fall back to any sub-module go.mod when no root go.mod exists - ecosystem_versions: read Go version from go.work or first workspace module when no root go.mod is present Fixes #6012 * Add specs and fixtures for go.work workspace support - GoWorkParser: unit tests for block/single-line use directives, inline comments, deduplication, bare `use .`, empty content, toolchain and godebug directives - FileFetcher: workspace with root module, workspace with no root go.mod, root-only workspace (use .) - FileParser: workspace dependency parsing and cross-module deduplication - FileUpdater: workspace update dispatch, all-module go get, go work sync failure propagation, no-root-mod workspace - Fixtures: workspace/ (root + libs + services), workspace_no_root_mod/ (api + worker), workspace_root_only/ (use . only) * Fix Sorbet, lint, and ecosystem_versions regressions caught by CI - Restore T.nilable in T.let memoization for @all_go_mods and @updated_workspace_module_files — required Sorbet idiom for ||= memoized ivars not declared in initialize - Revert go_version_from_file regex to \d+\.\d+ (major.minor only) to match original ecosystem_versions behaviour and fix e2e tests - Wrap two over-length lines in file_parser_spec.rb - Use %w() literals and explicit subject in go_work_parser_spec.rb - Add file_updater_spec.rb to RSpec/AnyInstance todo exclusion list (same treatment as bundler, bun, and other ecosystems) * Remove go.env and .ruby-version (local development artifacts) * Address Copilot review feedback - Reject absolute paths and traversals in GoModUpdater#workspace_module_paths as a defence-in-depth guard (paths already validated by FileFetcher, but worth being explicit in the updater too) - Pass mod_file.path to handle_parser_error when parsing workspace modules so DependencyFileNotParseable points at the failing go.mod, not go.work - Fetch go.work.sum when present; return updated version after go work sync * chore: bump go_work_parser.rb to typed: strong Sorbet typing mode check (spoom) detected all methods have complete signatures, qualifying for the stricter strong level. * fix: select go.mod by requirement file in PackageDetailsFetcher In workspace repos there is no root go.mod, so the fallback to the first nested go.mod could apply exclude directives from an unrelated module. For direct dependencies, use the requirement file path to find the correct go.mod. Indirect deps (empty requirements) retain the existing fallback behaviour. * fix: align fetch/parse scope with go.work use directives Previously, the fetcher unconditionally appended the root go.mod/go.sum whenever they existed, even in workspace repos where go.work does not include ".". The parser then ingested those out-of-scope modules via all_go_mods, surfacing dependencies that the updater would never touch. Fix the fetcher so workspace mode is fully handled by workspace_module_files (which now fetches "go.mod"/"go.sum" directly for the root "." path when present in the use list) and removes the unconditional root fallback. Fix the parser so all_go_mods derives its list from GoWorkParser.use_paths in workspace mode, keeping parse scope in lockstep with update scope. Addresses maintainer feedback from @thavaahariharangit. * fix: harden workspace_module_paths against missing dirs and false-positive path rejection - Replace substring `include?("..")` check with Pathname.cleanpath-based traversal guard, matching FileFetcher#valid_module_path? and avoiding false rejections on dirs whose names contain ".." (e.g. "tools..v2") - Filter use-paths against fetched go.mod dependency_files so Dir.chdir never runs against a module directory that wasn't fetched, preventing an unhandled Errno::ENOENT if go.work has a stale or missing use entry * fix: resolve RuboCop offenses in workspace_module_paths Extract valid_workspace_path? and workspace_mod_name helpers to bring Metrics/PerceivedComplexity back under threshold, and use to_set(&:name) to satisfy Style/MapToSet. * fix: fail fast when a go.work workspace module has a vendor directory Workspace vendoring is not yet supported — the workspace update path only updates go.mod/go.sum files and does not invoke vendor_updater, which would leave vendor/modules.txt and vendored sources stale. Detect vendor/modules.txt in any workspace module (root or submodule) before proceeding and raise DependencyFileNotResolvable with a clear message rather than silently producing an inconsistent vendor directory. * fix: wrap long line in check_workspace_not_vendored! to satisfy LineLength cop * fix: each File.join argument on its own line to satisfy multiline argument cops --------- Co-authored-by: Jamie Magee <jamagee@microsoft.com> Co-authored-by: Hariharan Thavachelvam <164553783+thavaahariharangit@users.noreply.github.com>
1 parent a7b8ee8 commit 4231fc8

33 files changed

Lines changed: 1143 additions & 43 deletions

File tree

.rubocop_todo.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ RSpec/AnyInstance:
5151
- 'maven/spec/dependabot/maven/metadata_finder_spec.rb'
5252
- 'npm_and_yarn/spec/dependabot/npm_and_yarn/update_checker_spec.rb'
5353
- 'python/spec/dependabot/python/file_updater_spec.rb'
54+
- 'go_modules/spec/dependabot/go_modules/file_updater_spec.rb'
5455
- 'updater/spec/dependabot/dependency_change_builder_spec.rb'
5556
- 'updater/spec/dependabot/file_fetcher_command_spec.rb'
5657

go_modules/lib/dependabot/go_modules/file_fetcher.rb

Lines changed: 106 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
require "sorbet-runtime"
55
require "dependabot/file_fetchers"
66
require "dependabot/file_fetchers/base"
7+
require "dependabot/go_modules/go_work_parser"
78

89
module Dependabot
910
module GoModules
@@ -13,41 +14,68 @@ class FileFetcher < Dependabot::FileFetchers::Base
1314

1415
sig { override.params(filenames: T::Array[String]).returns(T::Boolean) }
1516
def self.required_files_in?(filenames)
16-
filenames.include?("go.mod")
17+
filenames.include?("go.mod") || filenames.include?("go.work")
1718
end
1819

1920
sig { override.returns(String) }
2021
def self.required_files_message
21-
"Repo must contain a go.mod."
22+
"Repo must contain a go.mod or go.work."
2223
end
2324

2425
sig { override.returns(T::Hash[Symbol, T.untyped]) }
2526
def ecosystem_versions
27+
version = go_version_from_file(go_mod) ||
28+
go_version_from_file(go_work) ||
29+
all_workspace_go_mods.filter_map { |f| go_version_from_file(f) }.first ||
30+
"unknown"
31+
2632
{
2733
package_managers: {
28-
"gomod" => go_mod&.content&.match(/^go\s(\d+\.\d+)/)&.captures&.first || "unknown"
34+
"gomod" => version
2935
}
3036
}
3137
end
3238

3339
sig { override.returns(T::Array[DependencyFile]) }
3440
def fetch_files
35-
# Ensure we always check out the full repo contents for go_module
36-
# updates.
37-
SharedHelpers.in_a_temporary_repo_directory(
38-
directory,
39-
clone_repo_contents
40-
) do
41-
fetched_files = go_mod ? [go_mod] : []
42-
# Fetch the (optional) go.sum
43-
fetched_files << T.must(go_sum) if go_sum
44-
fetched_files << T.must(go_env) if go_env
41+
SharedHelpers.in_a_temporary_repo_directory(directory, clone_repo_contents) do
42+
fetched_files = collect_dependency_files
43+
validate_files!(fetched_files)
4544
fetched_files
4645
end
4746
end
4847

4948
private
5049

50+
sig { returns(T::Array[DependencyFile]) }
51+
def collect_dependency_files
52+
fetched_files = T.let([], T::Array[DependencyFile])
53+
54+
if go_work
55+
fetched_files << T.must(go_work)
56+
fetched_files << T.must(go_work_sum) if go_work_sum
57+
fetched_files.concat(workspace_module_files)
58+
else
59+
fetched_files << T.must(go_mod) if go_mod
60+
fetched_files << T.must(go_sum) if go_sum
61+
end
62+
63+
fetched_files << T.must(go_env) if go_env
64+
65+
fetched_files
66+
end
67+
68+
sig { params(files: T::Array[DependencyFile]).void }
69+
def validate_files!(files)
70+
return if files.any? { |f| f.name.end_with?("go.mod") }
71+
72+
error_msg = go_work ? "No go.mod files found in workspace" : "No go.mod files found"
73+
raise Dependabot::DependencyFileNotFound.new(
74+
"go.mod",
75+
error_msg
76+
)
77+
end
78+
5179
sig { returns(T.nilable(Dependabot::DependencyFile)) }
5280
def go_mod
5381
@go_mod ||= T.let(fetch_file_if_present("go.mod"), T.nilable(Dependabot::DependencyFile))
@@ -58,13 +86,78 @@ def go_sum
5886
@go_sum ||= T.let(fetch_file_if_present("go.sum"), T.nilable(Dependabot::DependencyFile))
5987
end
6088

89+
sig { returns(T.nilable(Dependabot::DependencyFile)) }
90+
def go_work_sum
91+
@go_work_sum ||= T.let(fetch_file_if_present("go.work.sum"), T.nilable(Dependabot::DependencyFile))
92+
end
93+
6194
sig { returns(T.nilable(Dependabot::DependencyFile)) }
6295
def go_env
6396
return @go_env if defined?(@go_env)
6497

6598
@go_env = T.let(fetch_support_file("go.env"), T.nilable(Dependabot::DependencyFile))
6699
@go_env
67100
end
101+
102+
sig { params(file: T.nilable(Dependabot::DependencyFile)).returns(T.nilable(String)) }
103+
def go_version_from_file(file)
104+
file&.content&.match(/^go\s+(\d+\.\d+)/)&.captures&.first
105+
end
106+
107+
sig { returns(T::Array[Dependabot::DependencyFile]) }
108+
def all_workspace_go_mods
109+
return [] unless go_work
110+
111+
workspace_module_paths.filter_map do |module_path|
112+
name = module_path == "." ? "go.mod" : File.join(module_path, "go.mod")
113+
fetch_file_if_present(name)
114+
end
115+
end
116+
117+
sig { returns(T.nilable(Dependabot::DependencyFile)) }
118+
def go_work
119+
@go_work ||= T.let(fetch_file_if_present("go.work"), T.nilable(Dependabot::DependencyFile))
120+
end
121+
122+
sig { returns(T::Array[String]) }
123+
def workspace_module_paths
124+
return [] unless go_work
125+
126+
content = T.must(T.must(go_work).content)
127+
GoWorkParser.use_paths(content)
128+
.select { |p| valid_module_path?(p) }
129+
end
130+
131+
sig { params(path: String).returns(T::Boolean) }
132+
def valid_module_path?(path)
133+
return false if path.empty?
134+
return false if Pathname.new(path).absolute?
135+
return false if path.include?("\0")
136+
137+
clean = Pathname.new(path).cleanpath.to_s
138+
return false if clean.start_with?("../")
139+
140+
true
141+
end
142+
143+
sig { returns(T::Array[DependencyFile]) }
144+
def workspace_module_files
145+
files = T.let([], T::Array[DependencyFile])
146+
147+
workspace_module_paths.each do |module_path|
148+
mod_name = module_path == "." ? "go.mod" : File.join(module_path, "go.mod")
149+
mod_file = fetch_file_if_present(mod_name)
150+
next unless mod_file
151+
152+
files << mod_file
153+
154+
sum_name = module_path == "." ? "go.sum" : File.join(module_path, "go.sum")
155+
sum_file = fetch_file_if_present(sum_name)
156+
files << sum_file if sum_file
157+
end
158+
159+
files
160+
end
68161
end
69162
end
70163
end

go_modules/lib/dependabot/go_modules/file_parser.rb

Lines changed: 101 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
require "open3"
77
require "dependabot/dependency"
88
require "dependabot/file_parsers/base/dependency_set"
9+
require "dependabot/go_modules/go_work_parser"
910
require "dependabot/go_modules/path_converter"
1011
require "dependabot/go_modules/replace_stubber"
1112
require "dependabot/errors"
@@ -52,11 +53,13 @@ def initialize(
5253
def parse
5354
dependency_set = Dependabot::FileParsers::Base::DependencySet.new
5455

55-
required_packages.each do |hsh|
56-
unless skip_dependency?(hsh) # rubocop:disable Style/Next
56+
if workspace?
57+
parse_workspace_dependencies(dependency_set)
58+
else
59+
required_packages.each do |hsh|
60+
next if skip_dependency?(hsh)
5761

58-
dep = dependency_from_details(hsh)
59-
dependency_set << dep
62+
dependency_set << dependency_from_details(hsh)
6063
end
6164
end
6265

@@ -190,9 +193,98 @@ def go_env
190193
@go_env ||= T.let(get_original_file("go.env"), T.nilable(Dependabot::DependencyFile))
191194
end
192195

196+
sig { returns(T.nilable(Dependabot::DependencyFile)) }
197+
def go_work
198+
@go_work ||= T.let(get_original_file("go.work"), T.nilable(Dependabot::DependencyFile))
199+
end
200+
201+
sig { returns(T::Boolean) }
202+
def workspace?
203+
!go_work.nil?
204+
end
205+
206+
sig { returns(T::Array[Dependabot::DependencyFile]) }
207+
def all_go_mods
208+
@all_go_mods ||= T.let(
209+
if go_work
210+
workspace_mod_names = GoWorkParser.use_paths(T.must(T.must(go_work).content)).map do |path|
211+
path == "." ? "go.mod" : "#{path}/go.mod"
212+
end
213+
dependency_files.select { |f| workspace_mod_names.include?(f.name) }
214+
else
215+
dependency_files.select { |f| f.name.end_with?("go.mod") }
216+
end,
217+
T.nilable(T::Array[Dependabot::DependencyFile])
218+
)
219+
end
220+
221+
sig { params(dependency_set: Dependabot::FileParsers::Base::DependencySet).void }
222+
def parse_workspace_dependencies(dependency_set)
223+
all_go_mods.each do |mod_file|
224+
parse_single_module(mod_file).each do |dep|
225+
dependency_set << dep
226+
end
227+
end
228+
end
229+
230+
sig { params(mod_file: Dependabot::DependencyFile).returns(T::Array[Dependabot::Dependency]) }
231+
def parse_single_module(mod_file)
232+
SharedHelpers.in_a_temporary_directory do |path|
233+
File.write("go.mod", mod_file.content)
234+
235+
command = "go mod edit -json"
236+
stdout, stderr, status = Open3.capture3(command)
237+
handle_parser_error(path, stderr, file_path: mod_file.path) unless status.success?
238+
239+
parsed = JSON.parse(stdout)
240+
packages = parsed["Require"] || []
241+
242+
packages.filter_map do |hsh|
243+
next if skip_dependency_in_manifest?(hsh, parsed)
244+
245+
source = { type: "default", source: hsh["Path"] }
246+
version = hsh["Version"]&.sub(/^v?/, "")
247+
248+
reqs = [{
249+
requirement: hsh["Version"],
250+
file: mod_file.name,
251+
source: source,
252+
groups: []
253+
}]
254+
255+
Dependency.new(
256+
name: hsh["Path"],
257+
version: version,
258+
requirements: hsh["Indirect"] ? [] : reqs,
259+
package_manager: "go_modules"
260+
)
261+
end
262+
end
263+
end
264+
265+
sig { params(dep: T::Hash[String, T.untyped], mod_manifest: T::Hash[String, T.untyped]).returns(T::Boolean) }
266+
def skip_dependency_in_manifest?(dep, mod_manifest)
267+
return true if dependency_is_replaced_in?(dep, mod_manifest)
268+
269+
path_uri = URI.parse("https://#{dep['Path']}")
270+
!path_uri.host&.include?(".")
271+
rescue URI::InvalidURIError
272+
false
273+
end
274+
275+
sig { params(details: T::Hash[String, T.untyped], mod_manifest: T::Hash[String, T.untyped]).returns(T::Boolean) }
276+
def dependency_is_replaced_in?(details, mod_manifest)
277+
return false unless mod_manifest["Replace"]
278+
279+
mod_manifest["Replace"].any? do |replace|
280+
replace["Old"]["Path"] == details["Path"] &&
281+
(!replace["Old"]["Version"] || replace["Old"]["Version"] == details["Version"])
282+
end
283+
end
284+
193285
sig { override.void }
194286
def check_required_files
195-
raise "No go.mod!" unless go_mod
287+
raise "No go.mod or go.work!" unless go_mod || go_work
196288
end
197289

198290
sig { params(details: T::Hash[String, T.untyped]).returns(Dependabot::Dependency) }
@@ -264,10 +356,11 @@ def go_mod_content
264356
end
265357
end
266358

267-
sig { params(path: T.any(Pathname, String), stderr: String).returns(T.noreturn) }
268-
def handle_parser_error(path, stderr)
359+
sig { params(path: T.any(Pathname, String), stderr: String, file_path: T.nilable(String)).returns(T.noreturn) }
360+
def handle_parser_error(path, stderr, file_path: nil)
269361
msg = stderr.gsub(path.to_s, "").strip
270-
raise Dependabot::DependencyFileNotParseable.new(T.must(go_mod).path, msg)
362+
resolved_path = file_path || go_mod&.path || go_work&.path || "go.mod"
363+
raise Dependabot::DependencyFileNotParseable.new(resolved_path, msg)
271364
end
272365

273366
sig { params(dep: T::Hash[String, T.untyped]).returns(T::Boolean) }

0 commit comments

Comments
 (0)