Flatten the Bundler layout to the repository top level#9634
Conversation
hsbt
force-pushed
the
claude/exciting-wescoff-37e23d
branch
from
77ba7e9 to
49c0360
Compare
|
Thanks for working on this @hsbt . I don't know if it's a problem in my environment, but I can no longer run This is the error I get: BacktraceI also get a ton of "previous definition" warning: Backtrace....Previous definition of HighSecurity was here From what I'm seeing, it seems to be related to automatically adding
My $LOAD_PATH on master when 4. is called looks like this On this branch, my $LOAD_PATH looks like this: |
hsbt
force-pushed
the
claude/exciting-wescoff-37e23d
branch
from
49c0360 to
a875111
Compare
Flatten bundler/lib, bundler/exe and bundler/bundler.gemspec into the repository's top-level lib/, exe/ and bundler.gemspec to converge on the flat layout ruby/ruby already uses. Bundler is fully require_relative based, so the source move is mechanical and updates only the load-path references in .rspec, .rubocop.yml, Rakefile, the spec path helper and the CI workflows. The Bundler docs stay under bundler/ for now and move in a later commit.
setup_command builds and installs the default Bundler gem from the source tree. With Bundler flattened into the top-level lib/ and bundler.gemspec, the separate "bundler/lib" library and the chdir into bundler/ are gone: the single lib/ install now carries both RubyGems and Bundler, and the gem is built straight from the top-level gemspec.
Move bundler/{CHANGELOG,LICENSE,README}.md to the top level as
CHANGELOG-bundler.md, LICENSE-bundler.md and README-bundler.md so they
no longer collide with the RubyGems documents while keeping the
CHANGELOG*/LICENSE*/README* prefixes that license scanners rely on.
bundler/.document is dropped since lib/bundler/.document already excludes
the runtime tree from RDoc, leaving bundler/ empty.
The single-document direction stays for a later docs merge; this only
de-collides the file names.
The bundler/ subtree is now empty, so drop the dead `bundler/...` arm of the ProjectFiles exclude and regenerate Manifest.txt. The bundler files that rubygems-update ships move from bundler/lib, bundler/exe and bundler/*.md to their top-level locations; the file count is unchanged.
Update the Bundler vendor_lib targets and the corresponding patch headers from bundler/lib/bundler/vendor to lib/bundler/vendor. The Bundler:: namespace prefixes are untouched; unifying them under Gem:: is a later phase.
ruby/ruby's sync_default_gems.rb still copies Bundler sources from bundler/lib, bundler/exe and bundler/bundler.gemspec. Rewrite those source paths to the new top-level locations so the ruby-core sync keeps finding them; the ruby/ruby-side destinations are unchanged.
Catch the leftover bundler/lib and bundler/exe references after the move: the dev binstubs (bin/mdl, bin/rubocop, bin/test-unit), .gitattributes, .codespellrc, the release tool version file, the rubygems_ext/ci_detector cross-reference NOTE comments, and the developer documentation.
After flattening, Bundler shares lib/ with RubyGems, so spec subprocesses load the in-development Bundler from $LOAD_PATH instead of an installed gem, the same way ruby-core already does. Four examples assumed a separately installed Bundler and need updating. The load-order example pre-activates bundler unconditionally (was ruby_core? only) so Bundler.setup does not append the default bundler gem. "does not reveal system gems" relied on bundler showing up in installed_specs as an installed gem; flattened, bundler loads from $LOAD_PATH and no longer appears there. Assert the example's actual intent instead: the bundled gem stays visible and the system gem stays hidden across Gem.refresh. "bundle update --bundler" used `the_bundle.include_gems`, whose check runs `ruby -e "require 'bundler'; Bundler.setup"`; the in-development Bundler on $LOAD_PATH wins and its setup mismatches the locked version, so both the bundler and myrack checks fail. Verify the resolved versions through the binstub instead (`bundle --version` and `bundle list`), where the self-manager activates the locked Bundler before reading the bundle. "shows culprit file and line" double-loaded vendored Thor because the system binstub activated the installed Bundler while $LOAD_PATH carried the in-development one. Run it through the dev binstub so Bundler resolves from a single place.
After flattening, lib/ holds both RubyGems and Bundler, so any dev binstub that puts the worktree lib/ on $LOAD_PATH now overlays the already-booted system RubyGems with the worktree one. bin/bundle hit this through the Bundler gemspec activation: the partial overlay mixed a worktree Gem with an older system Gem::ConfigFile, breaking native extension installs on Ruby 3.2/3.3 (undefined install_extension_in_lib) and corrupting spec names elsewhere. The same overlay breaks bin/rake, bin/ronn, bin/rubocop and bin/mdl, which unshift the worktree lib/ (or require it through rubygems_ext) but boot on the system RubyGems. On a host whose RubyGems differs from the worktree copy this double-loads files like rubygems/package and raises (undefined method `spec=' for class Gem::Package::Old). Route all of them through switch_rubygems, like bin/rspec and bin/test-unit already do, so they re-exec onto the worktree RubyGems before anything is loaded. Point the shared gem home job at the worktree copy too, since the dev binstub can no longer load a separately installed RubyGems, and the matrix is redundant for now.
hsbt
force-pushed
the
claude/exciting-wescoff-37e23d
branch
from
a875111 to
f50eb42
Compare
|
@Edouard-chin This only reproduces in a clean environment where the gems aren't installed yet, so it slipped past CI and my local environment. I've now applied the same |
|
Confirmed it works with Do you think we could add this |
|
Good point. Some of these are also invoked outside of rake ( I'd like to keep this PR focused, so let me revisit consolidating the switch as a follow-up after it merges. |
ruby/rubygems flattened the Bundler tree from bundler/ up to the repository root, so repoint the rubygems mappings accordingly. Guard the gemspec fixup with File.exist? too, since the synthetic parent tree built for the flattening commit has no lib/bundler/bundler.gemspec yet. ruby/rubygems#9634 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ruby/rubygems flattened the Bundler tree from bundler/ up to the repository root, so repoint the rubygems mappings accordingly. Guard the gemspec fixup with File.exist? too, since the synthetic parent tree built for the flattening commit has no lib/bundler/bundler.gemspec yet. ruby/rubygems#9634 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2 participants
This is the first phase of converging RubyGems and Bundler on a single layout. Bundler currently lives under bundler/ with its own lib, exe, gemspec, and docs, while ruby/ruby already vendors Bundler flattened at the repository top level. The two trees have to be reconciled by hand on every sync. Moving Bundler's runtime tree, executables, gemspec, and docs up to the root removes that divergence and lets a later phase share code directly between the two products.
Bundler is entirely require_relative based, so the source move itself is mechanical. The rest of the change repoints what referenced the old paths. That covers the gemspec file list and metadata URIs, the .rspec and .rubocop load paths, the Rakefile man and gem-build tasks, gem update --system, the automatiek vendoring config, the ruby-core sync shim, and the CI workflows. The docs keep distinct names at the root, CHANGELOG-bundler.md and friends, so they do not collide with the RubyGems ones.
Flattening puts Bundler on $LOAD_PATH next to RubyGems, so a few specs that assumed a separately installed Bundler are adjusted to match the flat layout.