Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# Changelog

## 4.0.15 / 2026-06-24

### Enhancements:

* Reduce peak memory usage of full index loading and bundle install. Pull request [#9618](https://github.com/ruby/rubygems/pull/9618) by hsbt
* Installs bundler 4.0.15 as a default gem.

### Bug fixes:

* Forward security policy to old-format gems. Pull request [#9611](https://github.com/ruby/rubygems/pull/9611) by hsbt

## 4.0.14 / 2026-06-10

### Enhancements:
Expand Down
16 changes: 16 additions & 0 deletions bundler/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# Changelog

## 4.0.15 / 2026-06-24

### Enhancements:

* Resolve Git LFS files in git sources from the real remote. Pull request [#9632](https://github.com/ruby/rubygems/pull/9632) by hsbt
* Suggest access issues, not only yanking, for missing locked gems. Pull request [#9631](https://github.com/ruby/rubygems/pull/9631) by hsbt
* Implement a make jobserver (continuation of #9210). Pull request [#9625](https://github.com/ruby/rubygems/pull/9625) by hsbt
* Reduce peak memory usage of full index loading and bundle install. Pull request [#9618](https://github.com/ruby/rubygems/pull/9618) by hsbt
* Bump up to rb-sys 0.9.128. Pull request [#9569](https://github.com/ruby/rubygems/pull/9569) by hsbt

### Bug fixes:

* Skip the make jobserver on Windows. Pull request [#9630](https://github.com/ruby/rubygems/pull/9630) by hsbt
* Exempt lockfile versions from cooldown on every resolution path. Pull request [#9619](https://github.com/ruby/rubygems/pull/9619) by hsbt
* Set `Bundler.settings[:ssl_ca_cert]` to download gems. Pull request [#9610](https://github.com/ruby/rubygems/pull/9610) by junaruga

## 4.0.14 / 2026-06-10

### Bug fixes:
Expand Down
19 changes: 15 additions & 4 deletions bundler/lib/bundler/definition.rb
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,16 @@ def prefer_local!
sources.prefer_local!
end

# Releases memory only needed during resolution, such as remote spec
# indexes and resolver state. Only safe to call once resolution is
# complete and the result has been materialized, since any further
# resolution will need to refetch remote specs.
def release_resolution_memory!
@resolver = nil
@resolution_base = nil
sources.release_resolution_memory!
end

# For given dependency list returns a SpecSet with Gemspec of all the required
# dependencies.
# 1. The method first resolves the dependencies specified in Gemfile
Expand Down Expand Up @@ -688,9 +698,10 @@ def materialize(dependencies)
"available locally before rerunning Bundler."
else
"Your bundle is locked to #{locked_gem} from #{locked_gem.source}, but that version can " \
"no longer be found in that source. That means the author of #{locked_gem} has removed it. " \
"You'll need to update your bundle to a version other than #{locked_gem} that hasn't been " \
"removed in order to install."
"no longer be found in that source. That means either the author of #{locked_gem} has removed it, " \
"or you no longer have access to that source. You'll need to update your bundle to a version other " \
"than #{locked_gem} that hasn't been removed, or check your credentials and access rights for " \
"#{locked_gem.source}, in order to install."
end

raise GemNotFound, message
Expand Down Expand Up @@ -1283,7 +1294,7 @@ def unlocked_resolution_base

def new_resolution_base(last_resolve:, unlock:)
new_resolution_platforms = @current_platform_missing ? @new_platforms + [Bundler.local_platform] : @new_platforms
Resolver::Base.new(source_requirements, expanded_dependencies, last_resolve, @platforms, locked_specs: @originally_locked_specs, unlock: unlock, prerelease: gem_version_promoter.pre?, prefer_local: @prefer_local, new_platforms: new_resolution_platforms)
Resolver::Base.new(source_requirements, expanded_dependencies, last_resolve, @platforms, locked_specs: @originally_locked_specs, unlock: unlock, prerelease: gem_version_promoter.pre?, prefer_local: @prefer_local, new_platforms: new_resolution_platforms, explicit_unlocks: @explicit_unlocks)
end

def new_resolver(base)
Expand Down
4 changes: 4 additions & 0 deletions bundler/lib/bundler/fetcher.rb
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,10 @@ def api_fetcher?
fetchers.first.api_fetcher?
end

def release_resolution_memory!
@fetchers&.each(&:release_resolution_memory!)
end

def gem_remote_fetcher
@gem_remote_fetcher ||= begin
require_relative "fetcher/gem_remote_fetcher"
Expand Down
3 changes: 3 additions & 0 deletions bundler/lib/bundler/fetcher/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ def api_fetcher?
false
end

def release_resolution_memory!
end

private

def log_specs(&block)
Expand Down
13 changes: 13 additions & 0 deletions bundler/lib/bundler/fetcher/compact_index.rb
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,13 @@ def api_fetcher?
true
end

# The client holds the parsed checksums of all info files in the
# index. Dropping it is always safe because it is rebuilt from the
# local cache on demand.
def release_resolution_memory!
@compact_index_client = nil
end

private

def compact_index_client
Expand All @@ -73,6 +80,12 @@ def compact_index_client
end

def fetch_gem_infos(names)
# Create the client and update the versions file on this thread.
# Otherwise the workers race to lazily create the client and update
# the versions file concurrently, e.g. when the client was released
# after resolution and is being rebuilt for `bundle cache`.
compact_index_client.available?

in_parallel(names) {|name| compact_index_client.info(name) }
rescue TooManyRequestsError # rubygems.org is rate limiting us, slow down.
@bundle_worker&.stop
Expand Down
2 changes: 2 additions & 0 deletions bundler/lib/bundler/fetcher/gem_remote_fetcher.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ def initialize(*)
super

@pool_size = Bundler.settings.installation_parallelization
ssl_ca_cert = Bundler.settings[:ssl_ca_cert]
@cert_files << ssl_ca_cert if ssl_ca_cert
end

def request(*args)
Expand Down
8 changes: 7 additions & 1 deletion bundler/lib/bundler/installer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -195,7 +195,13 @@ def install(options)
force = options[:force]
local = options[:local] || options[:"prefer-local"]
jobs = Bundler.settings.installation_parallelization
spec_installations = ParallelInstaller.call(self, @definition.specs, jobs, standalone, force, local: local)
specs = @definition.specs
# Installing default gems may need the remote index again to cache
# their .gem files, so keep resolution memory around in that case.
# The bundler spec itself is excluded because it comes from the
# metadata source and never goes through that path.
@definition.release_resolution_memory! if specs.none? {|s| s.default_gem? && s.source.is_a?(Source::Rubygems) }
spec_installations = ParallelInstaller.call(self, specs, jobs, standalone, force, local: local)
spec_installations.each do |installation|
post_install_messages[installation.name] = installation.post_install_message if installation.has_post_install_message?
end
Expand Down
45 changes: 42 additions & 3 deletions bundler/lib/bundler/installer/parallel_installer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -110,10 +110,49 @@ def failed_specs
end

def install_with_worker
installed_specs = {}
enqueue_specs(installed_specs)
with_jobserver do
installed_specs = {}
enqueue_specs(installed_specs)

process_specs(installed_specs) until finished_installing?
process_specs(installed_specs) until finished_installing?
end
end

def with_jobserver
# The jobserver hands tokens to child `make` processes through MAKEFLAGS
# using the GNU make `--jobserver-auth` protocol. nmake, the default make
# on mswin, instead reads MAKEFLAGS as bare option letters and aborts
# every native extension build with `fatal error U1065: invalid option
# '-'`. Skip the jobserver when nmake is in use. Other Windows toolchains
# such as mingw use GNU make and keep working through the inherited pipe.
return yield if nmake?

begin
r, w = IO.pipe
r.close_on_exec = false
w.close_on_exec = false
w.write("*" * @size)

old_makeflags = ENV["MAKEFLAGS"]
ENV["MAKEFLAGS"] = [old_makeflags, "--jobserver-auth=#{r.fileno},#{w.fileno}"].compact.join(" ")

yield
ensure
# Restore MAKEFLAGS before closing the pipe so a close failure can't
# leave the process with descriptors that point at a closed pipe.
old_makeflags ? ENV["MAKEFLAGS"] = old_makeflags : ENV.delete("MAKEFLAGS")

r&.close
w&.close
end
end

# Mirror how RubyGems' extension builder picks the make program so the
# jobserver is only set up when a GNU-compatible make will consume it.
def nmake?
make = ENV["MAKE"] || ENV["make"]
make ||= "nmake" if RUBY_PLATFORM.include?("mswin")
/\bnmake/i.match?(make.to_s)
end

def install_serially
Expand Down
14 changes: 7 additions & 7 deletions bundler/lib/bundler/mirror.rb
Original file line number Diff line number Diff line change
Expand Up @@ -160,18 +160,18 @@ def replies?(mirror)

def wait_for_writtable_socket(socket, address, timeout)
if IO.select(nil, [socket], nil, timeout)
probe_writtable_socket(socket, address)
probe_writtable_socket(socket)
else # TCP Handshake timed out, or there is something dropping packets
false
end
end

def probe_writtable_socket(socket, address)
socket.connect_nonblock(address)
rescue Errno::EISCONN
true
rescue StandardError # Connection failed
false
def probe_writtable_socket(socket)
# Check the pending error on the socket rather than calling
# +connect_nonblock+ a second time. On BSD-based systems such as macOS a
# second connect returns +EISCONN+ even when the asynchronous connect
# failed, which would make a down mirror look reachable.
socket.getsockopt(Socket::SOL_SOCKET, Socket::SO_ERROR).int == 0
end
end
end
Expand Down
30 changes: 17 additions & 13 deletions bundler/lib/bundler/resolver.rb
Original file line number Diff line number Diff line change
Expand Up @@ -437,25 +437,29 @@ def cooldown_hint(specs)
def cooldown_excluded?(spec)
return false unless spec.respond_to?(:created_at) && spec.created_at
return false unless spec.respond_to?(:remote) && spec.remote
return false if pinned_by_lockfile_floor?(spec)
return false if locked_by_lockfile?(spec)
days = spec.remote.effective_cooldown
return false if days.nil? || days <= 0
(cooldown_now - spec.created_at) < (days * 86_400)
end

# A spec sitting exactly at a `>= locked_version` prevent-downgrade floor is
# the version the lockfile currently pins. `bundle update` and `bundle
# outdated` install that floor so resolution never moves a gem backwards.
# Filtering it out for cooldown would then make resolution impossible
# whenever the locked version is itself inside the cooldown window, which is
# exactly what happens to a lockfile written before cooldown was enabled.
# Keep it eligible; gems being explicitly updated carry an exact `=`
# requirement instead and stay subject to the cooldown filter.
def pinned_by_lockfile_floor?(spec)
# A version already written to the lockfile has been adopted, and cooldown
# only governs the adoption of *new* versions, so it must never retract one
# the lockfile already pins. Keying this off the locked specs rather than the
# prevent-downgrade floor matters because that floor is absent on resolutions
# that re-pick a gem from scratch: the auxiliary full update run to compute
# `--update` targets, and the from-scratch retries after a conflict unlocks a
# gem. In those passes the locked version is the only candidate, so filtering
# it out makes an unrelated operation impossible whenever every published
# version matching the requirement sits inside the cooldown window.
#
# Gems named on a `bundle update GEM` command are the exception: the user
# asked to move them, so they stay subject to cooldown and a locked-but-fresh
# release is pushed back to an older one (or fails loudly when none exists).
def locked_by_lockfile?(spec)
return false unless defined?(@base) && @base
requirement = base_requirements[spec.name]
return false unless requirement && !requirement.exact?
requirement.requirements.any? {|op, version| op == ">=" && version == spec.version }
return false if @base.explicitly_unlocked?(spec.name)
@base.locked_specs[spec.name].any? {|locked| locked.version == spec.version }
end

def cooldown_now
Expand Down
9 changes: 9 additions & 0 deletions bundler/lib/bundler/resolver/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ class Base
attr_reader :packages, :requirements, :source_requirements, :locked_specs

def initialize(source_requirements, dependencies, base, platforms, options)
@explicit_unlocks = options.delete(:explicit_unlocks) || []
@source_requirements = source_requirements
@locked_specs = options[:locked_specs]

Expand Down Expand Up @@ -43,6 +44,14 @@ def get_package(name)
@packages[name]
end

# Gems the user named on a `bundle update GEM` / `bundle lock --update GEM`
# command line. These are the only ones meant to move off their locked
# version, so cooldown keeps applying to them while every other locked gem
# stays exempt.
def explicitly_unlocked?(name)
@explicit_unlocks.include?(name)
end

def base_requirements
@base_requirements ||= build_base_requirements
end
Expand Down
35 changes: 34 additions & 1 deletion bundler/lib/bundler/rubygems_gem_installer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@

module Bundler
class RubyGemsGemInstaller < Gem::Installer
# Cap how many jobserver slots a single gem's `make` may grab so that one
# gem with many recipes doesn't starve the others sharing the pool. Beyond
# a handful of jobs the extra parallelism rarely pays off in practice.
MAX_JOBS_PER_GEM = 3

def check_executable_overwrite(filename)
# Bundler needs to install gems regardless of binstub overwriting
end
Expand Down Expand Up @@ -101,10 +106,18 @@ def generate_bin_script(filename, bindir)
end

def build_jobs
Bundler.settings[:jobs] || super
@jobserver_read_io&.read_nonblock(MAX_JOBS_PER_GEM, @jobserver_tokens)
acquired_jobs = @jobserver_tokens.empty? ? nil : @jobserver_tokens.size

acquired_jobs || Bundler.settings[:jobs] || super
rescue IO::WaitReadable, EOFError
1
end

def build_extensions
@jobserver_tokens = +""
@jobserver_read_io, @jobserver_write_io = connect_to_jobserver

extension_cache_path = options[:bundler_extension_cache_path]
extension_dir = spec.extension_dir
unless extension_cache_path && extension_dir
Expand All @@ -128,6 +141,11 @@ def build_extensions
FileUtils.cp_r extension_dir, extension_cache_path
end
end
ensure
unless @jobserver_tokens.empty?
@jobserver_write_io.write(@jobserver_tokens)
@jobserver_write_io.flush
end
end

def spec
Expand All @@ -144,6 +162,21 @@ def gem_checksum

private

def connect_to_jobserver
return unless ENV["MAKEFLAGS"]
# We append our own --jobserver-auth, so read the last one. Otherwise a
# parent jobserver's descriptors (e.g. `bundle install` run under
# `make -j`) would be picked up instead of the pool ParallelInstaller created.
read_fd, write_fd = ENV["MAKEFLAGS"].scan(/--jobserver-auth=(\d+),(\d+)/).last

return unless read_fd && write_fd

# Pass explicit modes. On POSIX, IO.new detects the descriptor's access
# mode, but on Windows it can't, so the write end would default to read
# mode and raise "IOError: not opened for writing" when releasing slots.
[IO.new(read_fd.to_i, "r", autoclose: false), IO.new(write_fd.to_i, "w", autoclose: false)]
end

def prepare_extension_build(extension_dir)
SharedHelpers.filesystem_access(extension_dir, :create) do
FileUtils.mkdir_p extension_dir
Expand Down
6 changes: 6 additions & 0 deletions bundler/lib/bundler/source/git/git_proxy.rb
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,12 @@ def copy_to(destination, submodules = false)
FileUtils.rm_rf(p)
end
git "clone", "--no-checkout", "--quiet", path.to_s, destination.to_s
# The copy is cloned from the local bare cache, which holds no Git LFS
# objects, so point origin back at the real remote and let git-lfs derive
# its endpoint from there when checking out. Use the credential-filtered
# URI to avoid persisting secrets in the copy's .git/config; auth is left
# to git's credential helper.
git "remote", "set-url", "origin", credential_filtered_uri, dir: destination
File.chmod((File.stat(destination).mode | 0o777) & ~File.umask, destination)
rescue Errno::EEXIST => e
file_path = e.message[%r{.*?((?:[a-zA-Z]:)?/.*)}, 1]
Expand Down
Loading
Loading