Versions file not updated for RC-releases

[sudoremo]

Describe the problem as clearly as you can

• Running gem generate_index -d … --update does not include new Gems with RC-versions (e.g. 1.0.0.rc0).
• Running with --update, in contrast to omitting the flag, creates an additional line for each new Gem version instead of altering the existing line. This may not be a bug though.

Did you try upgrading RubyGems?

Yes.

Post steps to reproduce the problem

I have added a new unit test in this fork branch.

Which command did you run?

gem generate_index -d … --update

What were you expecting to happen?

Pre-releases should also be added to the index. Otherwise, they cannot be installed.

What actually happened?

Pre-releases are not added to the index / versions file.

[sudoremo]

May I bump this issue? Many thanks for having a look at it.

[sudoremo]

The root cause is in Gem::Indexer#update_index in lib/rubygems/indexer.rb. The compact index update only receives released specs, not prerelease specs:

if @build_compact
  Gem.time "Updated compact index files" do
    files += update_compact_index released, @dest_directory, @directory
  end
end

Since build_compact defaults to true, and modern Bundler prefers compact index over legacy specs files, prerelease gems become invisible after an incremental --update even though they are correctly added to prerelease_specs.4.8.gz.

A workaround is to pass --no-compact with --update, forcing Bundler to fall back to the legacy index where prereleases are indexed correctly.

[simi]

Running with --update, in contrast to omitting the flag, creates an additional line for each new Gem version instead of altering the existing line. This may not be a bug though.

That's intended. Compact Index API benefits from appends for HTTP caching being able to effectively download only the missing bit.

IMHO best would be to make possible to generate only Compact Index API (#107), which is the most simple and most performant format today compatible with Bundler.

[sudoremo]

Thanks for the quick response @simi!

To clarify: the issue we're reporting is specifically about the compact index path, not the legacy Marshal format. In update_index, the call to update_compact_index only receives released specs, not prerelease. So prerelease gems are never added to the compact index files (versions, info/<gemname>) during an incremental --update. Since modern Bundler prefers the compact index, prerelease gems are effectively invisible after an incremental update — even though they are correctly added to prerelease_specs.4.8.gz.

We believe the fix would be to pass specs (both released and prerelease) instead of just released to update_compact_index. Would you agree with that assessment, or do you see a different approach?

[simi]

Yes, pre-releases belongs to compact index.

[sudoremo]

We've opened a PR with a fix and a test: #108

It would be wonderful if this could be reviewed and released in a timely manner. Thank you very much!

[sudoremo]

@simi Have you had a chance to look at the PR yet? We'd be really greatful for a fix. Many thanks for your time.