Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bundler should give a better error when the Gemfile itself activates a conflicting gem #7178

Open
Bandes opened this issue Nov 20, 2023 · 17 comments · Fixed by #7386
Open

Bundler should give a better error when the Gemfile itself activates a conflicting gem #7178

Bandes opened this issue Nov 20, 2023 · 17 comments · Fixed by #7386

Comments

@Bandes
Copy link

Bandes commented Nov 20, 2023

I have been running into a strange situation which I believe is due to differing versions of the built-in gem set in different docker images.

My application runs on ruby 3.0.6 currently. My CI is CircleCI, and I use the cimg/ruby:3.0.6-browsers which appears to use set:1.0.3 However my devops team is running my application on Docker, using ruby:3.0.6alpine3.16 which appears to use set:1.0.1

Whichever of those versions I specify in Gemfile.lock will cause an error similar to this:

`check_for_activated_spec!': You have already activated set 1.0.1, but your Gemfile requires set 1.0.3. Since set is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports set as a default gem. (Gem::LoadError)

Similar things have happened to me on other projects that use more up-to-date versions of ruby.

I would have expected that bundler would be able to force the use of the version of the gem in Gemfile.lock, it hadn't occurred to me that I would need to inspect which versions were built-in to my images.

@deivid-rodriguez
Copy link
Member

We should be supporting gemfied set since Bundler 2.2.8, which first included #4297. Would it be possible that you are using an older Bundler?

@Bandes
Copy link
Author

Bandes commented Nov 20, 2023

I don't think so. My Gemfile.lock was built with 2.4.22

RUBY VERSION
   ruby 3.0.6p216

BUNDLED WITH
   2.4.22

@deivid-rodriguez
Copy link
Member

Not a good guess then, thanks for confirming.

Please provide repro steps so that we can take a look at this.

@Bandes
Copy link
Author

Bandes commented Nov 20, 2023

I'm having trouble finding ways to describe how to reproduce the issue without linking to our specific code or our specific images. I do have this which shows the docker versions we are using - the base image has 2.2.33 which we are updating to 2.4.22 before we bundle install

╰─ docker run --rm -it public.ecr.aws/docker/library/ruby:3.0.6-alpine3.16 sh                                                                                                                                                                                                                                                                                          ─╯
/ # bundle --version
Bundler version 2.2.33
/ # gem install bundler:2.4.22
Fetching bundler-2.4.22.gem
Successfully installed bundler-2.4.22
1 gem installed
/ # bundle --version
Bundler version 2.4.22
/ # gem list | grep set
set (default: 1.0.1)

@deivid-rodriguez
Copy link
Member

Maybe extracting only dependency manifests to a separate repository, removing your private code and private gems, and see if it still reproduces?

@indirect
Copy link
Member

I have suddenly started seeing this problem on a new machine, but I'm having trouble creating another copy of the repo with the same problem. The symptom I can observe is this error during Bundler.setup:

❯ b ruby -e "puts 'hi'"
/Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated date 3.3.4, but your Gemfile requires date 3.3.3. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:25:in `block in setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `map'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler.rb:162:in `setup'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `block in <top (required)>'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:159:in `with_level'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:111:in `silence'
        from /Users/andre/.gem/ruby/3.2.2/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `<top (required)>'
        from <internal:/Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /Users/andre/.rubies/ruby-3.2.2/lib/ruby/3.2.0/rubygems.rb:1370:in `<top (required)>'
        from <internal:gem_prelude>:2:in `require'
        from <internal:gem_prelude>:2:in `<internal:gem_prelude>'

The error only appears when installing to system gems, so I can make it appear and disappear repeatedly by running bundle config path .bundle (gone) and then bundle config unset path (error returns).

My system gems include both the locked version of date and a newer version:

❯ gem list date

*** LOCAL GEMS ***

date (3.3.4, default: 3.3.3)

@indirect
Copy link
Member

oh no 🤦🏻 I figured it out. my Rails app contains a (relatively) complicated gemfile that loads some ruby to evaluate whether to include other gems. The "configuration" class has this at the top:

require "erb"
require "open-uri"
require "psych"
require "set"

Turns out open-uri requires time, which requires date. So ultimately what this boils down to is exactly what the error message says: somehow, you have a newer version of this gem installed, and merely evaluating your Gemfile has required a version that does not match your lock. So Bundler explodes when it gets around to trying to load your locked version of the gem (whether that gem is set or date).

We should probably turn this into one of the advanced Bundler error messages, and say something like this:

Your Gemfile is locked to date version 3.3.3. When Bundler tried to require date, it discovered that other Ruby code had already required date version 3.3.4. As a result, it is impossible to set up your Gemfile.

This is most likely caused by your Gemfile running Ruby code that requires gems, which is not supported. You will need to remove whatever is inside your Gemfile requiring gems, so that Bundler can take care of loading the versions you need.

Maybe this is also an opportunity for us to overwrite require to raise an exception while we eval Gemfiles? At least then it would fail early, and let users know exactly where the bad code is. 😅 cc @deivid-rodriguez @martinemde

@indirect
Copy link
Member

The good news is that figuring all of this out suggests a workaround:

  1. bundle config path .bundle to create a separate space for gems just for the given app.
  2. bundle install to install gems just for this app, without any newer gems available to be required while evaluating the gemfile.
  3. bundle clean after every bundle update, to make sure that newer/older gems are not also installed into the separate app gem space to be picked up by the require lines in the gemfile.

I have tested this on my example app, and it seems to work.

@deivid-rodriguez
Copy link
Member

The idea of giving a better error message when a require fails with this error during Gemfile evaluation makes sense to me 👍.

@martinemde martinemde changed the title Bundler does not resolve a difference in version between built-in and Gemfile gems Bundler should give a better error when the Gemfile itself activates a conflicting gem Dec 4, 2023
@martinemde
Copy link
Member

I made a branch with a failing test for this case. I'm not confident about its approach, but it does trigger the error: https://github.com/rubygems/rubygems/compare/require-conflict-in-gemfile

Bandes added a commit to modolabs/bundler-set-error-example that referenced this issue Dec 6, 2023
I have been having trouble with a docker setup that was failing due to a mismatch in the `set` gem.
The issue on github is here:
rubygems/rubygems#7178

The intention of this repo is to provide a small test case that will demonstrate the problem
I am having.
@Bandes
Copy link
Author

Bandes commented Dec 7, 2023

Hi this repo https://github.com/modolabs/bundler-set-error-example should demonstrate the issue I am having. It seems like it might have something to do with spring because if I remove those gems from the gemfile I can no longer reproduce the issue. It might also be a problem with the dockerfile, but I'm not sure.

If these commands are executed in that repo, the error will occur for me

docker build --no-cache --build-arg BUNDLE_WITHOUT=nil . -t test-project
docker run -t test-project bundle exec rails runner '1+1'

@deivid-rodriguez
Copy link
Member

In your case, you're using a very old spring version that does not include rails/spring#659, that's why you run into this error. If you upgrade (or remove) spring, you should be fine!

@stanhu
Copy link
Contributor

stanhu commented Jan 5, 2024

This might be a separate issue, but I ran into a similar issue with the uri gem that is loaded by RubyGems only when:

  1. A local gemspec is used.
  2. The gemspec has a home page URL: https://github.com/ruby/ruby/blob/10b9679fa61f40c08ec28ec34251f7ec8648054d/lib/rubygems/specification_policy.rb#L430.

Consider this Gemfile:

source 'https://rubygems.org'

gem 'uri', '= 0.12.2'
gem 'test', path: 'test'

In test/test.gemspec:

# frozen_string_literal: true

Gem::Specification.new do |s|
  s.name = "test"
  s.version = "1.0.0"
  s.summary = "test"
  s.authors = ['John Doe']
  s.homepage = 'https://example.com'
end

If I simply run bundle exec irb, I see:

/Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated uri 0.13.0, but your Gemfile requires uri 0.12.2. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:25:in `block in setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/spec_set.rb:165:in `each'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `map'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/runtime.rb:24:in `setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler.rb:162:in `setup'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `block in <top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:159:in `with_level'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/ui/shell.rb:111:in `silence'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/setup.rb:10:in `<top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:56:in `require_relative'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:56:in `kernel_load'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli/exec.rb:23:in `run'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:492:in `exec'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor.rb:527:in `dispatch'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:34:in `dispatch'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/cli.rb:28:in `start'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/exe/bundle:37:in `block in <top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/bundler-2.4.22/exe/bundle:29:in `<top (required)>'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `load'
        from /Users/stanhu/.asdf/installs/ruby/3.2.2/bin/bundle:25:in `<main>'

In this example, the user doesn't have a whole lot of control over the loading of uri.

The only workaround I can see is to update uri to v0.13.0.

@deivid-rodriguez
Copy link
Member

Yeah, we should vendor the uri gem in Rubygems to avoid this problem.

@deivid-rodriguez
Copy link
Member

Closed unintentionally by #7386. Reopening.

@N0xFF
Copy link

N0xFF commented Feb 9, 2024

After updating to Rails 7.1 we started receiving an error:

/Ruby/3.1.4/x64/lib/ruby/gems/3.1.0/gems/bundler-2.5.6/lib/bundler/runtime.rb:304:in `check_for_activated_spec!': You have already activated mutex_m 0.1.1, but your Gemfile requires mutex_m 0.2.0. Since mutex_m is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports mutex_m as a default gem. (Gem::LoadError)

The cause was Spring. After disabling it the error is gone, but investigating the reason was hard. It would be better if we had a clearer error message.

@joshuapinter
Copy link

FYI, we were receiving the following error:

Error: The application encountered the following error: You have already activated strscan 3.0.7, but your Gemfile requires strscan 3.1.0. Since strscan is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports strscan as a default gem. (Gem::LoadError)

What worked was manually install strscan at 3.1.0 in the global gem space:

sudo gem install strscan:3.1.0

This ensures that one is used instead of the default 3.0.7:

gem list strscan

*** LOCAL GEMS ***

strscan (3.1.0, default: 3.0.7)

I tried other methods before doing this that I saw elsewhere, including:

  1. Upgrading rubygems.
  2. Setting this NGINX config: passenger_env_var RUBYOPT '-r bundler/setup';

Neither worked.

I did get it working by pinning my version of strscan to 3.0.7 in our Gemfile but that wasn't a long term solution because we needed to upgrade rexml due to a security risk and 3.0.7 depended on an older version of rexml, so we needed to upgrade strscan.

Hope that helps others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants