Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent cache key is generated for ruby 3.1.4 #12076

Closed
K-S-A opened this issue Jul 28, 2023 · 4 comments · Fixed by #12248
Closed

Inconsistent cache key is generated for ruby 3.1.4 #12076

K-S-A opened this issue Jul 28, 2023 · 4 comments · Fixed by #12248

Comments

@K-S-A
Copy link
Contributor

K-S-A commented Jul 28, 2023

Expected behavior

For ruby 3.1.x the value of the cache key should not depend on the presence of the server cache directory.
More precisely, the RuboCop::ResultCache.rubocop_required_features value should not change if the RuboCop::Server::Cache.dir already exists.

Actual behavior

The cache key is changed for two consecutive rubocop command executions and depends on the <rubocop_cache_root>/server directory existance.

Steps to reproduce the problem

RUBOCOP_CACHE_ROOT = "#{ENV.fetch('PWD')}/tmp".freeze

# ensure consistent initial state
`rm -rf #{RUBOCOP_CACHE_ROOT}/*`

puts `du -lah #{RUBOCOP_CACHE_ROOT}`

puts('============ first run ====================')
`rubocop --config .rubocop.yml --cache-root #{RUBOCOP_CACHE_ROOT}`
puts `du -lah #{RUBOCOP_CACHE_ROOT}`
puts('============ first run ====================')

puts('============ second run ====================')
puts `rubocop --config .rubocop.yml --cache-root #{RUBOCOP_CACHE_ROOT}`
puts `du -lah #{RUBOCOP_CACHE_ROOT}`
puts('============ second run ====================')

# # Put this lines in the `lib/rubocop.rb` file,
# # right after the `RuboCop::ResultCache.rubocop_required_features = $LOADED_FEATURES - before_us` line.
# # It will print out the difference for the `RuboCop::ResultCache.rubocop_required_features` after the second run.
# file_before = "#{Dir.pwd}/tmp/rubocop_required_features_before.txt"
# if File.exist?(file_before)
#   file_after = file_before.sub('_before.txt', '_after.txt')
#   File.write(file_after, RuboCop::ResultCache.rubocop_required_features.join("\n"))

#   diff = RuboCop::ResultCache.rubocop_required_features.difference(File.read(file_before).split("\n"))
#   puts("diff: #{diff}")
# else
#   File.write(file_before, RuboCop::ResultCache.rubocop_required_features.join("\n"))
# end


# # Example output:

# ruby double_cache_bug.rb
#   0B    ~/tmp
# ============ first run ====================
# 168K    ~/tmp/rubocop_required_features_before.txt
#   0B    ~/tmp/rubocop_cache/server/rubocop_cache
#   0B    ~/tmp/rubocop_cache/server
# 4.0K    ~/tmp/rubocop_cache/58851d0db3a86962b016f59f3bb0c63cc8be77ae/753f2e0a1b736120c2a1f088699bbf940a3c8afe/0a315cd08e2318901e19f47db30b94a8a292c349
# 4.0K    ~/tmp/rubocop_cache/58851d0db3a86962b016f59f3bb0c63cc8be77ae/753f2e0a1b736120c2a1f088699bbf940a3c8afe
# 4.0K    ~/tmp/rubocop_cache/58851d0db3a86962b016f59f3bb0c63cc8be77ae
# 4.0K    ~/tmp/rubocop_cache
# 172K    ~/tmp
# ============ first run ====================
# ============ second run ====================
# diff: ["~/3.1.4/lib/ruby/3.1.0/fileutils.rb"]
# Inspecting 1 file
# .

# 1 file inspected, no offenses detected
# 168K    ~/tmp/rubocop_required_features_before.txt
# 4.0K    ~/tmp/rubocop_cache/d42b30f00b6fa756e8fabe5b83bb34d4242dc188/753f2e0a1b736120c2a1f088699bbf940a3c8afe/0a315cd08e2318901e19f47db30b94a8a292c349
# 4.0K    ~/tmp/rubocop_cache/d42b30f00b6fa756e8fabe5b83bb34d4242dc188/753f2e0a1b736120c2a1f088699bbf940a3c8afe
# 4.0K    ~/tmp/rubocop_cache/d42b30f00b6fa756e8fabe5b83bb34d4242dc188
#   0B    ~/tmp/rubocop_cache/server/rubocop_cache
#   0B    ~/tmp/rubocop_cache/server
# 4.0K    ~/tmp/rubocop_cache/58851d0db3a86962b016f59f3bb0c63cc8be77ae/753f2e0a1b736120c2a1f088699bbf940a3c8afe/0a315cd08e2318901e19f47db30b94a8a292c349
# 4.0K    ~/tmp/rubocop_cache/58851d0db3a86962b016f59f3bb0c63cc8be77ae/753f2e0a1b736120c2a1f088699bbf940a3c8afe
# 4.0K    ~/tmp/rubocop_cache/58851d0db3a86962b016f59f3bb0c63cc8be77ae
# 8.0K    ~/tmp/rubocop_cache
# 168K    ~/tmp/rubocop_required_features_after.txt
# 344K    ~/tmp
# ============ second run ====================

RuboCop version

$ [bundle exec] rubocop -V
1.55.0 (using Parser 3.2.2.3, rubocop-ast 1.29.0, running on ruby 3.1.4) [arm64-darwin22]

Workaround

$ rubocop_server_cache_path="$(echo "$PWD" | tr '/' '+')"
$ echo "PWD: ${PWD}; rubocop_server_cache_path: ${rubocop_server_cache_path:1}"
$ mkdir -p "tmp/rubocop_cache/server/${rubocop_server_cache_path:1}"

I would be happy to submit the patch for this issue.

@ruslan-nazarets
Copy link

I have the same issue. Does anyone know how to fix it?

@koic
Copy link
Member

koic commented Sep 1, 2023

@K-S-A Yeah! Please open a patch with a reproduction test.

@meric426
Copy link
Contributor

I have the same issue. Also, changing a single file within the project seems to invalidate the entire cache, is this related?

K-S-A added a commit to K-S-A/rubocop that referenced this issue Oct 6, 2023
…Cache

The "fileutils.rb" item is dynamically added to the `$LOADED_FEATURES`
due to the autoloading when calling the `RuboCop::Server::Cache.dir`.

Explicit require is added to achieve idempotency when initializing
the `RuboCop::ResultCache.rubocop_required_features`.

This change resolves the issue where the top level cache folder was
named differently during two consecutive runs of rubocop.
@K-S-A
Copy link
Contributor Author

K-S-A commented Oct 7, 2023

@meric426, the rubocop cache consists of several "layers". Its invalidation can be caused neither by changed dependencies, nor by configuration changes, nor by updating the content of the scanned file(s). Please, share the minimum reproduction gist/repo to confirm if your case is related.

bbatsov pushed a commit that referenced this issue Oct 11, 2023
The "fileutils.rb" item is dynamically added to the `$LOADED_FEATURES`
due to the autoloading when calling the `RuboCop::Server::Cache.dir`.

Explicit require is added to achieve idempotency when initializing
the `RuboCop::ResultCache.rubocop_required_features`.

This change resolves the issue where the top level cache folder was
named differently during two consecutive runs of rubocop.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants