Excessive memory usage in Rubocop JUnit Reporter starting in 1.46.0 or 1.47.0 #11657
Comments
|
Can you identify where the problem is (by using the recently introduced profiling ability - #11396)? If you try to measure memory (additionally to cpu, If that won't help, you can try to disable extensions ( |
|
@fatkodima, thanks for suggestion on git bisect I can narrow it down to #11603 / efce268 Our memory doesn't go above 250MB during a complete run in all previous commits, but spirals into the GB once that commit is applied. Including @tdeo as the commit author in case any ideas arise for additional debugging I can help provide for the junit scenario. To confirm, 52e8e4f is the previous commit, and this does not show this behavior.
Note I did this against the following just to confirm as well against a more stock rubocop install:
Same issue re: memory usage when using the junit formatter after #11603 / efce268 , so we can rule extensions or custom cops as causes |
|
I can't reproduce that locally. I see that junit format generation already generates a gazillion of memory, before and after that PR. So I would try to optimize that separately. Is this happening only with junit format in your case? And can you run |
|
Would you happen to have a significant amount of inline enable / disable directives? Otherwise did you have very few enabled cops in the first place? |
|
@tdeo not sure how to compare vs other projects, but I'd say yeah, we have a good amount listed in our rubocop.yml and rubocop_todo.yml @fatkodima I'll try getting a profile up to see, but will need to make sure it's redacted enough not to give away any private info. In our case, I have only tried the normal outputted + Junit outputter, but only see this issue with Junit, it's far higher than normal (by a ton) so breaking our CI system which has a memory limit on any given. |
|
@fatkodima , actually I just tried json formatter, so it's not just junit. Same issue: memory goes super high with efce268 , but doesn't with any commit before. |
|
Yeah, it doesn't seem to be an issue limited to JUnit formatter if #11603 is the cause. |
|
<sorry for noise, changing which Github account I was using> I think I see the issue - using Rubocop 1.46.0 (or any commit after #11603 ) - the cache is not used at also appears to be ignored? Parallelization also appears to be ignored? without a formatter in debug mode: (i.e. and so on... and so on... |
|
actually, that may not be it in full. Even with 1.45.1, cache is not looked at with formatters (but is without formatters) per the logs. The memory goes up, however, in 1.46.0 with a formatter while the logs are still showing Scanning which doesn't happen during 1.45.1. |
|
if I run |
|
Also @fatkodima ,I tried doing a memory profile, but it's not giving accurate results. Using This uses cache per logs in v1.46.0: This does not: |
|
Actually, I just did a profile run to compare before and after that commit on a small folder, which shows the issue well: With JSON formatter and on commit 52e8e4f (i.e. before the commit I tracked it down too): With JSON formatter and on commit efce268 (i.e. commit git bisect found): The config and cop/team locations show up new at the top of the list. Both commands were same (so no cache used due to memory and profile):
|
|
Can you run a memory profile on a larger set of files? To have at least 500Mb of allocated memory. I do not think |
|
Can you check if #11665 fixes your problem? |
|
I broke the profile/formatter stopping cache and parallelization running issue out to it's own ticket: #11664 So here we can focus on the memory usage. As you can see, even at >750 MB, config.rb:37 stays on top after efce268
and |
@fatkodima that definitely impacted things (for the better!). Huge difference, both when running overall and for the specific memory profile case I was using. thanks for looking so quickly! For the specific comparison, for the memory profile for the same folders mentioned here: is now: |
|
@fatkodima With that change, a full run over our source code isn't going above 125 MB runtime memory (while not running profiling). Prior to this, it was at 25 GB or so last time before I killed it. |
Originally, the inefficiency was discovered when working through the bug report in the `rubocop` repository - rubocop/rubocop#11657. Tested on the `rubocop` repository. `git clone` it, point `rexml` to the local repository, `bundle install` etc and run inside it: ``` bundle exec rubocop --profile --memory --format junit --out results/rubocop.xml lib/rubocop/cop/layout ``` ### Memory #### Before ``` Total allocated: 630.15 MB (8838482 objects) Total retained: 53.50 MB (445069 objects) allocated memory by gem ----------------------------------- 294.26 MB rexml/lib 214.78 MB rubocop/lib 38.60 MB rubocop-ast/lib 31.62 MB parser-3.2.1.0 31.43 MB other 10.02 MB lib 3.11 MB rubocop-rspec-2.18.1 1.95 MB rubocop-performance-1.16.0 1.83 MB regexp_parser-2.7.0 1.61 MB ast-2.4.2 405.71 kB unicode-display_width-2.4.2 287.16 kB rubocop-capybara-2.17.1 244.96 kB rubocop-rake-0.6.0 5.00 kB rubygems allocated memory by file ----------------------------------- 123.30 MB rexml/lib/rexml/text.rb 101.92 MB rubocop/lib/rubocop/formatter/junit_formatter.rb 61.42 MB rexml/lib/rexml/namespace.rb 31.07 MB rexml/lib/rexml/attribute.rb 28.89 MB rubocop/lib/rubocop/config.rb 27.30 MB rexml/lib/rexml/element.rb 22.75 MB rexml/lib/rexml/formatters/pretty.rb 22.75 MB rexml/lib/rexml/entity.rb 22.75 MB <internal:kernel> 15.11 MB parser-3.2.1.0/lib/parser/source/buffer.rb 12.59 MB rubocop-ast/lib/rubocop/ast/node.rb 12.03 MB rubocop/lib/rubocop/cop/registry.rb 11.88 MB rubocop/lib/rubocop/cop/team.rb 5.90 MB rubocop/lib/rubocop/cop/commissioner.rb 5.87 MB parser-3.2.1.0/lib/parser/lexer-F1.rb 5.69 MB rexml/lib/rexml/parent.rb 5.44 MB rubocop/lib/rubocop/cop/base.rb 5.17 MB rubocop-ast/lib/rubocop/ast/builder.rb 4.56 MB (eval) 4.25 MB parser-3.2.1.0/lib/parser/builders/default.rb 3.75 MB <internal:pack> 3.59 MB ruby/3.2.0/lib/ruby/3.2.0/psych/tree_builder.rb 3.53 MB rubocop/lib/rubocop/path_util.rb 3.21 MB rubocop/lib/rubocop/cli.rb 2.45 MB parser-3.2.1.0/lib/parser/ruby26.rb 2.27 MB rubocop-ast/lib/rubocop/ast/node_pattern/compiler/sequence_subcompiler.rb 2.23 MB rubocop-ast/lib/rubocop/ast/processed_source.rb 2.05 MB rubocop-ast/lib/rubocop/ast/node/if_node.rb 2.00 MB rubocop-ast/lib/rubocop/ast/token.rb 1.73 MB rubocop-ast/lib/rubocop/ast/node_pattern/method_definer.rb 1.73 MB ruby/3.2.0/lib/ruby/3.2.0/erb/compiler.rb 1.61 MB ast-2.4.2/lib/ast/node.rb 1.54 MB rubocop/lib/rubocop/cop/variable_force.rb 1.53 MB rubocop/lib/rubocop/cop/internal_affairs/cop_description.rb 1.49 MB rubocop/lib/rubocop/cop/naming/inclusive_language.rb 1.47 MB rubocop-ast/lib/rubocop/ast/node/mixin/parameterized_node.rb 1.42 MB rubocop-ast/lib/rubocop/ast/node_pattern/compiler.rb 1.42 MB rubocop-ast/lib/rubocop/ast/node_pattern/compiler/node_pattern_subcompiler.rb 1.39 MB rubocop/lib/rubocop/cop/layout/redundant_line_break.rb 1.35 MB rubocop/lib/rubocop/cop/util.rb 1.29 MB regexp_parser-2.7.0/lib/regexp_parser/scanner.rb 1.29 MB rubocop/lib/rubocop/cop/mixin/range_help.rb 1.27 MB ruby/3.2.0/lib/ruby/3.2.0/psych/parser.rb 1.18 MB rubocop/lib/rubocop/cop/layout/comment_indentation.rb 1.17 MB rubocop-ast/lib/rubocop/ast/node/mixin/descendence.rb 1.10 MB ruby/3.2.0/lib/ruby/3.2.0/erb.rb 1.07 MB rubocop/lib/rubocop/cop/variable_force/variable_table.rb 1.04 MB rubocop/lib/rubocop/cop/layout/end_of_line.rb 1.01 MB rubocop/lib/rubocop/cop/mixin/end_keyword_alignment.rb 996.49 kB rubocop/lib/rubocop/cop/metrics/utils/abc_size_calculator.rb allocated memory by location ----------------------------------- 87.70 MB rubocop/lib/rubocop/formatter/junit_formatter.rb:65 61.19 MB rexml/lib/rexml/text.rb:385 36.04 MB rexml/lib/rexml/text.rb:134 35.83 MB rexml/lib/rexml/namespace.rb:19 26.06 MB rexml/lib/rexml/text.rb:374 22.75 MB rexml/lib/rexml/entity.rb:136 22.75 MB <internal:kernel>:49 17.16 MB rubocop/lib/rubocop/config.rb:37 15.77 MB rexml/lib/rexml/attribute.rb:127 15.30 MB rexml/lib/rexml/attribute.rb:125 13.08 MB rexml/lib/rexml/element.rb:331 11.37 MB rexml/lib/rexml/element.rb:2382 11.37 MB rubocop/lib/rubocop/formatter/junit_formatter.rb:56 9.89 MB parser-3.2.1.0/lib/parser/source/buffer.rb:205 9.86 MB rubocop/lib/rubocop/cop/team.rb:32 8.53 MB rexml/lib/rexml/namespace.rb:23 8.53 MB rexml/lib/rexml/namespace.rb:24 8.53 MB rexml/lib/rexml/namespace.rb:26 5.86 MB rubocop/lib/rubocop/cop/registry.rb:54 5.69 MB rexml/lib/rexml/formatters/pretty.rb:40 5.69 MB rexml/lib/rexml/formatters/pretty.rb:44 5.39 MB rubocop/lib/rubocop/config.rb:319 4.55 MB (eval):3 4.20 MB rubocop/lib/rubocop/config.rb:34 3.84 MB rubocop-ast/lib/rubocop/ast/node.rb:93 3.73 MB <internal:pack>:21 3.71 MB rubocop/lib/rubocop/cop/base.rb:346 3.58 MB ruby/3.2.0/lib/ruby/3.2.0/psych/tree_builder.rb:97 3.52 MB rubocop/lib/rubocop/path_util.rb:55 3.50 MB rubocop-ast/lib/rubocop/ast/builder.rb:99 3.21 MB rubocop/lib/rubocop/cli.rb:92 3.00 MB parser-3.2.1.0/lib/parser/lexer-F1.rb:14606 2.91 MB rubocop/lib/rubocop/cop/registry.rb:52 2.84 MB rexml/lib/rexml/parent.rb:116 2.84 MB rexml/lib/rexml/element.rb:330 2.84 MB rexml/lib/rexml/parent.rb:15 2.84 MB rexml/lib/rexml/formatters/pretty.rb:41 2.84 MB rexml/lib/rexml/formatters/pretty.rb:85 2.84 MB rexml/lib/rexml/formatters/pretty.rb:78 2.84 MB rexml/lib/rexml/formatters/pretty.rb:52 2.84 MB rubocop/lib/rubocop/formatter/junit_formatter.rb:52 2.84 MB rubocop-ast/lib/rubocop/ast/node.rb:236 1.89 MB parser-3.2.1.0/lib/parser/lexer-F1.rb:14602 1.86 MB parser-3.2.1.0/lib/parser/source/buffer.rb:117 1.74 MB rubocop-ast/lib/rubocop/ast/processed_source.rb:185 1.69 MB rubocop-ast/lib/rubocop/ast/token.rb:14 1.67 MB rubocop-ast/lib/rubocop/ast/builder.rb:98 1.66 MB rubocop/lib/rubocop/cop/commissioner.rb:125 1.52 MB rubocop/lib/rubocop/cop/base.rb:286 1.49 MB rubocop/lib/rubocop/cop/internal_affairs/cop_description.rb:80 ``` #### After ``` Total allocated: 367.43 MB (4224322 objects) 🔥 🔥 🔥 Total retained: 53.50 MB (445067 objects) allocated memory by gem ----------------------------------- 214.62 MB rubocop/lib 54.44 MB rexml/lib 38.60 MB rubocop-ast/lib 31.62 MB parser-3.2.1.0 10.02 MB lib 8.69 MB other 3.11 MB rubocop-rspec-2.18.1 1.95 MB rubocop-performance-1.16.0 1.83 MB regexp_parser-2.7.0 1.61 MB ast-2.4.2 405.71 kB unicode-display_width-2.4.2 287.16 kB rubocop-capybara-2.17.1 244.96 kB rubocop-rake-0.6.0 5.00 kB rubygems allocated memory by file ----------------------------------- 101.92 MB rubocop/lib/rubocop/formatter/junit_formatter.rb 28.89 MB rubocop/lib/rubocop/config.rb 27.30 MB rexml/lib/rexml/element.rb 15.77 MB rexml/lib/rexml/attribute.rb 15.11 MB parser-3.2.1.0/lib/parser/source/buffer.rb 12.59 MB rubocop-ast/lib/rubocop/ast/node.rb 12.03 MB rubocop/lib/rubocop/cop/registry.rb 11.88 MB rubocop/lib/rubocop/cop/team.rb 5.90 MB rubocop/lib/rubocop/cop/commissioner.rb 5.87 MB parser-3.2.1.0/lib/parser/lexer-F1.rb 5.69 MB rexml/lib/rexml/parent.rb 5.69 MB rexml/lib/rexml/formatters/pretty.rb 5.44 MB rubocop/lib/rubocop/cop/base.rb 5.17 MB rubocop-ast/lib/rubocop/ast/builder.rb 4.56 MB (eval) 4.25 MB parser-3.2.1.0/lib/parser/builders/default.rb 3.75 MB <internal:pack> 3.59 MB ruby/3.2.0/lib/ruby/3.2.0/psych/tree_builder.rb 3.53 MB rubocop/lib/rubocop/path_util.rb 3.05 MB rubocop/lib/rubocop/cli.rb 2.45 MB parser-3.2.1.0/lib/parser/ruby26.rb 2.27 MB rubocop-ast/lib/rubocop/ast/node_pattern/compiler/sequence_subcompiler.rb 2.23 MB rubocop-ast/lib/rubocop/ast/processed_source.rb 2.05 MB rubocop-ast/lib/rubocop/ast/node/if_node.rb 2.00 MB rubocop-ast/lib/rubocop/ast/token.rb 1.73 MB rubocop-ast/lib/rubocop/ast/node_pattern/method_definer.rb 1.73 MB ruby/3.2.0/lib/ruby/3.2.0/erb/compiler.rb 1.61 MB ast-2.4.2/lib/ast/node.rb 1.54 MB rubocop/lib/rubocop/cop/variable_force.rb 1.53 MB rubocop/lib/rubocop/cop/internal_affairs/cop_description.rb 1.49 MB rubocop/lib/rubocop/cop/naming/inclusive_language.rb 1.47 MB rubocop-ast/lib/rubocop/ast/node/mixin/parameterized_node.rb 1.42 MB rubocop-ast/lib/rubocop/ast/node_pattern/compiler.rb 1.42 MB rubocop-ast/lib/rubocop/ast/node_pattern/compiler/node_pattern_subcompiler.rb 1.39 MB rubocop/lib/rubocop/cop/layout/redundant_line_break.rb 1.35 MB rubocop/lib/rubocop/cop/util.rb 1.29 MB regexp_parser-2.7.0/lib/regexp_parser/scanner.rb 1.29 MB rubocop/lib/rubocop/cop/mixin/range_help.rb 1.27 MB ruby/3.2.0/lib/ruby/3.2.0/psych/parser.rb 1.18 MB rubocop/lib/rubocop/cop/layout/comment_indentation.rb 1.17 MB rubocop-ast/lib/rubocop/ast/node/mixin/descendence.rb 1.10 MB ruby/3.2.0/lib/ruby/3.2.0/erb.rb 1.07 MB rubocop/lib/rubocop/cop/variable_force/variable_table.rb 1.04 MB rubocop/lib/rubocop/cop/layout/end_of_line.rb 1.01 MB rubocop/lib/rubocop/cop/mixin/end_keyword_alignment.rb 996.49 kB rubocop/lib/rubocop/cop/metrics/utils/abc_size_calculator.rb 970.58 kB rubocop/lib/rubocop/cop/style/redundant_self.rb 947.97 kB rubocop/lib/rubocop/cop/layout/empty_comment.rb 938.93 kB rubocop/lib/rubocop/cop/mixin/empty_lines_around_body.rb 871.31 kB rubocop/lib/rubocop/cop/variable_force/variable.rb allocated memory by location ----------------------------------- 87.70 MB rubocop/lib/rubocop/formatter/junit_formatter.rb:65 17.16 MB rubocop/lib/rubocop/config.rb:37 15.77 MB rexml/lib/rexml/attribute.rb:127 13.08 MB rexml/lib/rexml/element.rb:331 11.37 MB rexml/lib/rexml/element.rb:2382 11.37 MB rubocop/lib/rubocop/formatter/junit_formatter.rb:56 9.89 MB parser-3.2.1.0/lib/parser/source/buffer.rb:205 9.86 MB rubocop/lib/rubocop/cop/team.rb:32 5.86 MB rubocop/lib/rubocop/cop/registry.rb:54 5.39 MB rubocop/lib/rubocop/config.rb:319 4.55 MB (eval):3 4.20 MB rubocop/lib/rubocop/config.rb:34 3.84 MB rubocop-ast/lib/rubocop/ast/node.rb:93 3.73 MB <internal:pack>:21 3.71 MB rubocop/lib/rubocop/cop/base.rb:346 3.58 MB ruby/3.2.0/lib/ruby/3.2.0/psych/tree_builder.rb:97 3.52 MB rubocop/lib/rubocop/path_util.rb:55 3.50 MB rubocop-ast/lib/rubocop/ast/builder.rb:99 3.05 MB rubocop/lib/rubocop/cli.rb:92 3.00 MB parser-3.2.1.0/lib/parser/lexer-F1.rb:14606 2.91 MB rubocop/lib/rubocop/cop/registry.rb:52 2.84 MB rexml/lib/rexml/parent.rb:116 2.84 MB rexml/lib/rexml/element.rb:330 2.84 MB rexml/lib/rexml/parent.rb:15 2.84 MB rexml/lib/rexml/formatters/pretty.rb:40 2.84 MB rexml/lib/rexml/formatters/pretty.rb:41 2.84 MB rubocop/lib/rubocop/formatter/junit_formatter.rb:52 2.84 MB rubocop-ast/lib/rubocop/ast/node.rb:236 1.89 MB parser-3.2.1.0/lib/parser/lexer-F1.rb:14602 1.86 MB parser-3.2.1.0/lib/parser/source/buffer.rb:117 1.74 MB rubocop-ast/lib/rubocop/ast/processed_source.rb:185 1.69 MB rubocop-ast/lib/rubocop/ast/token.rb:14 1.67 MB rubocop-ast/lib/rubocop/ast/builder.rb:98 1.66 MB rubocop/lib/rubocop/cop/commissioner.rb:125 1.52 MB rubocop/lib/rubocop/cop/base.rb:286 1.49 MB rubocop/lib/rubocop/cop/internal_affairs/cop_description.rb:80 1.47 MB parser-3.2.1.0/lib/parser/source/buffer.rb:274 1.41 MB ast-2.4.2/lib/ast/node.rb:77 1.35 MB parser-3.2.1.0/lib/parser/ruby26.rb:0 1.30 MB rubocop/lib/rubocop/cop/commissioner.rb:153 1.27 MB ruby/3.2.0/lib/ruby/3.2.0/psych/parser.rb:62 1.25 MB rubocop-ast/lib/rubocop/ast/node.rb:106 1.24 MB rubocop/lib/rubocop/cop/registry.rb:181 1.16 MB parser-3.2.1.0/lib/parser/source/buffer.rb:254 1.10 MB ruby/3.2.0/lib/ruby/3.2.0/erb.rb:429 1.07 MB rubocop-ast/lib/rubocop/ast/node_pattern/method_definer.rb:58 1.04 MB rubocop/lib/rubocop/cop/layout/end_of_line.rb:50 988.72 kB rubocop/lib/rubocop/config.rb:322 982.96 kB rubocop-ast/lib/rubocop/ast/node/mixin/parameterized_node.rb:91 975.88 kB rubocop-ast/lib/rubocop/ast/node/if_node.rb:141 ``` So, `-42%` of allocated memory and `-52%` of allocated objects. ### CPU #### Before ``` TOTAL (pct) SAMPLES (pct) FRAME 2620 (10.0%) 2620 (10.0%) Dir.pwd ==> 2314 (8.9%) 2314 (8.9%) String#gsub ==> 1538 (5.9%) 1531 (5.9%) String#scan ==> 4376 (16.8%) 960 (3.7%) REXML::Text.normalize 5223 (20.0%) 907 (3.5%) Class#new ==> 895 (3.4%) 895 (3.4%) Regexp#=== 879 (3.4%) 740 (2.8%) Enumerable#find 660 (2.5%) 660 (2.5%) IO#write ==> 732 (2.8%) 641 (2.5%) Kernel#clone ==> 618 (2.4%) 618 (2.4%) String#=~ ==> 2244 (8.6%) 579 (2.2%) REXML::Formatters::Pretty#write_element ==> 1086 (4.2%) 484 (1.9%) REXML::Namespace#name= 795 (3.0%) 381 (1.5%) Parser::Lexer#advance 362 (1.4%) 362 (1.4%) String#[] 677 (2.6%) 308 (1.2%) REXML::Attribute#to_string 574 (2.2%) 286 (1.1%) REXML::Namespace#name= 286 (1.1%) 268 (1.0%) REXML::Element#root 1844 (7.1%) 256 (1.0%) Racc::Parser#_racc_do_parse_c 556 (2.1%) 236 (0.9%) Kernel#require_relative 8190 (31.3%) 233 (0.9%) REXML::Attributes#[]= 3913 (15.0%) 230 (0.9%) RuboCop::Cop::Commissioner#trigger_responding_cops 26099 (99.9%) 224 (0.9%) Array#each 820 (3.1%) 223 (0.9%) RuboCop::Config#initialize 273 (1.0%) 222 (0.8%) Kernel#dup 6009 (23.0%) 200 (0.8%) Kernel#public_send 4961 (19.0%) 189 (0.7%) Hash#each_value 3749 (14.4%) 173 (0.7%) RuboCop::Formatter::JUnitFormatter#classname_attribute_value 13301 (50.9%) 165 (0.6%) RuboCop::Formatter::JUnitFormatter#add_testcase_element_to_testsuite_element 325 (1.2%) 139 (0.5%) RuboCop::Cop::Registry#clear_enrollment_queue 1554 (5.9%) 134 (0.5%) Array#select ``` #### After ``` TOTAL (pct) SAMPLES (pct) FRAME 1878 (12.1%) 1878 (12.1%) Dir.pwd 783 (5.1%) 783 (5.1%) String#gsub 3091 (20.0%) 739 (4.8%) Class#new 692 (4.5%) 607 (3.9%) Enumerable#find 702 (4.5%) 339 (2.2%) Parser::Lexer#advance 317 (2.0%) 317 (2.0%) IO#write 283 (1.8%) 283 (1.8%) String#[] 275 (1.8%) 275 (1.8%) String#match? 267 (1.7%) 262 (1.7%) String#scan 244 (1.6%) 230 (1.5%) REXML::Element#root 1551 (10.0%) 205 (1.3%) Racc::Parser#_racc_do_parse_c 236 (1.5%) 201 (1.3%) Kernel#dup 196 (1.3%) 179 (1.2%) REXML::Attribute#to_string 4037 (26.1%) 177 (1.1%) Kernel#public_send 3286 (21.2%) 176 (1.1%) RuboCop::Cop::Commissioner#trigger_responding_cops 15481 (100.0%) 176 (1.1%) Array#each 460 (3.0%) 166 (1.1%) Kernel#require_relative 661 (4.3%) 141 (0.9%) RuboCop::Config#initialize 2099 (13.6%) 141 (0.9%) REXML::Attributes#[]= 2866 (18.5%) 139 (0.9%) RuboCop::Formatter::JUnitFormatter#classname_attribute_value 292 (1.9%) 132 (0.9%) RuboCop::Cop::Registry#clear_enrollment_queue 126 (0.8%) 126 (0.8%) File.fnmatch? 874 (5.6%) 123 (0.8%) REXML::Formatters::Pretty#write_element 113 (0.7%) 113 (0.7%) Symbol#to_s 1348 (8.7%) 107 (0.7%) Array#select 103 (0.7%) 101 (0.7%) RuboCop::Cop::Registry#initialize 5611 (36.2%) 91 (0.6%) RuboCop::Formatter::JUnitFormatter#add_testcase_element_to_testsuite_element 269 (1.7%) 91 (0.6%) REXML::Text.normalize 89 (0.6%) 89 (0.6%) String#tr 161 (1.0%) 85 (0.5%) Parser::Lexer#emit ``` ### Time #### Before ``` $ time bundle exec rubocop --cache false --format junit --out results/rubocop.xml lib/rubocop/cop/layout bundle exec rubocop --cache false --format junit --out results/rubocop.xml 12.28s user 2.02s system 99% cpu 14.313 total ``` #### After ``` $ time bundle exec rubocop --cache false --format junit --out results/rubocop.xml lib/rubocop/cop/layout bundle exec rubocop --cache false --format junit --out results/rubocop.xml 10.17s user 1.97s system 99% cpu 12.150 total ``` **Note**: There is also a difference in time needed to run this gem's tests after this PR changes. Feel free to ask clarifying questions if some changes are not clear. Co-authored-by: Sutou Kouhei <kou@clear-code.com>
Expected behavior
Version 1.46.0 and Version 1.47.0 have same memory footprint as 1.45.1 and below and are relatively drop-in.
Actual behavior
When running with any of 1.45.1, 1.46.0 or 1.47.0 vs our source code base, we are fine when we run this and it takes a minute or so. Max memory is 200MB or so.
Similar on 1.45.1 when running in unit mode - Max memory is 200MB or so.
But memory goes into the gigabytes when we run this on 1.46.0 or 1.47.0 and it takes 30+ minutes if it even finishes:
For example, I kicked these off at the same time, as they are mid-running:
The top row is Rubocop 1.47.0, and bottom one is Rubocop 1.45.1, both running the junit version of the command . Same code repository, just git-cloned into two different folders
You can see the massive difference
Steps to reproduce the problem
This is a private source code base, so I can't share out unfortunately.
Never had an issue on rubocop upgrades before,
Upgrade to a rubocop 1.46.0 or 1.47.0 and run this command:
Note: I also tried upgrading rubocop-rails along with rubocop, same issue, so it's not that.
RuboCop version
The text was updated successfully, but these errors were encountered: