Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use 1k buffer in FileUtils.readLines to reduce heap allocation pressure #2612

Merged
merged 2 commits into from
Mar 31, 2024

Conversation

chrisribble
Copy link
Contributor

No description provided.

@chrisribble
Copy link
Contributor Author

chrisribble commented Mar 30, 2024

JMH results with various buffer sizes:

Benchmark                                      Mode  Cnt      Score     Error   Units
OshiBenchmark.readLines1k                      avgt    5     10.945 ±   1.473   us/op
OshiBenchmark.readLines1k:gc.alloc.rate        avgt    5    985.187 ± 134.199  MB/sec
OshiBenchmark.readLines1k:gc.alloc.rate.norm   avgt    5  11296.015 ±   0.002    B/op
OshiBenchmark.readLines1k:gc.count             avgt    5     69.000            counts
OshiBenchmark.readLines1k:gc.time              avgt    5     32.000                ms
OshiBenchmark.readLines2k                      avgt    5     10.561 ±   0.053   us/op
OshiBenchmark.readLines2k:gc.alloc.rate        avgt    5   1204.201 ±   6.095  MB/sec
OshiBenchmark.readLines2k:gc.alloc.rate.norm   avgt    5  13336.015 ±   0.001    B/op
OshiBenchmark.readLines2k:gc.count             avgt    5     84.000            counts
OshiBenchmark.readLines2k:gc.time              avgt    5     53.000                ms
OshiBenchmark.readLines4k                      avgt    5     10.928 ±   0.203   us/op
OshiBenchmark.readLines4k:gc.alloc.rate        avgt    5   1521.972 ±  28.238  MB/sec
OshiBenchmark.readLines4k:gc.alloc.rate.norm   avgt    5  17440.015 ±   0.001    B/op
OshiBenchmark.readLines4k:gc.count             avgt    5    106.000            counts
OshiBenchmark.readLines4k:gc.time              avgt    5     69.000                ms
OshiBenchmark.readLines512                     avgt    5     10.273 ±   0.148   us/op
OshiBenchmark.readLines512:gc.alloc.rate       avgt    5    952.852 ±  13.679  MB/sec
OshiBenchmark.readLines512:gc.alloc.rate.norm  avgt    5  10264.014 ±   0.001    B/op
OshiBenchmark.readLines512:gc.count            avgt    5     67.000            counts
OshiBenchmark.readLines512:gc.time             avgt    5     48.000                ms
OshiBenchmark.readLines8k                      avgt    5     12.006 ±   0.063   us/op
OshiBenchmark.readLines8k:gc.alloc.rate        avgt    5   2034.124 ±  10.731  MB/sec
OshiBenchmark.readLines8k:gc.alloc.rate.norm   avgt    5  25608.017 ±   0.001    B/op
OshiBenchmark.readLines8k:gc.count             avgt    5    142.000            counts
OshiBenchmark.readLines8k:gc.time              avgt    5     95.000                ms

The gains from going below 1k appear to be very small, but the delta from the default 8k to 1k is big: from 25 kB to 11 kB per invocation.

Copy link

codecov bot commented Mar 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.84%. Comparing base (80ba121) to head (89f5fbd).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2612      +/-   ##
==========================================
+ Coverage   86.23%   86.84%   +0.60%     
==========================================
  Files          30       30              
  Lines        1337     1338       +1     
  Branches      189      189              
==========================================
+ Hits         1153     1162       +9     
- Misses        103      104       +1     
+ Partials       81       72       -9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@dbwiddis dbwiddis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one question

@dbwiddis dbwiddis merged commit 7692f36 into oshi:master Mar 31, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants