Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(memory-allocator): use mimalloc-v3 on macOS and bump mimalloc to v2.1.7 on other platforms #9533

Merged
merged 3 commits into from
Mar 4, 2025

Conversation

h-a-n-a
Copy link
Contributor

@h-a-n-a h-a-n-a commented Mar 3, 2025

Summary

  1. Bumped mimalloc to v3 for macOS.

Lifted issue in #8976

According to my investigation, although memory is still growing, upgrading mimalloc to v3 will greatly mitigate the memory bloating issue in development mode. That is to say, memory growing is slower than it was on mimalloc-v2 for each time the compiler rebuilds:

With mimalloc dev3-bin:

hyperfine --warmup 2 --runs 5 './target/release/mimalloc-test'
Benchmark 1: ./target/release/mimalloc-test
  Time (mean ± σ):      9.230 s ±  0.157 s    [User: 28.923 s, System: 13.791 s]
  Range (min … max):    9.052 s …  9.450 s    5 runs

With mimalloc dev2-bin (rspack main):

hyperfine --warmup 2 --runs 5 './target/release/mimalloc-test'
Benchmark 1: ./target/release/mimalloc-test
  Time (mean ± σ):      9.360 s ±  0.126 s    [User: 29.537 s, System: 14.543 s]
  Range (min … max):    9.216 s …  9.525 s    5 runs

Case: building 10000 modules and rebuild 10 times, for each time a new console.log will be written to the entry module.

  1. Bumped mimalloc to v2.1.7 for non-macOS platforms.

This is the latest version of mimalloc-v2. We don't want to rollout too much change at once. So we're staying at v2 at least for now.

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

Sorry, something went wrong.

@github-actions github-actions bot added release: feature release: feature related release(mr only) team The issue/pr is created by the member of Rspack. labels Mar 3, 2025
Copy link

netlify bot commented Mar 3, 2025

Deploy Preview for rspack canceled.

Name Link
🔨 Latest commit 3d971bb
🔍 Latest deploy log https://app.netlify.com/sites/rspack/deploys/67c68934ba8230000842a09f

@h-a-n-a h-a-n-a requested a review from chenjiahan March 3, 2025 10:24

This comment was marked as outdated.

Copy link

codspeed-hq bot commented Mar 3, 2025

CodSpeed Performance Report

Merging #9533 will not alter performance

Comparing mimalloc-v3 (3d971bb) with main (1c977b4)

Summary

✅ 7 untouched benchmarks

chenjiahan
chenjiahan previously approved these changes Mar 3, 2025

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

Copy link
Contributor

github-actions bot commented Mar 4, 2025

📝 Benchmark detail: Open

Name Base (2025-03-04 ec71bd0) Current Change
10000_big_production-mode_disable-minimize + exec 36.7 s ± 403 ms 37 s ± 521 ms +0.87 %
10000_development-mode + exec 1.71 s ± 20 ms 1.66 s ± 28 ms -3.09 %
10000_development-mode_hmr + exec 681 ms ± 18 ms 666 ms ± 7.4 ms -2.24 %
10000_production-mode + exec 2.21 s ± 117 ms 2.22 s ± 136 ms +0.09 %
10000_production-mode_persistent-cold + exec 2.33 s ± 50 ms 2.25 s ± 140 ms -3.47 %
10000_production-mode_persistent-hot + exec 1.64 s ± 26 ms 1.57 s ± 32 ms -4.17 %
arco-pro_development-mode + exec 1.73 s ± 128 ms 1.73 s ± 121 ms +0.34 %
arco-pro_development-mode_hmr + exec 375 ms ± 1.9 ms 374 ms ± 2.6 ms -0.15 %
arco-pro_production-mode + exec 3.54 s ± 113 ms 3.45 s ± 102 ms -2.53 %
arco-pro_production-mode_generate-package-json-webpack-plugin + exec 3.58 s ± 191 ms 3.55 s ± 219 ms -1.10 %
arco-pro_production-mode_persistent-cold + exec 3.56 s ± 148 ms 3.53 s ± 107 ms -0.99 %
arco-pro_production-mode_persistent-hot + exec 2.25 s ± 115 ms 2.34 s ± 104 ms +3.90 %
arco-pro_production-mode_traverse-chunk-modules + exec 3.53 s ± 177 ms 3.47 s ± 166 ms -1.54 %
large-dyn-imports_development-mode + exec 1.95 s ± 58 ms 1.92 s ± 48 ms -1.78 %
large-dyn-imports_production-mode + exec 2 s ± 57 ms 1.99 s ± 57 ms -0.25 %
threejs_development-mode_10x + exec 1.43 s ± 35 ms 1.44 s ± 20 ms +0.11 %
threejs_development-mode_10x_hmr + exec 786 ms ± 10 ms 784 ms ± 14 ms -0.31 %
threejs_production-mode_10x + exec 5.03 s ± 77 ms 4.94 s ± 67 ms -1.76 %
threejs_production-mode_10x_persistent-cold + exec 5.13 s ± 357 ms 4.99 s ± 47 ms -2.79 %
threejs_production-mode_10x_persistent-hot + exec 4.47 s ± 335 ms 4.34 s ± 336 ms -2.84 %
10000_big_production-mode_disable-minimize + rss memory 8698 MiB ± 78.5 MiB 9460 MiB ± 43.5 MiB +8.76 %
10000_development-mode + rss memory 678 MiB ± 11.8 MiB 689 MiB ± 31.2 MiB +1.71 %
10000_development-mode_hmr + rss memory 1247 MiB ± 285 MiB 1470 MiB ± 239 MiB +17.91 %
10000_production-mode + rss memory 650 MiB ± 26.5 MiB 688 MiB ± 28.3 MiB +5.95 %
10000_production-mode_persistent-cold + rss memory 762 MiB ± 20 MiB 769 MiB ± 74.6 MiB +0.91 %
10000_production-mode_persistent-hot + rss memory 735 MiB ± 11.5 MiB 758 MiB ± 50.3 MiB +3.08 %
arco-pro_development-mode + rss memory 636 MiB ± 21.9 MiB 595 MiB ± 56.6 MiB -6.36 %
arco-pro_development-mode_hmr + rss memory 709 MiB ± 21.4 MiB 626 MiB ± 80.1 MiB -11.68 %
arco-pro_production-mode + rss memory 776 MiB ± 38.8 MiB 713 MiB ± 72.3 MiB -8.15 %
arco-pro_production-mode_generate-package-json-webpack-plugin + rss memory 792 MiB ± 25.6 MiB 731 MiB ± 21.6 MiB -7.73 %
arco-pro_production-mode_persistent-cold + rss memory 863 MiB ± 20.6 MiB 795 MiB ± 83.3 MiB -7.92 %
arco-pro_production-mode_persistent-hot + rss memory 832 MiB ± 32 MiB 807 MiB ± 52 MiB -2.98 %
arco-pro_production-mode_traverse-chunk-modules + rss memory 800 MiB ± 35.8 MiB 758 MiB ± 66.4 MiB -5.34 %
large-dyn-imports_development-mode + rss memory 697 MiB ± 17 MiB 667 MiB ± 10.3 MiB -4.40 %
large-dyn-imports_production-mode + rss memory 576 MiB ± 3.62 MiB 561 MiB ± 10.3 MiB -2.64 %
threejs_development-mode_10x + rss memory 651 MiB ± 59.5 MiB 649 MiB ± 5.77 MiB -0.24 %
threejs_development-mode_10x_hmr + rss memory 974 MiB ± 301 MiB 1022 MiB ± 321 MiB +4.87 %
threejs_production-mode_10x + rss memory 919 MiB ± 56.1 MiB 960 MiB ± 78.3 MiB +4.45 %
threejs_production-mode_10x_persistent-cold + rss memory 1050 MiB ± 71.5 MiB 1043 MiB ± 41.1 MiB -0.66 %
threejs_production-mode_10x_persistent-hot + rss memory 916 MiB ± 60.7 MiB 897 MiB ± 30.5 MiB -2.09 %

@h-a-n-a h-a-n-a enabled auto-merge (squash) March 4, 2025 05:01
@h-a-n-a h-a-n-a disabled auto-merge March 4, 2025 05:01
@h-a-n-a h-a-n-a changed the title feat(memory-allocator): use mimalloc-v3 on macOS and bump mimalloc to v2.1.9 on other platforms feat(memory-allocator): use mimalloc-v3 on macOS and bump mimalloc to v2.1.7 on other platforms Mar 4, 2025
@h-a-n-a
Copy link
Contributor Author

h-a-n-a commented Mar 4, 2025

Downgraded mimalloc from v2.1.9 to v2.1.7 as windows was not being able to compile.

@h-a-n-a h-a-n-a enabled auto-merge (squash) March 4, 2025 05:03
@h-a-n-a h-a-n-a requested a review from chenjiahan March 4, 2025 05:07
@h-a-n-a h-a-n-a merged commit 9e1bd1a into main Mar 4, 2025
31 checks passed
@h-a-n-a h-a-n-a deleted the mimalloc-v3 branch March 4, 2025 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release: feature release: feature related release(mr only) team The issue/pr is created by the member of Rspack.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants