Node 16 is end of life (EOL). Time to upgrade

[RayBB]

https://nodejs.org/en/blog/announcements/nodejs16-eol

PS: this update brings the nice fetch api so maybe we can move away from the deprecated request library.

• Node 18 release notes
• Node 20 release notes

Proposal & Constraints

I tried bumping to node 18 and 20 but got some errors in both.

Node 18 error

 => ERROR [21/21] RUN ln -s vendor/infogami/infogami infogami  && make  && python -m pip list --outdated                                         8.3s
------
 > [21/21] RUN ln -s vendor/infogami/infogami infogami  && make  && python -m pip list --outdated:
0.132 git submodule init
0.179 Submodule 'vendor/infogami' (https://github.com/internetarchive/infogami.git) registered for path 'vendor/infogami'
0.179 Submodule 'vendor/js/wmd' (https://github.com/internetarchive/wmd.git) registered for path 'vendor/js/wmd'
0.180 git submodule sync
0.209 Synchronizing submodule url for 'vendor/infogami'
0.209 Synchronizing submodule url for 'vendor/js/wmd'
0.209 git submodule update
0.241 Cloning into '/openlibrary/vendor/infogami'...
1.142 Cloning into '/openlibrary/vendor/js/wmd'...
1.832 Submodule path 'vendor/infogami': checked out 'c50a56933bbf0aec7a746a0cac2aefedb669cca4'
1.871 Submodule path 'vendor/js/wmd': checked out '2e681e2a5827420791ee691a082abf689b6fb3aa'
1.874 mkdir -p static/build
1.875 parallel --verbose -q npx lessc {} static/build/{/.}.css --clean-css="--s1 --advanced" ::: static/css/page-admin.less static/css/page-barcodescanner.less static/css/page-book-widget.less static/css/page-book.less static/css/page-design.less static/css/page-dev.less static/css/page-edit.less static/css/page-form.less static/css/page-home.less static/css/page-list-edit.less static/css/page-lists.less static/css/page-plain.less static/css/page-subject.less static/css/page-team.less static/css/page-user.less
2.016 npx lessc static/css/page-admin.less static/build/page-admin.css --clean-css\=--s1\ --advanced
2.038 npx lessc static/css/page-barcodescanner.less static/build/page-barcodescanner.css --clean-css\=--s1\ --advanced
2.040 npx lessc static/css/page-book-widget.less static/build/page-book-widget.css --clean-css\=--s1\ --advanced
2.042 npx lessc static/css/page-book.less static/build/page-book.css --clean-css\=--s1\ --advanced
2.620 npx lessc static/css/page-design.less static/build/page-design.css --clean-css\=--s1\ --advanced
2.927 npx lessc static/css/page-dev.less static/build/page-dev.css --clean-css\=--s1\ --advanced
3.091 npx lessc static/css/page-edit.less static/build/page-edit.css --clean-css\=--s1\ --advanced
3.398 npx lessc static/css/page-form.less static/build/page-form.css --clean-css\=--s1\ --advanced
3.411 npx lessc static/css/page-home.less static/build/page-home.css --clean-css\=--s1\ --advanced
3.893 npx lessc static/css/page-list-edit.less static/build/page-list-edit.css --clean-css\=--s1\ --advanced
4.462 npx lessc static/css/page-lists.less static/build/page-lists.css --clean-css\=--s1\ --advanced
4.478 npx lessc static/css/page-plain.less static/build/page-plain.css --clean-css\=--s1\ --advanced
4.758 npx lessc static/css/page-subject.less static/build/page-subject.css --clean-css\=--s1\ --advanced
5.283 npx lessc static/css/page-team.less static/build/page-team.css --clean-css\=--s1\ --advanced
5.704 npx lessc static/css/page-user.less static/build/page-user.css --clean-css\=--s1\ --advanced
6.708 mkdir -p static/build
6.709 rm -f static/build/*.js static/build/*.js.map
6.711 npm run build-assets:webpack
6.887 
6.887 > openlibrary@1.0.0 build-assets:webpack
6.887 > env NODE_ENV=production WEBPACK_PACKAGE=webpack5 npx webpack --mode=production
6.887 
7.477 Browserslist: caniuse-lite is outdated. Please run:
7.477   npx browserslist@latest --update-db
7.477   Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
7.687 Browserslist: caniuse-lite is outdated. Please run:
7.687   npx browserslist@latest --update-db
7.687   Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
8.260 node:internal/crypto/hash:69
8.260   this[kHandle] = new _Hash(algorithm, xofLen);
8.260                   ^
8.260 
8.260 Error: error:0308010C:digital envelope routines::unsupported
8.260     at new Hash (node:internal/crypto/hash:69:19)
8.260     at Object.createHash (node:crypto:133:10)
8.260     at BulkUpdateDecorator.hashFactory (/openlibrary/node_modules/webpack5/lib/util/createHash.js:145:18)
8.260     at BulkUpdateDecorator.digest (/openlibrary/node_modules/webpack5/lib/util/createHash.js:80:21)
8.260     at NormalModule._initBuildHash (/openlibrary/node_modules/webpack5/lib/NormalModule.js:872:53)
8.260     at /openlibrary/node_modules/webpack5/lib/NormalModule.js:912:10
8.260     at processResult (/openlibrary/node_modules/webpack5/lib/NormalModule.js:708:12)
8.260     at /openlibrary/node_modules/webpack5/lib/NormalModule.js:807:5
8.260     at /openlibrary/node_modules/webpack5/node_modules/loader-runner/lib/LoaderRunner.js:399:11
8.260     at /openlibrary/node_modules/webpack5/node_modules/loader-runner/lib/LoaderRunner.js:251:18 {
8.260   opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
8.260   library: 'digital envelope routines',
8.260   reason: 'unsupported',
8.260   code: 'ERR_OSSL_EVP_UNSUPPORTED'
8.260 }
8.260 
8.260 Node.js v18.17.1
8.293 make: *** [Makefile:24: js] Error 1
------
Dockerfile.olbase:83
--------------------
  82 |     # run make to initialize git submodules, build css and js files
  83 | >>> RUN ln -s vendor/infogami/infogami infogami \
  84 | >>>  && make \
  85 | >>>  && python -m pip list --outdated
  86 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c ln -s vendor/infogami/infogami infogami  && make  && python -m pip list --outdated" did not complete successfully: exit code: 2

Node 20 error

84.85 npm ERR! code 1
84.85 npm ERR! path /openlibrary/node_modules/iltorb
84.86 npm ERR! command failed
84.86 npm ERR! command sh -c node ./scripts/install.js || node-gyp rebuild
84.86 npm ERR! make: Entering directory '/openlibrary/node_modules/iltorb/build'
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/common/dictionary.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/common/transform.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/dec/bit_reader.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/dec/decode.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/dec/huffman.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/dec/state.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/backward_references.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/backward_references_hq.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/bit_cost.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/block_splitter.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/brotli_bit_stream.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/cluster.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/compress_fragment.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/compress_fragment_two_pass.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/dictionary_hash.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/encode.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/encoder_dict.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/entropy_encode.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/histogram.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/literal_cost.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/memory.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/metablock.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/static_dict.o
84.86 npm ERR!   CC(target) Release/obj.target/iltorb/brotli/c/enc/utf8_util.o
84.86 npm ERR!   CXX(target) Release/obj.target/iltorb/src/common/allocator.o
84.86 npm ERR! make: Leaving directory '/openlibrary/node_modules/iltorb/build'
84.86 npm ERR! info install installing standalone, skipping download.
84.86 npm ERR! gyp info it worked if it ends with ok
84.86 npm ERR! gyp info using node-gyp@9.4.0
84.86 npm ERR! gyp info using node@20.5.1 | linux | arm64
84.86 npm ERR! gyp info find Python using Python version 3.11.1 found at "/usr/local/bin/python3"
84.86 npm ERR! gyp http GET https://nodejs.org/download/release/v20.5.1/node-v20.5.1-headers.tar.gz
84.86 npm ERR! gyp http 200 https://nodejs.org/download/release/v20.5.1/node-v20.5.1-headers.tar.gz
84.86 npm ERR! gyp http GET https://nodejs.org/download/release/v20.5.1/SHASUMS256.txt
84.86 npm ERR! gyp http 200 https://nodejs.org/download/release/v20.5.1/SHASUMS256.txt
84.86 npm ERR! gyp info spawn /usr/local/bin/python3
84.86 npm ERR! gyp info spawn args [
84.86 npm ERR! gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
84.86 npm ERR! gyp info spawn args   'binding.gyp',
84.86 npm ERR! gyp info spawn args   '-f',
84.86 npm ERR! gyp info spawn args   'make',
84.86 npm ERR! gyp info spawn args   '-I',
84.86 npm ERR! gyp info spawn args   '/openlibrary/node_modules/iltorb/build/config.gypi',
84.86 npm ERR! gyp info spawn args   '-I',
84.86 npm ERR! gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
84.86 npm ERR! gyp info spawn args   '-I',
84.86 npm ERR! gyp info spawn args   '/home/openlibrary/.cache/node-gyp/20.5.1/include/node/common.gypi',
84.86 npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
84.86 npm ERR! gyp info spawn args   '-Dvisibility=default',
84.86 npm ERR! gyp info spawn args   '-Dnode_root_dir=/home/openlibrary/.cache/node-gyp/20.5.1',
84.86 npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
84.86 npm ERR! gyp info spawn args   '-Dnode_lib_file=/home/openlibrary/.cache/node-gyp/20.5.1/<(target_arch)/node.lib',
84.86 npm ERR! gyp info spawn args   '-Dmodule_root_dir=/openlibrary/node_modules/iltorb',
84.86 npm ERR! gyp info spawn args   '-Dnode_engine=v8',
84.86 npm ERR! gyp info spawn args   '--depth=.',
84.86 npm ERR! gyp info spawn args   '--no-parallel',
84.86 npm ERR! gyp info spawn args   '--generator-output',
84.86 npm ERR! gyp info spawn args   'build',
84.86 npm ERR! gyp info spawn args   '-Goutput_dir=.'
84.86 npm ERR! gyp info spawn args ]
84.86 npm ERR! gyp info spawn make
84.86 npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
84.86 npm ERR! In file included from ../../nan/nan.h:176,
84.86 npm ERR!                  from ../src/common/allocator.cc:1:
84.86 npm ERR! ../../nan/nan_callbacks.h:55:23: error: 'AccessorSignature' is not a member of 'v8'
84.86 npm ERR!    55 | typedef v8::Local<v8::AccessorSignature> Sig;
84.86 npm ERR!       |                       ^~~~~~~~~~~~~~~~~
84.86 npm ERR! ../../nan/nan_callbacks.h:55:40: error: template argument 1 is invalid
84.86 npm ERR!    55 | typedef v8::Local<v8::AccessorSignature> Sig;
84.86 npm ERR!       |                                        ^
84.86 npm ERR! In file included from ../src/common/allocator.cc:1:
84.86 npm ERR! ../../nan/nan.h: In function 'bool Nan::IdleNotification(int)':
84.86 npm ERR! ../../nan/nan.h:683:32: warning: 'bool v8::Isolate::IdleNotificationDeadline(double)' is deprecated: Use MemoryPressureNotification() to influence the GC schedule. [-Wdeprecated-declarations]
84.86 npm ERR!   683 |         idle_time_in_ms * 0.001);
84.86 npm ERR!       |                                ^
84.86 npm ERR! In file included from /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-initialization.h:13,
84.86 npm ERR!                  from /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8.h:34,
84.86 npm ERR!                  from /home/openlibrary/.cache/node-gyp/20.5.1/include/node/node.h:73,
84.86 npm ERR!                  from ../../nan/nan.h:56,
84.86 npm ERR!                  from ../src/common/allocator.cc:1:
84.86 npm ERR! /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-isolate.h:1293:8: note: declared here
84.86 npm ERR!  1293 |   bool IdleNotificationDeadline(double deadline_in_seconds);
84.86 npm ERR!       |        ^~~~~~~~~~~~~~~~~~~~~~~~
84.86 npm ERR! In file included from ../src/common/allocator.cc:1:
84.86 npm ERR! ../../nan/nan.h: In function 'void Nan::AsyncQueueWorker(Nan::AsyncWorker*)':
84.86 npm ERR! ../../nan/nan.h:2294:7: warning: cast between incompatible function types from 'void (*)(uv_work_t*)' {aka 'void (*)(uv_work_s*)'} to 'uv_after_work_cb' {aka 'void (*)(uv_work_s*, int)'} [-Wcast-function-type]
84.86 npm ERR!  2294 |     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
84.86 npm ERR!       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
84.86 npm ERR! ../../nan/nan.h: In function 'void Nan::SetAccessor(v8::Local<v8::ObjectTemplate>, v8::Local<v8::String>, Nan::GetterCallback, Nan::SetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, Nan::imp::Sig)':
84.86 npm ERR! ../../nan/nan.h:2542:16: error: no matching function for call to 'v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>&, void (*&)(v8::Local<v8::Name>, const v8::PropertyCallbackInfo<v8::Value>&), void (*&)(v8::Local<v8::Name>, v8::Local<v8::Value>, const v8::PropertyCallbackInfo<void>&), v8::Local<v8::Object>&, v8::AccessControl&, v8::PropertyAttribute&, Nan::imp::Sig&)'
84.86 npm ERR!  2542 |     , signature);
84.86 npm ERR!       |                ^
84.86 npm ERR! In file included from /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-function.h:15,
84.86 npm ERR!                  from /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8.h:33,
84.86 npm ERR!                  from /home/openlibrary/.cache/node-gyp/20.5.1/include/node/node.h:73,
84.86 npm ERR!                  from ../../nan/nan.h:56,
84.86 npm ERR!                  from ../src/common/allocator.cc:1:
84.86 npm ERR! /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-template.h:809:8: note: candidate: 'void v8::ObjectTemplate::SetAccessor(v8::Local<v8::String>, v8::AccessorGetterCallback, v8::AccessorSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)'
84.86 npm ERR!   809 |   void SetAccessor(
84.86 npm ERR!       |        ^~~~~~~~~~~
84.86 npm ERR! /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-template.h:814:22: note:   no known conversion for argument 7 from 'Nan::imp::Sig' {aka 'int'} to 'v8::SideEffectType'
84.86 npm ERR!   814 |       SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect,
84.86 npm ERR!       |       ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
84.86 npm ERR! /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-template.h:816:8: note: candidate: 'void v8::ObjectTemplate::SetAccessor(v8::Local<v8::Name>, v8::AccessorNameGetterCallback, v8::AccessorNameSetterCallback, v8::Local<v8::Value>, v8::AccessControl, v8::PropertyAttribute, v8::SideEffectType, v8::SideEffectType)'
84.86 npm ERR!   816 |   void SetAccessor(
84.87 npm ERR!       |        ^~~~~~~~~~~
84.87 npm ERR! /home/openlibrary/.cache/node-gyp/20.5.1/include/node/v8-template.h:821:22: note:   no known conversion for argument 7 from 'Nan::imp::Sig' {aka 'int'} to 'v8::SideEffectType'
84.87 npm ERR!   821 |       SideEffectType getter_side_effect_type = SideEffectType::kHasSideEffect,
84.87 npm ERR!       |       ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
84.87 npm ERR! make: *** [iltorb.target.mk:150: Release/obj.target/iltorb/src/common/allocator.o] Error 1
84.87 npm ERR! gyp ERR! build error 
84.87 npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
84.87 npm ERR! gyp ERR! stack     at ChildProcess.onExit (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:203:23)
84.87 npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:514:28)
84.87 npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
84.87 npm ERR! gyp ERR! System Linux 5.15.49-linuxkit-pr
84.87 npm ERR! gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
84.87 npm ERR! gyp ERR! cwd /openlibrary/node_modules/iltorb
84.87 npm ERR! gyp ERR! node -v v20.5.1
84.87 npm ERR! gyp ERR! node-gyp -v v9.4.0
84.87 npm ERR! gyp ERR! not ok
84.87 
84.87 npm ERR! A complete log of this run can be found in: /home/openlibrary/.npm/_logs/2023-09-01T19_57_55_465Z-debug-0.log

Related files

Merging #8256 will make this upgrade a little faster ;)

Stakeholders

@cclauss @jimman2003 (who did the last upgrade #6037)

[jimman2003]

According to this Webpack issue we need to update webpack 5 and remove webpack 4 (if it is used anymore)

[cclauss]

we need to update webpack 5

https://github.com/internetarchive/openlibrary/pulls/app%2Frenovate

• Update dependency webpack to v5 - autoclosed #8066
• Update dependency webpack5 to v5.88.2 - autoclosed #8062
• Update dependency webpack-cli to v5 - autoclosed #8067
• Update dependency workbox-webpack-plugin to v6.6.0 #8063

@jimchamp @cdrini As you guys know, when I attempt to do JS upgrades, stuff just breaks!

It might be useful to have a version of #8066 that passes our GitHub Actions tests.

[RayBB]

Ah... so it seems that somewhat like this issue of upgrading webpack might be blocked by upgrading Vue.
Whenever we touch webpack 5 we get this error:
vuejs/vue-cli#5986

Edit which seems somewhat blocked by eslint... which is somewhat blocked by babel? #6496

[jimman2003]

I also think vue-cli is one of the causes here , vue-cli v4 has a webpack 4 dependancy, v5 has a webpack 5 one