Action updating visual test reference snapshots fails

[krassowski]

As seen in a few recent PRs the action updating snapshots fails:

• Fixes focus indicator on input checkbox for Firefox #15612 https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637
• Dark High Contrast Theme for JupyterLab with Improved Sidebar Focus Indication #15623 https://github.com/jupyterlab/jupyterlab/actions/runs/7472322849
• 400% Zoom screen usability enhancements - Updated #14766 (comment) https://github.com/jupyterlab/jupyterlab/actions/runs/7461896604

It seems to have last worked on

• 29th Dec 2023 for 4.0.x backports (Backport PR #15524: Fix visual tests #15578 (comment), https://github.com/jupyterlab/jupyterlab/actions/runs/7356925384/job/20027751189)
• 27th Dec 2023 for main https://github.com/jupyterlab/jupyterlab/actions/runs/7338404549/job/19980875086

and was no longer working by 6th Jan 2024 (#15612 (comment))

Run jupyterlab/maintainer-tools/.github/actions/update-snapshots@v1
  with:
    github_token: ***
    server_url: http-get://localhost:8888/lab
    test_folder: galata
    artifact_name: updated-galata-snapshots
    report_name: update-galata-report
    npm_client: yarn
    browser: chromium
    continue_on_error: yes
    dry_run: no
    git_username: github-actions[bot]
    git_email: github-actions[bot]@users.noreply.github.com
    start_server_script: start
    update_script: test:update
  env:
    PYTHON_VERSION: 3.11
    NODE_VERSION: 18.x
    CACHE_PREFIX: Linux-Update Playwright Snapshots-update-galata-snapshots
    DEPENDENCY_TYPE: standard
    pythonLocation: /opt/hostedtoolcache/Python/3.11.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python[2](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:2)_ROOT_DIR: /opt/hostedtoolcache/Python/[3](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:3).11.7/x6[4](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:4)
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib
Run (yarn run start 1>/tmp/snapshots-update-server.log 2>&1) &
  (yarn run start 1>/tmp/snapshots-update-server.log 2>&1) &
  shell: /usr/bin/bash -l {0}
  env:
    PYTHON_VERSION: 3.11
    NODE_VERSION: 18.x
    CACHE_PREFIX: Linux-Update Playwright Snapshots-update-galata-snapshots
   ``` DEPENDENCY_TYPE: standard
    pythonLocation: /opt/hostedtoolcache/Python/3.11.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib
Run yarn install
/home/runner/work/_temp/[5](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:5)413518f-ee12-4f[6](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:6)3-b[7](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:7)df-[8](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:8)5cea0c4[9](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:9)52f.sh: line 1: yarn: command not found
/home/runner/work/_temp/5413518f-ee12-4f63-b7df-85cea0c4952f.sh: line 2: yarn: command not found
Error: Process completed with exit code 127.
Run echo "::group::Server log"
  echo "::group::Server log"
  cat /tmp/snapshots-update-server.log
  echo "::endgroup::"
  shell: /usr/bin/bash -l {0}
  env:
    PYTHON_VERSION: 3.[11](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:11)
    NODE_VERSION: 18.x
    CACHE_PREFIX: Linux-Update Playwright Snapshots-update-galata-snapshots
    DEPENDENCY_TYPE: standard
    pythonLocation: /opt/hostedtoolcache/Python/3.11.7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.11.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.7/x64/lib
Server log
  /home/runner/work/_temp/5f502a24-5f43-4dd6-8417-8[16](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:16)[30](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:31)c3f[60](https://github.com/jupyterlab/jupyterlab/actions/runs/7501823637/job/20423240394#step:8:63)7b.sh: line 1: yarn: command not found

[krassowski]

There was one commit to the main branch of https://github.com/jupyterlab/maintainer-tools in this range of dates, jupyterlab/maintainer-tools@fe20cc9 @blink1073 do you see any way it could have been related?

[blink1073]

Nothing in that diff looks relevant, and oddly enough yarn is available in the Base Setup section but not in the update-snapshots section.

[krassowski]

Comparing the successful and failing jobs, there is a difference in Install dependencies step (defined in jupyterlab workflow)

- + '[' linux-gnu == Linux ']'
+ + '[' linux-gnu == linux-gnu ']'
+ ++ which yarn
+ + sudo rm -rf /usr/local/bin/yarn
+ + yarn
+ ./scripts/ci_install.sh: line 19: yarn: command not found
  + mkdir -p /home/runner/.jupyter
  + git config --global user.name foo
  + git config --global user.email foo@bar.com
  + pip install -q --upgrade pip --user
  + pip --version

and later:

- + yarn --version
- 3.5.0
  + node -p process.versions

So it looks like this must have been introduced by #11801, CC @jtpio

[krassowski]

I guess we can try switching to using jlpm as an npm_client for the update-snapshots action, but it is still not great that there is unhandled and silent error in ci install script, this one:

yarn
./scripts/ci_install.sh: line 19: yarn: command not found

[jtpio]

switching to using jlpm as an npm-command for the update-snapshots action

Maybe this could also help when jlpm is updated to a newer of yarn, so we use the same version everywhere.

[brichet]

Unfortunately, I don't think this issue is fixed yet:
https://github.com/jupyterlab/jupyterlab/actions/runs/7538551202/job/20519297029, it includes the changes on the workflow file.

[krassowski]

Right sorry - it works for main snapshots but not for docs; PR on the way.