Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

CPU unit tests are failing in vision nightly builds #8051

Closed
izaitsevfb opened this issue Oct 17, 2023 · 5 comments 路 Fixed by #8052
Closed

CPU unit tests are failing in vision nightly builds #8051

izaitsevfb opened this issue Oct 17, 2023 · 5 comments 路 Fixed by #8052
Assignees

Comments

@izaitsevfb
Copy link

馃悰 Describe the bug

HUD: https://hud.pytorch.org/hud/pytorch/vision/nightly/1?per_page=50&name_filter=tests%20%2F%20unittests

Looks like the same issue across platforms:

2023-10-16T11:57:47.9045908Z 锟絒31mFAILED锟絒0m test/test_utils.py::锟絒1mtest_draw_boxes锟絒0m - AssertionError: Tensor-likes are not equal!
2023-10-16T11:57:47.9046508Z 
2023-10-16T11:57:47.9046642Z Mismatched elements: 240 / 30000 (0.8%)
2023-10-16T11:57:47.9047134Z Greatest absolute difference: 251 at index (0, 10, 5)
2023-10-16T11:57:47.9047689Z Greatest relative difference: inf at index (0, 8, 4)
2023-10-16T11:57:47.9048831Z 锟絒31m= 锟絒31m锟絒1m1 failed锟絒0m, 锟絒32m21402 passed锟絒0m, 锟絒33m16844 skipped锟絒0m, 锟絒33m1 xfailed锟絒0m, 锟絒33m251 warnings锟絒0m锟絒31m in 1206.38s (0:20:06)锟絒0m锟絒31m =锟絒0m
2023-10-16T11:57:47.9062170Z ##[error]Process completed with exit code 1.

cc @NicolasHug @pmeier @atalman

Versions

null

@pmeier
Copy link
Collaborator

pmeier commented Oct 17, 2023

We haven't merged anything the last few days. Meaning, this likely comes from core. Let me dig and see if I can find an offending PR.

@pmeier
Copy link
Collaborator

pmeier commented Oct 17, 2023

Could also come from Pillow. Error points to

vision/test/test_utils.py

Lines 108 to 111 in 1d6f787

if PILLOW_VERSION >= (8, 2):
# The reference image is only valid for new PIL versions
expected = torch.as_tensor(np.array(Image.open(path))).permute(2, 0, 1)
assert_equal(result, expected)

and Pillow==10.1 was released two days ago.

@pmeier
Copy link
Collaborator

pmeier commented Oct 17, 2023

Can reproduce with Pillow==10.1.0.

@pmeier pmeier self-assigned this Oct 17, 2023
@pmeier
Copy link
Collaborator

pmeier commented Oct 17, 2023

The default font has changed. See python-pillow/Pillow#7354

reference / new

reference new

@pmeier
Copy link
Collaborator

pmeier commented Oct 17, 2023

There currently seems to be no way to force the old font. I'll ask for clarification there. In the mean time, we could temporarily pin to pillow<10.1 until we have a plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants