Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patches for 0.59.1 #9480

Merged
merged 16 commits into from
Mar 18, 2024
Merged

Patches for 0.59.1 #9480

merged 16 commits into from
Mar 18, 2024

Conversation

sklam
Copy link
Member

@sklam sklam commented Mar 7, 2024

  • Cherrypick for 0.59.1
  • change log
  • skipped PPC test failure for test_issue9490_non_det_ssa_problem

TODO:

@sklam sklam added the skip_release_notes Skip towncrier requirement label Mar 7, 2024
kc611 and others added 2 commits March 7, 2024 11:28
Fix Issue numba#9432, caching of kernels using target-specific overloads (such as CG sync)
@sklam sklam added this to the 0.59.1 milestone Mar 7, 2024
@sklam
Copy link
Member Author

sklam commented Mar 7, 2024

BFID: numba_yaml_439

@sklam sklam mentioned this pull request Mar 8, 2024
30 tasks
@sklam sklam marked this pull request as ready for review March 8, 2024 15:04
@sklam sklam requested a review from gmarkall as a code owner March 8, 2024 15:04
@sklam sklam added the 4 - Waiting on reviewer Waiting for reviewer to respond to author label Mar 8, 2024
@sklam sklam requested a review from kc611 March 8, 2024 15:05
@sklam
Copy link
Member Author

sklam commented Mar 8, 2024

#9480 (comment)

passed. subsequent commits have no code changes.

@sklam sklam added the BuildFarm Passed For PRs that have been through the buildfarm and passed label Mar 8, 2024
kc611
kc611 previously approved these changes Mar 11, 2024
@sklam sklam added 2 - In Progress and removed 4 - Waiting on reviewer Waiting for reviewer to respond to author labels Mar 12, 2024
@sklam
Copy link
Member Author

sklam commented Mar 12, 2024

Waiting for release blocker fix

Fix non-deterministic bug caused by unstableness in SSA
@sklam
Copy link
Member Author

sklam commented Mar 15, 2024

numba_yaml_440

@sklam sklam removed 2 - In Progress BuildFarm Passed For PRs that have been through the buildfarm and passed labels Mar 15, 2024
@sklam sklam added 3 - Ready for Review Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm labels Mar 15, 2024
@sklam sklam requested a review from kc611 March 15, 2024 19:58
…count.

The problem is likely bad interaction between TBB and MKL when number of thread is high (72 on buildfarm).
@sklam
Copy link
Member Author

sklam commented Mar 16, 2024

BFID: numba_yaml_441

@sklam
Copy link
Member Author

sklam commented Mar 17, 2024

Note for reviewer:

After cherrypicks and changelog update, the following tests adjustments to test_issue9490_non_det_ssa_problem were made:

  • skip LLVM codegen failure on PPC (db03908)
AssertionError: subprocess failed with output:
Invalid PPC CTR loop!
UNREACHABLE executed at /opt/conda/conda-bld/llvmdev_1684265335202/work/llvm/lib/Target/PowerPC/PPCCTRLoops.cpp:179!
Fatal Python error: Aborted

I believe this is purely a LLVM codegen problem.

  • On linux-64 py3.9 npy1.25 , the buildfarm consistently fail the test due to a TBB problem (b39bdf2).
    GDB points to:
(gdb) info threads
  Id   Target Id                                    Frame
* 1    Thread 0x7ffff7c39280 (LWP 3954038) "python" __pthread_clockjoin_ex (threadid=140736834631424, thread_return=0x0,
    clockid=<optimized out>, abstime=<optimized out>, block=<optimized out>) at pthread_join_common.c:145
  2    Thread 0x7fffdc49c700 (LWP 3954042) "python" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  3    Thread 0x7fffdc09b700 (LWP 3954043) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  4    Thread 0x7fffdbc9a700 (LWP 3954044) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  5    Thread 0x7fffdb899700 (LWP 3954045) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  6    Thread 0x7fffdb498700 (LWP 3954047) "python" syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
  7    Thread 0x7fffdac96700 (LWP 3954048) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  8    Thread 0x7fffdb097700 (LWP 3954046) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  12   Thread 0x7fffd9c92700 (LWP 3954054) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  13   Thread 0x7fffd9490700 (LWP 3954053) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  14   Thread 0x7fffda895700 (LWP 3954050) "python" 0x00007ffff7d3f89b in sched_yield () at ../sysdeps/unix/syscall-template.S:78
  15   Thread 0x7fffd908f700 (LWP 3954055) "python" 0x00007ffff5b4349a in __kmp_acquire_ticket_lock ()
   from /home/siu/miniconda3/envs/testenv_bad/lib/python3.9/site-packages/mkl/../../../libiomp5.so

I believe this is bad interaction between TBB and MKL OMP on a machine that has 72 cpu cores. The resolution is to force the numba code to use only 1-thread.

Co-authored-by: Kaustubh <ckaustubhm06@gmail.com>
@sklam sklam added 5 - Ready to merge Review and testing done, is ready to merge and removed 3 - Ready for Review labels Mar 18, 2024
@sklam sklam merged commit 9ce83ef into numba:release0.59 Mar 18, 2024
21 checks passed
@sklam sklam deleted the misc/rel0.59.1_patches branch March 18, 2024 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to merge Review and testing done, is ready to merge Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm skip_release_notes Skip towncrier requirement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants