You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Working in my enterprise context where we process large folders of ".tiff" files (>100k images), i found that usage of src/PIL/TiffImagePlugin.py still cause the well known error "Too many open files" due to memory storage i suppose is related to file handlers. (from stackoverflow posts)
I found a fast-fix for that by adding the following snippet of code to TiffImagePlugin.py :
if_fp:
try:
os.close(_fp)
exceptOSError:
pass
I propose to add this block after the if-else block in TiffImagePlugin.py like that:
if "_debug_multipage" in encoderinfo:
...
if_fp:
try:
os.close(_fp)
exceptOSError:
pass
I found this solution to correctly solve my issue empirically, avoiding "too many open files" error when working with large images directories.
PS:
I am a neophyte to github pull/branch/merge/commit system, so excuse me if i did not proposed the fix in the correct way. I just would like to help you, since Pillow is a very useful and lovely library.
The text was updated successfully, but these errors were encountered:
Hi. You've created a issue, where people typically report problems they'd like fixed or ask questions they'd like answered. You have a problem that you'd like solved, so this is fine.
To formally suggest a code change, people create pull requests, but I have some comments.
Although you say "too many open files" is a "well known error", I would find it easier to deal with a specific problem. If you could post some example code, and what operating system and Pillow version you're using, that would be helpful.
Are you opening TIFFs, saving TIFFs, or both? If you're opening TIFFs, I wonder if PR Do not close provided file handles with libtiff #7199 would solve your problem. If you're unable to compile Pillow from source to try it out, let us know what operating system and Python version you're using, and we should be able to put together a wheel you can install to test it.
Would you agree that we've already run your proposed addition a few lines earlier?
Working in my enterprise context where we process large folders of ".tiff" files (>100k images), i found that usage of src/PIL/TiffImagePlugin.py still cause the well known error "Too many open files" due to memory storage i suppose is related to file handlers. (from stackoverflow posts)
I found a fast-fix for that by adding the following snippet of code to TiffImagePlugin.py :
I propose to add this block after the if-else block in TiffImagePlugin.py like that:
I found this solution to correctly solve my issue empirically, avoiding "too many open files" error when working with large images directories.
PS:
I am a neophyte to github pull/branch/merge/commit system, so excuse me if i did not proposed the fix in the correct way. I just would like to help you, since Pillow is a very useful and lovely library.
The text was updated successfully, but these errors were encountered: