About Error FileNotFoundError: [Errno 2] No such file or directory: '/proc/12825/stat' the some suggest

[IT-IvDragon]

when I was useing the function process_iter discover a not myself can catch the error。
For errors like FileNotFoundError: [Errno 2] No such file or directory: '/proc/xxx/stat'。
I searched for problems in lssues , but found that the latest version didn't fix them. Through the source code found that this function only catch the NoSuchProcess exception，and this error I think is a problem with pids functino，pids from /proc dir fetch a unusua path，lead to open the path /proc/xxx/stat raised an exception，so we can do try/catch the FileNotFoundError error at the open file location。

[giampaolo]

I cannot understand much from your report. Please paste your code and the full traceback message.

[A1ines]

I cannot understand much from your report. Please paste your code and the full traceback message.

I encountered the same problem.
[Errno 2] No such file or directory: '/proc/1535931/stat'
After investigating,
I found that there are some anomalies with the /proc/1535931 directory. It doesn't contain the stat file, which causes the read error.

Since it runs on a server and I don't have permission to delete the anomalous process directory, I had to use a simple but effective method: skipping this PID.

Successfully run after modification.

[IT-IvDragon]

you can in the function process_iter add try finally ，skip the pid when happen error。

[giampaolo]

Are you saying file /proc/1535931/stat does not exist but directory /proc/1535931 exists? If that's the case, what's the content of directory /proc/1535931?

[IT-IvDragon]

yes ，I met the problem just not exist stat file， the other file/directory in directory /proc/1535931 is exist，I think is Linux system because of some special circumstances not create the stat Or the /proc/1535931 directory wasn't deleted

[giampaolo]

Can you do ls -la /proc/1535931 and paste the output?

[IT-IvDragon]

I tried ls ，but cannot find the progress id ，just the /proc/1535931 dir

[giampaolo]

So the /proc/1535931 directory is empty?

[IT-IvDragon]

not empty, there are some files, but no stat file

[giampaolo]

OK, paste those files. Paste the output of ls -la /proc/1535931 command.

[IT-IvDragon]

@A1ines

[pierresouchay]

Same Error here, a random error that happens not very often at all (I would say once every 1k calls, we use this to list the most expensive processes in RAM periodically, and rarely, in out CI, it fails:

INFO -   File "/var/env/home/.venv/lib/python3.11/site-packages/psutil/_common.py", line 340, in wrapper
INFO -     ret = self._cache[fun]
INFO -           ^^^^^^^^^^^
INFO - AttributeError: 'Process' object has no attribute '_cache'
INFO - 
INFO - During handling of the above exception, another exception occurred:
INFO - 
INFO - Traceback (most recent call last):
INFO -   File "/var/env/home/.venv/lib/python3.11/site-packages/psutil/_pslinux.py", line 1517, in wrapper
INFO -     return fun(self, *args, **kwargs)
INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO -   File "/var/env/home/.venv/lib/python3.11/site-packages/psutil/_pslinux.py", line 1751, in memory_info
INFO -     with open_binary("%s/%s/statm" % (self._procfs_path, self.pid)) as f:
INFO -          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO -   File "/var/env/home/.venv/lib/python3.11/site-packages/psutil/_common.py", line 604, in open_binary
INFO -     return open(fname, "rb", **kwargs)
INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO - FileNotFoundError: [Errno 2] No such file or directory: '/proc/641/statm'

=> Looks like a race concurrency issue.

When this happens, while the call: proc.memory_info().rss fails with FileNotFoundError, the call proc.cmdline() fails with psutil.NoSuchProcess

[giampaolo]

@pierresouchay mmm this indeed appears to be a race condition. My interpretation is the following:

• /proc/PID/statm (or other similar files) disappears, causing FileNotFoundError. That means the process is gone:
  

  psutil/psutil/_pslinux.py

  Line 1975 in 0866e1e

  with open_binary("%s/%s/statm" % (self._procfs_path, self.pid)) as f:

• despite /proc/PID/statm is gone, /proc/PID/ directory still exists, so we let FileNotFoundError propagate instead of turning it into NoSuchProcess:
  

  psutil/psutil/_pslinux.py

  Lines 1723 to 1727 in 0866e1e

  	except FileNotFoundError:
  	self._raise_if_zombie()
  	if not os.path.exists("%s/%s" % (self._procfs_path, self.pid)):
  	raise NoSuchProcess(self.pid, self._name)
  	raise

My interpretation is that, when a process is terminated, /proc/PID/statm and /proc/PID may not disappear at the same time. /proc/PID may still stick around for some time.

[pierresouchay]

My interpretation is that, when a process is terminated, /proc/PID/statm and /proc/PID may not disappear at the same time. /proc/PID may still stick around for some time.

Yes, I have the same interpretation (and it makes sense, likely the kernel might not remove all the underlying files and /proc/PID in an atomic call (most likely iteratively cleaned up during the various phases of process cleanup

[giampaolo]

Mmm this is a hard one to fix. It's not obvious how we can distinguish between a /proc file that legitimately does not exist from "process is gone".

[giampaolo]

It would be very useful to know what files /proc/PID contains when such a race condition occurs. Is it empty? Does it have a portion of all files?

An easy way to solve this would be the following:

--- a/psutil/_pslinux.py
+++ b/psutil/_pslinux.py
@@ -1722,7 +1722,9 @@ def wrap_exceptions(fun):
             raise NoSuchProcess(self.pid, self._name)
         except FileNotFoundError:
             self._raise_if_zombie()
-            if not os.path.exists("%s/%s" % (self._procfs_path, self.pid)):
+            if not os.path.exists("%s/%s/stat" % (self._procfs_path, self.pid)):
                 raise NoSuchProcess(self.pid, self._name)
             raise

We assume that if /proc/PID/stat does not exist the process is gone.

[pierresouchay]

Does it have a portion of all files?

I have no clue... but it is hard to get, since it is a race condition. In my understanding, your patch looks like a safe bet: /proc/PID/stat not existing -> likely no such process even if /proc/PID is found (I think that's the case for zombie processes for instance)

[pierresouchay]

Thank you @giampaolo : you rock!