[GH-ISSUE #71] [Question] process.cpu_percent() got over 100% value #42

Closed
opened 2026-05-05 03:23:05 -06:00 by gitea-mirror · 4 comments
Owner

Originally created by @qpc001 on GitHub (Apr 11, 2023).
Original GitHub issue: https://github.com/XuehaiPan/nvitop/issues/71

Originally assigned to: @XuehaiPan on GitHub.

Required prerequisites

  • I have read the documentation https://nvitop.readthedocs.io.
  • I have searched the Issue Tracker that this hasn't already been reported. (comment there if it has.)
  • I have tried the latest version of nvitop in a new isolated virtual environment.

Questions

I am using the script here: 5a0da9239b/README.md (L658)

And I print a strange cpu_percent value:

image

But I print in top is:
image

Originally created by @qpc001 on GitHub (Apr 11, 2023). Original GitHub issue: https://github.com/XuehaiPan/nvitop/issues/71 Originally assigned to: @XuehaiPan on GitHub. ### Required prerequisites - [X] I have read the documentation <https://nvitop.readthedocs.io>. - [X] I have searched the [Issue Tracker](https://github.com/XuehaiPan/nvitop/issues) that this hasn't already been reported. (comment there if it has.) - [X] I have tried the latest version of nvitop in a new isolated virtual environment. ### Questions I am using the script here: https://github.com/XuehaiPan/nvitop/blob/5a0da9239be94859d2f02a3c9d4bda3075b1c0c5/README.md?plain=1#L658 And I print a strange cpu_percent value: ![image](https://user-images.githubusercontent.com/31476516/231132411-74ae4d4e-5b37-4c5f-9529-f0b24787002d.png) But I print in top is: ![image](https://user-images.githubusercontent.com/31476516/231132509-6566b9a0-6be3-4749-920b-c2ea36120bd2.png)
gitea-mirror 2026-05-05 03:23:05 -06:00
  • closed this issue
  • added the
    question
    label
Author
Owner

@qpc001 commented on GitHub (Apr 11, 2023):

The full script I am using is:

from nvitop import Device
from nvitop import Device, GpuProcess, NA, colored

devices = Device.all()  # or `Device.cuda.all()` to use CUDA ordinal instead
for device in devices:
    processes = device.processes()  # type: Dict[int, GpuProcess]
    sorted_pids = sorted(processes.keys())

    print(device)
    print(f'  - Fan speed:       {device.fan_speed()}%')
    print(f'  - Temperature:     {device.temperature()}C')
    print(f'  - GPU utilization: {device.gpu_utilization()}%')
    print(f'  - Total memory:    {device.memory_total_human()}')
    print(f'  - Used memory:     {device.memory_used_human()}')
    print(f'  - Free memory:     {device.memory_free_human()}')
    print(f'  - Processes ({len(processes)}): {sorted_pids}')
    if len(processes) > 0:
        processes = GpuProcess.take_snapshots(processes.values(), failsafe=True)
        processes.sort(key=lambda process: (process.username, process.pid))

        print(colored(f'  - Processes ({len(processes)}):', color='blue', attrs=('bold',)))
        fmt = '    {pid:<5}  {username:<8} {cpu:>5}  {host_memory:>8} {time:>8}  {gpu_memory:>8}  {sm:>3}  {command:<}'.format
        print(colored(fmt(pid='PID', username='USERNAME',
                          cpu='CPU%', host_memory='HOST-MEM', time='TIME',
                          gpu_memory='GPU-MEM', sm='SM%',
                          command='COMMAND'),
                      attrs=('bold',)))
        for snapshot in processes:
            print(fmt(pid=snapshot.pid,
                      username=snapshot.username[:7] + ('+' if len(snapshot.username) > 8 else snapshot.username[7:8]),
                      cpu=snapshot.cpu_percent, host_memory=snapshot.host_memory_human,
                      time=snapshot.running_time_human,
                      gpu_memory=(snapshot.gpu_memory_human if snapshot.gpu_memory_human is not NA else 'WDDM:N/A'),
                      sm=snapshot.gpu_sm_utilization,
                      command=snapshot.command))
    print('-' * 120)
<!-- gh-comment-id:1503072088 --> @qpc001 commented on GitHub (Apr 11, 2023): The full script I am using is: ``` from nvitop import Device from nvitop import Device, GpuProcess, NA, colored devices = Device.all() # or `Device.cuda.all()` to use CUDA ordinal instead for device in devices: processes = device.processes() # type: Dict[int, GpuProcess] sorted_pids = sorted(processes.keys()) print(device) print(f' - Fan speed: {device.fan_speed()}%') print(f' - Temperature: {device.temperature()}C') print(f' - GPU utilization: {device.gpu_utilization()}%') print(f' - Total memory: {device.memory_total_human()}') print(f' - Used memory: {device.memory_used_human()}') print(f' - Free memory: {device.memory_free_human()}') print(f' - Processes ({len(processes)}): {sorted_pids}') if len(processes) > 0: processes = GpuProcess.take_snapshots(processes.values(), failsafe=True) processes.sort(key=lambda process: (process.username, process.pid)) print(colored(f' - Processes ({len(processes)}):', color='blue', attrs=('bold',))) fmt = ' {pid:<5} {username:<8} {cpu:>5} {host_memory:>8} {time:>8} {gpu_memory:>8} {sm:>3} {command:<}'.format print(colored(fmt(pid='PID', username='USERNAME', cpu='CPU%', host_memory='HOST-MEM', time='TIME', gpu_memory='GPU-MEM', sm='SM%', command='COMMAND'), attrs=('bold',))) for snapshot in processes: print(fmt(pid=snapshot.pid, username=snapshot.username[:7] + ('+' if len(snapshot.username) > 8 else snapshot.username[7:8]), cpu=snapshot.cpu_percent, host_memory=snapshot.host_memory_human, time=snapshot.running_time_human, gpu_memory=(snapshot.gpu_memory_human if snapshot.gpu_memory_human is not NA else 'WDDM:N/A'), sm=snapshot.gpu_sm_utilization, command=snapshot.command)) print('-' * 120) ```
Author
Owner

@XuehaiPan commented on GitHub (Apr 11, 2023):

Hi @qpc001

From htop manual page (ref: Why is the CPU usage reported by top in Linux over 100%?):

PERCENT_CPU (CPU%)
     The  percentage of the CPU time that the process is currently using.
     This is the default way to represent CPU usage in Linux. Each process can
     consume up to 100% which means the full capacity of the core it is running
     on. This is sometimes called "Irix mode" e.g. in top(1).

The CPU percent is calculated by:

cpu_percent = (user_time + system_time) / elapsed_time

when you are running a multi-threaded program on a multi-core machine. It can be over 100%. E.g. 3000% on a 32-core machine.

Feel free to ask more questions.

<!-- gh-comment-id:1503086251 --> @XuehaiPan commented on GitHub (Apr 11, 2023): Hi @qpc001 From `htop` manual page (ref: [Why is the CPU usage reported by top in Linux over 100%?](https://superuser.com/questions/174660/why-is-the-cpu-usage-reported-by-top-in-linux-over-100)): ```text PERCENT_CPU (CPU%) The percentage of the CPU time that the process is currently using. This is the default way to represent CPU usage in Linux. Each process can consume up to 100% which means the full capacity of the core it is running on. This is sometimes called "Irix mode" e.g. in top(1). ``` The CPU percent is calculated by: ```python cpu_percent = (user_time + system_time) / elapsed_time ``` when you are running a multi-threaded program on a multi-core machine. It can be over 100%. E.g. `3000%` on a 32-core machine. Feel free to ask more questions.
Author
Owner

@qpc001 commented on GitHub (Apr 12, 2023):

Hi @qpc001

From htop manual page (ref: Why is the CPU usage reported by top in Linux over 100%?):

PERCENT_CPU (CPU%)
     The  percentage of the CPU time that the process is currently using.
     This is the default way to represent CPU usage in Linux. Each process can
     consume up to 100% which means the full capacity of the core it is running
     on. This is sometimes called "Irix mode" e.g. in top(1).

The CPU percent is calculated by:

cpu_percent = (user_time + system_time) / elapsed_time

when you are running a multi-threaded program on a multi-core machine. It can be over 100%. E.g. 3000% on a 32-core machine.

Feel free to ask more questions.

My point is that it is not possible for the CPU usage of this program to reach over 200%, rather than discussing why the program's CPU usage can exceed 100%.

<!-- gh-comment-id:1504395104 --> @qpc001 commented on GitHub (Apr 12, 2023): > Hi @qpc001 > > From `htop` manual page (ref: [Why is the CPU usage reported by top in Linux over 100%?](https://superuser.com/questions/174660/why-is-the-cpu-usage-reported-by-top-in-linux-over-100)): > > ``` > PERCENT_CPU (CPU%) > The percentage of the CPU time that the process is currently using. > This is the default way to represent CPU usage in Linux. Each process can > consume up to 100% which means the full capacity of the core it is running > on. This is sometimes called "Irix mode" e.g. in top(1). > ``` > > The CPU percent is calculated by: > > ```python > cpu_percent = (user_time + system_time) / elapsed_time > ``` > > when you are running a multi-threaded program on a multi-core machine. It can be over 100%. E.g. `3000%` on a 32-core machine. > > Feel free to ask more questions. My point is that it is not possible for the CPU usage of this program to reach over 200%, rather than discussing why the program's CPU usage can exceed 100%.
Author
Owner

@XuehaiPan commented on GitHub (Apr 12, 2023):

My point is that it is not possible for the CPU usage of this program to reach over 200%, rather than discussing why the program's CPU usage can exceed 100%.

@qpc001 I'm sorry that I misunderstood your question. But the answer would be similar.

  1. It can get CPU percent over 100%. As I mentioned:

    cpu_percent = (user_time + system_time) / elapsed_time
    

    It's really depending on your sample interval for elapsed_time. It can get a large value if your elapsed_time is relatively small. If you want to get a more accurate result. You should average over a time interval.

    process.cpu_percent(interval=1.0)  # blocking mode
    

    This would block your program for 1 second and gather the CPU time information during the sample interval.

    nvitop use process.cpu_percent() in non-blocking mode. That is, it will not block your program but use the CPU time information between two calls. You may get an inaccurate result in the first call because you don't have a previous call to compare with.

    process.cpu_percent()  # non-blocking: the first call, may be inaccurate
    # do something
    process.cpu_percent()  # non-blocking: the second call, returns the average CPU percent between the first and the second calls
    
  2. Here is the code for how psutil.Process calculate the CPU percent:

    90b35e3ae8/psutil/init.py#L946-L1036

    def cpu_percent(self, interval=None):
        """Return a float representing the current process CPU
        utilization as a percentage.
    
        When *interval* is 0.0 or None (default) compares process times
        to system CPU times elapsed since last call, returning
        immediately (non-blocking). That means that the first time
        this is called it will return a meaningful 0.0 value.
    
        When *interval* is > 0.0 compares process times to system CPU
        times elapsed before and after the interval (blocking).
    
        In this case is recommended for accuracy that this function
        be called with at least 0.1 seconds between calls.
    
        A value > 100.0 can be returned in case of processes running
        multiple threads on different CPU cores.
    
        The returned value is explicitly NOT split evenly between
        all available logical CPUs. This means that a busy loop process
        running on a system with 2 logical CPUs will be reported as
        having 100% CPU utilization instead of 50%.
    
        Examples:
    
          >>> import psutil
          >>> p = psutil.Process(os.getpid())
          >>> # blocking
          >>> p.cpu_percent(interval=1)
          2.0
          >>> # non-blocking (percentage since last call)
          >>> p.cpu_percent(interval=None)
          2.9
          >>>
        """
        blocking = interval is not None and interval > 0.0
        if interval is not None and interval < 0:
            raise ValueError("interval is not positive (got %r)" % interval)
        num_cpus = cpu_count() or 1
    
        def timer():
            return _timer() * num_cpus
    
        if blocking:
            st1 = timer()
            pt1 = self._proc.cpu_times()
            time.sleep(interval)
            st2 = timer()
            pt2 = self._proc.cpu_times()
        else:
            st1 = self._last_sys_cpu_times
            pt1 = self._last_proc_cpu_times
            st2 = timer()
            pt2 = self._proc.cpu_times()
            if st1 is None or pt1 is None:
                self._last_sys_cpu_times = st2
                self._last_proc_cpu_times = pt2
                return 0.0
    
        delta_proc = (pt2.user - pt1.user) + (pt2.system - pt1.system)
        delta_time = st2 - st1
        # reset values for next call in case of interval == None
        self._last_sys_cpu_times = st2
        self._last_proc_cpu_times = pt2
    
        try:
            # This is the utilization split evenly between all CPUs.
            # E.g. a busy loop process on a 2-CPU-cores system at this
            # point is reported as 50% instead of 100%.
            overall_cpus_percent = ((delta_proc / delta_time) * 100)
        except ZeroDivisionError:
            # interval was too low
            return 0.0
        else:
            # Note 1:
            # in order to emulate "top" we multiply the value for the num
            # of CPU cores. This way the busy process will be reported as
            # having 100% (or more) usage.
            #
            # Note 2:
            # taskmgr.exe on Windows differs in that it will show 50%
            # instead.
            #
            # Note 3:
            # a percentage > 100 is legitimate as it can result from a
            # process with multiple threads running on different CPU
            # cores (top does the same), see:
            # http://stackoverflow.com/questions/1032357
            # https://github.com/giampaolo/psutil/issues/474
            single_cpu_percent = overall_cpus_percent * num_cpus
            return round(single_cpu_percent, 1)
    

    The result also depends on how many CPUs are on your machine. nvitop uses psutil to gather process information about CPU and virtual RAM usage. If you think this would be a bug, please report it to giampaolo/psutil.

Here is the result from my side:

image

<!-- gh-comment-id:1504703694 --> @XuehaiPan commented on GitHub (Apr 12, 2023): > My point is that it is not possible for the CPU usage of this program to reach over 200%, rather than discussing why the program's CPU usage can exceed 100%. @qpc001 I'm sorry that I misunderstood your question. But the answer would be similar. 1. It can get CPU percent over 100%. As I mentioned: ```python cpu_percent = (user_time + system_time) / elapsed_time ``` It's really depending on your sample interval for `elapsed_time`. It can get a large value if your `elapsed_time` is relatively small. If you want to get a more accurate result. You should average over a time interval. ```python process.cpu_percent(interval=1.0) # blocking mode ``` This would block your program for 1 second and gather the CPU time information during the sample interval. `nvitop` use `process.cpu_percent()` in non-blocking mode. That is, it will not block your program but use the CPU time information between two calls. You may get an inaccurate result in the first call because you don't have a previous call to compare with. ```python process.cpu_percent() # non-blocking: the first call, may be inaccurate # do something process.cpu_percent() # non-blocking: the second call, returns the average CPU percent between the first and the second calls ``` 2. Here is the code for how `psutil.Process` calculate the CPU percent: https://github.com/giampaolo/psutil/blob/90b35e3ae85651d67a095205adc94aaa0a4ec98f/psutil/__init__.py#L946-L1036 ```python def cpu_percent(self, interval=None): """Return a float representing the current process CPU utilization as a percentage. When *interval* is 0.0 or None (default) compares process times to system CPU times elapsed since last call, returning immediately (non-blocking). That means that the first time this is called it will return a meaningful 0.0 value. When *interval* is > 0.0 compares process times to system CPU times elapsed before and after the interval (blocking). In this case is recommended for accuracy that this function be called with at least 0.1 seconds between calls. A value > 100.0 can be returned in case of processes running multiple threads on different CPU cores. The returned value is explicitly NOT split evenly between all available logical CPUs. This means that a busy loop process running on a system with 2 logical CPUs will be reported as having 100% CPU utilization instead of 50%. Examples: >>> import psutil >>> p = psutil.Process(os.getpid()) >>> # blocking >>> p.cpu_percent(interval=1) 2.0 >>> # non-blocking (percentage since last call) >>> p.cpu_percent(interval=None) 2.9 >>> """ blocking = interval is not None and interval > 0.0 if interval is not None and interval < 0: raise ValueError("interval is not positive (got %r)" % interval) num_cpus = cpu_count() or 1 def timer(): return _timer() * num_cpus if blocking: st1 = timer() pt1 = self._proc.cpu_times() time.sleep(interval) st2 = timer() pt2 = self._proc.cpu_times() else: st1 = self._last_sys_cpu_times pt1 = self._last_proc_cpu_times st2 = timer() pt2 = self._proc.cpu_times() if st1 is None or pt1 is None: self._last_sys_cpu_times = st2 self._last_proc_cpu_times = pt2 return 0.0 delta_proc = (pt2.user - pt1.user) + (pt2.system - pt1.system) delta_time = st2 - st1 # reset values for next call in case of interval == None self._last_sys_cpu_times = st2 self._last_proc_cpu_times = pt2 try: # This is the utilization split evenly between all CPUs. # E.g. a busy loop process on a 2-CPU-cores system at this # point is reported as 50% instead of 100%. overall_cpus_percent = ((delta_proc / delta_time) * 100) except ZeroDivisionError: # interval was too low return 0.0 else: # Note 1: # in order to emulate "top" we multiply the value for the num # of CPU cores. This way the busy process will be reported as # having 100% (or more) usage. # # Note 2: # taskmgr.exe on Windows differs in that it will show 50% # instead. # # Note 3: # a percentage > 100 is legitimate as it can result from a # process with multiple threads running on different CPU # cores (top does the same), see: # http://stackoverflow.com/questions/1032357 # https://github.com/giampaolo/psutil/issues/474 single_cpu_percent = overall_cpus_percent * num_cpus return round(single_cpu_percent, 1) ``` The result also depends on how many CPUs are on your machine. `nvitop` uses `psutil` to gather process information about CPU and virtual RAM usage. If you think this would be a bug, please report it to [giampaolo/psutil](https://github.com/giampaolo/psutil). Here is the result from my side: ![image](https://user-images.githubusercontent.com/16078332/231364469-dacf0d81-acf9-4b65-800d-5d16a1769c4b.png)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/nvitop#42
No description provided.