github-starred/nvitop
2
0
Fork 0
mirror of https://github.com/XuehaiPan/nvitop.git synced 2026-05-15 14:15:55 -06:00
Code Issues 29 Projects Packages Wiki Activity Actions

[GH-ISSUE #86] [BUG] unable to display power limit #53

New issue
Closed
opened 2026-05-05 03:23:44 -06:00 by gitea-mirror · 5 comments
gitea-mirror commented 2026-05-05 03:23:44 -06:00
Owner
Copy link

Originally created by @liblaf on GitHub (Aug 4, 2023).
Original GitHub issue: https://github.com/XuehaiPan/nvitop/issues/86

Originally assigned to: @XuehaiPan on GitHub.

Required prerequisites

  • I have read the documentation https://nvitop.readthedocs.io.
  • I have searched the Issue Tracker that this hasn't already been reported. (comment there if it has.)
  • I have tried the latest version of nvitop in a new isolated virtual environment.

What version of nvitop are you using?

1.2.0

Operating system and version

Arch Linux x86_64

NVIDIA driver version

535.86.05

NVIDIA-SMI

Fri Aug  4 16:37:22 2023       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.86.05              Driver Version: 535.86.05    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4060 ...    Off | 00000000:01:00.0 Off |                  N/A |
| N/A   34C    P8               2W / 115W |     25MiB /  8188MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2072      G   /usr/lib/Xorg                                20MiB |
+---------------------------------------------------------------------------------------+

Python environment

Installation
pipx install nvitop
Python version & relevant libraries
3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429] linux
nvidia-ml-py==11.525.131
nvitop==1.2.0

Problem description

nvitop cannot display power limit correctly.

$ nvitop --once
Fri Aug 04 16:42:38 2023
╒═════════════════════════════════════════════════════════════════════════════╕
│ NVITOP 1.2.0       Driver Version: 535.86.05      CUDA Driver Version: 12.2 │
├───────────────────────────────┬──────────────────────┬──────────────────────┤
│ GPU  Name        Persistence-M│ Bus-Id        Disp.A │ Volatile Uncorr. ECC │
│ Fan  Temp  Perf  Pwr:Usage/Cap│         Memory-Usage │ GPU-Util  Compute M. │
╞═══════════════════════════════╪══════════════════════╪══════════════════════╡
│   0  .. 4060 Laptop GPU  Off  │ 00000000:01:00.0 Off │                  N/A │
│ N/A   38C    P8      2W / N/A │      26MiB / 8188MiB │      0%      Default │
╘═══════════════════════════════╧══════════════════════╧══════════════════════╛

Steps to Reproduce

Command lines:

nvitop --once

Traceback

No response

Logs

[DEBUG] 2023-08-04 16:43:45,692 nvitop.api.libnvml::__determine_get_memory_info_version_suffix: NVML get memory info version 2 is available.
[DEBUG] 2023-08-04 16:43:45,694 nvitop.api.libnvml::__determine_get_running_processes_version_suffix: NVML get running process version 3 API with v3 type struct is available.

Expected behavior

nvitop should display power limit as nvidia-smi.

Additional context

No response

Originally created by @liblaf on GitHub (Aug 4, 2023). Original GitHub issue: https://github.com/XuehaiPan/nvitop/issues/86 Originally assigned to: @XuehaiPan on GitHub. ### Required prerequisites - [X] I have read the documentation <https://nvitop.readthedocs.io>. - [X] I have searched the [Issue Tracker](https://github.com/XuehaiPan/nvitop/issues) that this hasn't already been reported. (comment there if it has.) - [X] I have tried the latest version of nvitop in a new isolated virtual environment. ### What version of nvitop are you using? 1.2.0 ### Operating system and version Arch Linux x86_64 ### NVIDIA driver version 535.86.05 ### NVIDIA-SMI ```text Fri Aug 4 16:37:22 2023 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.86.05 Driver Version: 535.86.05 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 4060 ... Off | 00000000:01:00.0 Off | N/A | | N/A 34C P8 2W / 115W | 25MiB / 8188MiB | 0% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 2072 G /usr/lib/Xorg 20MiB | +---------------------------------------------------------------------------------------+ ``` ### Python environment ###### Installation ```shell pipx install nvitop ``` ###### Python version & relevant libraries ``` 3.11.3 (main, Jun 5 2023, 09:32:32) [GCC 13.1.1 20230429] linux nvidia-ml-py==11.525.131 nvitop==1.2.0 ``` ### Problem description `nvitop` cannot display power limit correctly. ```shell $ nvitop --once Fri Aug 04 16:42:38 2023 ╒═════════════════════════════════════════════════════════════════════════════╕ │ NVITOP 1.2.0 Driver Version: 535.86.05 CUDA Driver Version: 12.2 │ ├───────────────────────────────┬──────────────────────┬──────────────────────┤ │ GPU Name Persistence-M│ Bus-Id Disp.A │ Volatile Uncorr. ECC │ │ Fan Temp Perf Pwr:Usage/Cap│ Memory-Usage │ GPU-Util Compute M. │ ╞═══════════════════════════════╪══════════════════════╪══════════════════════╡ │ 0 .. 4060 Laptop GPU Off │ 00000000:01:00.0 Off │ N/A │ │ N/A 38C P8 2W / N/A │ 26MiB / 8188MiB │ 0% Default │ ╘═══════════════════════════════╧══════════════════════╧══════════════════════╛ ``` ### Steps to Reproduce Command lines: ```bash nvitop --once ``` ### Traceback _No response_ ### Logs ```text [DEBUG] 2023-08-04 16:43:45,692 nvitop.api.libnvml::__determine_get_memory_info_version_suffix: NVML get memory info version 2 is available. [DEBUG] 2023-08-04 16:43:45,694 nvitop.api.libnvml::__determine_get_running_processes_version_suffix: NVML get running process version 3 API with v3 type struct is available. ``` ### Expected behavior `nvitop` should display power limit as `nvidia-smi`. ### Additional context _No response_
gitea-mirror 2026-05-05 03:23:44 -06:00
  • closed this issue
  • added the
    bug
         label
gitea-mirror commented 2026-05-05 03:23:46 -06:00
Author
Owner
Copy link

@XuehaiPan commented on GitHub (Aug 4, 2023):

Hi @liblaf, thanks for raising this issue.

Could you try the following code in your Python REPL?

$ python3
>>> from nvitop import *
>>> d = Device(0)
>>> libnvml.nvmlDeviceGetPowerUsage(d.handle)  # power usage in milliwatts (mW)
59085
>>> d.power_usage()  # power usage in milliwatts (mW)
59147
>>> libnvml.nvmlDeviceGetPowerManagementLimit(d.handle)  # power limit in milliwatts (mW)
400000
>>> d.power_limit()  # power limit in milliwatts (mW)
400000

Then we can dive deeper into this error.

<!-- gh-comment-id:1665287764 --> @XuehaiPan commented on GitHub (Aug 4, 2023): Hi @liblaf, thanks for raising this issue. Could you try the following code in your Python REPL? ```python $ python3 >>> from nvitop import * >>> d = Device(0) >>> libnvml.nvmlDeviceGetPowerUsage(d.handle) # power usage in milliwatts (mW) 59085 >>> d.power_usage() # power usage in milliwatts (mW) 59147 >>> libnvml.nvmlDeviceGetPowerManagementLimit(d.handle) # power limit in milliwatts (mW) 400000 >>> d.power_limit() # power limit in milliwatts (mW) 400000 ``` Then we can dive deeper into this error.
gitea-mirror commented 2026-05-05 03:23:48 -06:00
Author
Owner
Copy link

@liblaf commented on GitHub (Aug 4, 2023):

$ ${HOME}/.local/pipx/venvs/nvitop/bin/python
Python 3.11.3 (main, Jun  5 2023, 09:32:32) [GCC 13.1.1 20230429] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from nvitop import *
>>> d = Device(0)
>>> libnvml.nvmlDeviceGetPowerUsage(d.handle)  # power usage in milliwatts (mW)
2348
>>> d.power_usage()  # power usage in milliwatts (mW)
2347
>>> libnvml.nvmlDeviceGetPowerManagementLimit(d.handle)  # power limit in milliwatts (mW)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 2394, in nvmlDeviceGetPowerManagementLimit
    _nvmlCheckReturn(ret)
  File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 855, in _nvmlCheckReturn
    raise NVMLError(ret)
pynvml.NVMLError_NotSupported: Not Supported
>>> d.power_limit()  # power limit in milliwatts (mW)
'N/A'

is this issue caused by a mismatch between the version of nvidia-ml-py==11.525.131 and the nvidia==535.86.05 driver?

<!-- gh-comment-id:1665301477 --> @liblaf commented on GitHub (Aug 4, 2023): ```python $ ${HOME}/.local/pipx/venvs/nvitop/bin/python Python 3.11.3 (main, Jun 5 2023, 09:32:32) [GCC 13.1.1 20230429] on linux Type "help", "copyright", "credits" or "license" for more information. >>> from nvitop import * >>> d = Device(0) >>> libnvml.nvmlDeviceGetPowerUsage(d.handle) # power usage in milliwatts (mW) 2348 >>> d.power_usage() # power usage in milliwatts (mW) 2347 >>> libnvml.nvmlDeviceGetPowerManagementLimit(d.handle) # power limit in milliwatts (mW) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 2394, in nvmlDeviceGetPowerManagementLimit _nvmlCheckReturn(ret) File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 855, in _nvmlCheckReturn raise NVMLError(ret) pynvml.NVMLError_NotSupported: Not Supported >>> d.power_limit() # power limit in milliwatts (mW) 'N/A' ``` is this issue caused by a mismatch between the version of `nvidia-ml-py==11.525.131` and the `nvidia==535.86.05` driver?
gitea-mirror commented 2026-05-05 03:23:48 -06:00
Author
Owner
Copy link

@liblaf commented on GitHub (Aug 4, 2023):

i tried to install the latest nvidia-ml-py==12.535.77 but it produces the same error.

<!-- gh-comment-id:1665318656 --> @liblaf commented on GitHub (Aug 4, 2023): i tried to install the latest `nvidia-ml-py==12.535.77` but it produces the same error.
gitea-mirror commented 2026-05-05 03:23:49 -06:00
Author
Owner
Copy link

@XuehaiPan commented on GitHub (Aug 4, 2023):

@liblaf Thanks for the feedback.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 2394, in nvmlDeviceGetPowerManagementLimit
    _nvmlCheckReturn(ret)
  File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 855, in _nvmlCheckReturn
    raise NVMLError(ret)
pynvml.NVMLError_NotSupported: Not Supported

Based on your traceback, your NVIDIA product may not support the power management feature.

According to the NVML documentation nvmlDeviceGetPowerManagementLimit:

Returns
NVML_SUCCESS if limit has been set
NVML_ERROR_UNINITIALIZED if the library has not been successfully initialized
NVML_ERROR_INVALID_ARGUMENT if device is invalid or limit is NULL
NVML_ERROR_NOT_SUPPORTED if the device does not support this feature
NVML_ERROR_GPU_IS_LOST if the target GPU has fallen off the bus or is otherwise inaccessible
NVML_ERROR_UNKNOWN on any unexpected error

Description
Retrieves the power management limit associated with this device.

For Fermi or newer fully supported devices.

The power limit defines the upper boundary for the card's power draw. If the card's total power draw reaches this limit the power management algorithm kicks in.

This reading is only available if power management mode is supported. See nvmlDeviceGetPowerManagementMode.

The power limit API may return NVML_ERROR_NOT_SUPPORTED if the power management feature is not available.

You can try the following command in your console:

$ nvidia-smi --query-gpu=index,power.management,power.draw,power.limit --format=csv
index, power.management, power.draw [W], power.limit [W]
0, Enabled, 59.40 W, 400.00 W
1, Enabled, 61.41 W, 400.00 W
2, Enabled, 63.99 W, 400.00 W
3, Enabled, 60.90 W, 400.00 W
4, Enabled, 63.85 W, 400.00 W
5, Enabled, 61.56 W, 400.00 W
6, Enabled, 62.22 W, 400.00 W
7, Enabled, 59.93 W, 400.00 W
<!-- gh-comment-id:1665349490 --> @XuehaiPan commented on GitHub (Aug 4, 2023): @liblaf Thanks for the feedback. > ```pytb > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 2394, in nvmlDeviceGetPowerManagementLimit > _nvmlCheckReturn(ret) > File "/home/liblaf/.local/pipx/venvs/nvitop/lib/python3.11/site-packages/pynvml.py", line 855, in _nvmlCheckReturn > raise NVMLError(ret) > pynvml.NVMLError_NotSupported: Not Supported > ``` Based on your traceback, your NVIDIA product may not support the power management feature. According to the NVML documentation [nvmlDeviceGetPowerManagementLimit](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceQueries.html#group__nvmlDeviceQueries_1gf754f109beca3a4a8c8c1cd650d7d66c): > **Returns** > [NVML_SUCCESS](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceEnumvs.html#group__nvmlDeviceEnumvs_1gg06fa9b5de08c6cc716fbf565e93dd3d0a24d587db29324840a89a40c00c727f3) if limit has been set > [NVML_ERROR_UNINITIALIZED](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceEnumvs.html#group__nvmlDeviceEnumvs_1gg06fa9b5de08c6cc716fbf565e93dd3d0e054644f6612bf2ffb3f0e68379ec612) if the library has not been successfully initialized > [NVML_ERROR_INVALID_ARGUMENT](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceEnumvs.html#group__nvmlDeviceEnumvs_1gg06fa9b5de08c6cc716fbf565e93dd3d09658b0b4a4ced32a9148abb63d234aff) if device is invalid or limit is NULL > [NVML_ERROR_NOT_SUPPORTED](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceEnumvs.html#group__nvmlDeviceEnumvs_1gg06fa9b5de08c6cc716fbf565e93dd3d0f9e8a97d37f2e4efdf330ef938663e81) if the device does not support this feature > [NVML_ERROR_GPU_IS_LOST](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceEnumvs.html#group__nvmlDeviceEnumvs_1gg06fa9b5de08c6cc716fbf565e93dd3d0eeab290c4ad003602d90e105a7fec88a) if the target GPU has fallen off the bus or is otherwise inaccessible > [NVML_ERROR_UNKNOWN](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceEnumvs.html#group__nvmlDeviceEnumvs_1gg06fa9b5de08c6cc716fbf565e93dd3d0e08ac94f623461b8f5156bbb2c1eeb2a) on any unexpected error > > > **Description** > Retrieves the power management limit associated with this device. > > For Fermi or newer fully supported devices. > > The power limit defines the upper boundary for the card's power draw. If the card's total power draw reaches this limit the power management algorithm kicks in. > > This reading is only available if power management mode is supported. See [nvmlDeviceGetPowerManagementMode](https://docs.nvidia.com/deploy/nvml-api/group__nvmlDeviceQueries.html#group__nvmlDeviceQueries_1g10365092adc37d7a17d261db8fe63fb6). The power limit API may return `NVML_ERROR_NOT_SUPPORTED` if the power management feature is not available. You can try the following command in your console: ```console $ nvidia-smi --query-gpu=index,power.management,power.draw,power.limit --format=csv index, power.management, power.draw [W], power.limit [W] 0, Enabled, 59.40 W, 400.00 W 1, Enabled, 61.41 W, 400.00 W 2, Enabled, 63.99 W, 400.00 W 3, Enabled, 60.90 W, 400.00 W 4, Enabled, 63.85 W, 400.00 W 5, Enabled, 61.56 W, 400.00 W 6, Enabled, 62.22 W, 400.00 W 7, Enabled, 59.93 W, 400.00 W ```
gitea-mirror commented 2026-05-05 03:23:51 -06:00
Author
Owner
Copy link

@liblaf commented on GitHub (Aug 4, 2023):

indeed

$ nvidia-smi --query-gpu=index,power.management,power.draw,power.limit --format=csv
index, power.management, power.draw [W], power.limit [W]
0, [N/A], 2.55 W, [N/A]
<!-- gh-comment-id:1665380307 --> @liblaf commented on GitHub (Aug 4, 2023): indeed ```shell $ nvidia-smi --query-gpu=index,power.management,power.draw,power.limit --format=csv index, power.management, power.draw [W], power.limit [W] 0, [N/A], 2.55 W, [N/A] ```
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
drop-windows-curses
pep-639
v1.6.2
v1.6.1
v1.6.0
v1.5.3
v1.5.2
v1.5.1
v1.5.0
v1.4.2
v1.4.1
v1.4.0
v1.3.2
v1.3.1
v1.3.0
v1.2.0
v1.1.2
v1.1.1
v1.1.0
v1.0.0
v0.11.0
v0.10.2
v0.10.1
v0.10.0
v0.9.0
v0.8.1
v0.8.0
v0.7.3
v0.7.2
v0.7.1
v0.7.0
v0.6.2
v0.6.1
v0.6.0
v0.5.6
v0.5.5
v0.5.4
v0.5.3
v0.5.2.2
v0.5.2.1
v0.5.2
v0.5.1.4
v0.5.1.3
v0.5.1.2
v0.5.1.1
v0.5.1
v0.5.0
v0.4.4.1
v0.4.4
v0.4.3
v0.4.2.2
v0.4.2.1
v0.4.2
v0.4.1
v0.4.0
v0.3.6.2
v0.3.6.1
v0.3.6
v0.3.5.6
v0.3.5.5
v0.3.5.4
v0.3.5.3
v0.3.5.2
v0.3.5.1
v0.3.5
v0.3.4.1
v0.3.4
v0.3.3
v0.3.2.1
v0.3.2
v0.3.1
v0.3.0.2
v0.3.0.1
v0.3.0
v0.2.5.2
v0.2.5.1
v0.2.5
v0.2.4.3
v0.2.4.2
v0.2.4.1
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2
v0.1
Labels
Clear labels   
api

   
bug

   
bug

   
cli / tui

   
dependencies

   
documentation

   
documentation

   
documentation

   
duplicate

   
enhancement

   
exporter

   
invalid

   
pull-request

Mirrored from GitHub Pull Request

  
pynvml

   
question

   
question

   
upstream

   
wontfix
No labels
api
bug
bug
cli / tui
dependencies
documentation
documentation
documentation
duplicate
enhancement
exporter
invalid
pull-request
pynvml
question
question
upstream
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/nvitop#53
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?