github-starred/nvitop

Fork 0

mirror of https://github.com/XuehaiPan/nvitop.git synced 2026-05-15 14:15:55 -06:00

[GH-ISSUE #103] [Question] Memory bandwidth utilization of GPUs? #62

New issue

Closed

opened 2026-05-05 03:24:08 -06:00 by gitea-mirror · 4 comments

gitea-mirror commented

2026-05-05 03:24:08 -06:00

Owner

Originally created by @walkieq on GitHub (Oct 24, 2023).
Original GitHub issue: https://github.com/XuehaiPan/nvitop/issues/103

Originally assigned to: @XuehaiPan on GitHub.

Required prerequisites

I have read the documentation https://nvitop.readthedocs.io.
I have searched the Issue Tracker that this hasn't already been reported. (comment there if it has.)
I have tried the latest version of nvitop in a new isolated virtual environment.

Questions

Is there a way to measure the runtime memory bandwidth utilization of GPUs?
Or is it possible to estimate/calculate the memory bandwidth utilization using the numbers reported in nvitop?

Originally created by @walkieq on GitHub (Oct 24, 2023). Original GitHub issue: https://github.com/XuehaiPan/nvitop/issues/103 Originally assigned to: @XuehaiPan on GitHub. ### Required prerequisites - [X] I have read the documentation <https://nvitop.readthedocs.io>. - [X] I have searched the [Issue Tracker](https://github.com/XuehaiPan/nvitop/issues) that this hasn't already been reported. (comment there if it has.) - [X] I have tried the latest version of nvitop in a new isolated virtual environment. ### Questions Is there a way to measure the runtime memory bandwidth utilization of GPUs? Or is it possible to estimate/calculate the memory bandwidth utilization using the numbers reported in nvitop?

gitea-mirror

2026-05-05 03:24:08 -06:00

closed this issue
added the
api

question
labels

gitea-mirror commented

2026-05-05 03:24:12 -06:00

Author

Owner

@XuehaiPan commented on GitHub (Oct 24, 2023):

Is there a way to measure the runtime memory bandwidth utilization of GPUs?

@walkieq The memory bandwidth utilization rate can be retrieved by:

from nvitop import Device

devices = Device.all()
for device in devices:
    print(f'Memroy usage percentage:           {device.memory_percent() / 100:%}')
    print(f'Memory bandwidth utilization rate: {device.memory_utilization() / 100:%}')

Also, you can get the memory throughput via:

device.pcie_throughput()
device.pcie_tx_throughput()
device.pcie_rx_throughput()
device.pcie_tx_throughput_human()
device.pcie_rx_throughput_human()

device.nvlink_throughput()
device.nvlink_tx_throughput()
device.nvlink_rx_throughput()
device.nvlink_tx_throughput_human()
device.nvlink_rx_throughput_human()

@XuehaiPan commented on GitHub (Oct 24, 2023): > Is there a way to measure the runtime memory bandwidth utilization of GPUs? @walkieq The memory bandwidth utilization rate can be retrieved by: ```python from nvitop import Device devices = Device.all() for device in devices: print(f'Memroy usage percentage: {device.memory_percent() / 100:%}') print(f'Memory bandwidth utilization rate: {device.memory_utilization() / 100:%}') ``` Also, you can get the memory throughput via: ```python device.pcie_throughput() device.pcie_tx_throughput() device.pcie_rx_throughput() device.pcie_tx_throughput_human() device.pcie_rx_throughput_human() device.nvlink_throughput() device.nvlink_tx_throughput() device.nvlink_rx_throughput() device.nvlink_tx_throughput_human() device.nvlink_rx_throughput_human() ```

gitea-mirror commented

2026-05-05 03:24:12 -06:00

Author

Owner

@walkieq commented on GitHub (Oct 25, 2023):

Thank you so much!
I have tested the API but I got some interesting results.

I am running the following:

from nvitop import Device

devices = Device.all()
for device in devices:
    print(f'Memroy usage percentage:           {device.memory_percent() / 100:%}')
    print(f'Memory bandwidth utilization rate: {device.memory_utilization() / 100:%}')
    print(device.pcie_throughput())
    print(device.pcie_tx_throughput())
    print(device.pcie_rx_throughput())
    print(device.pcie_tx_throughput_human())
    print(device.pcie_rx_throughput_human())

and the results are:

Memroy usage percentage:           94.600000%
Memory bandwidth utilization rate: 0.000000%
ThroughputInfo(tx=177000, rx=177000)
175000
182000
177.7MiB/s
168.9MiB/s

It looks like the device.memory_utilization() is different to the device.pcie_throughput(). Which one shall I refer to?

@walkieq commented on GitHub (Oct 25, 2023): Thank you so much! I have tested the API but I got some interesting results. I am running the following: ```python from nvitop import Device devices = Device.all() for device in devices: print(f'Memroy usage percentage: {device.memory_percent() / 100:%}') print(f'Memory bandwidth utilization rate: {device.memory_utilization() / 100:%}') print(device.pcie_throughput()) print(device.pcie_tx_throughput()) print(device.pcie_rx_throughput()) print(device.pcie_tx_throughput_human()) print(device.pcie_rx_throughput_human()) ``` and the results are: ``` Memroy usage percentage: 94.600000% Memory bandwidth utilization rate: 0.000000% ThroughputInfo(tx=177000, rx=177000) 175000 182000 177.7MiB/s 168.9MiB/s ``` It looks like the device.memory_utilization() is different to the device.pcie_throughput(). Which one shall I refer to?

gitea-mirror commented

2026-05-05 03:24:13 -06:00

Author

Owner

@XuehaiPan commented on GitHub (Oct 25, 2023):

It looks like the device.memory_utilization() is different to the device.pcie_throughput(). Which one shall I refer to?

Device.memory_utilization refers to:

Percent of time over the past sample period during which global (device) memory was being read or written.

The sample period may be between 1 second and 1/6 second depending on the product.

Device.pcie_throughput refers to:

The current PCIe throughput in KiB/s.

This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval.

You can see the detailed definition via:

nvidia-smi --help-query-gpu | less --pattern=utilization

Both the memory utilization rate and the PCIe throughput have their own meaning. You can also calculate the PCIe bandwidth utilization rate by:

PCIe utilization rate = 100 * current PCIe throughput / theoretical PCIe throughput (via PCIe generation and PCIe width)

@XuehaiPan commented on GitHub (Oct 25, 2023): > It looks like the device.memory_utilization() is different to the device.pcie_throughput(). Which one shall I refer to? [Device.memory_utilization](https://nvitop.readthedocs.io/en/latest/api/device.html#nvitop.Device.memory_utilization) refers to: > Percent of time over the past sample period during which global (device) memory was being read or written. > > The sample period may be between 1 second and 1/6 second depending on the product. [Device.pcie_throughput](https://nvitop.readthedocs.io/en/latest/api/device.html#nvitop.Device.pcie_throughput) refers to: > The current PCIe throughput in KiB/s. > > This function is querying a byte counter over a 20ms interval and thus is the PCIe throughput over that interval. You can see the detailed definition via: ```bash nvidia-smi --help-query-gpu | less --pattern=utilization ``` <img width="1080" alt="image" src="https://github.com/XuehaiPan/nvitop/assets/16078332/b21a8464-eaab-40af-a9d6-300850560394"> <img width="1080" alt="image" src="https://github.com/XuehaiPan/nvitop/assets/16078332/c20e43e9-1716-45e3-83fd-dad951e30b75"> Both the memory utilization rate and the PCIe throughput have their own meaning. You can also calculate the PCIe bandwidth utilization rate by: ``` PCIe utilization rate = 100 * current PCIe throughput / theoretical PCIe throughput (via PCIe generation and PCIe width) ```

gitea-mirror commented

2026-05-05 03:24:14 -06:00

Author

Owner

@XuehaiPan commented on GitHub (Nov 29, 2023):

Closing due to inactivity. Please feel free to ask for a reopening if you have more questions.

@XuehaiPan commented on GitHub (Nov 29, 2023): Closing due to inactivity. Please feel free to ask for a reopening if you have more questions.

gitea-mirror referenced this issue

2026-05-05 03:26:40 -06:00

[PR #62] [CLOSED] chore(pre-commit): [pre-commit.ci] autoupdate #143