0

I'm deeply confused about the state of CUDA profiling. Right now I have a single kernel that I'd like to optimize, and I'm struggling to find what the right software is. As near as I can tell there are the following options from Nvidia:

  1. nvprof to get some coarse information printed to the command line, or save out a profile.
  2. Nvidia Visual Profiler (nvvp) to view a profile saved by nvprof. As near as I can tell this only gives the timeline of when kernels launched/completed, and some other coarse information about what the GPU is up to (PCIe data transfers, etc.).
  3. Nvidia Nsight. As near as I can tell Nvidia has made the baffling decision to dilute the branding of this name, and seems to use the name "Nvidia Nsight" for a variety of packages of software of wildly varying featurefulness? So far I have found:
    • Some Nsight package that seems to be a modified version of Eclipse.
    • Nsight "Visual Studio Edition".
    • Nsight "Visual Studio Code Edition". From looking at screenshots it seems like the vscode Nsight version implements almost no features compared to what I see elsewhere for screenshots of Nsight. I'm confused why it's even called "Nsight", but maybe I'm missing how to open up the fuller view
    • Nsight Compute. I have no idea how this package relates to the others.
    • Nsight Graphics. Again, no idea.

So far I've used nvprof and nvvp, but they seem to only really be helpful to figure bottlenecks between kernels, not within a single kernel invocation. My questions:

  1. What is the best software to profile and optimize a single kernel invocation (not minimize bottlenecks between multiple kernel invocations/data transfers, etc.)?
  2. Can I do this with nvvp?
  3. If not, can I do this with one of the Nsight packages? If so, why are there so many different options for Nsight, and why do they seem to have wildly varying features? Which one should I use?

Really I'd just love to have all of my confusions resolved, and would adore an answer like "Yes, most people use X, ignore Y, Z and W, those are old packages that Nvidia doesn't maintain anymore, and are really only still up for legacy users.".

1
  • Questions 2 and 3 are off-topic (not about recommending software but about using some particular program). Question 1 is fine, except that you should define "best" by defining what are your requirements for such a program you're looking for.
    – Alejandro
    May 11, 2022 at 22:37

1 Answer 1

0

nvprof and nvvp are the legacy profilers, while the Nsight profilers are newer and are regularly updated with new features. So as long as the Nsight profilers support your GPU architecture, you should probably use them.

I do not know how exactly Nvidia categorizes its software products into "Nsight" or not, but Nsight certainly is not a single product/piece of software and not everything called "Nsight" has something to do with profiling. As you noted, there are multiple IDE plugins under this moniker which give better syntax highlighting, a debugging GUI (wrapping cuda-gdb) etc.

The two available profiers for use in the compute context (vs 3D/Ray-Tracing with "Nsight Graphics") are Nsight Systems (nsys) and Nsight Compute (ncu). Both can be called in CLI mode for data-collection on a remote server, or with a GUI (nsys-ui and ncu-ui) to view the collected data or interactively collect data.

Nsight Systems gives you a timeline for the whole application, i.e. as OP described it, it "minimize[s] bottlenecks between multiple kernel invocations/data transfers, etc."

For more information on the relation between legacy and Nsight profilers see the Nvidia blog post Migrating to NVIDIA Nsight Tools from NVVP and Nvprof

TL;DR:

The right tool to optimize a single kernel is Nsight Compute.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.