Step-by-step guide to create CUDA Ubuntu VM in Proxmox 8

I have been learning a lot of GPU computing using CUDA lately. For ease of use, I am using a VM with discrete GPU for my GPU computing needs. Then, I can access this VM from the internet, so I can do GPU computing form anywhere in the world. However, I needed to recreate one and I forgot how I did it. So today I will document it all and share it with you guys!

Why Ubuntu?

I normally use Debian for my VMs, however based on my experience, using Ubuntu is easier if you want to use proprietary NVIDIA drivers. The official Ubuntu apt repository already provide different versions of NVIDIA drivers and can be install with just one simple apt install command.

Creating VM

First step is of course to create the VM. I clicked on create VM button on Proxmox GUI. There I select no media for the OS, then choose the q35 machine and OVMF (UEFI) bios. I also remove any disk and choose my desired number of CPU cores with host CPU type for best performance. The rest can be filled as you normally choose.

Ubuntu Cloud Image

The easiest way to use Ubuntu in Proxmox is to download the .img file into the Proxmox machine. For Ubuntu 24, it is downloadable here as noble-server-cloudimg-amd64.img. Then we can import the image to our VM with this command in Proxmox shell:

qm importdisk <VM ID> <path to .img> <storage location>

# this is my command
qm importdisk 303 ./noble-server-cloudimg-amd64.img local-lvm

Now, if we go back to the VM hardware settings in the Proxmox GUI, we can see a new unused disk. Attach the new disk as you normally would (for me with discard and SSD emulation options). Now select this disk as boot disk in options. Don't forget to add a Cloudinit drive as scsi in hardware settings, then fill the necessary information for that. For me, I filled the user, SSH public key, and set ip=dhcp.

Then, we just need to add our NVIDIA card to the VM. In the hardware settings, add a new PCI Device. I choose raw device, then select the NVIDIA card. I also checked the All Functions and PCI-Express boxes.

Install NVIDIA Driver

Now, you should able to start the VM and access its shell. Ubuntu 24 cloud image doesn't comes with qemu-guest-agent, so I need to check my router for its IP. After getting the IP, I can just SSH into the VM with ssh <user>@<ip>. There, I immediately install qemu-guest-agent and reboot. Here, don't forget to enable QEMU guest agent in the options of Proxmox GUI. We can install NVIDIA driver now with this simple command:

sudo apt install nvidia-headless-580-server-open nvidia-utils-580-server

I choose the headless version, because I only use this VM for GPU computing. Also, I choose the open version, because that is supposedly the newer one(?). Anyway, after a reboot, nvidia-smi command should show information about your NVIDIA card.

Is that it..? Not yet! Because I use the card in a headless server, I need to enable the so called persistence mode. The easiest way to achieve this, is to have this cronjob as root user: @reboot /usr/bin/nvidia-smi -pm 1 && /usr/bin/nvidia-smi -rgc.However, although this works now, according to this documentation, the persistence mode is currently near EOL and soon to be deprecated.

Conclusion

Once you have done all the steps above and have an functional CUDA VM, you can just save this image as backup and create new VM with the backup if you need one. So, this should be things that you only need to do once per one cloudimage. Now, I can focus my time on my GPU computing journey!