(Not) Upgrading To Proxmox 9

Proxmox 9 is here! It is based on the newly released stable Debian 13 Trixie. Proxmox 8, which is based on Debian 12, will only have support until 2026. So I just decided to upgrade as soon as possible. What could go wrong, right!?

Upgrading

In my experience, upgrading Debian is pretty easy. In essence, you just need to change the apt sources then do apt dist-upgrade. Of course, for each upgrade we need to check the release note and the official way guide to upgrade it. The Debian team always documented it really well and the guide for upgrading to Trixie is available here. Of course, in this post we are talking about Proxmox, not only Debian.

This will be my first time upgrading an Proxmox instance. Like the Debian team, the Proxmox team also documented it really well here. They also provided a script to check your installation before upgrading. This is available in the terminal with this command pve8to9. Since my Proxmox installation is pretty basic, I pretty much can upgrade without doing anything special.

The upgrade is also similar to Debian upgrade. I just need to change the apt sources and the do a dist-upgrade. After backing up all of my VMs, I started doing the upgrade and it upgraded successfully. After a reboot, I am greeted with Proxmox 9 GUI.

Problem

A few days went by after the upgrade without me noticing any problem. In my previous post, I wrote about how I also use this Proxmox machine as my couch gaming machine as an VM with an iGPU passthrough. On the weekend, as I was about to play some games, I cannot start my gaming VM. This is the error that I got:

kvm: -device vfio-pci,host=0000:c6:00.0,id=hostpci0,bus=ich9-pcie-port-1,addr=0x0,romfile=/usr/share/kvm/vbios_8845hs.bin: vfio 0000:c6:00.0: Invalid MSI-X relocation BAR 1, consumed by 64-bit BAR 0

This was hard to debug, since in the last few weeks I haven't started my gaming VM. I first thought that the problem has something to do with my BIOS instead, since the error was memory related. After spending hours trying out different BIOS settings, I noticed that this is my first time trying to start my gaming VM with Proxmox 9.

I also tried using a discrete GPU, which is connected with OCuLink to the PC. It was working with my old Proxmox 8 instance, but similar with the iGPU, this setup also doesn't work. The VM can boot, but checking with lspci on the guest machine shows me that the GPU is not being passed on correctly.

Solution

Well I haven't found any solution 🥲. I'm sure that the problem has something to do with the Proxmox upgrade at this point. Googling around the error that I got didn't help. Googling about problems with PCIe passthrough in Proxmox 9 did reveal that some people also has problem with it, but they have different error log. One QEMU argument actually allows me to boot the gaming VM. However, after the BIOS screen showed up, nothing happen and my screen was all black.

# I can boot the VM with this arg, but only received black screen after BIOS
# add the following line to /etv/pve/nodes/<node>/qemu-server/<id>.conf
# hostpci0 is my GPU
args: -set device.hostpci0.x-msix-relocation=bar2

After unsuccessfully trying out lot of things, I decided to rollback my Proxmox to version 8. Fortunately, this was easy to do. I just needed to backup all VMs first to an external NAS, then installed a fresh Proxmox 8 on the machine. After that, restoring all VMs was just a few clicks away. And I was right! The problem was Proxmox 9. The gaming VM now starts without any problem. So I will stay with Proxmox 8 for now and wait until next year before I consider upgrading to Proxmox 9 again