PCIE passthrough Proxmox and Windows 11

calendar 28-Sep-2022 · 6 min read · virtualization proxmox windows 11 guide  ·
Share on: linkedin copy

PCI(E) passthrough

If you wish to use a GPU with or pass physical disks to a VM. You will have to pass the devices through to the VM. PCI(E) Passthrough requires hardware that allows for it to work but if you have that you are good to go! A poppular use for PCI(E) passthrough is taking the graphics card and passing it into a VM to either play games or running calculations on. Other uses is passing physical hard drives into virtual machines to use as a NAS.

Video

BIOS

To make use of PCI(E) passthrough, you will have to enable IOMMU (AMD) or VT-D (Intel) in the BIOS of your machine. Depending on motherboard and processor manufacturer. The names for what you have to enable will be different and the same with how everything looks.

I recommend looking at this link if you are unsure. They show how to enable IOMMU or VT-D on a range of motherboards. https://us.informatiweb.net/tutorials/it/bios/enable-iommu-or-vt-d-in-your-bios.html

VM requirements

For PCI(E) passthrough to work properly you will have to there are two hardware settings that have other default settings in Proxmox.

  • BIOS: This needs to be set to OVMF (UEFI).
  • Machine: This needs to be set to q35.

You might get issues if you change these settings after the operating system has been installed. Here is a guide on how to setup a Windows 11 VM in a way that is compatible with PCI(E) passthrough https://divgitally.com/post/proxmox-windows11-vm/#create-and-configure-vm

Configuring PCI(E) passthrough

In addition to adding the PCI(E) device to the VM, you will also have to run some commands in the shell/terminal of the Proxmox node.

Edit GRUB

First is editing the GRUB file to add a few arguments to run during the startup of the Proxmox node.

Type or copy in the line below and press the enter key.

1nano /etc/default/grub

This will open the GRUB file in a text editor.

Now find the line with GRUB_CMDLINE_LINUX_DEFAULT="quiet" What you must add to this line is different Depending on if you have an AMD or Intel CPU.

  • AMD:
1GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"
  • Intel:
1GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"

When that has been added you can save by pressing CRTL + S and exit by clicking CTRL + X You will also have to update GRUB. This can be done with the command:

1update-grub

Edit Modules

Type or copy in the line below

1nano /etc/modules

There are four lines that need to be added to this file. This is the same for both Intel and AMD processors

1vfio
2vfio_iommu_type1
3vfio_pci
4vfio_virqfd

When they have been added you can save by pressing CRTL + S and exit by clicking CTRL + X

Additional GPU configuration.

Error 43

Not all graphics cards like to play nice being passed through to a VM. These cards will give you an error with the number 43. To avoid this there are some additional arguments to be added to the config file of the VM.

Open the config file with the command below.

1nano /etc/pve/qemu-server/"ID of VM".conf

The ID is for example 100.

In this file you will have to add two arguments to the cpu line and add the args line.

1cpu: host,hidden=1,flags=+pcid
2args: -cpu 'host,+kvm\_pv\_unhalt,+kvm\_pv\_eoi,hv\_vendor\_id=NV43FIX,kvm=off'
Block drivers from being used in Proxmox.

You might get a situation where Proxmox loads the device drivers and will not release it for use in the VM. You can then block the drivers from being loaded in Proxmox.

Open the blacklist file with the line below:

1nano /etc/modprobe.d/blacklist.conf

At the end of the file, you can add "Blacklist" followed by the driver's name.

1blacklist radeon
2blacklist nouveau
3blacklist nvidia

These examples are for GPU's. If you have an issue with other devices, you might have to add the driver for that device. To find the driver's name you will have to use your preferred search engine to search a bit.

After this you will have to run the command below to update the blocked drivers followed by a restart of the Proxmox node.

1update-initramfs -u

Done with terminal (maybe)

That should be all you need to use the shell for to set this up. To be on the safe side you should reboot the Proxmox node before proceeding.

Adding the PCI(E) device to the VM.

The final part should be adding the PCI(E) device.

  1. Select the desired VM.
  2. Navigate to hardware.
  3. Click Add.
  4. Select PCI Device in the dropdown menu.
  5. What you select here depends on the type of PCI(E) device you wish to pass through. Make sure to check all applicable boxes.
    • Device: Some devices will have multiple entries here. GPUs for example will usually split video and audio. In this case you should only select the first and check the All Functions box.
    • All Functions: Checking this will make it, so you do not have to include all the devices from one PCI(E) device. As my previous example this will include the audio device when you have selected the video device of a GPU.
    • Primary GPU: If you are passing through a GPU, check this to make sure video is processed by your GPU. This will also block the console window from functioning.
    • ROM-Bar: This makes the firmware of the PCI(E) device visible for you VM.
    • PCI-Express: Checking this will make sure the card is read as PCI Express for higher speeds.
    • ID overrides The final four is meant to override the ID of a device. This can be used to force a driver to work with a device that usually would not work.
  6. Click Add.

Finally

It is now time to start up your VM and hope everything works as it should. It will usually take a minute or two for the operating system to recognize and apply the proper drivers for everything to work correctly. Hopefully everything will work without issue, and you can proceed with downloading and installing the latest drivers. If the opposite is true, it is time to troubleshoot. My tip is to go over the files you have modified to make sure that the additions have been saved. Also run update commands again before rebooting the system.

If you are still experiencing difficulties, you might have to install the operating system again or look at the links below. The reddit one is specifyed to passing through graphics cards.

These are some links you might find usefull.

Proxmox wiki https://pve.proxmox.com/wiki/PCI(e)_Passthrough

Reddit Homelab https://www.reddit.com/r/homelab/comments/b5xpua/the_ultimate_beginners_guide_to_gpu_passthrough/

Windows 11 setup https://divgitally.com/post/proxmox-windows11-vm/

Translations: