Debian QEMU – VGA passthrough – Virt-Manager

Binding e unbinding dei driver
One new feature in the 2.6.13-rc3 kernel release, is the ability to bind and unbind drivers from devices manually from user space. Previously, the only way to disconnect a driver from a device was usually to unload the whole driver from memory, using rmmod.

In the sysfs tree, every driver now has bind and unbind files associated with it:

$ tree /sys/bus/usb/drivers/ub/
|-- 1-1:1.0 -> ../../../../devices/pci0000:00/0000:00:1d.7/usb1/1-1/1-1:1.0
|-- bind
|-- module -> ../../../../module/ub
`-- unbind

In order to unbind a device from a driver, simply write the bus id of the device to the unbind file:

echo -n "1-1:1.0" > /sys/bus/usb/drivers/ub/unbind

and the device will no longer be bound to the driver:

$ tree /sys/bus/usb/drivers/ub/
|-- bind
|-- module -> ../../../../module/ub
`-- unbind

To bind a device to a driver, the device must first not be controlled by any other driver. To ensure this, look for the “driver” symlink in the device directory:

$ tree /sys/bus/usb/devices/1-1:1.0
|-- bAlternateSetting
|-- bInterfaceClass
|-- bInterfaceNumber
|-- bInterfaceProtocol
|-- bInterfaceSubClass
|-- bNumEndpoints
|-- bus -> ../../../../../../bus/usb
|-- modalias
`-- power
`-- state

Then, simply write the bus id of the device you wish to bind, into the bind file for that driver:

echo -n "1-1:1.0" > /sys/bus/usb/drivers/usb-storage/bind

And check that the binding was successful:

$ tree /sys/bus/usb/devices/1-1:1.0
|-- bAlternateSetting
|-- bInterfaceClass
|-- bInterfaceNumber
|-- bInterfaceProtocol
|-- bInterfaceSubClass
|-- bNumEndpoints
|-- bus -> ../../../../../../bus/usb
|-- driver -> ../../../../../../bus/usb/drivers/usb-storage
|-- host2
| `-- power
| `-- state
|-- modalias
`-- power
`-- state

As the example above shows, this capability is very useful for switching devices between drivers which handle the same type of device (both the ub and usb-storage drivers handle USB mass storage devices, like flash drives.)

A number of “enterprise” Linux distributions offer multiple drivers of different version levels in their kernel packages. This manual binding feature will allow configuration tools to pick and choose which devices should be bound to which drivers, allowing users to upgrade only specific devices if they wish to.

In order for a device to bind successfully with a driver, that driver must already support that device. This is why you can not just arbitrarily bind any device to any driver. To help with the issue of adding new devices support to drivers after they are built, the PCI system offers a dynamic_id file in sysfs so that user space can write in new device ids that the driver should bind too. In the future, this ability to add new driver IDs to a running kernel will be moved into the driver core to make it available for all buses.

Metodo alternativo

echo 1002 6739 | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id


In this directory, there is a new_id file entry that can be used to dynamically add VID PID pair like this :

echo VID PID >new_id

Per caricare i moduli VFIO all’accensione e assegnarli ai dispositivi:
Edit the initramfs at /etc/initram-fs/modules to ensure it has VFIO modules loaded on boot:

sudo vim /etc/initram-fs/modules

# Add to file

sudo update-initramfs -u

Create a new module file located at /etc/modprobe.d/local.conf. This step binds the video cards to VFIO on boot so they are not claimed by the host OS (this script also binds VFIO post boot):

sudo vim /etc/modprobe.d/local.conf

# Add to file
options vfio-pci ids=10de:1b80,10de:10f0
options vfio-pci disable_vga=1

Reboot the server.

Note: 10de:1b80, 10de:10f0 are specific to our GPU hardware (GTX 1080s). These numbers refer to the video card and onboard audio. You can find your specific model / vendor numbers by running:

lspci -nnk | grep -i nvidia

4b:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] (rev a1)
4b:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f0] (rev a1)

At this point your physical host should be setup and ready to passthrough GPUs to guest VMs. You can verify IOMMU and VFIO are working by running the following:

sudo find /sys/kernel/iommu_groups/ -type l

# Your output should be a long listing of lines like this


lspci -nnk

# Find your VGA controllers, it should look similar to this

4b:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] (rev a1)
    Subsystem: ASUSTeK Computer Inc. Device [1043:8591]
    Kernel driver in use: vfio-pci

GPU primaria: estrazione ROM
Primary GPU Workaround

Another quirk that needs to be addressed is only necessary if you are passing through your primary GPU but interesting nonetheless. This behavior occurs with the GTX 1080, but did not with a Geforce 210 in my testing, so your results may vary.

Like motherboards, GPUs have their own BIOS (aka ROM) and when the computer boots the primary GPU is using a “shadowed” copy of the ROM file. This causes issues when doing passthrough. To workaround this you need to get a copy of a non-shadowed ROM file which is specific to the GPU model. In our case we had other non-primary GPUs we could dump the ROM from. You can also try 3rd party websites that provide ROMs but I did not have success with those.

First unbind a non-primary GPU from vfio-pci (if it is bound), for example:

echo “0000:4b:00.0” | sudo tee /sys/bus/pci/drivers/vfio-pci/unbind

Then dump the ROM contents to a file:

echo 0 | sudo tee /sys/devices/pci0000:00/0000:00:03.0/0000:4b:00.0/rom

sudo cat /sys/devices/pci0000:00/0000:00:03.0/0000:4b:00.0/rom > gpu.rom

echo 1 | sudo tee /sys/devices/pci0000:00/0000:00:03.0/0000:4b:00.0/rom

Note: The echo 0 and echo 1 basically just switches the ROM into a readable state and then back again if you’re curious.

After you have a good ROM file you need to add this code to your VMs XML in the definition of the GPU which added earlier:

Patch dei driver NVIDIA

  1. Using KVM/qemu, boot your Windows 10 VM using the kvm=off option described above.
  2. Inside Windows, download the Nvidia graphics driver for your graphics card from here
    The latest driver for my Nvidia GTX 970, as of this writing, is 372.54.
  3. Run the Nvidia driver installer. It will unpack the files to C:/NVIDIA/DisplayDriver/{version}/Win10_64/International/Display.Driver, with the driver as of this writing it will be C:\NVIDIA\DisplayDriver\372.54\Win10_64\International\Display.DriverExit the Nvidia installer once it unpacked the files !!!
  4. Download the Windows WDK from here: the webpage that opens, go to step 2, then click “Install Windows Driver Kit (WDK) 10”. You do NOT need to install the other stuff!

    Run the installer and install the Windows Driver Kit (WDK).

  5. Download Python from As of this writing the latest version is 3.5.2Run the installer.
  6. To enable testsigning:
    1. Right-click on the Windows menu icon and select “Command Prompt (Admin)”
    2. On the command prompt, enter
      Bcdedit.exe -set TESTSIGNING ON</code>

      and press Enter

    Later, if you wish to disable testsigning, use the following command:

    Bcdedit.exe -set TESTSIGNING OFF

    Important: Please be aware of the security risks when you run testsigned drivers.

  7. Reboot your Windows 10 VM.
  8. Open and download the two script files (a Powershell script and a Python script):
    • gencert.ps1

    Both files must reside in the same location. (Note: I opened the file, copied the content into an empty Notepad file and saved it.)

  9. Open the Windows File Explorer and go to the folder where you saved the two files.Click “File”, then select “Open command prompt” -> “Open command prompt as administrator”.
  10. Before running the script, open the file in Notepad or with IDLE (right click on the file and select) and check that the driver version you are going to install is actually supported by this patch. See below for reference:
    # TODO: Wildcard Search
    PATCHES = {
    “41FF978804000085C0”: “31C0909090909085C0”, # 361.91 – 368.39
    “41FF97B804000085C0”: “31C0909090909085C0”, # 372.54
  11. Copy and paste the following onto the command line: C:\NVIDIA\DisplayDriver\372.54\Win10_64\International\Display.Driver

    and hit Enter.Note: Replace the version (here “372.54”) in the path name with the Nvidia driver version you downloaded.

    The script will run for a few minutes, after which you see “Warnings: None”. Hit Enter.

    The Python script will now execute the Powershell script and testsign the driver.

  12. Using the File Explorer, go to

    and run “setup.exe” to install the patched Nvidia driver.

  13. Shutdown Windows and modify your qemu start script:
    -cpu host,hv_vapic,hv_time,hv_relaxed,hv_spinlocks=0x1fff \

    Note: Remove the “kvm=off” option and instead add “hv_vapic,hv_time,hv_relaxed,hv_spinlocks=0x1fff”

Questa voce è stata pubblicata in Debian e contrassegnata con , , , , , , , , . Contrassegna il permalink.


Inserisci i tuoi dati qui sotto o clicca su un'icona per effettuare l'accesso:


Stai commentando usando il tuo account Chiudi sessione / Modifica )

Foto Twitter

Stai commentando usando il tuo account Twitter. Chiudi sessione / Modifica )

Foto di Facebook

Stai commentando usando il tuo account Facebook. Chiudi sessione / Modifica )

Google+ photo

Stai commentando usando il tuo account Google+. Chiudi sessione / Modifica )

Connessione a %s...