More USB ports is always better, but have you ever tried to add more? Over the years I’ve tried several PCI Express (PCIe) addin cards designed to give your computer up to 7 additional ports but have never been able to get one working. They’ve all delivered power but won’t talk data.
I think I own about 4 different such cards spanning back to when I was still running Windows. Even the ones that came with driver disks never worked, and moving to Linux changed nothing.
Admittedly they were all quite cheap and from AliExpress, maybe you just get what you pay for.
But if you look at AliExpress listings there’s a strange pattern: variants are described based on the USB controller IC manufacturer. Clearly VIA is not in vogue as examples with their chips are usually cheaper.
I bought this most recent PCIe card in August 2022 and experienced the same problems as all the others, but left it in the machine just to deliver power.
Recently however I actually needed more USB devices and was incentivised to get this working.
Finding the problem
kernel: [ 241.221228] usb 1-2: new high-speed USB device number 23 using xhci_hcd kernel: [ 241.365238] usb 1-2: device descriptor read/64, error -71 kernel: [ 241.643109] usb 1-2: device descriptor read/all, error -71 kernel: [ 241.981216] usb 1-2: new high-speed USB device number 24 using xhci_hcd kernel: [ 246.913690] usb 1-2: Device not responding to setup address. kernel: [ 247.121869] usb 1-2: Device not responding to setup address. kernel: [ 247.329166] usb 1-2: device not accepting address 24, error -71 kernel: [ 247.330981] usb usb1-port2: attempt power cycle kernel: [ 247.741167] usb 1-2: new high-speed USB device number 25 using xhci_hcd kernel: [ 247.776977] usb 1-2: device descriptor read/all, error -71 kernel: [ 248.113170] usb 1-2: new high-speed USB device number 26 using xhci_hcd kernel: [ 248.172991] usb 1-2: device descriptor read/all, error -71 kernel: [ 248.174822] usb usb1-port2: unable to enumerate USB device
Checking my systems kernel logs I saw new messages appearing every time I connected a USB device to the expansion card – a good sign, proving this isn’t a lost cause. Lots of slightly different errors but something to research at least.
I also used lspci to identify the USB controller on the card – the VIA VL805.
Thanks to this information I quickly found several bugs reported to different Linux vendors (Redhat, Ubuntu) which describe a conflict with AMD Virtualisation.
To confirm this I went back to my logs and sure enough found the same AMD-Vi error messages that appear in the Ubuntu bug report.
kernel: [ 288.309768] xhci_hcd 0000:07:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0010 address=0xbdc76000 flags=0x0000]
Fumbling for a fix
I now know I’m not crazy or unlucky – there really is a problem with AMD systems and these VIA chips.
Unfortunately those bug reports didn’t have any solutions beyond disabling IOMMU – and the more recent Ubuntu report claimed that disabling IOMMU might now make Linux fail to boot!
Another discussion about this bug on the Arch Linux forum had someone claiming to solve the problem by upgrading the firmware on the PCIe card having been told to do so by VIA support no less!
The download link in the post is dead but it isn’t too hard to find this firmware since it’s apparently the same chip used on the Raspberry Pi 4. A user has uploaded a firmware archive to Github, and some more firmware versions hosted on spox.org.
While the page on Spox suggests this can be done in Virtualbox using PCI passthrough (aka IOMMU, the very feature seemingly causing this issue) I decided to quickly move the PCIe card to my test bench and boot Hirens to run the Windows firmware flashing tool. This worked fine and so I gleefully popped the card back in my main computer and…
Take two?
…it still doesn’t work. Same errors.
I’m invested in making this work now, I’ve spent time working this out.
Thankfully I found a post on Minisforum that not only explains the issue in detail, but provides a reasonable solution that let’s us get everything working.
To paraphrase the description in 27leguas’s post: some PCI devices can only handle 32-bit addressing for legacy reasons and thus can only address 4GB of total memory, the IOMMU sits between PCI devices and system memory and remaps the 32-bit addresses to 64-bit native memory addresses allowing devices to access more memory.
While software IOMMU works perfectly well it can have a significant performance impact, especially when using virtualisation.
Making it work
The solution is a kernel parameter I had seen plenty of times while researching this bug.
While hardware IOMMU can be disabled in the BIOS you can also tell the Linux kernel to ignore the hardware device and use a software solution. A lot of the suggested fixes I saw involved adding iommu=soft or off, forcing software IOMMU.
As it turns out there’s another parameter we can set, amd_iommu=force_enable.
It’s not quite certain what this changes, but it fixes our USB problems! It’s also suggested we add iommu=pt, which enables passthrough. The only negative is it prevents Secure Memory Encryption working on Ryzen Pro processors – probably not an issue for many people, but worth mentioning.
To apply these changes we just need to add them to the kernel flags in the bootloader.
Assuming you’re using GRUB, you just need to edit (as root) the file /etc/default/grub and add the new options to the GRUB_CMDLINE_LINUX_DEFAULT line. For example:
GRUB_DEFAULT=0 GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=force_enable iommu=pt" GRUB_CMDLINE_LINUX=""
Save the file and run update-grub to apply the changes to the bootloader. Reboot and everything should work!
It’s been a couple of months and this PCIe card still works great. I use it as a dumping ground for low demand devices that just need to go somewhere – think bluetooth and keyboard dongles, maybe even a Zune.
I hope you enjoyed the post and that maybe it helped you out. I’ll see you next time.