dpdk - 无法将 DPDK 绑定(bind)接口(interface)添加到 OVS 网桥

Question

我尝试使用 ovs-vsctl 将 PCI 接口(interface) (I219-V) 添加到 OVS 桥，但没有成功:

root@pc:/home/me# ovs-vsctl add-port br0 tryport -- set Interface tryport type=dpdk options:dpdk-devargs=0000:00:1f.6
ovs-vsctl: Error detected while setting up 'tryport': Error attaching device '0000:00:1f.6' to DPDK.  See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is "/usr/local/var/log/openvswitch".

OVS 和 DPDK 版本:

ovs-vsctl: 2.15.90
DB Schema 8.2.0
libdpdk: 21.02.0

添加失败:

2021-02-19T18:12:32.543Z|00126|dpdk|ERR|EAL: Driver cannot attach the device (0000:00:1f.6)
2021-02-19T18:12:32.543Z|00127|dpdk|ERR|EAL: Failed to attach device on primary process
2021-02-19T18:12:32.543Z|00128|netdev_dpdk|WARN|Error attaching device '0000:00:1f.6' to DPDK
2021-02-19T18:12:32.543Z|00129|netdev|WARN|tryport: could not set configuration (Invalid argument)
2021-02-19T18:12:32.543Z|00130|dpdk|ERR|Invalid port_id=32
2021-02-19T18:12:32.547Z|00131|dpdk|ERR|EAL: Driver cannot attach the device (0000:00:1f.6)
2021-02-19T18:12:32.547Z|00132|dpdk|ERR|EAL: Failed to attach device on primary process
2021-02-19T18:12:32.547Z|00133|netdev_dpdk|WARN|Error attaching device '0000:00:1f.6' to DPDK
2021-02-19T18:12:32.547Z|00134|netdev|WARN|tryport2: could not set configuration (Invalid argument)
2021-02-19T18:12:32.547Z|00135|dpdk|ERR|Invalid port_id=32

这是 OVS DB 的内容，EAL 似乎正确初始化:

root@pc:/home/me# ovs-vsctl list Open_Vswitch
_uuid               : 4150fd20-ecbd-4ee3-a438-0c74d0411831
bridges             : []
cur_cfg             : 9
datapath_types      : [netdev, system]
datapaths           : {}
db_version          : "8.2.0"
dpdk_initialized    : true
dpdk_version        : "DPDK 21.02.0"
external_ids        : {hostname=pc, rundir="/usr/local/var/run/openvswitch", system-id=""}
iface_types         : [bareudp, dpdk, dpdkvhostuser, dpdkvhostuserclient, erspan, geneve, gre, gtpu, internal, ip6erspan, ip6gre, lisp, patch, stt, system, tap, vxlan]
manager_options     : []
next_cfg            : 9
other_config        : {dpdk-init="true"}
ovs_version         : "2.15.90"
ssl                 : []
statistics          : {}
system_type         : Ubuntu
system_version      : "20.04-focal"

这是接口(interface)的当前设置，设备使用 dpdk-devbind 绑定(bind)到 vfio-pci:

➜  ~ dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:00:1f.6 'Ethernet Connection (10) I219-V 0d4f' drv=vfio-pci unused=

Network devices using kernel driver
===================================
0000:00:14.3 'Wireless-AC 9462 02f0' if=wlp0s20f3 drv=iwlwifi unused=vfio-pci *Active*

IOMMU 已启用:

➜  ~ sudo dmesg | grep -e DMAR -e IOMMU
[    0.015758] ACPI: DMAR 0x000000006CAFE000 0000A8 (v01 LENOVO TP-N2Q   00001250 PTEC 00000002)
[    0.055802] DMAR: IOMMU enabled
[    0.157186] DMAR: Host address width 39
[    0.157188] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.157196] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.157198] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.157203] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.157205] DMAR: RMRR base: 0x0000006e81a000 end: 0x0000006e839fff
[    0.157207] DMAR: RMRR base: 0x00000079000000 end: 0x0000007d7fffff
[    0.157210] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.157212] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.157213] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.159739] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.890022] DMAR: No ATSR found
[    0.890108] DMAR: dmar0: Using Queued invalidation
[    0.890113] DMAR: dmar1: Using Queued invalidation
[    0.891306] DMAR: Intel(R) Virtualization Technology for Directed I/O

我的 grub 和大页面设置:

➜  ~ cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.10.7-051007-generic root=UUID=455abfd8-5778-4930-8062-0ba7a48be98e ro quiet splash iommu=pt intel_iommu=on vt.handoff=7

grep HugePages_ /proc/meminfo
HugePages_Total:    2048
HugePages_Free:     1536
HugePages_Rsvd:        0
HugePages_Surp:        0

关于如何进一步调试问题的任何提示？不幸的是，EAL 错误报告“驱动程序无法附加设备”并不是非常确定。

Answer 1

安装包 libdpdk-dev 解决了这个问题。