On 23/07/2015 08:09, Rapelly, Varun wrote:
> Hi,
>
> I'm facing problems with "iommu=pt intel_iommu=on" in ProLiant DL380p Gen8
> server[RHEL 7.1]. But on ProLiant DL380 G7[RHEL 7.1] server, not facing this
> issue.
>
> When I add the above options to the kernel boot line and configure OVS bridge
> [with -dpdk option], not able to send the packets out from the OVS bridge.
> But when I pass iommu=pt intel_iommu=off, i'm able to send & receive packets.
> Following are the dmesg details for both the scenarios on gen8 machine.
>
> [root at ARTHA ~]# nic
>
> Network devices using DPDK-compatible driver
> ============================================
> 0000:0a:00.0 'I350 Gigabit Network Connection' drv=igb_uio unused=
>
> Network devices using kernel driver
> ===================================
> 0000:03:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno1 drv=tg3
> unused=igb_uio *Active*
> 0000:03:00.1 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno2 drv=tg3
> unused=igb_uio
> 0000:03:00.2 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno3 drv=tg3
> unused=igb_uio
> 0000:03:00.3 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno4 drv=tg3
> unused=igb_uio
> 0000:0a:00.1 'I350 Gigabit Network Connection' if=ens3f1 drv=igb
> unused=igb_uio
>
> [root at ARTHA ~]# cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64
> root=UUID=89019831-4506-451e-8259-68171411ac4b ro crashkernel=auto rhgb quiet
> iommu=pt intel_iommu=on default_hugepagesz=1G hugepagesz=1G hugepages=16
> isolcpus=2-7
>
> [root at ARTHA ~]# dmesg | grep -e DMAR -e IOMMU [deleted few lines below as
> it was a big list]
> {
> [ 0.000000] ACPI: DMAR 00000000bddad200 00450 (v01 HP ProLiant
> 00000001 \xffffffd2? 0000162E)
> [ 0.000000] Intel-IOMMU: enabled
> [ 0.019553] dmar: IOMMU 0: reg_base_addr f8ffe000 ver 1:0 cap
> d2078c106f0466 ecap f020de
> [ 0.019652] IOAPIC id 8 under DRHD base 0xf8ffe000 IOMMU 0
> [ 0.019653] IOAPIC id 0 under DRHD base 0xf8ffe000 IOMMU 0
> [ 0.633163] IOMMU 0 0xf8ffe000: using Queued invalidation
> [ 0.633505] IOMMU: Setting identity map for device 0000:0a:00.0 [0xe8000 -
> 0xe8fff]
> [ 0.633539] IOMMU: Setting identity map for device 0000:0a:00.1 [0xe8000 -
> 0xe8fff]
> [ 0.633713] IOMMU: Prepare 0-16MiB unity mapping for LPC
> }
>
> After creating ovs bridge and adding ports to it
>
> [ 150.845001] SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses
> transition SIDs
> [ 150.867256] igb_uio: module verification failed: signature and/or required
> key missing - tainting kernel
> [ 150.867415] igb_uio: Use MSIX interrupt by default
> [ 150.991486] igb 0000:0a:00.0: removed PHC on ens3f0
> [ 150.991490] igb 0000:0a:00.0: DCA disabled
> [ 151.051927] igb_uio 0000:0a:00.0: irq 85 for MSI/MSI-X
> [ 151.052122] igb_uio 0000:0a:00.0: uio device registered with irq 55
> [ 151.086101] gre: GRE over IPv4 demultiplexor driver
> [ 151.095108] openvswitch: Open vSwitch switching datapath
> [ 153.946783] device ovs-netdev entered promiscuous mode
> [ 153.956208] device ovs entered promiscuous mode
> [ 154.132619] dmar: DRHD: handling fault status reg 2
> [ 154.132625] dmar: DMAR:[DMA Read] Request device [0a:00.0] fault addr
> 6633d000
> DMAR:[fault reason 06] PTE Read access is not set
> [ 154.211283] dmar: DRHD: handling fault status reg 102
> [ 154.211287] dmar: DMAR:[DMA Read] Request device [0a:00.0] fault addr
> 6633d000
> DMAR:[fault reason 06] PTE Read access is not set
> [ 154.394040] dmar: DRHD: handling fault status reg 202
> [ 154.394046] dmar: DMAR:[DMA Read] Request device [0a:00.0] fault addr
> 6637d000
> DMAR:[fault reason 06] PTE Read access is not set
>
>
> [root at ARTHA ~]# ping -I ovs 10.54.218.1
> PING 10.54.218.1 (10.54.218.1) from 10.54.218.89 ovs: 56(84) bytes of data.
> ^C
> --- 10.54.218.1 ping statistics ---
> 3 packets transmitted, 0 received, 100% packet loss, time 1999ms
>
>
> [root at ARTHA ~]# cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-3.10.0-229.el7.x86_64
> root=UUID=89019831-4506-451e-8259-68171411ac4b ro crashkernel=auto rhgb quiet
> iommu=pt intel_iommu=off default_hugepagesz=1G hugepagesz=1G hugepages=16
> isolcpus=2-7
> [root at ARTHA ~]# dmesg | grep -e DMAR -e IOMMU
> [ 0.000000] ACPI: DMAR 00000000bddad200 00450 (v01 HP ProLiant
> 00000001 \xffffffd2? 0000162E)
> [ 0.000000] Intel-IOMMU: disabled
> [ 0.019551] dmar: IOMMU 0: reg_base_addr f8ffe000 ver 1:0 cap
> d2078c106f0466 ecap f020de
> [ 0.019651] IOAPIC id 8 under DRHD base 0xf8ffe000 IOMMU 0
> [ 0.019652] IOAPIC id 0 under DRHD base 0xf8ffe000 IOMMU 0
>
> [ 263.084959] SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses
> transition SIDs
> [ 263.113272] igb_uio: module verification failed: signature and/or required
> key missing - tainting kernel
> [ 263.113455] igb_uio: Use MSIX interrupt by default
> [ 263.240329] igb 0000:0a:00.0: removed PHC on ens3f0
> [ 263.240333] igb 0000:0a:00.0: DCA disabled
> [ 263.298807] igb_uio 0000:0a:00.0: irq 85 for MSI/MSI-X
> [ 263.299001] igb_uio 0000:0a:00.0: uio device registered with irq 55
> [ 263.332111] gre: GRE over IPv4 demultiplexor driver
> [ 263.341157] openvswitch: Open vSwitch switching datapath
> [ 266.172120] device ovs-netdev entered promiscuous mode
> [ 266.181865] device ovs entered promiscuous mode
>
> [root at ARTHA ~]# ping -I ovs 10.54.218.1
> PING 10.54.218.1 (10.54.218.1) from 10.54.218.89 ovs: 56(84) bytes of data.
> 64 bytes from 10.54.218.1: icmp_seq=1 ttl=255 time=0.762 ms
> 64 bytes from 10.54.218.1: icmp_seq=2 ttl=255 time=1.27 ms
> ^C
> --- 10.54.218.1 ping statistics ---
> 2 packets transmitted, 2 received, 0% packet loss, time 1000ms
> rtt min/avg/max/mdev = 0.762/1.017/1.273/0.257 ms
>
> [root at ARTHA ~]# cat /proc/cpuinfo| egrep "vmx|svm"
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep
> mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl
> vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt
> tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt
> pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep
> mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl
> vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt
> tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb xsaveopt
> pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms
>
> Is it a known issue or something to do with the configuration?
>
>
> Regards,
> Varun
>
I think this could be an HP issue. You can read more in the following link:
http://dpdk.org/ml/archives/dev/2015-March/015504.html
Sergio
