It would be great if we could figure out the reason. The contiv-vpp documentation needed to spell out that you need 1GB hugepages precisely because 2MB pages don't work.
And there's also dpdk documentation [0] which doesn't mention the hugepages problem, making it seem like it shoudn't be an issue, but maybe that's a documentation overight. Juraj [0] https://doc.dpdk.org/guides-18.11/nics/thunderx.html From: Gorka Garcia [mailto:ggar...@marvell.com] Sent: Wednesday, December 12, 2018 11:03 AM To: Juraj Linkeš <juraj.lin...@pantheon.tech>; dmar...@me.com; Nitin Saxena <nitin.sax...@cavium.com> Cc: vpp-dev@lists.fd.io; Sirshak Das <sirshak....@arm.com> Subject: RE: [vpp-dev] 2MB vs 1GB hugepages on ARM ThunderX I am not sure on the reason for this, but it is documented here: https://github.com/contiv/vpp/blob/master/docs/arm64/MANUAL_INSTALL_CAVIUM.md “To mention the most important thing from DPDK setup instructions you need to setup 1GB hugepages. The allocation of hugepages should be done at boot time or as soon as possible after system boot to prevent memory from being fragmented in physical memory. Add parameters hugepagesz=1GB hugepages=16 default_hugepagesz=1GB to the file /etc/default/grub” Gorka From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> On Behalf Of Juraj Linkeš Sent: Wednesday, December 12, 2018 9:07 AM To: dmar...@me.com<mailto:dmar...@me.com>; gorka.gar...@cavium.com<mailto:gorka.gar...@cavium.com>; Nitin Saxena <nitin.sax...@cavium.com<mailto:nitin.sax...@cavium.com>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>; Sirshak Das <sirshak....@arm.com<mailto:sirshak....@arm.com>> Subject: [EXT] Re: [vpp-dev] 2MB vs 1GB hugepages on ARM ThunderX External Email ________________________________ External Email Thanks Damjan. Nitin, Gorka, do you have any input on this? Juraj From: Damjan Marion via Lists.Fd.Io [mailto:dmarion=me....@lists.fd.io] Sent: Tuesday, December 11, 2018 5:21 PM To: Juraj Linkeš <juraj.lin...@pantheon.tech<mailto:juraj.lin...@pantheon.tech>> Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] 2MB vs 1GB hugepages on ARM ThunderX Dear Juraj, I don't think anybody have experience with ThunderX to help you here. The facts that other NICs work OK indicates that that this particular driver requires something special. What it is, you will probably need to ask Cavium/Marvell guys... -- Damjan On 11 Dec 2018, at 07:56, Juraj Linkeš <juraj.lin...@pantheon.tech<mailto:juraj.lin...@pantheon.tech>> wrote: Hi folks, I've ran into an issue with hugepages on a Cavium ThunderX soc. I was trying to bind a physical interface to VPP. When using 1GB hugepages the interface seems to be working fine (well, at least I saw the interface in VPP and I was able to configure it and use ping with it), but when using 2MB hugepages the interface appeared in error state. The output from show hardware told me this: VirtualFunctionEthernet1/0/1 1 down VirtualFunctionEthernet1/0/1 Ethernet address 40:8d:5c:e7:b1:12 Cavium ThunderX carrier down flags: pmd pmd-init-fail maybe-multiseg rx: queues 1 (max 96), desc 1024 (min 0 max 65535 align 1) tx: queues 1 (max 96), desc 1024 (min 0 max 65535 align 1) pci: device 177d:a034 subsystem 177d:a134 address 0002:01:00.01 numa 0 module: unknown max rx packet len: 9204 promiscuous: unicast off all-multicast off vlan offload: strip off filter off qinq off rx offload avail: vlan-strip ipv4-cksum udp-cksum tcp-cksum jumbo-frame crc-strip scatter rx offload active: jumbo-frame crc-strip scatter tx offload avail: ipv4-cksum udp-cksum tcp-cksum outer-ipv4-cksum tx offload active: rss avail: ipv4 ipv4-tcp ipv4-udp ipv6 ipv6-tcp ipv6-udp port vxlan geneve nvgre rss active: ipv4 ipv4-tcp ipv4-udp ipv6 ipv6-tcp ipv6-udp tx burst function: (nil) rx burst function: (nil) Errors: rte_eth_rx_queue_setup[port:0, errno:-22]: Unknown error -22 I dug around a bit and this seems to be what -22 means: #define EINVAL 22 /* Invalid argument */ -EINVAL: The size of network buffers which can be allocated from the memory pool does not fit the various buffer sizes allowed by the device controller. Is this something you've seen before? Is this a bug? Do I need to do something extra if I want to use 2MB hugepages? Thanks, Juraj -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11564): https://lists.fd.io/g/vpp-dev/message/11564 Mute This Topic: https://lists.fd.io/mt/28720621/675642 Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [dmar...@me.com<mailto:dmar...@me.com>] -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11577): https://lists.fd.io/g/vpp-dev/message/11577 Mute This Topic: https://lists.fd.io/mt/28720621/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
