On 02-Jul-20 3:49 AM, 陈毅强 wrote:
Hi,ALL
We fail to start dpdk in a newly deployed machine.Failed progress works in
proc-type=secondary mode. (DPDK version 16.04).Some logs are shown below
=========================================================================================
Primary progress:
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 4
PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7fffec3ff000
EAL: PCI memory mapped at 0x7ffff7ff4000
PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 12, SFP+: 3
PMD: eth_ixgbe_dev_init(): port 2 vendorID=0x8086 deviceID=0x10fb
EAL: PCI device 0000:82:00.1 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffed25d9000
EAL: PCI memory mapped at 0x7ffff7ff0000
Secondary progress logs:
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied.
Rx Burst Bulk Alloc function will be used on port=0.
EAL: PCI device 0000:04:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff5c04000
EAL: PCI memory mapped at 0x7ffff6004000
PMD: ixgbe_set_tx_function(): Using simple tx code path
PMD: ixgbe_set_rx_function(): Port[1] doesn't meet Vector Rx preconditions or
RTE_IXGBE_INC_VECTOR is not enabled
PMD: ixgbe_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied.
Rx Burst Bulk Alloc function will be used on port=1.
EAL: PCI device 0000:82:00.0 on NUMA socket 1
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7ffff65d8000
EAL: Cannot mmap device resource file
/sys/bus/pci/devices/0000:82:00.0/resource0 to address: 0x7fffec3ff000
EAL: Error - exiting with code: 1
Cause: Requested device 0000:82:00.0 cannot be used
==================================================================================
Relevant code is list be below:
pci_uio_map_secondary(struct rte_pci_device *dev)
{
int fd, i;
struct mapped_pci_resource *uio_res;
struct mapped_pci_res_list *uio_res_list =
RTE_TAILQ_CAST(rte_uio_tailq.head, mapped_pci_res_list);
TAILQ_FOREACH(uio_res, uio_res_list, next) {
/* skip this element if it doesn't match our PCI address */
if (rte_eal_compare_pci_addr(&uio_res->pci_addr, &dev->addr))
continue;
for (i = 0; i != uio_res->nb_maps; i++) {
/*
* open devname, to mmap it
*/
fd = open(uio_res->maps[i].path, O_RDWR);
if (fd < 0) {
RTE_LOG(ERR, EAL, "Cannot open %s: %s\n",
uio_res->maps[i].path, strerror(errno));
return -1;
}
void *mapaddr = pci_map_resource(uio_res->maps[i].addr,
fd, (off_t)uio_res->maps[i].offset,
(size_t)uio_res->maps[i].size, 0);
/* fd is not needed in slave process, close it */
close(fd);
if (mapaddr != uio_res->maps[i].addr) {
RTE_LOG(ERR, EAL,
"Cannot mmap device resource file %s to address: %p\n",
uio_res->maps[i].path,
uio_res->maps[i].addr);
return -1;
}
}
return 0;
}
RTE_LOG(ERR, EAL, "Cannot find resource for device\n");
return 1;
}
And We check the system config . ASLR is not enabled on our system. I wondered
why mapaddr != uio_res->maps[i].addr .Can anyone tell me why?
Thanks in advance.
Hi,
This is a known issue in DPDK and unfortunately it's not really
solvable. I've made attempts in the past to address it with an RFC [1]
patchset to enable deterministic hotplug, but it seems that this isn't a
problem the majority care about enough to fix it.
You might want to try with different --base-virtaddr values, but the
root cause is that we do not control our memory address space, kernel
does. Since secondary processes are predicated upon duplicating primary
process's address space, this inevitably means that sometimes the
duplication isn't possible because something else has already taken the
address, and we cannot do anything about it.
Enabling/disabling ASLR will not help or hurt with this issue; rather,
it'll make it more deterministic (i.e. it'll either always work or
always fail). So, playing around with various values for --base-virtaddr
is pretty much your only option at this point.
[1] http://patches.dpdk.org/bundle/aburakov/reliable_device_map/
--
Thanks,
Anatoly
