[dpdk-dev] [PATCH 0/2] dpdk: Allow for dynamic enablement of some isolated features
On Fri, Aug 01, 2014 at 02:08:22PM -0700, Bruce Richardson wrote: > On Fri, Aug 01, 2014 at 04:43:52PM -0400, Neil Horman wrote: > > On Fri, Aug 01, 2014 at 12:22:22PM -0700, Bruce Richardson wrote: > > > On Fri, Aug 01, 2014 at 11:06:29AM -0400, Neil Horman wrote: > > > > On Thu, Jul 31, 2014 at 01:25:06PM -0700, Bruce Richardson wrote: > > > > > On Thu, Jul 31, 2014 at 04:10:18PM -0400, Neil Horman wrote: > > > > > > On Thu, Jul 31, 2014 at 11:36:32AM -0700, Bruce Richardson wrote: > > > > > > > Thu, Jul 31, 2014 at 02:10:32PM -0400, Neil Horman wrote: > > > > > > > > On Thu, Jul 31, 2014 at 10:32:28AM -0400, Neil Horman wrote: > > > > > > > > > On Thu, Jul 31, 2014 at 03:26:45PM +0200, Thomas Monjalon > > > > > > > > > wrote: > > > > > > > > > > 2014-07-31 09:13, Neil Horman: > > > > > > > > > > > On Wed, Jul 30, 2014 at 02:09:20PM -0700, Bruce > > > > > > > > > > > Richardson wrote: > > > > > > > > > > > > On Wed, Jul 30, 2014 at 03:28:44PM -0400, Neil Horman > > > > > > > > > > > > wrote: > > > > > > > > > > > > > On Wed, Jul 30, 2014 at 11:59:03AM -0700, Bruce > > > > > > > > > > > > > Richardson wrote: > > > > > > > > > > > > > > On Tue, Jul 29, 2014 at 04:24:24PM -0400, Neil > > > > > > > > > > > > > > Horman wrote: > > > > > > > > > > > > > > > Hey all- > > > > > > > > > > > > > > With regards to the general approach for runtime detection of > > > > > > > software > > > > > > > functions, I wonder if something like this can be handled by the > > > > > > > packaging system? Is it possible to ship out a set of shared libs > > > > > > > compiled up for different instruction sets, and then at rpm > > > > > > > install > > > > > > > time, symlink the appropriate library? This would push the whole > > > > > > > issue > > > > > > > of detection of code paths outside of code, work across all our > > > > > > > libraries and ensure each user got the best performance they > > > > > > > could get > > > > > > > form a binary? > > > > > > > Has something like this been done before? The building of all the > > > > > > > libraries could be scripted easy enough, just do multiple builds > > > > > > > using > > > > > > > different EXTRA_CFLAGS each time, and move and rename the .so's > > > > > > > after > > > > > > > each run. > > > > > > > > > > > > > > > > > > > Sorry, I missed this in my last reply. > > > > > > > > > > > > In answer to your question, the short version is that such a thing > > > > > > is roughly > > > > > > possible from a packaging standpoint, but completely unworkable > > > > > > from a > > > > > > distribution standpoint. We could certainly build the dpdk > > > > > > multiple times and > > > > > > rename all the shared objects to some variant name representative > > > > > > of the > > > > > > optimzations we build in for certain cpu flags, but then we woudl > > > > > > be shipping X > > > > > > versions of the dpdk, and any appilcation (say OVS that made use of > > > > > > the dpdk > > > > > > would need to provide a version linked against each variant to be > > > > > > useful when > > > > > > making a product, and each end user would need to manually select > > > > > > (or run a > > > > > > script to select) which variant is most optimized for the system at > > > > > > hand. Its > > > > > > just not a reasonable way to package a library. > > > > > > > > > > Sorry, perhaps I was not clear, having the user have to select the > > > > > appropriate library was not what I was suggesting. Instead, I was > > > > > suggesting that the rpm install "librte_pmd_ixgbe.so.generic", > > > > > "librte_pmd_ixgbe.so.sse42" and "librte_pmd_ixgbe.so.avx". Then the > > > > > rpm > > > > > post-install script would look at the cpuflags in cpuinfo and then > > > > > symlink librte_pmd_ixgbe.so to the best-match version. That way the > > > > > user > > > > > only has to link against "librte_pmd_ixgbe.so" and depending on the > > > > > system its run on, the loader will automatically resolve the symbols > > > > > from the appropriate instruction-set specific .so file. > > > > > > > > > > > > > This is an absolute packaging nightmare, it will potentially break all > > > > sorts of > > > > corner cases, and support processes. To cite a few examples: > > > > > > > > 1) Upgrade support - What if the minimum cpu requirements for dpdk are > > > > advanced > > > > at some point in the future? The above strategy has no way to know > > > > that a given > > > > update has more advanced requirements than a previous update, and when > > > > the > > > > update is installed, the previously linked library for the old base will > > > > dissappear, leaving broken applications behind. > > > > > > Firstly, I didn't know we could actually specify minimum cpu > > > requirements for packaging, that is something that could be useful :-) > > You misread my comment :). I didn't say we could specify minimum cpu > > requirements at packaging (you can't, beyond general arch), I said "what if > > the > >
[dpdk-dev] Debugging EAL PCI / Driver Init
I did a bit more experimentation and found the following. If I unmark the rte_igb_pmd_init function as static, and call it directly from my code, the driver will load, and the port count increments to 2: EAL: PCI device :01:00.0 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: PCI memory mapped at 0x7f09d45f2000 EAL: PCI memory mapped at 0x7f09d473 PMD: eth_igb_dev_init(): port_id 0 vendorID=0x8086 deviceID=0x1521 EAL: PCI device :01:00.1 on NUMA socket -1 EAL: probe driver: 8086:1521 rte_igb_pmd EAL: PCI memory mapped at 0x7f09d0f0 EAL: PCI memory mapped at 0x7f09d472c000 PMD: eth_igb_dev_init(): port_id 1 vendorID=0x8086 deviceID=0x1521 So it seems like when you enable these options: # Combine to one single library CONFIG_RTE_BUILD_COMBINE_LIBS=y CONFIG_RTE_LIBNAME="intel_dpdk" You don't really get a working DPDK inside of "-lintel_dpdk". Someone suggested linking with "-Xlinker -lintel_dpdk" but that didn't seem to help. Is there a secret to getting a single integrated static library, where all of the PMD's end up in the PCI driver list so rte_eal_pci_probe can find them? Or some secret to linking against the combined library which works properly? Matthew. On Fri, Aug 01, 2014 at 10:51:38AM -0700, Matthew Hall wrote: > Hello, > > I am running into a problem where Eth driver init works fine in a sample app > and finds my NICs, and the NICs appear in rte_eal_pci_dump(stdout) but they > don't show up in rte_eth_dev_count() even after rte_eal_pci_probe() is called > the same as the sample apps, so my app won't boot. > > I have a lot of experience using the older versions of the DPDK where you had > to call the PMD init functions manually but no experience with the later > versions where the DPDK is supposed to init the PMDs itself automatically. > > What do I have to do to dump the most possible debug output on why the driver > list for my PCI devices always seems empty? Any places I should look to see > the issue? Maybe I didn't link it together with the right DPDK libs? I used > the combined DPDK static lib libintel_dpdk.a to make things simpler as I had > seen recommended in various places. > > Thanks, > Matthew.
[dpdk-dev] Debugging EAL PCI / Driver Init
Also, when using the separate libraries the problem still happens: -lethdev -lrte_cfgfile -lrte_cmdline -lrte_distributor -lrte_hash -lrte_ip_frag -lrte_lpm -lrte_malloc -lrte_mbuf -lrte_mempool -lrte_pmd_e1000 -lrte_pmd_pcap -lrte_pmd_virtio_uio -lrte_pmd_vmxnet3_uio -lrte_port -lrte_ring -lrte_table -lrte_timer -lrte_eal -lbsd -ldl -lpcap -lpthread So it seems there is a special order or link technique which must be used, or rte_eal's PCI code won't be able to load the PMD's during PCI probing. Is it documented anywhere how to get that to work? Thanks, Matthew. On Sat, Aug 02, 2014 at 08:29:04AM -0700, Matthew Hall wrote: > I did a bit more experimentation and found the following. If I unmark the > rte_igb_pmd_init function as static, and call it directly from my code, the > driver will load, and the port count increments to 2: > > EAL: PCI device :01:00.0 on NUMA socket -1 > EAL: probe driver: 8086:1521 rte_igb_pmd > EAL: PCI memory mapped at 0x7f09d45f2000 > EAL: PCI memory mapped at 0x7f09d473 > PMD: eth_igb_dev_init(): port_id 0 vendorID=0x8086 deviceID=0x1521 > > EAL: PCI device :01:00.1 on NUMA socket -1 > EAL: probe driver: 8086:1521 rte_igb_pmd > EAL: PCI memory mapped at 0x7f09d0f0 > EAL: PCI memory mapped at 0x7f09d472c000 > PMD: eth_igb_dev_init(): port_id 1 vendorID=0x8086 deviceID=0x1521 > > So it seems like when you enable these options: > > # Combine to one single library > CONFIG_RTE_BUILD_COMBINE_LIBS=y > CONFIG_RTE_LIBNAME="intel_dpdk" > > You don't really get a working DPDK inside of "-lintel_dpdk". Someone > suggested linking with "-Xlinker -lintel_dpdk" but that didn't seem to help. > > Is there a secret to getting a single integrated static library, where all of > the PMD's end up in the PCI driver list so rte_eal_pci_probe can find them? > Or > some secret to linking against the combined library which works properly? > > Matthew. > > On Fri, Aug 01, 2014 at 10:51:38AM -0700, Matthew Hall wrote: > > Hello, > > > > I am running into a problem where Eth driver init works fine in a sample > > app > > and finds my NICs, and the NICs appear in rte_eal_pci_dump(stdout) but they > > don't show up in rte_eth_dev_count() even after rte_eal_pci_probe() is > > called > > the same as the sample apps, so my app won't boot. > > > > I have a lot of experience using the older versions of the DPDK where you > > had > > to call the PMD init functions manually but no experience with the later > > versions where the DPDK is supposed to init the PMDs itself automatically. > > > > What do I have to do to dump the most possible debug output on why the > > driver > > list for my PCI devices always seems empty? Any places I should look to see > > the issue? Maybe I didn't link it together with the right DPDK libs? I used > > the combined DPDK static lib libintel_dpdk.a to make things simpler as I > > had > > seen recommended in various places. > > > > Thanks, > > Matthew.
[dpdk-dev] Debugging EAL PCI / Driver Init
On Sun, Aug 03, 2014 at 01:37:06AM +0900, Masaru Oki wrote: > cc links library funtion from archive only if call from other object. > but new dpdk pmd library has constractor section and not call directly. > ld always links library funtion with constractor section. > use -Xlinker, or use ld instead of cc. Hello Oki-san, The trick to fix it was this, I finally found it in the example Makefiles with V=1 flag. -Wl,--whole-archive -Wl,--start-group -lintel_dpdk -Wl,--end-group -Wl,--no-whole-archive Thank you for the advice you provided, I couldn't have fixed it without your suggestions... it got me to look more closely at the linking. Importantly, "-Wl,--whole-archive" includes the entire archive whether or not it's called from other objects, so we don't lose the constructors, just like you said. Matthew.
