[dpdk-dev] IRC Channel, Come Guys ;)
Hello, 2014-04-01 23:35, Fred Pedrisa: > I've created an unofficial channel in freenode irc network, so for those > interested in joining there for a better chatting here are the informations > > Server : irc.freenode.net > Channel : ##dpdk (Yes, you must type two sharps). Why not use #dpdk ? -- Thomas
[dpdk-dev] [PATCH 03/16] pkg: add recipe for RPM
Hello,
Sorry for the long delay.
2014-02-24 08:52, Chris Wright:
> > pkg/rpm.spec | 143
>
> This should be dpdk.spec
Actually it should be dpdk-core.spec.
Since it is a file hosted in the project, is it mandatory to have such naming?
Could you explain why?
When building it with "rpmbuild -ta dpdk.tar.gz", the .spec name has no
importance.
> > +ExclusiveArch: i686, x86_64
> > +%define target %{_arch}-default-linuxapp-gcc
>
> What is this for?
Multi-targets builds are supported in DPDK Makefiles. So the target must be
explicited when building.
Is it OK to define such variable here?
> > +%description
> > +Dummy main package. Make only subpackages.
>
> I think the core package should have the .so, the -devel package w/
> headers, and no -static package. Including a .a file is discouraged in
> Fedora
OK
> > +%package core-runtime
>
> perhaps just 'runtime'
[...]
> > +%package core-devel
>
> name this devel
No, dpdk-core is a name to differentiate the main component and the extensions
such as dpdk-memnic.
So we should have dpdk-core and dpdk-core-devel. Do you agree?
> > +# debuginfo packaging is broken
> > +%define debug_package %{nil}
>
> Can you add a better comment why it's broken?
Yes, I think it's due to direct use of ld linker.
> > +make O=%{target} T=%{target} config
>
> Did you try to update the build system to output into $RPM_BUILD_ROOT
> directly? It's much preferred compared to the cp/rmdir below
Probably we could change the build system but it should be another patch.
> > +%install
> > +rm -rf %{buildroot}
> > +make O=%{target} DESTDIR=%{destdir}
>
> Why make again (IOW, why not in %build section)?
This one is for installing files only. Isn't it the role of %install?
> > +mv%{destdir}/%{target}/kmod/*.ko %{buildroot}%{moddir}
>
> Normally I'd use install command (-d will create directories)
Yes but here the strategy is to dispatch some files and keep the remaining
ones in %{datadir}.
Thanks for the review
--
Thomas
[dpdk-dev] [PATCH 03/16] pkg: add recipe for RPM
Hello,
Sorry for the long delay.
2014-02-26 14:07, Thomas Graf:
> On 02/04/2014 04:54 PM, Thomas Monjalon wrote:
> > +Version: 1.5.2r1
> > +Release: 1
>
> What kind of upgrade strategy do you have in mind?
>
> I'm raising this because Fedora and other distributions will require
> a unique package name for every version of the package that is not
> backwards compatible.
>
> Typically libraries provide backwards compatible within a major release,
> i.e. all 1.x.x releases would be compatible. I realize that this might
> not be applicable yet but maybe 1.5.x?
>
> Depending on the versioning schema the name would be dpdk15, dpdk16, ...
> or dpdk152, dpdk153, ...
We are working on this but at the moment there is no restriction on API/ABI
breakage. So I think it's too early to define such rule.
> > +BuildRequires: kernel-devel, kernel-headers, doxygen
>
> Is a python environment required as well?
Python is only needed to run some tools on the target. But is is optional.
Do you think it should be written somewhere?
> > +%description
> > +Dummy main package. Make only subpackages.
>
> I would just call the main package "libdpdk152" so you don't have to
> repeat the encoding versioning in all the subpackages.
>
> > +
> > +%package core-runtime
>
> What about calling it just "libdpdk"?
In this case, it should be libdpdk-core in order to distinguish it from dpdk
extensions. But the name of the project is dpdk so it seems simpler to call it
dpdk-core.
Is the "lib" prefix mandatory for libraries?
> > +%files core-runtime
> > +%dir %{datadir}
> > +%{datadir}/config
> > +%{datadir}/tools
> > +%{moddir}/*
> > +%{_sbindir}/*
> > +%{_bindir}/*
> > +%{_libdir}/*.so
>
> This brings up the question of multiple parallel DPDK installations.
> A specific application linking to library version X will also require
> tools of version X, right? A second application linking against version
> Y will require tools version Y. Right now, these could not be installed
> in parallel. Any chance we can make the runtime version independent?
Are you thinking about installing different major versions? In my
understanding, we cannot install 2 different minor versions of a package.
As long as there is no stable API, there is no major versions defined.
So don't you think we should speak about it later?
> Same applies to header files. A good option here would be to install
> them to /usr/include/libdpdk{version}/ and have a dpdk-1.5.2.pc which
> provides Cflags: -I${includedir}/libdpdk${version}
Yes same applies :)
I agree that a .pc file would be a good idea. But we also must allow to build
with the DPDK framework.
> You'll also need for all packages and subpackages installing shared
> libraries:
>
> %post -p /sbin/ldconfig
> %postun -p /sbin/ldconfig
OK
Thanks for the review
--
Thomas
[dpdk-dev] [vmxnet3-usermap PATCH 07/16] pkg: add recipe for RPM
2014-02-26 14:22, Thomas Graf: > On 02/04/2014 04:54 PM, Thomas Monjalon wrote: > > +BuildRequires: dpdk-core-devel, kernel-devel, kernel-headers > > +Requires: dpdk-core-runtime > > How does the compatibility mapping look like? I assume a given vmxnet3 > version can only be linked against certain dpdk versions? We need to > encode that mapping in the spec file somehow. Since vmxnet3-usermap-1.1, any dpdk >= 1.3 is supported. But RPM packaging is not supported for these old versions. So do you think it's needed to encode a restriction for these old versions? -- Thomas
[dpdk-dev] [PATCH 03/16] pkg: add recipe for RPM
On 04/02/2014 11:53 AM, Thomas Monjalon wrote:
> 2014-02-26 14:07, Thomas Graf:
>>> +BuildRequires: kernel-devel, kernel-headers, doxygen
>>
>> Is a python environment required as well?
>
> Python is only needed to run some tools on the target. But is is optional.
> Do you think it should be written somewhere?
Not sure what target is in this context. You only need to list it if
a python environment must be present at build time.
>> What about calling it just "libdpdk"?
>
> In this case, it should be libdpdk-core in order to distinguish it from dpdk
> extensions. But the name of the project is dpdk so it seems simpler to call it
> dpdk-core.
> Is the "lib" prefix mandatory for libraries?
Not at all. You are free to name the package. The mandatory part is that
runtime and development files must be separated and the development
files such as headers will go into a -devel package.
dpdk-core
dpdk-core-devel
>> This brings up the question of multiple parallel DPDK installations.
>> A specific application linking to library version X will also require
>> tools of version X, right? A second application linking against version
>> Y will require tools version Y. Right now, these could not be installed
>> in parallel. Any chance we can make the runtime version independent?
>
> Are you thinking about installing different major versions? In my
> understanding, we cannot install 2 different minor versions of a package.
> As long as there is no stable API, there is no major versions defined.
> So don't you think we should speak about it later?
That's right, you can't install multiple versions of the same package
unless you name them differently. This is why we need to come up with a
strategy how to handle naming and upgrades now, before we push it into
Fedora.
Example:
Let's assume we push DPDK 1.6.0 into Fedora as 'dpdk-core' with NVR
dpdk-core-1.6.0-1. Let's assume we later push Open vSwitch 2.2 into
Fedora which will consume DPDK 1.6.0 via "Requires: dpdk-core = 1.6.0"
We can't do dpdk-core >= 1.6.0 because there is no compatibility.
DPDK 1.6.1 gets released and we push it as dpdk-core-1.6.1-1. We then
push dpdk-pktgen into Fedora which is based on DPDK 1.6.1 and requires
"dpdk-core = 1.6.1". Users won't be able to install both OVS and
dpdk-ptkgen in parallel at this point because they can't install both
1.6.0 and 1.6.1.
Fedora inclusion will require a strategy to resolve this. A unique name
for each release is an option (Every DPDK release is currently a new
major release). This can slowly transform into compatible releases once
stable ABIs are in place.
Unique names is not enough though as multiple packages would still
attempt to install the same file, e.g. header files. This would be
typically resolved by installing headers and other non versioned files
with a prefix as outlined below. This leaves the problems of tool
versioning as they also seem to be bound to specific DPDK versions but
can't be prefixed as they need to be part of $PATH.
So while we don't have to enforce stable ABIs at this point we have to
account for the lack of it in the packaging names and packaging
structure.
Packages could be named
dpdk-1.6.0-core
dpdk-1.6.0-core-devel
https://fedoraproject.org/wiki/Packaging:NamingGuidelines#General_Naming
>> Same applies to header files. A good option here would be to install
>> them to /usr/include/libdpdk{version}/ and have a dpdk-1.5.2.pc which
>> provides Cflags: -I${includedir}/libdpdk${version}
>
> Yes same applies :)
> I agree that a .pc file would be a good idea. But we also must allow to build
> with the DPDK framework.
Definitely.
[dpdk-dev] [vmxnet3-usermap PATCH 07/16] pkg: add recipe for RPM
On 04/02/2014 12:08 PM, Thomas Monjalon wrote: > 2014-02-26 14:22, Thomas Graf: >> On 02/04/2014 04:54 PM, Thomas Monjalon wrote: >>> +BuildRequires: dpdk-core-devel, kernel-devel, kernel-headers >>> +Requires: dpdk-core-runtime >> >> How does the compatibility mapping look like? I assume a given vmxnet3 >> version can only be linked against certain dpdk versions? We need to >> encode that mapping in the spec file somehow. > > Since vmxnet3-usermap-1.1, any dpdk >= 1.3 is supported. > But RPM packaging is not supported for these old versions. > So do you think it's needed to encode a restriction for these old versions? The restriction is only needed if RPM packages for these old versions actually exist. On a more general note: While it is extremely nice to have this spec file to ease the building process. Fedora does not allow inclusion of external kernel modules: https://fedoraproject.org/wiki/Packaging:Guidelines#No_External_Kernel_Modules
[dpdk-dev] [PATCH 03/16] pkg: add recipe for RPM
On 04/02/2014 11:01 AM, Thomas Monjalon wrote: > Hello, > > Sorry for the long delay. > > 2014-02-24 08:52, Chris Wright: >>> pkg/rpm.spec | 143 >> >> This should be dpdk.spec > > Actually it should be dpdk-core.spec. > Since it is a file hosted in the project, is it mandatory to have such naming? > Could you explain why? > When building it with "rpmbuild -ta dpdk.tar.gz", the .spec name has no > importance. You are right, it doesn't matter for external building but this would likely get pointed at in the Fedora package review process. https://fedoraproject.org/wiki/Packaging:NamingGuidelines#Spec_file_name
[dpdk-dev] IRC Channel, Come Guys ;)
On Wed, Apr 02, 2014 at 10:13:30AM +0200, Thomas Monjalon wrote: > Hello, > > 2014-04-01 23:35, Fred Pedrisa: > > I've created an unofficial channel in freenode irc network, so for those > > interested in joining there for a better chatting here are the informations > > > > Server : irc.freenode.net > > Channel : ##dpdk (Yes, you must type two sharps). > > Why not use #dpdk ? > Agreed, two hashes seems like your working around something that needs to be fixed. #dpdk is open and already has interested parties on it. Neil > -- > Thomas >
[dpdk-dev] [PATCH v5] eal_common_cpuflags: Fix %rbx corruption, and simplify the code
On Tue, Mar 25, 2014 at 01:51:04PM -0700, H. Peter Anvin wrote: > On 03/25/2014 12:52 PM, Neil Horman wrote: > > Neil Horman reported that on x86-64 the upper half of %rbx would get > > clobbered when the code was compiled PIC or PIE, because the > > i386-specific code to preserve %ebx was incorrectly compiled. > > > > However, the code is really way more complex than it needs to be. For > > one thing, the CPUID instruction only needs %eax (leaf) and %ecx > > (subleaf) as parameters, and since we are testing for bits, we might > > as well list the bits explicitly. Furthermore, we can use an array > > rather than doing a switch statement inside a structure. > > > > Reported-by: Neil Horman > > Signed-off-by: H. Peter Anvin > > Signed-off-by: Neil Horman > > > > Looks good to me. > > Reviewed-by: H. Peter Anvin > > -hpa > > > Bump, did this get lost somewhere? Its been over a week and I don't see it in the tree Regards Neil
[dpdk-dev] [PATCH 03/16] pkg: add recipe for RPM
On Wed, Apr 02, 2014 at 11:53:51AM +0200, Thomas Monjalon wrote:
> Hello,
>
> Sorry for the long delay.
>
> 2014-02-26 14:07, Thomas Graf:
> > On 02/04/2014 04:54 PM, Thomas Monjalon wrote:
> > > +Version: 1.5.2r1
> > > +Release: 1
> >
> > What kind of upgrade strategy do you have in mind?
> >
> > I'm raising this because Fedora and other distributions will require
> > a unique package name for every version of the package that is not
> > backwards compatible.
> >
> > Typically libraries provide backwards compatible within a major release,
> > i.e. all 1.x.x releases would be compatible. I realize that this might
> > not be applicable yet but maybe 1.5.x?
> >
> > Depending on the versioning schema the name would be dpdk15, dpdk16, ...
> > or dpdk152, dpdk153, ...
>
> We are working on this but at the moment there is no restriction on API/ABI
> breakage. So I think it's too early to define such rule.
>
Now that you have DSO builds in place, theres no reason not to take the extra
step of versioning your API's, making backwards compatibility fairly
straightforward. Monolithic builds are still somewhat problematic regarding API
stability, but you could certainly offer stability in the DSOs.
> > > +BuildRequires: kernel-devel, kernel-headers, doxygen
> >
> > Is a python environment required as well?
>
> Python is only needed to run some tools on the target. But is is optional.
> Do you think it should be written somewhere?
>
> > > +%description
> > > +Dummy main package. Make only subpackages.
> >
> > I would just call the main package "libdpdk152" so you don't have to
> > repeat the encoding versioning in all the subpackages.
> >
> > > +
> > > +%package core-runtime
> >
> > What about calling it just "libdpdk"?
>
The version name should be left out of the library name, whatever you do.
Packaging can be responsible for versioning.
> In this case, it should be libdpdk-core in order to distinguish it from dpdk
> extensions. But the name of the project is dpdk so it seems simpler to call
> it
> dpdk-core.
> Is the "lib" prefix mandatory for libraries?
>
Not strictly, but IIRC if you don't add the lib, the linker won't find it with
the -l option, so you'll want to add it.
> > > +%files core-runtime
> > > +%dir %{datadir}
> > > +%{datadir}/config
> > > +%{datadir}/tools
> > > +%{moddir}/*
> > > +%{_sbindir}/*
> > > +%{_bindir}/*
> > > +%{_libdir}/*.so
> >
> > This brings up the question of multiple parallel DPDK installations.
> > A specific application linking to library version X will also require
> > tools of version X, right? A second application linking against version
> > Y will require tools version Y. Right now, these could not be installed
> > in parallel. Any chance we can make the runtime version independent?
>
> Are you thinking about installing different major versions? In my
> understanding, we cannot install 2 different minor versions of a package.
> As long as there is no stable API, there is no major versions defined.
> So don't you think we should speak about it later?
>
If the versioning is done properly (i.e shared libraries get version ids
attached to the library files properly), you can install as many library
versions as you like. You can only install a single -devel package, since it
links lib.so to a specific version.
> > Same applies to header files. A good option here would be to install
> > them to /usr/include/libdpdk{version}/ and have a dpdk-1.5.2.pc which
> > provides Cflags: -I${includedir}/libdpdk${version}
>
> Yes same applies :)
> I agree that a .pc file would be a good idea. But we also must allow to build
> with the DPDK framework.
>
> > You'll also need for all packages and subpackages installing shared
> > libraries:
> >
> > %post -p /sbin/ldconfig
> > %postun -p /sbin/ldconfig
>
> OK
>
> Thanks for the review
> --
> Thomas
>
[dpdk-dev] [PATCH v5] eal_common_cpuflags: Fix %rbx corruption, and simplify the code
2014-04-02 07:16, Neil Horman: > On Tue, Mar 25, 2014 at 01:51:04PM -0700, H. Peter Anvin wrote: > > On 03/25/2014 12:52 PM, Neil Horman wrote: > > > Neil Horman reported that on x86-64 the upper half of %rbx would get > > > clobbered when the code was compiled PIC or PIE, because the > > > i386-specific code to preserve %ebx was incorrectly compiled. > > > > > > However, the code is really way more complex than it needs to be. For > > > one thing, the CPUID instruction only needs %eax (leaf) and %ecx > > > (subleaf) as parameters, and since we are testing for bits, we might > > > as well list the bits explicitly. Furthermore, we can use an array > > > rather than doing a switch statement inside a structure. > > > > > > Reported-by: Neil Horman > > > Signed-off-by: H. Peter Anvin > > > Signed-off-by: Neil Horman > > > > Looks good to me. > > > > Reviewed-by: H. Peter Anvin > > > > -hpa > > Bump, did this get lost somewhere? Its been over a week and I don't see it > in the tree No, it's not lost. But this patch is not trivial and there were 5 versions with acknowledgements in the middle. So I think it was not a bad idea to wait few days in order to be sure this version is OK :) -- Thomas
[dpdk-dev] [PATCH v5] eal_common_cpuflags: Fix %rbx corruption, and simplify the code
2014-03-25 13:51, H. Peter Anvin: > On 03/25/2014 12:52 PM, Neil Horman wrote: > > Neil Horman reported that on x86-64 the upper half of %rbx would get > > clobbered when the code was compiled PIC or PIE, because the > > i386-specific code to preserve %ebx was incorrectly compiled. > > > > However, the code is really way more complex than it needs to be. For > > one thing, the CPUID instruction only needs %eax (leaf) and %ecx > > (subleaf) as parameters, and since we are testing for bits, we might > > as well list the bits explicitly. Furthermore, we can use an array > > rather than doing a switch statement inside a structure. > > > > Reported-by: Neil Horman > > Signed-off-by: H. Peter Anvin > > Signed-off-by: Neil Horman > > Looks good to me. > > Reviewed-by: H. Peter Anvin Applied for version 1.6.0r2. Thanks for this difficult fix. -- Thomas
[dpdk-dev] Issues on FDIR and multi-processes
Hi everyone, we are working with Flow DIRector on a multiprocess application. This application consists of N secondary processes, each one receiving packets directly from its own queue on the NIC. It works fine when each secondary process runs on a different CPU core; instead, if we try to run two secondary processes on the same CPU core, these processes aren't able to read from their respectively queue. Moreover we observe that one process initially receives same packet; then all the processes no longer receive anything. Do you have any suggestion about these strange behaviour? Is it a DPDK limitation or a FDIR limitation? Best regards. Mauro
[dpdk-dev] Issues on FDIR and multi-processes
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Mauro Annarumma > Sent: Wednesday, April 02, 2014 4:02 PM > To: dev at dpdk.org > Subject: [dpdk-dev] Issues on FDIR and multi-processes > > Hi everyone, > we are working with Flow DIRector on a multiprocess application. This > application consists of N secondary processes, > each one receiving packets directly from its own queue on the NIC. It works > fine when each secondary process runs on > a different CPU core; instead, if we try to run two secondary processes on > the same CPU core, these processes aren't > able to read from their respectively queue. Running two co-operating Intel DPDK processes on the same lcore is not a supported configuration of Intel DPDK multiprocess. [Please see section 20.3 of the Intel DPDK Programmer's Guide document, which lists out some limitations of multi-process support]. Basically, each process in a multiprocess configuration is equivalent to a thread in a normal application deployment, so just as having two threads on the same core causes problems with mempools and other objects which use an "lcore_id" value unique in each thread to work correctly, so running two processes on the same lcore can cause exactly the same issues. If you really do want to use this model and have multiple threads on the same lcore, at minimum you need to ensure that each thread/process has a unique lcore_id value <= RTE_MAX_LCORE. /Bruce
