Hi Gaetan From: Gaëtan Rivet, Friday, January 19, 2018 5:00 PM > Hi Matan, > > On Fri, Jan 19, 2018 at 01:35:10PM +0000, Matan Azrad wrote: > > Hi Konstantin > > > > From: Ananyev, Konstantin, Friday, January 19, 2018 3:09 PM > > > > -----Original Message----- > > > > From: Matan Azrad [mailto:ma...@mellanox.com] > > > > Sent: Friday, January 19, 2018 12:52 PM > > > > To: Ananyev, Konstantin <konstantin.anan...@intel.com>; Thomas > > > > Monjalon <tho...@monjalon.net>; Gaetan Rivet > > > <gaetan.ri...@6wind.com>; > > > > Wu, Jingjing <jingjing...@intel.com> > > > > Cc: dev@dpdk.org; Neil Horman <nhor...@tuxdriver.com>; > Richardson, > > > > Bruce <bruce.richard...@intel.com> > > > > Subject: RE: [PATCH v3 7/7] app/testpmd: adjust ethdev port > > > > ownership > > > > > > > > Hi Konstantin > > > > > > > > From: Ananyev, Konstantin, Friday, January 19, 2018 2:38 PM > > > > > To: Matan Azrad <ma...@mellanox.com>; Thomas Monjalon > > > > > <tho...@monjalon.net>; Gaetan Rivet <gaetan.ri...@6wind.com>; > > > Wu, > > > > > Jingjing <jingjing...@intel.com> > > > > > Cc: dev@dpdk.org; Neil Horman <nhor...@tuxdriver.com>; > > > > > Richardson, Bruce <bruce.richard...@intel.com> > > > > > Subject: RE: [PATCH v3 7/7] app/testpmd: adjust ethdev port > > > > > ownership > > > > > > > > > > Hi Matan, > > > > > > > > > > > -----Original Message----- > > > > > > From: Matan Azrad [mailto:ma...@mellanox.com] > > > > > > Sent: Thursday, January 18, 2018 4:35 PM > > > > > > To: Thomas Monjalon <tho...@monjalon.net>; Gaetan Rivet > > > > > > <gaetan.ri...@6wind.com>; Wu, Jingjing <jingjing...@intel.com> > > > > > > Cc: dev@dpdk.org; Neil Horman <nhor...@tuxdriver.com>; > > > Richardson, > > > > > > Bruce <bruce.richard...@intel.com>; Ananyev, Konstantin > > > > > > <konstantin.anan...@intel.com> > > > > > > Subject: [PATCH v3 7/7] app/testpmd: adjust ethdev port > > > > > > ownership > > > > > > > > > > > > Testpmd should not use ethdev ports which are managed by other > > > > > > DPDK entities. > > > > > > > > > > > > Set Testpmd ownership to each port which is not used by other > > > > > > entity and prevent any usage of ethdev ports which are not > > > > > > owned by > > > Testpmd. > > > > > > > > > > > > Signed-off-by: Matan Azrad <ma...@mellanox.com> > > > > > > --- > > > > > > app/test-pmd/cmdline.c | 89 +++++++++++++++++++------------- > ---- > > > ---- > > > > > ----- > > > > > > app/test-pmd/cmdline_flow.c | 2 +- > > > > > > app/test-pmd/config.c | 37 ++++++++++--------- > > > > > > app/test-pmd/parameters.c | 4 +- > > > > > > app/test-pmd/testpmd.c | 63 ++++++++++++++++++++---------- > -- > > > > > > app/test-pmd/testpmd.h | 3 ++ > > > > > > 6 files changed, 103 insertions(+), 95 deletions(-) > > > > > > > > > > > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c > > > > > > index > > > > > > 31919ba..6199c64 100644 > > > > > > --- a/app/test-pmd/cmdline.c > > > > > > +++ b/app/test-pmd/cmdline.c > > > > > > @@ -1394,7 +1394,7 @@ struct cmd_config_speed_all { > > > > > > &link_speed) < 0) > > > > > > return; > > > > > > > > > > > > - RTE_ETH_FOREACH_DEV(pid) { > > > > > > + RTE_ETH_FOREACH_DEV_OWNED_BY(pid, my_owner.id) { > > > > > > > > > > Why do we need all these changes? > > > > > As I understand you changed definition of RTE_ETH_FOREACH_DEV(), > > > > > so no testpmd should work ok default (no_owner case). > > > > > Am I missing something here? > > > > > > > > Now, After Gaetan suggestion RTE_ETH_FOREACH_DEV(pid) will iterate > > > over all valid and ownerless ports. > > > > > > Yes. > > > > > > > Here Testpmd wants to iterate over its owned ports. > > > > > > Why? Why it can't just iterate over all valid and ownerless ports? > > > As I understand it would be enough to fix current problems and would > > > allow us to avoid any changes in testmpd (which I think is a good thing). > > > > Yes, I understand that this big change is very daunted, But I think the > current a lot of bugs in testpmd(regarding port ownership) even more > daunted. > > > > Look, > > Testpmd initiates some of its internal databases depends on specific > > port iteration, In some time someone may take ownership of Testpmd > ports and testpmd will continue to touch them. > > > > If I look back on the fail-safe, its sole purpose is to have seamless hotplug > with existing applications. >
Yes. > Port ownership is a genericization of some functions introduced by the fail- > safe, that could structure DPDK further. Not only. Port ownership is a new concept saying that not all the ports are only for the application and defines well the new port usage synchronization rules. It can be a solution for failsafe scenario, but it solves a big generic problem regardless fail-safe. > It should allow applications to have a seamless integration with subsystems > using port ownership. Without this, port ownership cannot be used. I do not think it is accurate. We can use different solution to solve the fail-safe case (seamless) by using the DEFFERED state as you did. Port ownership is not only for failsafe case - it is a generic new concept which BTW can fix the fail-safe case(full fix). So, application should use port ownership regardless the failsafe using, just to be sure no one touch its ports. > Testpmd should be fixed, but follow the most common design patterns of > DPDK applications. Going with port ownership seems like a paradigm shift. I think this patch is a classic fix for it and for the full generic issue. Do you have simpler fix to the races? > > In addition > > Using the old iterator in some places in testpmd will cause a race for run- > time new ports(can be created by failsafe or any hotplug code): > > - testpmd finds an ownerless port(just now created) by the old > > iterator and start traffic there, > > - failsafe takes ownership of this new port and start traffic there. > > Problem! > > Testpmd does not handle detection of new port. If it did, testing fail-safe > with it would be wrong. It used the old iterator everywhere. So it see(and uses) all the valid ports all the time. As the new concept - it should be changed to use its owned ports, It is a simple classic solution just to use the new iterator. > At startup, RTE_ETH_FOREACH_DEV already fixed the issue of registering > DEFERRED ports. There are still remaining issues regarding this, but I think > they should be fixed. The architecture does not need to be completely > moved to port ownership. I think port ownership fixes the issues nicely, don't you think so? > If anything, this should serve as a test for your API with common > applications. > I think you'd prefer to know and debug with testpmd instead of firing up VPP > or something like that to determine what went wrong with using the fail- > safe. > > Yes as your examples in docs. > > In addition > > As a good example for well-done application (free from ownership bugs) I > tried here to adjust Tespmd to the new rules and BTW to fix a lot of bugs. > > Testpmd has too much cruft, it won't ever be a good example of a well-done > application. > > If you want to demonstrate ownership, I think you should start an example > application demonstrating race conditions and their mitigation. > > I think that would be interesting for many DPDK users. I will think about that regardless the testpmd adjustment (need to find time :)) > > > > So actually applications which are not aware to the port ownership still are > exposed to races, but if there use the old iterator(with the new change) the > amount of races decreases. > > > > Thanks, Matan. > > > Konstantin > > > > > > > > > > > I added to Testpmd ability to take an ownership of ports as the > > > > new ownership and synchronization rules suggested, Since Tespmd is > > > > a DPDK entity which wants that no one will touch its owned ports, > > > > It must allocate > > > an unique ID, set owner for its ports (see in main function) and > > > recognizes them by its owner ID. > > > > > > > > > Konstantin > > > > > > > Regards, > -- > Gaëtan Rivet > 6WIND
