On 10/18/2021 12:13 PM, Dmitry Kozlyuk wrote:
-----Original Message-----
From: Ferruh Yigit <ferruh.yi...@intel.com>
Sent: 18 октября 2021 г. 11:42
To: Dmitry Kozlyuk <dkozl...@nvidia.com>; dev@dpdk.org; Andrew Rybchenko
<andrew.rybche...@oktetlabs.ru>; Ori Kam <or...@nvidia.com>; Raslan
Darawsheh <rasl...@nvidia.com>; NBU-Contact-Thomas Monjalon
<tho...@monjalon.net>
Cc: Qi Zhang <qi.z.zh...@intel.com>; jer...@marvell.com; Maxime Coquelin
<maxime.coque...@redhat.com>
Subject: Re: [PATCH 2/5] ethdev: add capability to keep shared objects on
restart
External email: Use caution opening links or attachments
On 10/16/2021 9:32 PM, Dmitry Kozlyuk wrote:
-----Original Message-----
From: Ferruh Yigit <ferruh.yi...@intel.com>
Sent: 15 октября 2021 г. 19:27
To: Dmitry Kozlyuk <dkozl...@nvidia.com>; dev@dpdk.org; Andrew
Rybchenko
<andrew.rybche...@oktetlabs.ru>; Ori Kam <or...@nvidia.com>; Raslan
Darawsheh <rasl...@nvidia.com>
Cc: NBU-Contact-Thomas Monjalon <tho...@monjalon.net>; Qi Zhang
<qi.z.zh...@intel.com>; jer...@marvell.com; Maxime Coquelin
<maxime.coque...@redhat.com>
Subject: Re: [PATCH 2/5] ethdev: add capability to keep shared objects
on
restart
External email: Use caution opening links or attachments
On 10/15/2021 1:35 PM, Dmitry Kozlyuk wrote:
-----Original Message-----
From: Ferruh Yigit <ferruh.yi...@intel.com>
[...]
Introducing UNKNOWN state seems wrong to me.
What should an application do when it is reported?
Now there's just no way to learn how the PMD behaves,
but if it provides a response, it can't be "I don't know what I do".
I agree 'unknown' state is not ideal, but my intentions is prevent
drivers that not implemented this new feature report wrong
capability.
Without capability, application already doesn't know how underlying
PMD behaves, so this is by default 'unknown' state.
I suggest keeping that state until driver explicitly updates its
state
to the correct value.
My concern is that when all the drivers are changed to report a proper
capability, UNKNOWN remains in the API meaning "there's a bug in
DPDK".
When all drivers are changed, of course we can remove the 'unknown'
flag.
Instead of UNKNOWN response we can declare that rte_flow_flush()
must be called unless the application wants to keep the rules
and has made sure it's possible, or the behavior is undefined.
(Can be viewed as "UNKNOWN by default", but is simpler.)
This way neither UNKNOWN state is needed,
nor the bit saying the flow rules are flushed.
Here is why, let's consider KEEP and FLUSH combinations:
(1) FLUSH=0, KEEP=0 is equivalent to UNKNOWN, i.e. the application
must explicitly flush the rules itself
in order to get deterministic behavior.
(2) FLUSH=1, KEEP=0 means PMD flushes all rules on the device stop.
(3) FLUSH=0, KEEP=1 means PMD can keep at least some rules,
exact support must be checked with
rte_flow_create()
when the device is stopped.
(4) FLUSH=1, KEEP=1 is forbidden.
What is 'FLUSH' here? Are you proposing a new capability?
If the application doesn't need the PMD to keep flow rules,
it can as well flush them always before the device stop
regardless of whether the driver does it automatically or not.
It's even simpler and probably as efficient. Testpmd does this.
If the application wants to take advantage of rule-keeping ability,
it just tests the KEEP bit. If it is unset that's the previous case,
application should call rte_flow_flush() before the device stop to be
sure.
Otherwise, the application can test capability to keep flow rule kinds
it is interested in (see my reply to Andrew).
Overall this is an optimization, application can workaround without
this
capability.
If driver doesn't set KEEP capability, it is not clear what does it
mean, driver doesn't keep rules or driver is not updated yet.
I suggest to update comment to clarify the meaning of the missing KEEP
flag.
And unless we have two explicit status flags application can never be
sure that driver doesn't keep rules after stop. I am don't know if
application wants to know this.
Other concern is how PMD maintainers will know that there is something
to update here, I am sure many driver maintainers won't even be aware
of
this, your patch doesn't even cc them. Your approach feels like you are
thinking only single PMD and ignore rest.
My intention was to have a way to follow drivers that is not updated,
by marking them with UNKNOWN flag. But this also doesn't work with new
drivers, they may forget setting capability.
What about following:
1) Clarify KEEP flag meaning:
having KEEP: flow rules are kept after stop
missing KEEP: unknown behavior
2) Mark all PMDs with useless flag:
dev_capa &= ~KEEP
Maintainer can remove or update this later, and we can easily track it.
Item 1) is almost what I did in v2. The difference (or clarification) is
that
if the bit is set, it doesn't mean that all rules are kept.
It allows the PMD to not support keeping some kinds of rules.
Please see the doc update about how the kind is defined
and how the application can test what is unsupported.
This complication is needed so that if a PMD cannot keep some exotic
kind of rules,
it is not forced to remove the capability completely,
blocking optimizations even if the application doesn't use problematic
rule kinds.
It makes the capability future-proof.
The second flag (FLUSH) would not be of much help.
Consider it is not set, but the PMD can keep some kinds of rules.
The application still needs to test all the kinds it needs.
But it needs to do the same if the KEEP bit is set.
Only if it is set the application can skip the tests and
rte_flow_flush(),
but these optimizations are small compared to keeping the rules itself.
Item 2) needs not to be done, because the absence of the bit is *the*
useless value:
it means the unspecified same behavior as it is presently.
It is worth noting that currently any application that relies on the PMD
to keep or flush the rules is non-portable, because PMD is allowed to do
anything.
To get a reliable behavior application must explicitly clear the rules.
Regarding you concern about maintainers forgetting to update PMDs,
I think there are better task-tracking tools then constants in the code
(the authors of golang's context.TODO may disagree :)
Hi Dmitry,
This is a valid concern, and adding code to the drivers proved that it
works.
There are multiple authors updating the ethdev layer and expecting PMD
maintainers will do required changes. For your case you are updating the
PMD
you concern but how other PMD maintainers will even be aware that there is
something to change in their PMD?
By your change you are putting some responsibility to other maintainers,
without even cc'ing them. And it is for sure they are not reading all
emails
in the mail list, they can't.
Task-tracking is an option, it the past I tried to upstream some todo doc
for PMDs. But I can see the additional maintenance cost to trace features
from a central point, comparing the distributing it to PMDS (adding code
to PMDs).
I think best method is whoever doing the ethdev layer do the relevant
change
in the PMDs, but has the obvious problem that not able to know enough
about
the PMDs to update them.
We have used the following option, and it worked in the past:
- When an ethdev feature require change in PMDs, ehtdev supports both new
and old method
- PMDs set a flag by default to request old method, so there is no update
in the PMD default behavior
- When PMD does the required changes, it removes the flag
- This lets me (or other maintainer), to trace the update status and ping
relevant maintainers
- When all PMDs updated, ethdev support for old method removed
- This method allows PMD maintainers do the change on their own time
Hi Ferruh,
Thanks for sharing the experience.
You suggest updating PMDs with an explicit reset of this bit,
despite that it will be zero anyway, to attract maintainers' attention.
ack, but please with a brief comment to clarify intention.
From user's perspective it will be all the same: KEEP bit reset,
not a special value saying the PMD is not updated
that we would need to deprecate and remove later.
ack, only it needs to be clear for application that PMD not advertising
KEEP flag means behavior is undefined, it does NOT mean PMD flush rules.
Which you already said updated like this in v2, but I am just stressing it.
If this understanding is correct, then for sure I can add a patch
updating the relevant PMDs.
