On Thu, 2 Feb 2023, Ferruh Yigit wrote:
On 2/2/2023 8:50 AM, Ivan Malov wrote:
On Thu, 2 Feb 2023, Ferruh Yigit wrote:
On 2/1/2023 3:22 PM, Jerin Jacob wrote:
On Wed, Feb 1, 2023 at 8:20 PM Ferruh Yigit <ferruh.yi...@amd.com>
wrote:
On 2/1/2023 1:48 PM, Jerin Jacob wrote:
On Wed, Feb 1, 2023 at 5:06 PM Ferruh Yigit <ferruh.yi...@amd.com>
wrote:
On 2/1/2023 11:15 AM, Jerin Jacob wrote:
On Wed, Feb 1, 2023 at 4:35 PM Thomas Monjalon
<tho...@monjalon.net> wrote:
01/02/2023 11:58, Andrew Rybchenko:
On 2/1/23 13:48, Jerin Jacob wrote:
On Wed, Feb 1, 2023 at 2:59 PM Andrew Rybchenko
<andrew.rybche...@oktetlabs.ru> wrote:
Frankly speaking I don't understand why default value is so
important if we have a way to change it. Reasons should be
really strong to change existing defaults.
The only reason is, typically testpmd will be used performance
benchmarking as an industry standard. It is difficult to
tell/educate
the QA or customers
that, "BTW if you need to get better performance add more flag to
testpmd command line".
I disagree.
When you do performance benchmark, you tune settings accordingly.
IMO, We tune the system resources like queue depth not the disabling
features for raw performance.
queue depth etc people know to tune so it is obvious. What is not
obvious is, testpmd only
negotiated some features by default.I am not using that feature,
hence
I need to explicitly
disable it.
When 'rte_eth_rx_metadata_negotiate()' API is NOT used at all, and I
believe that is the case for almost all applications since API is a
relatively new one, PMD default behavior should be to enable Rx
metadata
flow rules, in case user requests them later.
So, enabling all in application is same with not calling the API
at all.
In this perspective, disabling Rx metadata is additional
optimization/tuning that application can do if it is sure that Rx
metadata flow rules won't be used at all.
And API is more meaningful when it is used to disable Rx metadata.
I think it is reasonable to enable all Rx metadata by default in
testpmd
with a capability to disable it when wanted.
OR
May be we don't call 'rte_eth_rx_metadata_negotiate()' API by
default in
testpmd, it is only called when it is requested explicitly from user,
enable or disable.
Second option looks good to me.
When
1) user request for action which is needed negotiate(),
AND
2) rte_eth_rx_metadata_negotiate() != ENOSUP
then, testpmd print a warning that need to enable
rte_eth_rx_metadata_negotiate().
We are not suggesting same thing.
What you described above assumes PMD disabled Rx metadata flow rule
support by default, and it needs to be enabled explicitly by
'rte_eth_rx_metadata_negotiate()' API. This API becomes mandatory for
functionality.
As far as I understand PMD wants to disable this flow rule by default
because of performance concerns. But this creates inconsistency between
PMDs, because rest of them will enable this flow rule by default (if it
is supported) and be ready to use it when proper flow rule created.
With this approach some PMDs will need
'rte_eth_rx_metadata_negotiate()'
to enable Rx metadata flow rules, some won't. This can be confusing for
applications that *some* PMDs require double enabling with specific API
call.
Instead what I was trying to suggest is reverse,
all PMDs enable the Rx metadata flow rule by default, and don't require
double enabling.
But if application knows that it won't use Rx metadata flow rule, it
can
disable it to optimize the performance.
This makes 'rte_eth_rx_metadata_negotiate()' functionally optional, and
for testpmd context it can be called via a command on demand by user
for
optimization purpose.
This won't solve concern I have outlined earlier[1].
Yes, it won't.
I think, The part of the problem there is no enough adaption of
rte_eth_rx_metadata_negotiate(),
The view is total different from PMD maintainer PoV vs testpmd
application PoV.
Agree,
and I assume it is different for user application too, which may
prioritize consistency and portability.
Overall, I am not fan of the 'rte_eth_rx_metadata_negotiate()' API, I
think it is confusing.
Forgive me, in which way is it confusing?
All other flow rules enabled by creating flow rule, but this one
Hold on.. Did you just say "flow rules"? But this API is *not* about
flow rules. I suggest you please re-read description of the API.
It reads: "Negotiate the NIC's ability to deliver specific
kinds of metadata to the PMD". Nothing about flows there.
Why is it decoupled from flow library this way? Because there is
a drastic difference between generating and plumbing metadata
INSIDE the NIC's flow engine, on the one hand, and delivering
these data from the NIC to the host driver, on the other.
Let me explain. Say, one creates the following flow rules:
(a) flow create 0 transfer group 0 pattern [A] / end \
actions mark id XXX jump group 1 / end
(b) flow create 0 transfer group 0 pattern [B] / end \
actions mark id YYY jump group 1 / end
(c) flow create 0 transfer group 1 pattern mark id is XXX / end \
actions represented_port ethdev_id 1 / end
(d) flow create 0 transfer group 1 pattern mark id is YYY / end \
actions drop / end
In this example, metadata of type "MARK" is used to partition
flow group (table) 1 into multiple lookup sections. So the
mark value is GENERATED by the flow engine and then it is
CONSUMED by this very flow engine. The application may
NOT necessarily want to receive the mark with mbufs...
And it is only when the application wants these metadata
DELIVERED to it that it has to call the negotiate API.
The short of it, nothing prevents the driver from accepting
flow create requests that leverage some meta items/actions.
Drivers do not need the negotiate API to *configure flow*.
They only need this API in order to let the application
choose whether metadata will be DELIVERED (!) or not.
requires an API to enable it, so I believe it is inconsistent in that way.
Please see above. Everything is consistent as *flow library*
and *negotiate metadata delivery* API are totally decoupled.
From application perspective, assume that it doesn't know NIC details,
should it call this API or not? Without API call should application
assume Rx metadata flow rules are enabled or disabled?
Frankly, applications like testpmd need never call this API.
Simply because seeing, for example, MARK values in mbufs is
useless to it. This API is needed by other applications.
For example, OvS has partial MARK+RSS offload. It adds
flows that distribute packets across multiple queues
and set some MARK values for them. OvS is interested
in getting this values with mbufs since they affect
further lookups in software... So, OvS, knowing it
wants these metadata DELIVERED (!), should invoke
this metadata negotiate API to ensure that.
As I understand intention is to get hint from application if it will
require Rx metadata flow rules so that PMD can optimize better, but if
No, nothing about flow rules. Just delivery of metadata with mbufs.
PMD doesn't enable Rx metadata flow rules when this API call is missing
Again, PMD shall not make decisions on whether to enable or disable
support for some FLOW primitives based on interactions via this API.
This API exists solely to let PMD configure delivery of metadata,
i.e. not the way it is generated in the first instance.
than it becomes a mandatory API to configure the device. But I think it
should be optional for optimization.
Also if application not sure to use this flow rule or not, it will by
default enable all in any case, this will reduce the benefit. As done in
testpmd.
Please see above. PMD does not need this API, I take it.
API works in both ways, it request to enable some Rx metadata flow rule,
but based on what PMD returns application can know what device supports,
this also inconsistent with how other flow rules work, we don't have API
to get capability but detect them via flow create/validate.
Can there be a case API returns a flow rule negotiated, but it fails
when tried to create the flow rule, isn't this confusing for application
I think if we continue this approach there can be multiple enable and
capability learning APIs for various flow rules or flow rule groups, and
this can make flow API much more harder to use for applications.
See my explanations above. This API is not about flows. Period.
Just to avoid back and forth. We will call off this patch and remove
rte_eth_rx_metadata_negotiate()
PMD callback from cnxk driver. Keep it as old behavior, so we don't
need to care
about rte_eth_rx_metadata_negotiate().
When you remove 'rx_metadata_negotiate' callback, what will be the PMD
behavior? I assume PMD will do the required preparations as if all Rx
metadata is enabled.
And what is the performance impact, is removing callback improve the
performance?
[1]
The only reason is, typically testpmd will be used performance
benchmarking as an industry standard. It is difficult to tell/educate
the QA or customers
that, "BTW if you need to get better performance add more flag to
testpmd command line".
To make that worst, only some PMD needs to give the additional
parameter to get better number.
And also, testpmd usage will be treated as application modeling.
To make that worst, only some PMD needs to give the additional
parameter to get better number.
And also, testpmd usage will be treated as application modeling.
Since this feature only used on sfc and cnxk driver, What is the
situation with sfc driver?
Keeping it as negotiated and not use the feature, will impact
the per
core performance of sfc or
is it just PCI bandwidth thing which really dont show any
difference in testpmd?
Yes, sfc could run faster if no Rx metadata are negotiated. So,
it is better to negotiate nothing by default. But it is always
painful to change defaults. You need to explain that now you
need to negotiate Rx metadata to use mark, flag and tunnel
offloads.
Yes, it will be required on sfc and cnxk only.
As an sfc maintainer I don't mind to change testpmd defaults.
If we change testpmd defaults to "do nothing",
then we should disable MBUF_FAST_FREE as well.
if you see MBUF_FAST_FREE, it does nothing. Actually,
!MBUF_FAST_FREE is doing more work.