Hi Ori, Andrew,
On 05/10/2021 12:19, Andrew Rybchenko wrote:
Hi Ori,
On 10/5/21 9:20 AM, Ori Kam wrote:
Hi Ivan,
-----Original Message-----
From: Ivan Malov <ivan.ma...@oktetlabs.ru>
Cc: dev@dpdk.org
Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow API
Hi Ori,
On 04/10/2021 14:37, Ori Kam wrote:
Hi Ivan,
-----Original Message-----
From: Ivan Malov <ivan.ma...@oktetlabs.ru>
Sent: Monday, October 4, 2021 2:06 PM
Cc: dev@dpdk.org
Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow
API
Hi Ori,
On 04/10/2021 08:45, Ori Kam wrote:
Hi Ivan,
-----Original Message-----
From: Ivan Malov <ivan.ma...@oktetlabs.ru>
Sent: Sunday, October 3, 2021 9:11 PM
Subject: Re: [PATCH v1 02/12] ethdev: add eswitch port item to flow
API
On 03/10/2021 15:40, Ori Kam wrote:
Hi Andrew and Ivan,
-----Original Message-----
From: Andrew Rybchenko <andrew.rybche...@oktetlabs.ru>
Sent: Friday, October 1, 2021 4:47 PM
Subject: [PATCH v1 02/12] ethdev: add eswitch port item to flow
API
From: Ivan Malov <ivan.ma...@oktetlabs.ru>
For use with "transfer" flows. Supposed to match traffic entering
the e-switch from the external world (network, guests) via the
port which is logically connected with the given ethdev.
Must not be combined with attributes "ingress" / "egress".
This item is meant to use the same structure as ethdev item.
In case the app is not working with representors, meaning each
switch port is mapped to ethdev.
both items (ethdev and eswitch port ) have the same meaning?
No. Ethdev means ethdev, and e-switch port is the point where this
ethdev is plugged to. For example, "transfer + ESWITCH_PORT" for a
regular PF ethdev typically means the network port (maybe you can
recall the idea that a PF ethdev "represents" the network port it's
associated with).
I believe, that diagrams which these patches add to
"doc/guides/prog_guide/rte_flow.rst" may come in handy to
understand the meaning. Also, you can take a look at our larger
diagram from the Sep 14 gathering.
Lets look at the following system:
E-Switch has 3 ports - PF, VF1, VF2
The ports are distributed as follows:
DPDK application:
ethdev(0) pf,
ethdev(1) representor to VF1
ethdev(2) representor to VF2
ethdev(3) VF1
VM:
VF2
As we know all representors are realy connected to the PF(at least
in this example)
This example tries to say that the e-switch has 3 ports in total,
and, given your explanation, one may indeed agree that *in this
example* representors re-use e-switch port of ethdev=0 (with some
metadata to distinguish packets, etc.). But one can hardly assume
that *all* representors with any vendor's NIC are connected to the
e-switch the same way. It's vendor specific. Well, at least,
applications don't have this knowledge and don't need to.
So matching on ethdev(3) means matching on traffic sent from DPDK
port
3 right?
Item ETHDEV (ethdev_id=3) matches traffic sent by DPDK port 3. Looks
like we're on the same page here.
Good.
And matching on eswitch_port(3) means matching in traffic that goes
into VF1 which is the same traffic as ethdev(3) right?
I didn't catch the thought about "the same traffic". Direction is not the
same.
Item ESWITCH_PORT (ethdev_id=3) matches traffic sent by DPDK port 1.
This is the critical part for my understanding.
Matching on ethdev_id(3) means matching on traffic that is coming from
DPDK port3.
Right.
So from E-Switch view point it is traffic that goes into VF1?
No. Above you clearly say "coming from DPDK port3". That is, from the VF1.
*Not* going into it. Port 3 (ethdev_id=3) *is* VF1.
But taffic that goes from DPDK port 3 goes into VF1,
what am I missing?
Terms like "PF", "VF", "PHY_PORT" describe the nature of the e-switch
ports. In your example, DPDK port 3 is just based on VF1. The
application doesn't have such knowledge. To it, this is a regular ethdev.
In order to gain correct understanding, you should imagine an observer
standing inside the e-switch. When that observer sees packets entering
the e-switch FROM the VF1, we effectively talk about packets sent by the
ethdev sitting on top of that VF1, that is, packets coming FROM DPDK
port 3. And vice versa: if you say "traffic that goes into VF1", you
mean traffic that LEAVES the e-switch via VF1, that is, traffic, going
TO DPDK port 3.
"VF1" is just a name of a logical "window" between the e-switch and its
surroundings. If this VF is passed to a guest machine, then this VF is a
"window" between the e-switch and the guest machine. If you plug this VF
to the DPDK application (create an ethdev on top of this VF), then this
VF is a "window" between the e-switch and the PMD. That's it.
DPDK port 3 is a VF1 itself. So, is it loopback? I think no.
Let me repeat your example:
DPDK application:
ethdev(0) pf,
ethdev(1) representor to VF1
ethdev(2) representor to VF2
ethdev(3) VF1
VM:
VF2
Traffic that goes from DPDK port 3 (which is VF1 bound to DPDK)
goes to its representor by default, i.e. ethdev(1).
+1
While matching on E-Switch_port(3) means matching on traffic coming
from VF1?
No. It means matching on traffic coming from ethdev 1. From the VF1's
representor.
And by the same logic matching on ethdev_id(1) means matching on
taffic that was sent from DPDK port 1 and matching on E-Switch_port(1)
means matching on traffic coming from
VF1
In this case, you've got this right. But please see my above notes. By the
looks of it, you might have run into confusion over there.
That is the issue I'm not sure I understand, and I think that
if I don't underdand this, this miss underdanding will be shared by others.
Please see my diagram below.
In order to address the confusion and misunderstanding we
should find out the source of it. We always match on source
(inbound port) and the result is a destination (outgoing port).
So, if item is ETHDEV, the source is DPDK application via
corresponding ethdev port. If ETHDEV is an action, the
destination is the DPDK application via specified ethdev port.
Above is regardless of the ethdev port type (representor or
not). Always, no exceptions.
So in this case eswitch_port(3) is equal ot eswitch_port(1) right?
While ethdev(1) is not equal to ethdev(3)
No.
Item ETHDEV (ethdev_id=1) equals item ESWITCH_PORT (ethdev_id=3).
Item ETHDEV (ethdev_id=3) equals item ESWITCH_PORT (ethdev_id=1).
I think this was my first undestaning, lets see if I can explain it using my
words.
ETHDEV - means that the source port that we are matching on is the closest one
the dpdk application.
Yes, it sounds right.
Example like above ETHDEV(ethdev_id=1) means matching
on traffic coming from the PF with some metadata that marks this DPDK port.
Some vendors support allocation of separate logical e-switch ports for
representors. Other vendors can't support that, and, in this case, they
indeed have to devise "PF + metadata" solution. But DPDK framework (I
mean, vendor-agnostic code) can't assume any of these options as a
"standard". Neither can applications do that. It's truly vendor-specific.
No, no, no. It is the problem of too much knowledge and
diving too deep. Neither you nor API user should not
think about it. Representor ethdev(1) is a DPDK ethdev
port that's it. An application can send traffic via the
ethdev port and receive traffic from it. How the traffic
goes inside is vendor-specific implementation detail.
Application just know that if it sends traffic to VF1
representor ethdev(1), the traffic will be received
from VF1 by default. If it sends traffic from VF1,
the traffic will be received from VF1 representor
by default. It is the definition of the representor.
+1
ETHDEV(ethdev_id=3) means matching on traffic coming from VF1.
Yes.
ESWITCH_PORT meaning matching on the port that is connected to the DPDK port
(the other side of the wire). Example ESWITCH_PORT(ethdev_id=1) means matching
on traffic coming from VF1
Yes, since ethdev(1) is VF1 representor.
While matching on ESWITCH_PORT(ethdev_id=3) means matching on PF with some
metadata.
Again, yes and no. No, since you should not think about it this way. It
is just traffic sent via VF1 representor (since ethdev(3) is VF1 and its
other side of the logical wire is VF1
representor). No vendor-specific implementation details.
Everything assume that representors are on the PF and use some metadata.
No, it is a vendor-specific implementation details.
Flow API definitions should not mention it.
+1
Did I get it right?
Let's try to look at the overall idea one more time. Just to get us on
the same page.
In general, for any given ethdev you have:
ETHDEV (A)
|
| (PMD area)
|
CLOSEST ESWITCH_PORT (B)
|
| (e-switch area)
|
REMOTE ESWITCH_PORT (C)
|
X
The point "B" is located closest to the ethdev (A). Not to the
application in general, but to the given ethdev.
The point "C" is the most remote point from the ethdev's perspective.
Again, it's not the most remote from the application. It's the most
remote from this specific ethdev.
From the application perspective, the nature of (B) and (C) is unknown.
But the application knows for sure that these points just exist. To it,
these points are some *logical* e-switch ports.
Now.
When you say "item ETHDEV", you tell the PMD that you want to match
packets entering the *PMD area* FROM point (A). But you also add
"transfer" attribute. This attribute makes the PMD translate your
request to the e-switch viewpoint by "transferring" point "A" one level
below. What was "A" becomes "B". This way, an imaginary observer
standing inside the *e-switch area* intercepts packets which enter this
area FROM (B).
Summary: this way, you match packets going *DOWN* from (A).
When you say "item ESWITCH_PORT", you tell the PMD that you want to
match packets entering the *PMD area* FROM point (B). Once again,
"transfer" translates this to the e-switch viewpoint, that is, (B)
becomes (C). So, the e-switch knows that it should intercept packets
entering its area FROM point (C).
Summary: this way, you match packets going *UP* from (X).
Some use case examples:
- The ethdev (A) is a DPDK port based on a PF associated with a network
port. In this case, (B) == PF and (C) == (X) == network port.
- The ethdev (A) is a representor of VF1 which is passed to a VM. In
this case, (B) is just an e-switch logical port. It's nature is vendor
specific. It can be a PF, it can be not. It depends. (C) == VF, (X) == VM.
- The ethdev (A) is a representor of VF1 which is plugged to the same
DPDK application. In this case, (B) is, once again, some logical
e-switch port. (C) == VF. But (X) here is another ethdev. So, this way,
ethdev (A) is a representor's ethdev and ethdev (X) is a VF's ethdev.
In all cases the application doesn't care about the nature of the
e-switch ports (B) and (C). It just knows that they exist.
I think you get overall idea right, but explanations
are too complicated. It is simpler in fact.
And just to complete the picture, matching on ethdev(2) will result in
traffic coming from the dpdk port and matching on eswitch_port(2) will
match on traffic coming from VF2
Exactly.
But, Ori, let me draw your attention to the following issue. In order to
simplify understanding, I suggest that we refrain from saying "traffic that
GOES TO". Where it goes depends on default rules that are supposed to be
maintained by the PMD when ports get plugged / unplugged.
The flow items ETHDEV and ESWITH_PORT define the SOURCE of traffic.
That's it. They define where the traffic "goes FROM".
Say, the DPDK application sends a packet from ethdev 0. This packet enters
the e-switch. Match engine sits in the e-switch and intercepts the packet. It
doesn't care where the packet *would go* if it wasn't intercepted. It cares
about where the packet comes from. And it comes from ethdev 0. So, in the
focus, we have the SOURCE of the packet.
Agree with you we should only look at coming from,
but something in the patch made me thing otherwise (not sure what part)
I've reread the documentation and failed to find,
but it will be hard for me to do it, since I read
it too many times already.
Thanks,
Andrew.
--
Ivan M
