19/11/2019 11:59, Andrew Rybchenko: > On 11/19/19 12:50 PM, Thomas Monjalon wrote: > > 19/11/2019 10:24, Andrew Rybchenko: > >> On 11/8/19 4:30 PM, Thomas Monjalon wrote: > >>> 08/11/2019 14:27, Andrew Rybchenko: > >>>> On 11/8/19 4:17 PM, Thomas Monjalon wrote: > >>>>> 08/11/2019 13:00, Andrew Rybchenko: > >>>>>> On 11/8/19 2:03 PM, Thomas Monjalon wrote: > >>>>>>> 08/11/2019 11:42, Andrew Rybchenko: > >>>>>>>> On 11/8/19 1:28 PM, Thomas Monjalon wrote: > >>>>>>>>> 08/11/2019 09:35, Andrew Rybchenko: > >>>>>>>>>> The problem: > >>>>>>>>>> ~~~~~~~~~~~~ > >>>>>>>>>> PMD wants to know before port start if application wants to > >>>>>>>>>> to use flow MARK/FLAG in the future. It is required because: > >>>>>>>>>> > >>>>>>>>>> 1. HW may be configured in a different way to reserve resources > >>>>>>>>>> for MARK/FLAG delivery > >>>>>>>>>> > >>>>>>>>>> 2. Datapath implementation choice may depend on it (e.g. vPMD > >>>>>>>>>> is faster, but does not support MARK) > >>>>>>>>> Thank you for the clear problem statement. > >>>>>>>>> I agree with it. This is a real design issue. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Discussed solutions: > >>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~ > >>>>>>>> May be it is not 100% clear since below are alternatives. > >>>>>>>> > >>>>>>>>>> A. Explicit Rx offload suggested by the patch. > >>>>>>>>>> > >>>>>>>>>> B. Implicit by validation of a flow rule with MARK/FLAG actions > >>>>>>>>>> used. > >>>>>>>>>> > >>>>>>>>>> C. Use dynamic field/flag (i.e. application registers dynamic field > >>>>>>>>>> and/or flag and PMD uses lookup to solve the problem) plus part > >>>>>>>>>> of (B) to discover if a feature is supported. > >>>>>>>>> The dynamic field should be registered via a new API function > >>>>>>>>> named '<feature>_init'. > >>>>>>>>> It means the application must explicit request the feature. > >>>>>>>>> I agree this is the way to go. > >>>>>>>> If I understand your statement correctly, but (C) is not ideal since > >>>>>>>> it > >>>>>>>> looks global. If registered dynamic field of mbuf and is flag that > >>>>>>>> the feature should be enabled, it is a flag to all ports/devices. > >>>>>>>> > >>>>>>>>>> All solutions require changes in applications which use these > >>>>>>>>>> features. There is a deprecation notice in place which advertises > >>>>>>>>>> DEV_RX_OFFLOAD_FLOW_MARK addition, but may be it is OK to > >>>>>>>>>> substitute > >>>>>>>>>> it with solution (B) or (C). Solution (C) requires changes since > >>>>>>>>>> it should be combined with (B) in order to understand if > >>>>>>>>>> the feature is supported. > >>>>>>>>> I don't understand. > >>>>>>>>> Application request and PMD support are two different things. > >>>>>>>>> PMD support must be via rte_flow validation on a case by case > >>>>>>>>> anyway. > >>>>>>>> I mean that application wants to understand if the feature is > >>>>>>>> supported. Then, it wants to enable it. In the case of (B), > >>>>>>>> if I understand the solution correctly, there is no explicit > >>>>>>>> way to enable, PMD just detects it because of discovery is done > >>>>>>>> (that's what I mean by "implicit" and it is a drawback from my > >>>>>>>> point of view, but still could be considered). (C) solves the > >>>>>>>> problem of (B). > >>>>>>>> > >>>>>>>>>> Advantages and drawbacks of solutions: > >>>>>>>>>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > >>>>>>>>>> 1. The main drawback of (A) is a "duplication" since we already > >>>>>>>>>> have a way to request flow MARK using rte_flow API. > >>>>>>>>>> I don't fully agree that it is a duplication, but I agree > >>>>>>>>>> that it sounds like duplication and complicates a bit flow > >>>>>>>>>> MARK usage by applications. (B) complicates it as well. > >>>>>>>>>> > >>>>>>>>>> 2. One more drawback of the solution (A) is the necessity of > >>>>>>>>>> similar solution for META and it eats one more offload bit. > >>>>>>>>>> Yes, that's true and I think it is not a problem. > >>>>>>>>>> It would make it easier for applications to find out if > >>>>>>>>>> either MARK or META is supported. > >>>>>>>>>> > >>>>>>>>>> 3. The main advantage of the solution (A) is simplicity. > >>>>>>>>>> It is simple for application to understand if it supported. > >>>>>>>>>> It is simple in PMD to understand that it is required. > >>>>>>>>>> It is simple to disable it - just reconfigure. > >>>>>>>>>> Also it is easier to document it - just mention that > >>>>>>>>>> the offload should be supported and enabled. > >>>>>>>>>> > >>>>>>>>>> 4. The main advantage of the solution (B) is no "duplication". > >>>>>>>>>> I agree that it is valid argument. Solving the problem > >>>>>>>>>> without extra entities is always nice, but unfortunately > >>>>>>>>>> it is too complex in this case. > >>>>>>>>>> > >>>>>>>>>> 5. The main drawback of the solution (B) is the complexity. > >>>>>>>>>> It is necessary to choose a flow rule which should be used > >>>>>>>>>> as a criteria. It could be hardware dependent. > >>>>>>>>>> Complex logic is require in PMD if it wants to address the > >>>>>>>>>> problem and control MARK delivery based on validated flow > >>>>>>>>>> rules. It adds dependency between start/stop processing and > >>>>>>>>>> flow rules validation code. > >>>>>>>>>> It is pretty complicated to document it. > >>>>>>>>>> > >>>>>>>>>> 6. Useless enabling of the offload in the case of solution (A) > >>>>>>>>>> if really used flow rules do not support MARK looks like > >>>>>>>>>> drawback as well, but easily mitigated by a combination > >>>>>>>>>> with solution (B) and only required if the application wants > >>>>>>>>>> to dive in the level of optimization and complexity and > >>>>>>>>>> makes sense if application knows required flow rules in > >>>>>>>>>> advance. So, it is not a problem in this case. > >>>>>>>>>> > >>>>>>>>>> 7. Solution (C) has drawbacks of the solution (B) for > >>>>>>>>>> applications to understand if these features are supported, > >>>>>>>>>> but no drawbacks in PMD, since explicit criteria is used to > >>>>>>>>>> enable/disable (dynamic field/flag lookup). > >>>>>>>>>> > >>>>>>>>>> 8. Solution (C) is nice since it avoids "duplication". > >>>>>>>>>> > >>>>>>>>>> 9. The main drawback of the solution (C) is asymmetry. > >>>>>>>>>> As it was discussed in the case of RX_TIMESTAMP > >>>>>>>>>> (if I remember it correctly): > >>>>>>>>>> - PMD advertises RX_TIMESTAMP offload capability > >>>>>>>>>> - application enables the offload > >>>>>>>>>> - PMD registers dynamic field for timestamp > >>>>>>>>>> Solution (C): > >>>>>>>>>> - PMD advertises nothing > >>>>>>>>>> - application uses solution (B) to understand if > >>>>>>>>>> these features are supported > >>>>>>>>>> - application registers dynamic field/flag > >>>>>>>>>> - PMD does lookup and solve the problem > >>>>>>>>>> The asymmetry could be partially mitigated if RX_TIMESTAMP > >>>>>>>>>> solution is changed to require an application to register > >>>>>>>>>> dynamic fields and PMD to do lookup if the offload is > >>>>>>>>>> enabled. So, the only difference will be in no offload > >>>>>>>>>> in the case of flow MARK/FLAG and usage of complex logic > >>>>>>>>>> to understand if it is supported or no. > >>>>>>>>>> May be it would be really good since it will allow to > >>>>>>>>>> have dynamic fields registered before mempool population. > >>>>>>>>>> > >>>>>>>>>> 10. Common drawback of solutions (B) and (C) is no granularity. > >>>>>>>>>> Solution (A) may be per queue while (B) and (C) cannot be > >>>>>>>>>> per queue. Moreover (C) looks global - for all devices. > >>>>>>>>>> It could be really painful. > >>>>>>>>>> > >>>>>>>>>> (C) is nice, but I still vote for simplicity and > >>>>>>>>>> granularity of (A). > >>>>>>>>> I vote for clear separation of application needs and PMD support, > >>>>>>>>> by using the method C (dynamic fields). > >>>>>>>>> I agree timestamp must use the same path. > >>>>>>>>> I agree it's complicate because we don't know in advance whether > >>>>>>>>> a flow rule will be accepted, but that's the reality, config is > >>>>>>>>> complex. > >>>>>>>> Do you think that global nature of the (C) is acceptable? > >>>>>>> That's a good question. > >>>>>>> Maybe the feature request should be per port. > >>>>>>> In this case, we are back to solution A with a flag per port? > >>>>>> Offloads are natively per-queue as well, so (A) keeps the choice > >>>>>> between per-port vs per-queue to PMDs as usual. > >>>>>> > >>>>>>> Note that A and C will not guarantee that the offload will be > >>>>>>> possible. > >>>>>> Yes, definitely. > >>>>>> > >>>>>>> We need B (flow rule validation) anyway. > >>>>>> Strictly speaking (B) (checking flow rules before device > >>>>>> startup) is required if an application can predict flow > >>>>>> rules and wants to ensure that MARK offload will be usable. > >>>>>> Otherwise, it may be skipped. > >>>>> No no, I mean flow rule validation MUST be used anyway > >>>>> during the runtime before applying a rule. > >>>>> I agree it is hard to predict. I speak only about real rules. > >>>> OK, I see. Of course, flow rule validation is required at runtime. > >>>> I was rather concentrated on the stated problem solutions. > >>>> > >>>>>>> It seems A, B, C are not alternatives but all required > >>>>>>> as pieces of a puzzle... > >>>>>> Unfortunately true in the most complex case. > >>>>>> Right now it will be A with B if required as explained above. > >>>>>> C will come a bit later when the field migrates to dynamic. > >>>>>> > >>>>>> May be it is even better if application registers dynamic > >>>>>> fields before an attempt to enable offload to be sure that > >>>>>> it will not fail because of impossibility to register > >>>>>> dynamic field (lack of space). I'm not sure, but it is not > >>>>>> not that important. > >>>>> Yes of course, lack of mbuf space is another reason for > >>>>> disabling the feature. > >>>>> > >>>>>> If we finally go way A, should we add offloads for META back? > >>>>>> I guess separate Rx and Tx are required. > >>>>> I would prefer to add it as dynamic flags. > >>>>> Why rushing on a very temporary solution while it is not a new issue? > >>>> Basically it means that we go just (B)+(C) in the case of META. > >>>> I have no strong opinion but thought that it could be better to > >>>> align the solution. Of course, we can wait with it. As I understand > >>>> META is an experimental feature. > >>> Yes it is experimental and I think it is too late to align now. > >>> > >>> Anyway, we will probably to discuss again these offloads TAG/MARK/META, > >>> as requested by several people. > >>> > >> The series implements (A) to help to solve the problem described above. > >> What is the fate of the series in v19.11 in accordance with the > >> discussion? > > I am against adding anything related to a feature union'ed in mbuf. > > The feature must move to dynamic field first. > > > > In addition, such capability is very weak. > > I am not sure it is a good idea to have some weak capabilities, > > meaning a feature could be available but not in all cases. > > I think we should discuss more generally how we want to handle > > the rte_flow capabilities conveniently and reliably. > > It is really unexpected outcome from the above discussion.
I'm sorry, I thought I was clear in my request to switch to dynamic first. > It is just possibility to deliver and handle marks on datapath and > request to have it. It says almost nothing about rte_flow rules > supported etc. I'll be happy to take part in the discussion. > > > So regarding 19.11, as this feature is not new, it can wait 20.02. > > OK, it is not critical for me, so I don't mind, however, I've seen > patches which try to use it [1] except net/octeontx2 in the second > patch of the series. > > [1] https://patches.dpdk.org/patch/62415/
