On 4/26/21 11:46 PM, Honnappa Nagarahalli wrote:
> <snip>
> 
>>>>
>>>>>>>>>>>> On Thu, Mar 11, 2021 at 12:01 AM Honnappa
>>>>>>>>>>>> Nagarahalli <honnappa.nagaraha...@arm.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>         Performance of L3fwd example application
>>>>>>>>>>>>> is one of the key
>>>>>>>>>>>> benchmarks in DPDK. However, the application does
>>>>>>>>>>>> not have many debugging statistics to understand the
>>>>>>>>>>>> performance issues. We have added L3fwd as another
>>>>>>>>>>>> mode/stream to testpmd which provides
>>>>>>>>>> enough
>>>>>>>>>>>> statistics at various levels. This has allowed us to
>>>>>>>>>>>> debug the performance issues effectively.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is more work to be done to get it to
>>>>>>>>>>>>> upstreamable state. I am
>>>>>>>>>>>> wondering if such a patch is helpful for others and
>>>>>>>>>>>> if the community would be interested in taking a
>>>>>>>>>>>> look. Please let me know
>>>>>>>>> what you think.
>>>>>>>>>>>>
>>>>>>>>>>>> We are using app/proc-info/ to attach and analyze
>>>>>>>>>>>> the
>>>>> performance.
>>>>>>>>>>>> That helps to analyze the unmodified application. I
>>>>>>>>>>>> think, if something is missing in proc-info app, in
>>>>>>>>>>>> my opinion it is better to enhance proc-info so that
>>>>>>>>>>>> it can help other third-party
>>>>>>> applications.
>>>>>>>>>>>>
>>>>>>>>>>>> Just my 2c.
>>>>>>>>>>> Thanks Jerin. We will explore that.
>>>>>>>>>>
>>>>>>>>>> I agree it is dangerous to rely too much on testpmd for
>> everything.
>>>>>>>>>> Please tell us what in testpmd could be useful out of it.
>>>>>>>>>>
>>>>>>>>> Things that are very helpful in testpmd are: 1) HW
>>>>>>>>> statistics from the NIC 2) Forwarding stats 3) Burst stats
>>>>>>>>> (indication of headroom
>>>>>>>>> availability) 4) Easy to set parameters like RX and TX
>>>>>>>>> queue depths (among others) without having to recompile.
>>>>>>>>
>>>>>>>> [Kathleen Capella]
>>>>>>>> Thank you for the suggestion of app/proc-info. I've tried it
>>>>>>>> out with l3fwd and see that it does have the HW stats from
>>>>>>>> the NIC and the forwarding
>>>>>>> stats.
>>>>>>>> However, it does not have the burst stats testpmd offers,
>>>>>>>> nor the
>>>>>>>
>>>>>>> One option to see such  level of debugging would be to have
>>>>>>> - Create a memzone in the primary process
>>>>>>> - Application under test can update the stats in memzone based
>>>>>>> on the code flow
>>>>>>> - proc-info can read the counters updated by application under
>>>>>>> test using the memzone object got through
>> rte_memzone_lookup()
>>>>>> Agreed. Currently, using app/proc-info does not provide this
>>>>>> ability. We
>>>>> cannot add this capability to app/proc-info as these stats would
>>>>> be specific to L3fwd application.
>>>>>
>>>>> I meant creating generic counter-read/write infra via memzone to
>>>>> not make it as l3fwd specific.
>>>> Currently, app/proc-info is able to print the stats as they are 
>>>> standardized
>> via the API. But for statistics that are generated in the application, they 
>> are
>> very specific to that application. For ex: burst stats in testpmd are very
>> specific to it and another application might implement the same in a very
>> different manner.
>>>>
>>>> In needs to be something like the app/proc-info just needs to be a dumb
>> displaying utility and the application has to do all the heavy lifting of 
>> copying
>> the exact display strings to the memory.
>>>
>>> Yes.
>>>
>>>>
>>>>>>>
>>>>>>> Another approach will be using rte_trace()[1] for
>>>>>>> debugging/tracing by adding tracepoints in l3fwd for such events.
>>>>>>> It has a timestamp and the trace format is opensource trace
>>>>>>> format(CTF(Common trace format)), so that we can use post
>>>>>>> posting tools to analyze.
>>>>>>> [1]
>>>>>>> https://doc.dpdk.org/guides/prog_guide/trace_lib.html
>>>>>> This is good for analyzing an incident. I think it is an
>>>>>> overhead for
>>>>> development purposes.
>>>>>
>>>>> Consider if one wants to add burst stats, one can add stats
>>>>> increment under RTE_TRACE_POINT_FP, it will be emitted whenever
>>>>> code flow through that path. Set of events of can be viewed in
>>>>> trace viewer[1]. Would that be enough?
>>>>> Adding traces to l3fwd can be upstreamed as it is useful for
>>>>> others for debugging.
>>>>>
>>>>> [1]
>>>>> https://github.com/jerinjacobk/share/blob/master/dpdk_trace.JPG
>>>> This needs post processing of the trace info to derive the information, is 
>>>> it
>> correct? For ex: for burst stats, there will be several traces generated
>> collecting the number of packets returned by rte_eth_rx_burst which needs
>> to be post processed.
>>>
>>> Or You can have an additional variable to acculate it.
>>>
>>>> Also, adding traces is equivalent to adding statistics in L3fwd.
>>>
>>> Yes.
>>>
>>> If the sole purpose only stats then it is better to add status in
>>> l3fwd without performance impact. I thought some thing else.
>>>
>>>>
>>>>>>>
>>>>>>>> ability to easily change parameters without having to
>>>>>>>> recompile, which helps reduce debugging time significantly.
>>>> We will not be able to fix this above issue.
>>>
>>> It depends on what you want to debug. Trace can be disabled at runtime.
>>
>>
>> DPDK has existing API's for application metrics but they are rarely used.
>>
>> Why not implement rte_metrics in l3fwd and proc-info?
> This discussion has ended up as a stats discussion. But, we also need to be 
> able to change the configurable parameters easily.
> If we implement the stats and ability to change the configurable parameters, 
> then it is essentially bringing in some of the capabilities from testpmd to 
> the sample application. I think that will result in lot more code in the 
> sample application and will make it complicated.
> 
> Instead our proposal is to take L3fwd to testpmd and use all the infra code 
> that testpmd provides. We see that this approach results in less amount of 
> code added to DPDK overall.
> 

+1

Reply via email to