> On Mar 19, 2020, at 12:18 PM, Ciara Power <ciara.po...@intel.com> wrote:
>
> This patchset extensively reworks the telemetry library adding new
> functionality and simplifying much of the existing code, while
> maintaining backward compatibility.
>
> This work is based on the previously sent RFC for a "process info"
> library: https://patchwork.dpdk.org/project/dpdk/list/?series=7741
> However, rather than creating a new library, this patchset takes
> that work and merges it into the existing telemetry library, as
> mentioned above.
>
> The telemetry library as shipped in 19.11 is based upon the metrics
> library and outputs all statistics based on that as a source. However,
> this limits the telemetry output to only port-level statistics
> information, rather than allowing it to be used as a general scheme for
> telemetry information across all DPDK libraries.
>
> With this patchset applied, rather than the telemetry library being
> responsible for pulling ethdev stats and pushing them into the metrics
> library for retrieval later, each library e.g. ethdev, rawdev, and even
> the metrics library itself (for backwards compatiblity) now handle their
> own stats. Any library or app can register a callback function with
> telemetry, which will be called if requested by the client connected via
> the telemetry socket. The callback function in the library/app then
> formats its stats, or other data, into a JSON string, and returns it to
> telemetry to be sent to the client.
>
> To maintain backward compatibility, e.g. to allow the dpdk telemetry
> collectd plugin to continue to work, some of the existing telemetry
> code is kept, but is moved into the metrics library, and callbacks are
> registered with telemetry for the legacy commands that were supported
> previously.
>
> The new version of the library, apart from the legacy interface support
> for backward compatibility, does not have an external dependency on the
> Jansson library, allowing the library to be enabled by default.
>
> Note: In this version of the patchset, telemetry output is provided by
> the ethdev, rawdev and eal libraries, but this may be expanded further
> in later versions which are planned ahead of the merge deadline for
> 20.05
>
I am looking at the patch and noticed we are adding new lines to the JSON
output, they are not required. The spaces in the text could be removed as well
as it will reduce the amount of data being sent. The output can be put into a
JSON formatter to see the data better if needed.
Some required information I believe needs to be added to the patch.
- Command to get the Link status of a port Speed/Duplex/LinkState via a new
command ‘/ethdev/linkstate,X’ or I added it to the /ethdev/list command. I also
added the total devices and available devices via the DPDK APIs.
{
"/ethdev/list": {
"ports": [{
"port": 0,
"duplex": "Full",
"state": "UP",
"rate": 40000
}, {
"port": 1,
"duplex": "Full",
"state": "UP",
"rate": 100000
}, {
"port": 2,
"duplex": "Full",
"state": "UP",
"rate": 40000
}, {
"port": 3,
"duplex": "Full",
"state": "UP",
"rate": 100000
}],
"avail": 4,
"total": 4
}
}
The next one is the format of the /ethdev/stats are not very usable as most of
the fields are not supported in drivers.
{
"/ethdev/stats": {
"rx_good_packets": 0,
"tx_good_packets": 0,
"rx_good_bytes": 0,
"tx_good_bytes": 0,
"rx_missed_errors": 0,
"rx_errors": 0,
"tx_errors": 0,
"rx_mbuf_allocation_errors": 0,
"rx_q0packets": 0,
"rx_q0bytes": 0,
"rx_q0errors": 0,
"tx_q0packets": 0,
"tx_q0bytes": 0,
"rx_unicast_packets": 0,
"rx_multicast_packets": 0,
"rx_broadcast_packets": 0,
"rx_dropped_packets": 0,
"rx_unknown_protocol_packets": 5,
"tx_unicast_packets": 0,
"tx_multicast_packets": 0,
"tx_broadcast_packets": 0,
"tx_dropped_packets": 0,
"tx_link_down_dropped": 0,
"rx_crc_errors": 0,
"rx_illegal_byte_errors": 0,
"rx_error_bytes": 0,
"mac_local_errors": 0,
"mac_remote_errors": 0,
"rx_length_errors": 0,
"tx_xon_packets": 0,
"rx_xon_packets": 0,
"tx_xoff_packets": 0,
"rx_xoff_packets": 0,
"rx_size_64_packets": 0,
"rx_size_65_to_127_packets": 5,
"rx_size_128_to_255_packets": 0,
"rx_size_256_to_511_packets": 0,
"rx_size_512_to_1023_packets": 0,
"rx_size_1024_to_1522_packets": 0,
"rx_size_1523_to_max_packets": 0,
"rx_undersized_errors": 0,
"rx_oversize_errors": 0,
"rx_mac_short_dropped": 0,
"rx_fragmented_errors": 0,
"rx_jabber_errors": 0,
"tx_size_64_packets": 0,
"tx_size_65_to_127_packets": 6,
"tx_size_128_to_255_packets": 0,
"tx_size_256_to_511_packets": 0,
"tx_size_512_to_1023_packets": 0,
"tx_size_1024_to_1522_packets": 0,
"tx_size_1523_to_max_packets": 0,
"rx_flow_director_atr_match_packets": 0,
"rx_flow_director_sb_match_packets": 0,
"tx_low_power_idle_status": 0,
"rx_low_power_idle_status": 0,
"tx_low_power_idle_count": 0,
"rx_low_power_idle_count": 0,
"rx_priority0_xon_packets": 0,
"rx_priority1_xon_packets": 0,
"rx_priority2_xon_packets": 0,
"rx_priority3_xon_packets": 0,
"rx_priority4_xon_packets": 0,
"rx_priority5_xon_packets": 0,
"rx_priority6_xon_packets": 0,
"rx_priority7_xon_packets": 0,
"rx_priority0_xoff_packets": 0,
"rx_priority1_xoff_packets": 0,
"rx_priority2_xoff_packets": 0,
"rx_priority3_xoff_packets": 0,
"rx_priority4_xoff_packets": 0,
"rx_priority5_xoff_packets": 0,
"rx_priority6_xoff_packets": 0,
"rx_priority7_xoff_packets": 0,
"tx_priority0_xon_packets": 0,
"tx_priority1_xon_packets": 0,
"tx_priority2_xon_packets": 0,
"tx_priority3_xon_packets": 0,
"tx_priority4_xon_packets": 0,
"tx_priority5_xon_packets": 0,
"tx_priority6_xon_packets": 0,
"tx_priority7_xon_packets": 0,
"tx_priority0_xoff_packets": 0,
"tx_priority1_xoff_packets": 0,
"tx_priority2_xoff_packets": 0,
"tx_priority3_xoff_packets": 0,
"tx_priority4_xoff_packets": 0,
"tx_priority5_xoff_packets": 0,
"tx_priority6_xoff_packets": 0,
"tx_priority7_xoff_packets": 0,
"tx_priority0_xon_to_xoff_packets": 0,
"tx_priority1_xon_to_xoff_packets": 0,
"tx_priority2_xon_to_xoff_packets": 0,
"tx_priority3_xon_to_xoff_packets": 0,
"tx_priority4_xon_to_xoff_packets": 0,
"tx_priority5_xon_to_xoff_packets": 0,
"tx_priority6_xon_to_xoff_packets": 0,
"tx_priority7_xon_to_xoff_packets": 0
}
}
Most of the fields above are device dependent and very difficult to this data
from device to device. The array based information like queue stats needs to be
an array in json. The packet size counters are great, but not all of the ports
support hardware versions. I would like to see us add these to all PMDs even if
we have to do them in software and then added to the basic /ethdev/stats output.
I create a simpler layout with just the basic values from the dpdk stats get
API. Even the queue data is not valid for most ports, but I hope we can add
software support to drivers that do not have hardware queue stats. The
following is much easier to parse and use then the above format. Note I added
the portid value to structure. I used my own names for the fields, just easier
to type, but we can use the ones above.
{
“/ethdev/stats”: {
"portid": 0,
"ipackets": 0,
"opackets": 0,
"ibytes": 0,
"obytes": 0,
"imissed": 0,
"ierrors": 0,
"oerrors": 0,
"rx_nombuf": 0,
"q_ipackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"q_opackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"q_ibytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"q_obytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
"q_errors": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
}
}
If we need device specific data then we need to create a new command to dump
this data instead e.g. ‘/ethdev/xstats,X'.
—Keith
