Sid Young writes: > Are there any reliable/solid Lustre Specific metrics tools that can > push data to ELK OR Can generate JSON strings of metrics I can push > into more bespoke monitoring solutions... > > I am more interested in I/O metrics from the lustre side of things as > I can gather Disk/CPU/memory metrics with Metricbeat as needed already > in the legacy HPC.
Lustre Job Stats [0] may provide some or all of what you are looking for. The job stats data are output in yaml format, which are fairly easy to transform to inputs for Elasticsearch (or more generally as JSON for other systems). In our case we used Python. The yaml inputs are imported as Python dictionaries, which can then be used directly as input data to the Elasticsearch Python module. We happen to add some additional entries to the dictionary objects before submitting them to Elasticsearch. We also find it useful to clear the job stats after each read to simplify analysis. I do not have any publicly-available code to share, but I think the Python implementation is not overly complex. [0] https://doc.lustre.org/lustre_manual.xhtml#dbdoclet.jobstats -- Nathan Smith Research Systems Engineer Advanced Computing Center Oregon Health & Science University _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
