Thank you for the detailed answer, specially on point 2. This clarifies the use case of custom collectors. Based on your answer, I am convinced that a custom collector is what I should be using. The linux app is immutable but maintains and provides an interface to poll metrics. Its like using top to get the CPU and memory usage. The data exporter will be sitting on the host next to the application and will make calls to the app when the server wants to scrape to collect the metrics from the app.
On Wednesday, June 22, 2022 at 11:58:53 PM UTC-7 Brian Candler wrote: > On Thursday, 23 June 2022 at 05:04:42 UTC+1 [email protected] wrote: > >> 2. I have added a timer loop, which collects the metrics at the defined >> interval, I am wondering if there is a way to manage this interval from the >> prometheus server. >> https://prometheus.io/docs/instrumenting/writing_exporters/#scheduling >> This link says that the exporter should not collect metrics at arbitrary >> intervals, however it doesn't say how to sync it with the prometheus server. >> > > If you need to "do work" to collect metrics, you would do it whenever you > receive a scrape. That's what a Custom Collector is for: it's code which > is executed at scrape time, which has whatever special logic you need to > generate metrics on demand. > > However in the normal case, you are simply setting gauges or incrementing > counters in response to external events as part of the application's normal > flow. You would do those gauge or counter updates whenever the events > occur. Then the metrics are being maintained continuously, and so the > scrape only needs to return the current values of those metrics. In that > case, no custom collector is required. > > The example code you posted has a dummy polling loop to update metrics > intermittently. That's not what you'd do in real life. Rather, you'd > replace "run_metrics_loop" with "run_my_application", and insert code to > update metrics within the application, at strategic points where it does > work (say when it processes a REST API request, or whatever else it does > that needs measuring) > > >> 3. I am writing the exporter in python, is there a standard directory >> layout/coding practice that I should follow? >> > > Often you're integrating the exporter into an existing application, in > which case you just insert it into the layout of that application. > > You said you're instrumenting a "linux app": is that linux app also > written in Python? If so, add your exporter to the application. If not, > how are you collecting data from it? > > If the app is not in Python, and you can modify the app's source code, but > you don't want to integrate an exporter in whatever language the app is > written in, then here are some other approaches which don't involve writing > a separate exporter: > - the app can send update messages to statsd_exporter, which maintains > gauges and counter state > - the app can write metrics to text files, to be picked up by > node_exporter textfile collector > > If you can't modify the app, then at very worst it will probably emit log > messages. In that case, you can use mtail or grok_exporter to read the log > files and generate metrics. But that's a last resort. > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/55e998d4-413b-4965-a40b-7f82f2c32d32n%40googlegroups.com.
