Re: Emit Custom metrics in Spark Structured Streaming job

2018-07-10 Thread subramgr
Hi I am working on implementing my idea but here is how it goes: 1. Use this library https://github.com/groupon/spark-metrics 2. Have a cron job which periodically curl /metrics/json endpoint at driver and all other nodes 3. Parse the response and send the data through a telegraf agent installed

Re: Emit Custom metrics in Spark Structured Streaming job

2018-07-10 Thread chandan prakash
Hi Subramanian, Did you find any solution for this ? I am looking for something similar too. Regards, Chandan On Wed, Jun 27, 2018 at 9:47 AM subramgr wrote: > I am planning to send these metrics to our KairosDB. Let me know if there > are > any examples that I can take a look > > > > -- > Sent

Re: Emit Custom metrics in Spark Structured Streaming job

2018-06-26 Thread subramgr
I am planning to send these metrics to our KairosDB. Let me know if there are any examples that I can take a look -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spar