Thanks for sharing Nicolas, This is super-useful. On Mon, Sep 12, 2016 at 10:56 AM, Nicolas Maquet <nmaq...@gmail.com> wrote:
> Hey Tommy, > > At Movio, we feed all of our Samza metrics into Prometheus and use Grafana > to create visual dashboards. We configure all of our jobs to produce > metrics into a single topic and we have a monitoring tool that consumes > that topic and exposes the data to Prometheus ( > https://github.com/movio/samza-prometheus-exporter). > > We've been using this setup in production for 6 months now and it's been > working really well for us. > > Nick > > > On 13 Sep 2016 1:04 a.m., "Tommy Becker" <tobec...@tivo.com> wrote: > > > One of the challenges we've faced with deploying Samza is effectively > > monitoring it. We currently use the MetricsSnapshotReporter and consume > the > > resulting stream pulling out specific values that we want to monitor, but > > this approach isn't scalable. We're exploring pumping the metrics into an > > ELK stack, but the format isn't ideal (many keys contain dots, which ES > > doesn't like). Additionally, many of the keys are "dynamic" in that the > > names are determined by runtime configuration, which makes pulling > specific > > values out a bit of a pain. I'd love to get some information on how other > > folks are monitoring their Samza jobs. > > > > -- > > Tommy Becker > > Senior Software Engineer > > > > Digitalsmiths > > A TiVo Company > > > > www.digitalsmiths.com<http://www.digitalsmiths.com> > > tobec...@tivo.com<mailto:tobec...@tivo.com> > > > > ________________________________ > > > > This email and any attachments may contain confidential and privileged > > material for the sole use of the intended recipient. Any review, copying, > > or distribution of this email (or any attachments) by others is > prohibited. > > If you are not the intended recipient, please contact the sender > > immediately and permanently delete this email and any attachments. No > > employee or agent of TiVo Inc. is authorized to conclude any binding > > agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo > > Inc. may only be made by a signed written agreement. > > > -- Jagadish V, Graduate Student, Department of Computer Science, Stanford University
