One of the challenges we've faced with deploying Samza is effectively monitoring it. We currently use the MetricsSnapshotReporter and consume the resulting stream pulling out specific values that we want to monitor, but this approach isn't scalable. We're exploring pumping the metrics into an ELK stack, but the format isn't ideal (many keys contain dots, which ES doesn't like). Additionally, many of the keys are "dynamic" in that the names are determined by runtime configuration, which makes pulling specific values out a bit of a pain. I'd love to get some information on how other folks are monitoring their Samza jobs.
-- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com<http://www.digitalsmiths.com> tobec...@tivo.com<mailto:tobec...@tivo.com> ________________________________ This email and any attachments may contain confidential and privileged material for the sole use of the intended recipient. Any review, copying, or distribution of this email (or any attachments) by others is prohibited. If you are not the intended recipient, please contact the sender immediately and permanently delete this email and any attachments. No employee or agent of TiVo Inc. is authorized to conclude any binding agreement on behalf of TiVo Inc. by email. Binding agreements with TiVo Inc. may only be made by a signed written agreement.
