Re: Samza + MongoDb Sharding

2017-07-07 Thread Jagadish Venkatraman
Thank you for clarifying. Yes, You should partition your input streams as well. Typically, it does not hurt to over-partition them. https://samza.apache.org/learn/documentation/0.13/container/samza-container.html Thanks, Jagdish On Fri, Jul 7, 2017 at 10:55 PM Angelica Garcia-Gutierrez < agar.

Re: Question Concerning Custom Metrics and Timers

2017-07-07 Thread Jagadish Venkatraman
Hey Jeremiah, The "timer" metric in Samza maintains a sliding window of its data points. Each time you invoke timer.update(val), a data-point gets added to the sliding window. The reported value of a timer metric is the average of all data points in the window. In contrast, the "gauge" metric alw

Question Concerning Custom Metrics and Timers

2017-07-07 Thread Jeremiah Adams
I am making early attempts at gathering metrics around methods and sql calls. I am using the Snapshot Reporter to send metrics to a kafka topic. The docs state that metrics are sent to kafka every 60 seconds (https://samza.apache.org/learn/documentation/latest/container/metrics.html). When I

Re: Samza + MongoDb Sharding

2017-07-07 Thread Angelica Garcia-Gutierrez
That is correct, I am augmenting each message in the input stream with further information from the MongoDB. On 7/6/17, 5:43 PM, "Jagadish Venkatraman" wrote: >What is your input source you are reading the stream from? IIUC, you are >trying to make a remote call to mongo to decorate/ augment