Thanks guys. This is really helpful! I assume we have a plan to publish these docs on the samza wiki.
-David On Wed, Feb 24, 2016 at 10:34 AM, Xinyu Liu <xi...@linkedin.com.invalid> wrote: > Thanks, Shadi. The doc is really useful! > > @Milinda: thanks for pointing it out. Process-calls includes both > process-envelopes and process-null-envelopes, so it should be > process-envelopes in David's example. > > Thanks, > Xinyu > > On Wed, Feb 24, 2016 at 9:52 AM, Abdollahian Noghabi, Shadi < > abdol...@illinois.edu> wrote: > > > I have attached the document to SAMZA-702.< > > https://issues.apache.org/jira/browse/SAMZA-702> > > > > > > On Feb 24, 2016, at 9:33 AM, Milinda Pathirage <mpath...@umail.iu.edu > > <mailto:mpath...@umail.iu.edu>> wrote: > > > > Hi Shadi, > > > > Attachment is not there in your mail. I think mailing list dropped the > > attachment. IMHO, we should create a JIRA issue and attach the doc to the > > issue so that we can move it to Samza docs. > > > > On Wed, Feb 24, 2016 at 12:27 PM, Abdollahian Noghabi, Shadi < > > abdol...@illinois.edu<mailto:abdol...@illinois.edu>> wrote: > > > > I have a document with some of the metrics. I had gathered these around > > last summer, so they may be out-of-date. I have attached the document to > > this email. Hope it can help. > > > > > > > > > > > > > > On Feb 24, 2016, at 7:10 AM, Milinda Pathirage <mpath...@umail.iu.edu > > <mailto:mpath...@umail.iu.edu>> > > wrote: > > > > Hi David and Xinyu, > > > > If you want to get the number of messages processed, "process-envelopes" > > is > > the correct metrics. "process-calls" gives measure the number of times > > RunLoop#process method is called. So "process-calls" get updated even > > without processing any messages (This happens when no new messages in > > input > > stream). "process-ns" can be used as the average time taken to process a > > message. But this average also includes time taken to process null > > messages. So I don't trust the accuracy of that metric. > > > > Each metric emitted by Samza contains a header which includes job name, > > job > > id, container name and metric timestamp. You can use it to calculate > > messages per second values. > > > > If you are using KV store, KeyValueStoreMetrics contains metrics such as > > bytes read, bytes write, puts and gets for each store. > > > > Thanks > > Milinda > > > > On Tue, Feb 23, 2016 at 8:26 PM, xinyu liu <xinyuliu...@gmail.com > <mailto: > > xinyuliu...@gmail.com>> > > wrote: > > > > Hi, David, > > > > I didn't find a wiki page that contains the descriptions of all Samza > > metrics. You can find the basic metrics by googling the following > > classes: > > SamzaContainerMetrics, TaskInstanceMetrics, SystemConsumersMetrics and > > SystemProducersMetrics. For your example, you can use the > > "process-calls" > > in SamzaContainerMetrics to get the processed message count, and divide > > the > > delta by time to get the messages processed per sec. In practice, you > > can > > either use JConsole to connect to the running Samza container or consume > > the MetricsSnapshot topic to get the detailed metrics. > > > > Thanks, > > Xinyu > > > > On Tue, Feb 23, 2016 at 4:51 PM, David Yu <david...@optimizely.com > <mailto: > > david...@optimizely.com>> > > wrote: > > > > Hi, > > > > Where can I find the detailed descriptions of the out of the box > > metrics > > provided by MetricsSnapshotReporterFactory and JmxReporterFactory? > > > > I'm interested in seeing the basic metrics of the my samza job (e.g. > > messages_processed_per_sec). But it's hard to ping point to the > > specific > > metric that shows me that. > > > > Thanks, > > David > > > > > > > > > > > > -- > > Milinda Pathirage > > > > PhD Student | Research Assistant > > School of Informatics and Computing | Data to Insight Center > > Indiana University > > > > twitter: milindalakmal > > skype: milinda.pathirage > > blog: http://milinda.pathirage.org > > > > > > > > > > -- > > Milinda Pathirage > > > > PhD Student | Research Assistant > > School of Informatics and Computing | Data to Insight Center > > Indiana University > > > > twitter: milindalakmal > > skype: milinda.pathirage > > blog: http://milinda.pathirage.org > > > > >
