Re: Understanding metrics

2017-03-15 Thread Ankit Malhotra
Hi Xinyu, Super weird, but I can’t seem to find the commit-ns nor the window-ns metric for our jobs. There are window-calls and commit-calls but the timings are missing. task.commit.ms=9 task.window.ms=9 Ankit On 3/14/17, 8:00 PM, "xinyu liu" wrote: Hi, Ankit, When runni

Re: Understanding metrics

2017-03-14 Thread xinyu liu
Hi, Ankit, When running your job in multithreading, block-ns here actually includes the process_ns. This is because after your task.process() is submitted to the thread pool, the run loop thread will be blocked until the process() is complete for one of the task. It's interesting that block-ns (0.

Re: Understanding metrics

2017-03-14 Thread Ankit Malhotra
Wait, block-ns = 0.3ms (300,000ns). Also, why are we not adding in choose-ns? Thanks Ankit On 3/14/17, 6:26 PM, "Jagadish Venkatraman" wrote: I would expect (process_ns + block_ns) to be almost the same as 0.15 which makes sense. process_ns = 0.12 ms block_ns = 0.03 ms

Re: Understanding metrics

2017-03-14 Thread Jagadish Venkatraman
I would expect (process_ns + block_ns) to be almost the same as 0.15 which makes sense. process_ns = 0.12 ms block_ns = 0.03 ms process_ns + block_ns ~ 0.15ms This corresponds to the number of process calls roughly 1/7000 ~ 0.15ms per process call. >> Each process call is a separate thread. Give

Understanding metrics

2017-03-14 Thread Ankit Malhotra
Hi, We are trying to understand metrics that are being populated by our samza job and are a little confused what each of these metrics mean especially since we’re running the job with a thread pool. · We have 3 input streams · job.container.thread.pool.size=3 · 1 cont