Re: Sample Code for querying Flink's default metrics

2019-12-13 Thread Pankaj Chand
Additionally, when an old job completes and I run a new job on the Flink Yarn session mode cluster, when I query for metrics before they become available for the new job, I sometimes get the last metrics for the old job instead. This happens even if I wait for the TaskManager to be released by Flin

Re: Sample Code for querying Flink's default metrics

2019-12-12 Thread Pankaj Chand
Thank you, Chesnay! On Thu, Dec 12, 2019 at 5:46 AM Chesnay Schepler wrote: > Yes, when a cluster was started it takes a few seconds for (any) metrics > to be available. > > On 12/12/2019 11:36, Pankaj Chand wrote: > > Hi Vino, > > Thank you for the links regarding backpressure! > > I am current

Re: Sample Code for querying Flink's default metrics

2019-12-12 Thread Chesnay Schepler
Yes, when a cluster was started it takes a few seconds for (any) metrics to be available. On 12/12/2019 11:36, Pankaj Chand wrote: Hi Vino, Thank you for the links regarding backpressure! I am currently using code to get metrics by calling REST API via curl. However, many times the REST API

Re: Sample Code for querying Flink's default metrics

2019-12-12 Thread Pankaj Chand
Hi Vino, Thank you for the links regarding backpressure! I am currently using code to get metrics by calling REST API via curl. However, many times the REST API via curl gives an empty JSON object/array. Piped through JQ (for filtering JSON) it produces a null value. This is breaking my code. Exa

Re: Sample Code for querying Flink's default metrics

2019-12-09 Thread vino yang
Hi Pankaj, > Is there any sample code for how to read such default metrics? Is there any way to query the default metrics, such as CPU usage and Memory, without using REST API or Reporters? What's your real requirement? Can you use code to call REST API? Why does it not match your requirements?