Re: Hello, a question about Dashborad in Flink

2016-01-29 Thread Stephan Ewen
Hi! The REST monitoring interface and extended web dashboard were added in version 0.10 Greetings, Stephan On Fri, Jan 29, 2016 at 9:55 AM, Philip Lee wrote: > Great, > > you menat the difference between narrow shuffle and global shuffle? > > I use Flink version 0.9, > but it did not not work

Re: Hello, a question about Dashborad in Flink

2016-01-29 Thread Philip Lee
Great, you menat the difference between narrow shuffle and global shuffle? I use Flink version 0.9, but it did not not work to access REST interface when I use "ssh tunnel" to remote server. it is from version of probelm? Best, Phil On Fri, Jan 29, 2016 at 9:46 AM, Fabian Hueske wrote: > T

Re: Hello, a question about Dashborad in Flink

2016-01-29 Thread Fabian Hueske
The REST interface does also provide metrics about the number of records and the size of the input and output of all tasks. See: - /jobs//vertices/ - /jobs//vertices//subtasks//attempts/ in https://ci.apache.org/projects/flink/flink-docs-release-0.10/internals/monitoring_rest_api.html#details-of-a-

Re: Hello, a question about Dashborad in Flink

2016-01-28 Thread Philip Lee
Thanks, Is there any way to measure shuffle data (read and write) on Flink or Dashboard? I did not find the network usage metric in it. Best, Phil On Mon, Jan 25, 2016 at 5:06 PM, Fabian Hueske wrote: > You can start a job and then periodically request and store information > about the runnin

Re: Hello, a question about Dashborad in Flink

2016-01-25 Thread Fabian Hueske
You can start a job and then periodically request and store information about the running job and vertices from using corresponding REST calls [1]. The data will be in JSON format. After the job finished, you can stop requesting data. Next you parse the JSON, extract the information you need and g

Hello, a question about Dashborad in Flink

2016-01-25 Thread Philip Lee
Hello, According to http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Apache-Flink-Web-Dashboard-Completed-Job-history-td4067.html, I cannot retrieve the job history from Dashboard after turnning off JM. But as Fabian mentioned here, "However, you can query all stats that are di