How to get throughput and processing latency when testing Flink with hibench?

2021-05-17 Thread penguin.
When using hibench to test Flink, the jobs submitted are built-in applications of hibench, that is, the code logic of programs like wordcount cannot be changed. How can I get the throughput and processing delay of Flink? In addition, in the /report/hibench.report file of hibench, we can't get t

Testing Flink with identity program in hibench

2021-05-17 Thread penguin.
Thanks for reading this email. According to the introduction, the identity program in hibench reads data from Kafka and then writes it back to Kafka. When using the identity program in hibench to test the Flink, set the parallelism to 20 in the flink.conf file in the conf directory of hiben

Re:Re: The problem of getting data by Rest API

2021-05-09 Thread penguin.
rval of 10 seconds. On 5/7/2021 3:57 AM, penguin. wrote: On the Web UI page, we can see that the relevant data is updated every 3S, such as the read-bytes of each operator. But when I get data through Rest API, the data is updated every 6 seconds or even more than 10 seconds. Why? The related da

The problem of getting data by Rest API

2021-05-06 Thread penguin.
On the Web UI page, we can see that the relevant data is updated every 3S, such as the read-bytes of each operator. But when I get data through Rest API, the data is updated every 6 seconds or even more than 10 seconds. Why? The related data of read bytes obtained through Rest API is as follows:

Monitor the Flink

2021-01-17 Thread penguin.
Hello, In the Flink cluster, How to monitor each taskslot of taskmanager? For example, the CPU and memory usage of each slot and the traffic between slots. What is the way to get the traffic between nodes? thank you very much! penguin

Re:Re: Re: Idea import Flink source code

2021-01-13 Thread penguin.
Hi, I click the reply button every time... Does this mean that only the replied person can see the email? If Maven fails to download plugins or dependencies, is mvn -clean install -DskipTests a must? I'll try first. penguin 在 2021-01-13 16:35:10,"Matthias Pohl&qu

Task scheduling of Flink

2021-01-05 Thread penguin.
Hello! Do you know how to modify the task scheduling method of Flink?