Hi,
can anyone tell me where the default hybrid hash join function for partitioning
(shuffle phase) is implemented?
Even after deeper dinning I was not able to figure out where it is located.
Might be somewhere here?
—>
https://github.com/apache/flink/tree/master/flink-runtime/src/main/java/or
; Thanks ,vino.
>
> [1]:
> https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/HashPartition.java
>
> <https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/operators/hash/H
Hi all,
I’m currently debugging and enjoy it executing my task directly from the IDE.
The IDE starts a LocalExecutionEnvironment and immediately runs the task.
Now my errors seem to only appear when executing in the
RemoteExecutionEnvironment. (it’s around the RemoteInputChannels et. al.)
To run i
e task manager and then attach IDE to
> the JVM process of the Task Manager.
>
> Best,
> Andrey
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/Remote+Debugging+of+Flink+Clusters
>
> > On Wed, Feb 13, 2019 at 12:15 PM Benjamin Burkhardt
> > wrote:
>
> @Test
> public void testFoobar() {
> MyApplication.main(new String[0]);
> }
> }
>
> The class MiniClusterWithClientResource lives in the flink-test-utils package.
>
> Cheers,
> Till
>
> > On Fri, Feb 15, 2019 at 10:29 AM Benjamin Burkhardt
> > wrote:
>
Hi all,
I’m looking for a metric which allows me keeping track of the records or bytes
each TaskManager has received or processed for the current task.
Can anyone help me getting this?
Thanks.
Benjamin
.
Benjamin
Am 2. Apr. 2019, 10:37 +0200 schrieb Yun Tang :
> Hi Benjamin
>
> I think 'numBytesInLocalPerSecond' and 'numBytesInRemotePerSecond' which
> indicate 'The number of bytes this task reads from a local source per second'
> and 'The n
vertices/ to know all vertices similarly.
>
> However, AFAIK, if you use REST API to query I'm afraid you cannot directly
> know the received records per task manager, and you have to gather these
> metrics per task.
>
> Best
> Yun Tang
> From: Benjamin Burkhard
