Measure task execution time

2018-09-26 Thread Alejandro
Hello, I am trying to measure the total time my flink app takes. I've read it can be done with this: val r = env.execute("NAME") System.out.println("The job took " + r.getNetRuntime(TimeUnit.SECONDS) + " s to execute") The thing is, it works for some task, where all computations occur in the sa

Re: Get last element of a DataSe

2018-09-25 Thread Alejandro Alcalde
a:368) [info] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) [info] at java.lang.Thread.run(Thread.java:748) It seems the problem is not with the code above, but it is caused by it, since cuts is empty. *-- Alejandro Alcalde - elbauldelprogramador.com <http://elbauldelpro

Get last element of a DataSe

2018-09-25 Thread Alejandro Alcalde
by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 Best *-- Alejandro Alcalde - elbauldelprogramador.com <http://elbauldelprogramador.com>*

Re: CombinableGroupReducer says The Iterable can be iterated over only once

2018-09-14 Thread Alejandro
Then how I am suppose to implement that function? On 09/14/2018 05:29 PM, æšćŠ› wrote: > A java.util.Iterable is expected to provide iterators again and again. > > On Fri, Sep 14, 2018 at 10:53 PM Alejandro Alcalde > wrote: > >> Hello all, >> >> I am trying

CombinableGroupReducer says The Iterable can be iterated over only once

2018-09-14 Thread Alejandro Alcalde
ouble, Double)], out: Collector[(Double, Double)]): Unit = { ??? } } Where am I transversing `in` a second time? may be is the call to `asScala`? Bests *-- Alejandro Alcalde - elbauldelprogramador.com <http://elbauldelprogramador.com>*

Compute Symmetrical Uncertainty in parallel

2018-09-14 Thread Alejandro
In this answer Optimizing Flink transformation (https://stackoverflow.com/questions/52151715/optimizing-flink-transformation/52225586#52225586): def symmetricalUncertainty(xy: DataSet[(Double, Double)]): Double = { val su = xy.reduceGroup { in ⇒ val invec = in.toVector val x = invec map (_