Yes, of course A IDA Discretization [info] When computing its discretization [info] - Should be computed correctly *** FAILED *** [info] org.apache.flink.runtime.client.JobExecutionException: java.lang.Exception: The user defined 'open()' method caused an exce ption: Index: 0, Size: 0
[info] at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:623) [info] at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:235) [info] at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91) [info] at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:816) [info] at org.apache.flink.api.scala.ExecutionEnvironment.execute(ExecutionEnvironment.scala:525) [info] at org.apache.flink.api.scala.DataSet.collect(DataSet.scala:601) [info] at com.elbauldelprogramador.discretizers.IDADiscretizerTransformerSpec$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mc V$sp$2.apply(IDADiscretizerTransformerSpec.scala:56) [info] at com.elbauldelprogramador.discretizers.IDADiscretizerTransformerSpec$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mc V$sp$2.apply(IDADiscretizerTransformerSpec.scala:48) [info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) [info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) [info] ... [info] Cause: java.lang.Exception: The user defined 'open()' method caused an exception: Index: 0, Size: 0 [info] at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:498) [info] at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) [info] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) [info] at java.lang.Thread.run(Thread.java:748) [info] ... [info] Cause: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 [info] at java.util.ArrayList.rangeCheck(ArrayList.java:657) [info] at java.util.ArrayList.get(ArrayList.java:433) [info] at org.apache.flink.ml.package$BroadcastSingleElementMapper.open(package.scala:82) [info] at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36) [info] at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:494) [info] at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) [info] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711) [info] at java.lang.Thread.run(Thread.java:748) It seems the problem is not with the code above, but it is caused by it, since cuts is empty. *-- Alejandro Alcalde - elbauldelprogramador.com <http://elbauldelprogramador.com>* On Tue, Sep 25, 2018 at 1:33 PM Fabian Hueske <fhue...@gmail.com> wrote: > Hi, > > Can you post the full stacktrace? > > Thanks, Fabian > > Am Di., 25. Sep. 2018 um 12:55 Uhr schrieb Alejandro Alcalde < > algu...@gmail.com>: > >> Hi, >> >> I am trying to improve the efficiency of this code: >> >> discretized.map(_._2) >> .name("Map V") >> .reduce((_, b) ⇒ b) >> .name("Get Last V") >> >> I am just interested in the last element of discretized. >> >> I've seen this SO question: >> https://stackoverflow.com/questions/45076310/how-to-get-only-the-last-element-of-the-large-dataset-in-flink >> >> I've tried with this code: >> >> discretized.mapPartition(in ⇒ if (!in.hasNext) in.map(_._2) else Seq()) >> .setParallelism(1) >> .name("Get Last V") >> >> But I am getting this error: >> >> Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 >> >> Best >> >> *-- Alejandro Alcalde - elbauldelprogramador.com >> <http://elbauldelprogramador.com>* >> >
