Re: Flink Serialization and case class fields limit

t; > Best, > Andrey > > [1] https://underscore.io/blog/posts/2016/10/11/twenty-two.html > > > On 16 Nov 2018, at 13:10, Andrea Sella wrote: > > Hey squirrels, > > I've started to study more in-depth Flink Serialization and its "type > system". >

Flink Serialization and case class fields limit

Hey squirrels, I've started to study more in-depth Flink Serialization and its "type system". I have a generated case class using scalapb that has more than 30 fields; I've seen that Flink still uses the CaseClassSerializer, the TypeInformation is CaseClassTypeInfo, even if in the docs[1] is writ

Re: OutputFormat in streaming job

SinkFunction, which is better suited for > streaming jobs. > > Cheers, Fabian > > 2016-05-06 14:10 GMT+02:00 Andrea Sella : > >> Hi Fabian, >> >> ATM I am not interesting to guarantee exactly-once processing, thank you >> for the clarification. >

Re: OutputFormat in streaming job

exactly-once processing. In case > of a failure, it might emit some records a second time. Hence the results > will be written at least once. > > Hope this helps, > Fabian > > 2016-05-06 12:45 GMT+02:00 Andrea Sella : > >> Hi, >> >> I created a custom Outp

OutputFormat in streaming job

Hi, I created a custom OutputFormat to send data to a remote actor, there are issues to use an OutputFormat into a stream job? Or it will treat like a Sink? I prefer to use it in order to create a custom ActorSystem per TM in the configure method. Cheers, Andrea

Re: Flink and YARN ship folder

-yt params it will work fine. is it a bug? Cheers, Andrea 2016-03-14 15:00 GMT+01:00 Andrea Sella : > Hi Robert, > > Ok, thank you. > > 2016-03-14 11:13 GMT+01:00 Robert Metzger : > >> Hi Andrea, >> >> You don't have to manually replicate any operatio

Re: Integration Alluxio and Flink

older. > > On Tue, Mar 15, 2016 at 5:40 PM, Andrea Sella > wrote: > >> Hi Till, >> >> I put the jar as dependency of my job on build.sbt. I need to do >> somenthing else? >> >> val flinkDependencies = Seq( >> "org.apache.flink" %% "

Re: JobManager Dashboard and YARN execution

t; On Wed, Mar 9, 2016 at 1:58 PM, Stephan Ewen wrote: > >> Hi! >> >> Yes, the dashboard is available in both cases. It is proxied through the >> Yarn UI, you can find the link from there... >> >> You can always access JobManager logs through the UI. >&g

Re: Integration Alluxio and Flink

ricsoftware.minlog", "minlog"). exclude("org.apache.hadoop","hadoop-yarn-common") ) `show compile:dependencyClasspath` shows alluxio-core-client as dependency. https://github.com/alkagin/alluxio-wordcount/ Thanks, Andrea 2016-03-15 17:09 GMT+01:00 Till

Re: Integration Alluxio and Flink

.8 whereas alluxio-core-client uses version 3.2.2. I think you > have to exclude or shade this dependency away. > > Cheers, > Till > ​ > > On Mon, Mar 14, 2016 at 5:12 PM, Andrea Sella > wrote: > >> Hi Till, >> I tried to downgrade the Alluxio's ne

Re: Integration Alluxio and Flink

; documentation indicates that it works with Flink (but I cannot tell for > which version). > > Cheers, > Till > > On Mon, Mar 14, 2016 at 3:18 PM, Andrea Sella > wrote: > >> Hi to all, >> >> I'm trying to integrate Alluxio and Apache Flink, I followed

Integration Alluxio and Flink

Hi to all, I'm trying to integrate Alluxio and Apache Flink, I followed Running Flink on Alluxio to setup Flink. I tested in local mode executing: bin/flink run ./examples/batch/WordCount.jar --input alluxio:///flink/README.

Re: Flink and YARN ship folder

Sat, Mar 12, 2016 at 3:25 PM, Andrea Sella > wrote: > >> Hi Ufuk, >> >> I'm trying to execute the WordCount batch example with input and output >> on Alluxio, i followed Running Flink on Alluxio >> <http://www.alluxio.org/documentation/en/Running-Flink-on-

Re: Flink and YARN ship folder

check the classpath of the JVM in the YARN logs of the > JobManager/TaskManager processes. > > – Ufuk > > > On Fri, Mar 11, 2016 at 5:33 PM, Andrea Sella > wrote: > > Hi, > > > > There is a way to add external dependencies to Flink Job, running on > YARN,

Flink and YARN ship folder

Hi, There is a way to add external dependencies to Flink Job, running on YARN, not using HADOOP_CLASSPATH? I am looking for a similar idea to standalone mode using lib folder. BR, Andrea

JobManager Dashboard and YARN execution

Hi, I am experimenting the integration between Apache Flink and YARN. When I am running a Flink job using yarn-cluster, is the Dashboard available to monitor the execution? Is it also available for long running session? Is it possibile retrieve logs from the dashboard or I have to pass via yarn a

Re: Webclient script misses building from source

er dashboard by > setting: > > jobmanager.web.submit.enable: false > > in flink-conf.yaml. > > Cheers, > Aljoscha > > On 08 Mar 2016, at 14:21, Andrea Sella > wrote: > > > > Hi, > > > > I've built Flink from source but I was not able to find in

Webclient script misses building from source

Hi, I've built Flink from source but I was not able to find in build-target/bin the script start-webclient.sh to launch the WebUI. The script is available just in the binaries or I have to add an argument to trigger its generation? Thanks in advance, Andrea

Underlying TaskManager's Actor System

Hi, There is a way to access to the underlying TaskManager's Actor System? Thank you in advance, Andrea

Re: Unexpected behavior with Scala App trait.

/2.10.x/src/library/scala/App.scala#L31 > > > On Jan 19, 2016, at 12:08 AM, Andrea Sella > wrote: > > > > Hi, > > > > I was implementing TF-IDF example of flink-training when I faced a > problem with NPE during the deploy of my Job. > > > > Source code:

Unexpected behavior with Scala App trait.

Hi, I was implementing TF-IDF example of flink-training when I faced a problem with NPE during the deploy of my Job. Source code: https://github.com/alkagin/flink-tfidf-example I used 0.10.1 version and started in local mode. During the deploy of TFIDFNPE Job, which it extends App, Flink throws