Re: What's the meaning of "Registered `TaskManager` at akka://flink/deadLetters " ?

Hi, I attached log file, Thanks On Mon, Jan 15, 2018 at 3:36 PM, Piotr Nowojski wrote: > Hi, > > Could you post full job manager and task manager logs from startup until > the first signs of the problem? > > Thanks, Piotrek > > > On 15 Jan 2018, at 11:21, Reza Samee wrote: > > Thanks for resp

Re: using a Yarn cluster for both Spark and Flink

Why not? Isn't a resource manager meant for this? You should however clearly define service level agreements as a flink streaming job might require certain maximum latency opposed to a spark batch job. Soheil Pourbafrani schrieb am Do. 18. Jan. 2018 um 08:30: > Is it standard approach to set up a

Flink standalone scheduler

Does Flink standalone scheduler support dynamic resource allocations? Something like Yarn scheduler?

using a Yarn cluster for both Spark and Flink

Is it standard approach to set up a Yarn cluster for running both Spark and Flink applications?

Re: Flink CEP exception during RocksDB update

Thank you Kostas. Since this error is not easily reproducible on my end I’ll continue testing this and confirm the resolution once I am able to do so. Thanks, Varun Sent from my iPhone > On Jan 15, 2018, at 10:21 AM, Kostas Kloudas > wrote: > > Hi Varun, > > This can be related to this iss

Far too few watermarks getting generated with Kafka source

Hi, I have a job where we read data from either Kafka or a file (for testing), decode the entries and flat map them into events, and then add a timestamp and watermark assigner to the events in a later operation. This seems to generate periodic watermarks when running from a file, but when Kafka is

Submitting jobs via Java code

Hi, I am a beginner in Flink and I'm trying to deploy a simple example using a java client in a remote Flink server (1.4.0). I'm using org.apache.flink.client.program.Client this is the used code: Configuration config = new Configuration(); config.setString("jobmanager.rpc.address", "192.168.14

Re: Which collection to use in Scala case class

The issue of Scala case classes that can not be restored is a known issue in Flink 1.4. We need to investigate if it can be fixed easily. @Shashank: Could you give us a little reproducible example? Just a case class with a java.util.List in it? @Gordon: Is there a Jira issue for this? I might

Re: Which collection to use in Scala case class

Can you show us the exception you get when restoring? Looping in Time who knows more about how types are analyzed/serializers are constructed. On 17.01.2018 10:25, shashank agarwal wrote: Hello, A quick question which scala collection should I use in my scala case class which won't go throu

Re: Problem while debugging a python job

Thanks. That did it. best Mathias On 17.01.2018 13:11, Chesnay Schepler wrote: > All dependencies of flink-python are set to /provided/ so that they > aren't included in the flink-python jar (which would duplicate all > classes already contained in flink-dist). > > You can either temporarily mo

Re: State backend questions

According to this thread it is not yet possible to switch to/from RocksDBStatebackend, so I would suggest to with RocksDB from the start. For tuning RocksDB, see here

Re: Problem while debugging a python job

All dependencies of flink-python are set to /provided/ so that they aren't included in the flink-python jar (which would duplicate all classes already contained in flink-dist). You can either temporarily modify the dependencies and remove the /provided /scope, or create a simple test class tha

Re: History Server

The HS isn't aware of running jobs at all, so yes users would have to switch once the job is done. On 17.01.2018 00:45, Eron Wright wrote: As a follow-up question, how well does the history server work for observing a running job? I'm trying to understand whether, in the cluster-per-job mode

Which collection to use in Scala case class

Hello, A quick question which scala collection should I use in my scala case class which won't go through generic serializer. I was using java.utill.List in my scala case class before, Will this create the problem in savepoint and restore. cause, my restore is not working so i am trying to replac

Re: Failed to serialize remote message [class org.apache.flink.runtime.messages.JobManagerMessages$JobFound

Hi, yes you're right. The different standby JobManagers should have different web addresses. Cheers, Till On Tue, Jan 16, 2018 at 6:32 PM, jelmer wrote: > I think i found the issue. I'd just like to verify that my reasoning is > correct > > We had the following keys in our flink-conf.yaml > >