Server side error: Cannot find required BLOB at /tmp/blobStore

Hi, My Flink application which reads from Kafka and writes to HDFS is failing repeatedly with below exception: Caused by: java.io.IOException: Server side error: Cannot find required BLOB at /tmp/blobStore-** Can someone please help me on, what could be the root cause of this issue? I

Re: Server side error: Cannot find required BLOB at /tmp/blobStore

/tmp/blobStore Is it the path for checkpoints/savepoints storage ? On Tue, 11 Sep 2018, 10:01 Raja.Aravapalli, wrote: > Hi, > > > > My Flink application which reads from Kafka and writes to HDFS is failing > repeatedly with below exception: > > > > Caused by: java.io.IOException: Server side e

Re: [EXTERNAL] Re: Server side error: Cannot find required BLOB at /tmp/blobStore

I have not passed visibly anything from my end, to application any path either HDFS or Local, that starts with “/tmp” ☹ Regards, Raja. From: miki haiat Date: Tuesday, September 11, 2018 at 12:44 PM To: Raja Aravapalli Cc: user Subject: [EXTERNAL] Re: Server side error: Cannot find required B

Re: Server side error: Cannot find required BLOB at /tmp/blobStore

Hi. >From my presentation on Flink forward you can validate this •We used EMR on Amazon’s Linux AMI •We didn't change the default blob server location (/tmp) •Default a cron job cleaning up in /tmp •Solution change blob server location with blob.storage.directory Den tir. 11. sep. 2018 kl. 09.14

[Kerberos] JAAS module content not generated? javax.security.auth.callback.UnsupportedCallbackException: Could not login: the client is being asked for a password, but the Kafka client code does not c

Hi, We are using Flink 1.5.3 where the Kafka producer talks with a kerberized kafka (kerberos only, no SSL). It fails to connect to kafka with a root Exception: javax.security.auth.callback.UnsupportedCallbackException: Could not login: the client is being asked for a password, but the Kafka

Re: [EXTERNAL] Re: Server side error: Cannot find required BLOB at /tmp/blobStore

Thanks a lot Lasse. So, the recommendation is to update the property “blob.storage.directory”, to some maintained path, other than “/tmp”, so the default cleanup will be eliminated ? Also can you pls confirm if this is a hdfs path or a Local unix path ? Please acknowledge if I have understood

Re: WriteTimeoutException in Cassandra sink kill the job

Have you configured checkpointing in your job. If enabled, the job should revert back to the last stored checkpoint in case of a failure and process the failed record again. -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Create a file in parquet format

Hi I am working in a task. The purpose is to create a sink in Parquet format. Then, I am using the "Streaming Flink Sink", but I cannot complete the task. Do you know any example in github, blog, that I can use to complete the task. Many Thx BR Jose

Re: Exception when run flink-storm-example

Hi Till， legacy mode worked！ Thanks a lot. And what's difference between legacy and new? Is there any document and release note? There may be both flink job and flink-storm in the my cluster, I don't know the influence about legacy mode. | | Hanjing | | | 签名由网易邮箱大师定制 On 9/11/2018 14

How to get Cluster metrics in FLIP-6 mode

Hi, I found that these metrics[1] disappeared in my JM's prometheus reporter when I used FLIP-6 to deploy standalone cluster. (flink 1.5.3 release) Cluster ScopeMetricsDescriptionType *JobManager* numRegisteredTaskManagers The number of registered taskmanagers. Gauge numRunningJobs The number of

Re: JAXB Classloading errors when using PMML Library (AWS EMR Flink 1.4.2)

Hi, Do you also have pmml-model-moxy as a dependency in your job? Using mvn dependency:tree, I do not see that pmml-evaluator has a compile time dependency on jaxb-api. The jaxb-api dependency actually comes from pmml- model-moxy. The exclusion should be therefore defined on pmml-model-moxy. You

Re: How to get Cluster metrics in FLIP-6 mode

Hi Tony, You are right that these metrics are missing. There is already a ticket for that [1]. At the moment you can obtain these information from the REST API (/overview) [2]. Since FLIP-6, the JM is no longer responsible for these metrics but for backwards compatibility we can leave them in the

Re: Exception when run flink-storm-example

You can check these release notes https://flink.apache.org/news/2018/05/25/release-1.5.0.html for more information. Cheers, Till On Tue, Sep 11, 2018 at 12:02 PM jing wrote: > Hi Till， > legacy mode worked！ > Thanks a lot. And what's difference between legacy and new? Is there > any doc

Re: Create a file in parquet format

Hi Jose, You can find an example here: https://github.com/apache/flink/blob/1a94c2094b8045a717a92e232f9891b23120e0f2/flink-formats/flink-parquet/src/test/java/org/apache/flink/formats/parquet/avro/ParquetStreamingFileSinkITCase.java#L58 Best, Gary On Tue, Sep 11, 2018 at 11:59 AM, jose farfan

Re: How to get Cluster metrics in FLIP-6 mode

Hi Gary, Thanks for your information. Best, Tony Wei 2018-09-11 20:26 GMT+08:00 Gary Yao : > Hi Tony, > > You are right that these metrics are missing. There is already a ticket for > that [1]. At the moment you can obtain these information from the REST API > (/overview) [2]. > > Since FLIP-6,

ElasticSearch Checkpointing taking too much time

I am using flink 1.5.3, In this i am using elastic search sink. In this checkpoints and savepoints are failing, I have already given 50 minutes timeouts. After looking into details only elastic search sink checkpoints are taking time 30-35 mins. But state size and buffer size is 0 in that. Don't kn

Re: ElasticSearch Checkpointing taking too much time

Update: I am using parallelism 1 on this... is this issue? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Deadlock in SafetyNetCloseableRegistry?

Hi,all I starts a flink program and it runs on yarn. At first it doesn’t aquire enough resources so this is thrown. “org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Could not allocate all requires slots within timeout of 30 ms. Slots required: 16, slots allocated

Between Checkpoints in Kafka 11

Hi, I was going through the blog post on how TwoPhaseCommitSink function works with Kafka 11. One of the things I don’t understand is: What is the behavior of the Kafka 11 Producer between two checkpoints? Say that the time interval between two checkpoints is set to 15 minutes. Will Flink buffer a

Speakers needed for Apache DC Roadshow

We need your help to make the Apache Washington DC Roadshow on Dec 4th a success. What do we need most? Speakers! We're bringing a unique DC flavor to this event by mixing Open Source Software with talks about Apache projects as well as OSS CyberSecurity, OSS in Government and and OSS Career

Re: Deadlock in SafetyNetCloseableRegistry?

Hi, all Sorry for attaching this again. The flink version is 1.6 and the dead lock stack is "CloseableReaperThread" #54 daemon prio=5 os_prio=0 tid=0x7f4d6d3af000 nid=0x32f6 in Object.wait() [0x7f4d3fdfe000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(N

Re: Hashjoin implementation

Hi vino, thanks. I was running a join operation on two DataSets and writing the result to disk and the results were correct. I just was not able to identify the moment when the Hashtable is built. (HashPartition.java is not used in this case?) Do you have an idea why I cannot find it? Here

How does flink read a DataSet?

Hi All, Just like Spark does Flink read a dataset and keep it in memory and keep applying transformations? Or all records read by Flink async parallel reads? Furthermore, how does Flink deal with Regards, Taher Koitawala GS Lab Pune +91 8407979163

Re: How does flink read a DataSet?

Furthermore, how does Flink deal with Task Managers dying when it is using the DataSet API. Is checkpointing done on dataset too? Or the whole dataset has to re-read. Regards, Taher Koitawala GS Lab Pune +91 8407979163 On Tue, Sep 11, 2018 at 11:18 PM, Taher Koitawala wrote: > Hi All, >

Re: Problem with querying state on Flink 1.6.

Kostas - Till's advice got me past my first problem. I'm still having issues with the client side. I've got your example code from [1] in a github project [2]. My problem differs from David Anderson's above in that my call to QueryableStateClient is using a remote machine, not localhost (my client

Re: Triggering Savepoints with the Monitoring API

Thank you vino! On Mon, Sep 10, 2018, 11:08 PM vino yang wrote: > Hi Austin, > > It seems that your scene is very suitable for a usage scenario of Flink's > Savepoint: A/B Test (or upgrade application). Yes, Flink can support this > requirement, but you should understand that these two jobs will

Re: ElasticSearch Checkpointing taking too much time

Hi shashank, The parallelism won't be the problem. Did the checkpoint succeed finally? I think it may be that the data processing is blocked so that the checkpoint can not been successful. You can check if there are any error logs in the TaskManager or jstack the taskmanager to see what's wrong th

Re: Flink RMQSource Consumer: How I get the RabbitMQ UserId

Hi Marke, Should not use the code like this : *delivery.getProperties().getUserId();* to get the userId from Delivery object? And for second code example, Since you got the object of TimeSeriesType type, should not define *DataStream* instead of *DataStream*. Regarding userId, I just said that

Re: ElasticSearch Checkpointing taking too much time

Hi shashank, Hequn's solution is right. In addition, what type of statebackend you use, please make sure that JM/TM can access related systems (such as HDFS). If you still can't locate the problem, you can set the log level to DEBUG and share your log information. Thanks, vino. Hequn Cheng 于201

Re: Exception when run flink-storm-example

Hi hanjing, *There may be both flink job and flink-storm in the my cluster, I don't know the influence about legacy mode.* > For storm-compatible jobs, because of technical limitations, you need to use a cluster that supports legacy mode. But for Jobs implemented using the Flink-related API, I

Re: Hashjoin implementation

Hi Benjamin, Do you mean that you want to see HashPartition.java when you write the program? Oh, maybe you have confused something. The only thing you use to write a program is the Flink DataSet API, which is just a way to describe the job logic. And the class you are looking for, it's in the flin

Re: How does flink read a DataSet?

Hi Taher, Stream processing and batch processing are very different. The principle of batch processing determines that it needs to process bulk data, such as memory-based sorting, join, and so on. So, in this case, it needs to wait for the relevant data to arrive before it is calculated, but this

Re: ElasticSearch Checkpointing taking too much time

Hi Hequn, Actually there are no error logs and to turn on debug mode I have to restart the app, Actually, I am using around 25-30 operators all others are completing successfully in less time only elastic search sink is taking too much time. I am using around 6 Elastic search sinks all are takin

Re: ElasticSearch Checkpointing taking too much time

Hi, vino, I have tried bot HDFS and filesystem and other checkpoints completed successfully so access is not the issue. For debug mode, I have to restart the app. I'll check and let you know thanks -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: ElasticSearch Checkpointing taking too much time

-- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/