Re: [QUESTION] OutOfMemoryError when writing into HDFS

2017-06-11 Thread Mu Kong
Hi Ted, Thanks for the useful link there. Actually, I just found out that the error might be triggered by a kerberos issue in our system. I'll use the ticket cache and execute kinit in crontab to see if it is the case. But thanks for the link you provided. I'll definitely check it :) Best regar

Re: [QUESTION] OutOfMemoryError when writing into HDFS

2017-06-11 Thread Ted Yu
Can you see if the following post helps in troubleshooting ? https://blog.fastthread.io/2016/07/06/troubleshoot-outofmemoryerror-unable-to-create-new-native-thread/ Thanks On Sun, Jun 11, 2017 at 6:01 PM, Mu Kong wrote: > Hi all, > > Thanks a lot for your work for the community! > > This quest

[QUESTION] OutOfMemoryError when writing into HDFS

2017-06-11 Thread Mu Kong
Hi all, Thanks a lot for your work for the community! This question is more like a discuss. Currently, I'm experiencing an *OutOfMemoryError* when writing files from Kafka into HDFS using *BucketingSink*. The log looks like this: 2017-06-10 08:58:42,248 INFO org.apache.flink.runtime.taskmanager