Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-16 Thread Tzu-Li (Gordon) Tai
)     ... 17 more Thanks     From: Sathi Chowdhury Date: Friday, April 14, 2017 at 1:29 AM To: "user@flink.apache.org" Subject: Re: Flink errors out and job fails--IOException from CollectSink.open()   I am consistently seeing the same behavior…tried with elevated memory for job m

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-16 Thread Ted Yu
data.port: 4964 >> taskmanager.heap.mb: 39000 >> taskmanager.numberOfTaskSlots: 1 >> taskmanager.network.numberOfBuffers: 16368 >> taskmanager.memory.preallocate: false >> parallelization.degree.default: 4 >> even though the jobmanager is restarting the flink jo

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-16 Thread Sathi Chowdhury
ink.apache.org>> Subject: Re: Flink errors out and job fails--IOException from CollectSink.open() I am consistently seeing the same behavior…tried with elevated memory for job manager and taskmanager taskmanager.rpc.port: 6123 taskmanager.data.port: 4964 taskmanager.heap.mb: 3900

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-15 Thread Sathi Chowdhury
com.esotericsoftware.kryo.io.Output.flush(Output.java:163) ... 17 more Thanks From: Sathi Chowdhury Date: Friday, April 14, 2017 at 1:29 AM To: "user@flink.apache.org" Subject: Re: Flink errors out and job fails--IOException from CollectSink.open() I am consistently seeing the same behavior…tried wit

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-14 Thread Sathi Chowdhury
;user@flink.apache.org" Subject: Re: Flink errors out and job fails--IOException from CollectSink.open() In my jobmanager log I see this exception , probably is the root cause why the whole job is killed…is there any memory problem in jobmanager ? any clue for this error below? I ran the yarn-sessio

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-13 Thread Sathi Chowdhury
13, 2017 at 5:44 PM To: Ted Yu mailto:yuzhih...@gmail.com>> Cc: "user@flink.apache.org<mailto:user@flink.apache.org>" mailto:user@flink.apache.org>> Subject: Re: Flink errors out and job fails--IOException from CollectSink.open() Hi Ted, Sorry for my big font earlier

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-13 Thread Ted Yu
t; {StartingSequenceNumber: 495722540787610437508121973762 > 49737935721565982386290786,}}'}, starting state set to the restored > sequence number LATEST_SEQUENCE_NUM > > 2017-04-13 23:28:23,472 INFO org.apache.flink.runtime. > taskmanager.Task - Freeing task resources for Source:

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-13 Thread Sathi Chowdhury
resources for Source: Custom Source (1/1) (8a7301a437cb2d052208ee42c994104b). From: Sathi Chowdhury Date: Thursday, April 13, 2017 at 5:44 PM To: Ted Yu Cc: "user@flink.apache.org" Subject: Re: Flink errors out and job fails--IOException from CollectSink.open() Hi Ted, Sorry for my big font

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-13 Thread Sathi Chowdhury
s.StreamTask.invoke(StreamTask.java:343) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:655) at java.lang.Thread.run(Thread.java:745) Thanks Sathi From: Ted Yu Date: Thursday, April 13, 2017 at 5:02 PM To: Sathi Chowdhury Cc: "user@flink.apache.org

Re: Flink errors out and job fails--IOException from CollectSink.open()

2017-04-13 Thread Ted Yu
Can you give us a bit more information ? release of flink snippet of your code Thanks

Flink errors out and job fails--IOException from CollectSink.open()

2017-04-13 Thread Sathi Chowdhury
Has some one encountered this error …as I am using DataStream api to read from a kinesis stream .This happens intermittently and flink job dies. reamShard{streamName='dev-ingest-kinesis-us-west-2', shard='{ShardId: shardId-0009,HashKeyRange: {StartingHashKey: 30625413022884461711703714