Hi Amit ,
Thanks for response . Meanwhile I figured out the issue .
I had /Class X extending RichMapFunction/ and this class was preparing some
heavy data required for map function . I just moved that code to *open()*
function and it worked fine .
So I have one doubt , was it because flink was
Hi,
Could you share log of job and impacted task manager? How much memory
you have allocated to the Job Manager?
--
Thanks,
Amit
On Mon, May 21, 2018 at 8:46 PM, sohimankotia wrote:
> Hi,
>
> I am running flink batch job .
>
> My job is running fine if i use 4 task manger and 8 slots = 32 parall
Hi Alex,
StreamingExecutionEnvironment#readFile is a helper function to create
file reader data streaming source. It uses
ContinuousFileReaderOperator and ContinuousFileMonitoringFunction
internally.
As both file reader operator and monitoring function uses
checkpointing so is readFile [1], you c
I want to add checkpointing to my program that reads from a set of files in a
directory. Without checkpointing I use readFile():
DataStream text = env.readFile(
new TextInputFormat(new Path(inputPath)),
inputPath,
Hi,
I am running flink batch job .
My job is running fine if i use 4 task manger and 8 slots = 32 parallelism
with 6GB memory per task manager.
As soon I increase task mangers to 5 with 6 task per task manager = 30
parallelism (6GB memory per task manager)
I am getting oom error . I am not able
Hi,
I want to store my data in one hdfs and the flink checkpoints in another
hdfs. I didn't find a way to do it, anyone can point me a direction?
Thanks in advance,
Raul
Hi guys,
Were you able to RCA this? I think I'm hitting the same issue on 1.4.0, but
not really able to reproduce it through a test case.
In an IterativeCondition (using AfterMatchSkipStrategy.skipPastLastEvent),
while looking up previously matched events, it is hitting a
java.util.NoSuchElementEx
Hi
I have tried this "homework" about 2 months now.. I have tried for example CEP
(with Scala) and iterative condition, but I don't really understand how it
would work. Or does it work in my case ?
It seems difficult to store values of previous state to use next state(s). And
even between diff
Thanks Gary!
Sure, there are issues with updates in S3. You may want to look over
EMRFS guarantees of the consistent view [1]. I'm not sure, is it
possible in non-EMR AWS system or not.
I'm creating a JIRA issue regarding data loss possibility in S3. IMHO,
Flink docs should mention about possible
