Windowing isn't applied per key

2017-09-29 Thread Marcus Clendenin
I have a job that is performing an aggregation over a time window. This windowing is supposed to be happening by key, but the output I am seeing is creating an overall window on everything coming in. Is this happening because I am doing a map of the data before I am running the keyBy command? This

Taskmanager unable to rejoin job manager

2017-09-14 Thread Marcus Clendenin
Hi all, I am having an issue where one of our task managers that is running in high availability mode is timing out on the connection to zookeeper. This is causing it to retry the connection to zookeeper, which succeeds. The issue is once the taskmanager is back connected to zookeeper it is then

Re: RocksDB error with flink 1.2.0

2017-04-28 Thread Marcus Clendenin
I changed the max number of open files and got past this error but now I'm seeing errors that it's unable to flush the file. I am checkpointing using hdfs, should I be using local file system? Is there any better way to use the cep patterns with multiple patterns or are you suggesting creating my