Hi, I am using 1.10.0 version of Flink on EMR.
I am not using the Default Flink Sink. I have a Sink Function on the Stream and with in the invoke function i am creating a Data Structure (VO) and putting it in the Map. The EMR Step function i am running is. a Spring based FLink Job and i have a scheduler which runs every min and looks for items in the Map and generates JSON based in the VO from the Map and send it to Elastic Search and removes it from the HashMap once it is sent to ES successfully. I am using M5.2x large for worker nodes and M5.4xlarge for Master Node I have set the ulimit to 500K for all users (*) . Both soft and hard limit on Master and worker nodes. Thanks again for your response. Sateesh -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
