Some documentation on application profiling with Flink 1.3 (can be manually inserted into the scripts for Flink 1.2): https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/application_profiling.html
> On Jun 22, 2017, at 9:24 AM, Stefan Richter <s.rich...@data-artisans.com> > wrote: > > Hi, > > the answer highly depends on what you job is doing and there is no > information about that. Also what is your target in performance? Are you > using batch or streaming? If you feel like the performance is lower than > expected, I suggest that you do some profiling to figure out the hotspots. > For example, you could see that your job spends most time in type > serialization, which is a common bottleneck. In this case, maybe you can > write a faster custom serializer. Or rewriting the job (e.g. use early > aggregation where possible etc.) can yield much more performance improvement > then tuning magic numbers with no further knowledge about your job. > > Best, > Stefan > >> Am 22.06.2017 um 12:08 schrieb Samim Ahmed <samim1...@gmail.com >> <mailto:samim1...@gmail.com>>: >> >> Hi All, >> >> This query regarding the flink performance improvement . >> >> Flink Configuration: >> using flink in clustor mode with 3 salves and a master configuration >> slots used 30 (as the system has 30 core) >> task manager memory 30GB >> parallelism used : 30 >> jobmanager.heap.mb: 20480 >> taskmanager.heap.mb: 20480 >> taskmanager.numberOfTaskSlots: 30 >> taskmanager.network.numberOfBuffers: 20000 >> >> Input info: >> Input file : 1ROP(5min) data with 3333 Nodes and 665K eps >> Total number of events :: 199498294 >> >> Observation : >> Total time taken to complete the task = 6m24s >> >> Can you please suggest what else I need to modify to get the high >> performance in terms of lese execution time. Thanks in advance >> >> >> -- >> Regards, >> Samim Ahmed >> Mumbai >> 09004259232
