[
https://issues.apache.org/jira/browse/FLINK-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623879#comment-14623879
]
Yanping Wang commented on FLINK-2343:
-------------------------------------
Hi, Stephan
There were some bug fixes and optimizations back ported to jdk7 update 60 and
later but not all, and not up to date. The most recent or last jdk7, which is
update 79,
http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html
that probably is the best G1 in jdk7 you can get. As jdk7 is no longer to be
updated, update 79 was marked the end of jdk7.
The most helpful optimizations and fixes were in upcoming jdk8 update 60
(scheduled to be default java download in sep/oct 2015) , and jdk9 a year later.
The spark graph computing workload performance I measured 2X performance gain
on 160GB heap was compare jdk7u79 vs. jdk8u60. Also there is an article on
tuning G1 for Spark was based on jdk8u40 from my fellow engineers working on
Spark (my name was listed at ACK section :-)
https://databricks.com/blog/2015/05/28/tuning-java-garbage-collection-for-spark-applications.html
One good point by using G1 is, if we find problems, we can ask Oracle Java
developers to fix and to optimize. For CMS, there is no development efforts
there, as it will be replaced by G1.
By the way, in the experiment I did and also mentioned by the paper above, CMS
was bad because the Spark workload was a throughput workload with low latency
requirement. Throughput was run down so much when CMS running into full GC to
compact a large heap.
If you run G1 on Flink and see funning things, let me know I think I can try
making it better.
Thanks
-yanping
> Change default garbage collector in streaming environments
> ----------------------------------------------------------
>
> Key: FLINK-2343
> URL: https://issues.apache.org/jira/browse/FLINK-2343
> Project: Flink
> Issue Type: Improvement
> Components: Start-Stop Scripts
> Affects Versions: 0.10
> Reporter: Stephan Ewen
> Assignee: Stephan Ewen
> Fix For: 0.10
>
>
> When starting Flink, we don't pass any particular GC related JVM flags to the
> system. That means, it uses the default garbage collectors, which are the
> bulk parallel GCs for both old gen and new gen.
> For streaming applications, this results in vastly fluctuating latencies.
> Latencies are much more constant with either the {{CMS}} or {{G1}} GC.
> I propose to make the CMS the default GC for streaming setups.
> G1 may become the GC of choice in the future, but fro various articles I
> found, it is still somewhat in "beta" status (see for example here:
> http://jaxenter.com/kirk-pepperdine-on-the-g1-for-java-9-118190.html )
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
