Hi Uwe, OK, thanks for the info. We'll see if we can download the Lucene test suite and check it out.
FWIW, we use G1GC in our production runtime (~70 12-16 core Cisco UCS and HP Gen7/Gen8 nodes with 20+ GB heaps using Java 7 and Lucene 4.8.1 with pairs of 30 index partitions with 15M-23M docs each) and have not experienced any VM crashes (well, maybe a couple, but not directly traceable to G1 to my knowledge). We have found some undocumented pauses in G1 due to very large object arrays and filed a bug report which was confirmed and also affects CMS (we worked around this in our code using memory mapping of some files whose contents we previously held all in RAM). I think the only index corruption we've ever seen was in our index creation workflow (~30 HP Gen7 nodes with 27GB heaps) but this was using Parallel GC since it is a batch system, so that corruption (which we've not seen recently and never found a cause for) was definitely not due to G1GC. G1GC has bugs as does CMS but we've found it to work pretty well so far in our runtime system. Of course YMMV, thanks again for the info. Jim ________________________________________ From: Uwe Schindler [u...@thetaphi.de] Sent: Tuesday, January 27, 2015 3:02 PM To: java-user@lucene.apache.org Subject: RE: Lucene Version Upgrade (3->4) and Java JVM Versions(6->8) Hi., About G1GC. We consistently see problems when running the Lucene Testsuite with G1GC enabled. The people from Elasticsearch concluded: "There is a newer GC called the Garbage First GC (G1GC). This newer GC is designed to minimize pausing even more than CMS, and operate on large heaps. It works by dividing the heap into regions and predicting which regions contain the most reclaimable space. By collecting those regions first (garbage first), it can minimize pauses and operate on very large heaps. Sounds great! Unfortunately, G1GC is still new, and fresh bugs are found routinely. These bugs are usually of the segfault variety, and will cause hard crashes. The Lucene test suite is brutal on GC algorithms, and it seems that G1GC hasn’t had the kinks worked out yet. We would like to recommend G1GC someday, but for now, it is simply not stable enough to meet the demands of Elasticsearch and Lucene." (http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/_don_8217_t_touch_these_settings.html) In fact, the problems with G1GC can sometimes lead to index corruption, and are hard to reproduce. So better don't use... Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: McKinley, James T [mailto:james.mckin...@cengage.com] > Sent: Tuesday, January 27, 2015 8:58 PM > To: java-user@lucene.apache.org > Subject: RE: Lucene Version Upgrade (3->4) and Java JVM Versions(6->8) > > Why do you say not to use G1GC? We are using Java 7 & G1GC with Lucene > 4.8.1 in production. Thanks. > > Jim > ________________________________________ > From: Uwe Schindler [u...@thetaphi.de] > Sent: Tuesday, January 27, 2015 2:49 PM > To: java-user@lucene.apache.org; 'kiwi clive' > Subject: RE: Lucene Version Upgrade (3->4) and Java JVM Versions(6->8) > > Java 8 update 20 or later is also fine. At current time, always use latest > update > release and you are be fine with Java 7 and Java 8. Don't use older releases > and don't use G1 Garbage Collector. > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -----Original Message----- > > From: kiwi clive [mailto:kiwi_cl...@yahoo.com.INVALID] > > Sent: Tuesday, January 27, 2015 8:03 PM > > To: java-user@lucene.apache.org > > Subject: Re: Lucene Version Upgrade (3->4) and Java JVM Versions(6->8) > > > > Hi Hoss, > > Many thanks for the information. This looks very encouraging as the > > Java7 bug I remember was fixed and as far as I know, we should not be > > affected by the others. > > I'll put a few tests together and put my toe in the water :-) Clive > > > > From: Chris Hostetter <hossman_luc...@fucit.org> > > To: "java-user@lucene.apache.org" <java-user@lucene.apache.org>; kiwi > > clive <kiwi_cl...@yahoo.com> > > Sent: Tuesday, January 27, 2015 4:01 PM > > Subject: Re: Lucene Version Upgrade (3->4) and Java JVM > > Versions(6->8) > > > > > > > > > > : I seem to remember reading that certain versions of lucene were > > : incompatible with some java versions although I cannot find anything > > to > > : verify this. As we have tens of thousands of large indexes, > > backwards > > : compatibility without the need to reindex on an upgrade is of prime > > : importance to us. > > > > All known JVM bugs affecting Lucene are listed here... > > > > https://wiki.apache.org/lucene-java/JavaBugs > > > > > > -Hoss > > http://www.lucidworks.com/ > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
