Re: TimSort bug and its workaround

2015-03-02 Thread Colin P. McCabe
Thanks for bringing this up. If you can find any place where an array might realistically be larger than 67 million elements, then I guess file a JIRA for it. Also this array needs to be of objects, not of primitives (quicksort is used for those in jdk7, apparently). I can't think of any such pl

Re: TimSort bug and its workaround

2015-02-26 Thread Tsuyoshi Ozawa
Maybe we should discuss whether the elements of array can be larger than 67108864 in our use cases - e.g. FairScheduler uses Collection.sort(), but the number of job isn't larger than 67108864 in many use cases, so we can keep using it. It's also reasonable that we choose to use safe algorithms for

TimSort bug and its workaround

2015-02-26 Thread Tsuyoshi Ozawa
Hi hadoop developers, Last 2 weeks, a bug of JDK about TimSort, related to Collections#sort, is reported. How can we deal with this problem? http://envisage-project.eu/timsort-specification-and-verification/ https://bugs.openjdk.java.net/browse/JDK-8072909 The bug causes ArrayIndexOutOfBoundsEx