I talked to Lianhui offline and he said it is not that big of a deal to revert the patch.
On Tue, Apr 19, 2016 at 9:52 AM, Mark Grover <m...@apache.org> wrote: > Thanks. > > I'm more than happy to wait for more people to chime in here but I do feel > that most of us are leaning towards Option B anyways. So, I created a JIRA > (SPARK-14731) for reverting SPARK-12130 in Spark 2.0 and file a PR shortly. > Mark > > On Tue, Apr 19, 2016 at 7:44 AM, Tom Graves <tgraves...@yahoo.com.invalid> > wrote: > >> It would be nice if we could keep this compatible between 1.6 and 2.0 so >> I'm more for Option B at this point since the change made seems minor >> and we can change to have shuffle service do internally like Marcelo >> mention. Then lets try to keep compatible, but if there is a forcing >> function lets figure out a good way to run 2 at once. >> >> >> Tom >> >> >> On Monday, April 18, 2016 5:23 PM, Marcelo Vanzin <van...@cloudera.com> >> wrote: >> >> >> On Mon, Apr 18, 2016 at 3:09 PM, Reynold Xin <r...@databricks.com> wrote: >> > IIUC, the reason for that PR is that they found the string comparison to >> > increase the size in large shuffles. Maybe we should add the ability to >> > support the short name to Spark 1.6.2? >> >> Is that something that really yields noticeable gains in performance? >> >> If it is, it seems like it would be simple to allow executors register >> with the full class name, and map the long names to short names in the >> shuffle service itself. >> >> You could even get fancy and have different ExecutorShuffleInfo >> implementations for each shuffle service, with an abstract >> "getBlockData" method that gets called instead of the current if/else >> in ExternalShuffleBlockResolver.java. >> >> >> -- >> Marcelo >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> >> >> >> >
