Hi, all See SPARK-1529 for related discussion.
// maropu On Wed, Apr 27, 2016 at 12:27 PM, Saisai Shao <sai.sai.s...@gmail.com> wrote: > Quite curious about the benefits of using HDFS as shuffle service, also > what's the problem of using current shuffle service? > > > Thanks > Saisai > > On Wed, Apr 27, 2016 at 4:31 AM, Timothy Chen <tnac...@gmail.com> wrote: > >> Are you suggesting to have shuffle service persist and fetch data with >> hdfs, or skip shuffle service altogether and just write to hdfs? >> >> Tim >> >> >> > On Apr 26, 2016, at 11:20 AM, Michael Gummelt <mgumm...@mesosphere.io> >> wrote: >> > >> > Has there been any thought or work on this (or any other networked file >> system)? It would be valuable to support dynamic allocation without >> depending on the shuffle service. >> > >> > -- >> > Michael Gummelt >> > Software Engineer >> > Mesosphere >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org >> For additional commands, e-mail: dev-h...@spark.apache.org >> >> > -- --- Takeshi Yamamuro
