Re: Perf issue with Beam on spark (spark runner)

2021-10-12 Thread Alexey Romanenko
l Mejía <mailto:ieme...@gmail.com>>, Kyle Weaver <mailto:kcwea...@google.com>>, Yuchu Cao <mailto:yuc...@trulia.com>> > Subject: Re: Perf issue with Beam on spark (spark runner) > > > > A long time ago when I was experimenting with the Spark runner for a

Re: Perf issue with Beam on spark (spark runner)

2021-08-06 Thread Alexey Romanenko
plittable. Thanks! > > From: Alexey Romanenko > Date: Thursday, August 5, 2021 at 6:40 AM > To: Tao Li > Cc: "user@beam.apache.org" , Andrew Pilloud > , Ismaël Mejía , Kyle Weaver > , Yuchu Cao > Subject: Re: Perf issue with Beam on spark (spark runner) >

Re: Perf issue with Beam on spark (spark runner)

2021-08-05 Thread Tao Li
check out ParquetIO splittable. Thanks! From: Alexey Romanenko Date: Thursday, August 5, 2021 at 6:40 AM To: Tao Li Cc: "user@beam.apache.org" , Andrew Pilloud , Ismaël Mejía , Kyle Weaver , Yuchu Cao Subject: Re: Perf issue with Beam on spark (spark runner) It’s very likely that

Re: Perf issue with Beam on spark (spark runner)

2021-08-05 Thread Alexey Romanenko
ser@beam.apache.org <mailto:user@beam.apache.org>" <mailto:user@beam.apache.org>> > Cc: Kyle Weaver mailto:kcwea...@google.com>>, Yuchu Cao > mailto:yuc...@trulia.com>> > Subject: Re: Perf issue with Beam on spark (spark runner) > > Thanks eve