On 8 Dec 2017, at 17:05, Andrew Duffy
mailto:adu...@palantir.com>> wrote:
Hey Steve,
Happen to have a link to the TPC-DS benchmark data w/random S3 reads? I've done
a decent amount of digging, but all I've found is a reference in a slide deck
Is that one of mine?
We haven't done any benchma
t;
Cc: Apache Spark Dev
Subject: Re: Leveraging S3 select
On 29 Nov 2017, at 21:45, Lalwani, Jayesh
mailto:jayesh.lalw...@capitalone.com>> wrote:
AWS announced at re:Invent that they are launching S3 Select. This can allow
Spark to push down predicates to S3, rather than read the e
On 29 Nov 2017, at 21:45, Lalwani, Jayesh
mailto:jayesh.lalw...@capitalone.com>> wrote:
AWS announced at re:Invent that they are launching S3 Select. This can allow
Spark to push down predicates to S3, rather than read the entire file in
memory. Are there any plans to update Spark to use S3 S
AWS announced at re:Invent that they are launching S3 Select. This can allow
Spark to push down predicates to S3, rather than read the entire file in
memory. Are there any plans to update Spark to use S3 Select?
The information contained i
