On 14 Feb 2018, at 19:56, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
Newbie question:
I want to add system/integration tests for the new functionality. There are a
set of existing tests around Spark Catalog that I can leverage. Great. The
provider I’m writing is backed by a web servi
On 14 Feb 2018, at 13:51, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
Thanks a lot Steve. I’ll go through the Jira’s you linked in detail. I took a
quick look and am sufficiently scared for now. I had run into that warning from
the S3 stream before. Sigh.
things like that are trouble
,
-Ameen
From: Steve Loughran
Date: Tuesday, February 13, 2018 at 5:01 PM
To: "Tayyebi, Ameen"
Cc: Apache Spark Dev
Subject: Re: A new external catalog
On 13 Feb 2018, at 21:20, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
Yes, I’m thinking about upgrading to these:
1.9.0
Spark Dev
Subject: Re: A new external catalog
On 13 Feb 2018, at 21:20, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
Yes, I’m thinking about upgrading to these:
1.9.0
1.11.272
From:
1.7.3
1.11.76
272 is the earliest that has Glue.
How about I let the build system run the tes
;Tayyebi, Ameen" mailto:tayye...@amazon.com>>
Cc: Apache Spark Dev mailto:dev@spark.apache.org>>
Subject: Re: A new external catalog
On 13 Feb 2018, at 19:50, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
The biggest challenge is that I had to upgrade the AWS SDK to a
at 3:34 PM
To: "Tayyebi, Ameen"
Cc: Apache Spark Dev
Subject: Re: A new external catalog
On 13 Feb 2018, at 19:50, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
The biggest challenge is that I had to upgrade the AWS SDK to a newer version
so that it includes the Glue c
On 13 Feb 2018, at 19:50, Tayyebi, Ameen
mailto:tayye...@amazon.com>> wrote:
The biggest challenge is that I had to upgrade the AWS SDK to a newer version
so that it includes the Glue client since Glue is a new service. So far, I
haven’t see any jar hell issues, but that’s the main drawback
