+1 from me too. Same outcome as in RC1 for me.
On Sun, Feb 2, 2020 at 9:31 PM Dongjoon Hyun wrote:
>
> Please vote on releasing the following candidate as Apache Spark version
> 2.4.5.
>
> The vote is open until February 5th 11PM PST and passes if a majority +1 PMC
> votes are cast, with a mini
+1
I re-ran some of existing benchmarks in branch-2.4 on Linux/MacOS, and
haven't found any regressions compared to 2.4.4.
Maxim Gekk
On Tue, Feb 4, 2020 at 11:07 AM Takeshi Yamamuro
wrote:
> +1;
> I run the tests with
> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
> -
+1;
I run the tests with
`-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
-Psparkr`
on macOS (Java 8).
All the things look fine in my env.
Bests,
Takeshi
On Tue, Feb 4, 2020 at 12:35 PM Hyukjin Kwon wrote:
> +1 from me too.
>
> 2020년 2월 4일 (화) 오후 12:26, Wenchen Fan 님이 작성:
>
+1 from me too.
2020년 2월 4일 (화) 오후 12:26, Wenchen Fan 님이 작성:
> AFAIK there is no ongoing critical bug fixes, +1
>
> On Mon, Feb 3, 2020 at 11:46 PM Dongjoon Hyun
> wrote:
>
>> Yes, it does officially since 2.4.0.
>>
>> 2.4.5 is a maintenance release of 2.4.x line and the community didn't
>> supp
AFAIK there is no ongoing critical bug fixes, +1
On Mon, Feb 3, 2020 at 11:46 PM Dongjoon Hyun
wrote:
> Yes, it does officially since 2.4.0.
>
> 2.4.5 is a maintenance release of 2.4.x line and the community didn't
> support Hadoop 3.x on 'branch-2.4'. We didn't run test at all.
>
> Bests,
> Don
Yes, it does officially since 2.4.0.
2.4.5 is a maintenance release of 2.4.x line and the community didn't
support Hadoop 3.x on 'branch-2.4'. We didn't run test at all.
Bests,
Dongjoon.
On Sun, Feb 2, 2020 at 22:58 Ajith shetty wrote:
> Is hadoop-3.1 profile supported for this release.? i see
Is hadoop-3.1 profile supported for this release.? i see lot of UTs failing
under this profile.
https://github.com/apache/spark/blob/v2.4.5-rc2/pom.xml
Example:
[INFO] Running org.apache.spark.sql.hive.JavaMetastoreDataSourcesSuite
[ERROR] Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time e
I'll start with my +1.
Today, I verified the artifacts with GPG, and built and tested RC2 with the
followings.
- Profile: -Pyarn -Phadoop-2.7 -Pkubernetes -Pkinesis-asl -Phive
-Phive-thriftserver
- OS: CentOS (7.5.1804)
- Java: OpenJDK 1.8.0_242
* All Scala/Java UTs and JDBC IT passed.
Please vote on releasing the following candidate as Apache Spark version
2.4.5.
The vote is open until February 5th 11PM PST and passes if a majority +1
PMC votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 2.4.5
[ ] -1 Do not release this package because .
-1
Let us include the correctness fix:
https://github.com/apache/spark/pull/27229
Thanks,
Xiao
On Thu, Jan 16, 2020 at 8:46 AM Dongjoon Hyun
wrote:
> Thank you, Jungtaek!
>
> Bests,
> Dongjoon.
>
>
> On Wed, Jan 15, 2020 at 8:57 PM Jungtaek Lim
> wrote:
>
>> Once we decided to cancel the RC1
Thank you, Jungtaek!
Bests,
Dongjoon.
On Wed, Jan 15, 2020 at 8:57 PM Jungtaek Lim
wrote:
> Once we decided to cancel the RC1, what about including SPARK-29450 (
> https://github.com/apache/spark/pull/27209) into RC2?
>
> SPARK-29450 was merged into master, and Xiao figured out it fixed a
> re
Once we decided to cancel the RC1, what about including SPARK-29450 (
https://github.com/apache/spark/pull/27209) into RC2?
SPARK-29450 was merged into master, and Xiao figured out it fixed a
regression, long lasting one (broken at 2.3.0). The link refers the PR for
2.4 branch.
Thanks,
Jungtaek L
Sure. Wenchen and Hyukjin.
I observed all of the above reported issues and have been waiting to
collect more information before cancelling RC1 vote.
The other stuff I've observed is that Marcelo and Sean also requested
reverting the existing commit.
- https://github.com/apache/spark/pull/24732 (s
If we go for RC2, we should include both:
https://github.com/apache/spark/pull/27210
https://github.com/apache/spark/pull/27184
just for the sake of being complete and making the maintenance simple.
2020년 1월 16일 (목) 오후 12:38, Wenchen Fan 님이 작성:
> Recently we merged several fixes to 2.4:
> http
Recently we merged several fixes to 2.4:
https://issues.apache.org/jira/browse/SPARK-30325 a driver hang issue
https://issues.apache.org/jira/browse/SPARK-30246 a memory leak issue
https://issues.apache.org/jira/browse/SPARK-29708 a correctness issue(for
a rarely used feature, so not merged t
+1
On Wed, 15 Jan 2020, 08:24 Takeshi Yamamuro, wrote:
> +1;
>
> I checked the links and materials, then I run the tests with
> `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
> -Psparkr`
> on macOS (Java 8).
> All the things look fine and I didn't see the error on my env
>
+1;
I checked the links and materials, then I run the tests with
`-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes
-Psparkr`
on macOS (Java 8).
All the things look fine and I didn't see the error on my env
that Sean said above.
Thanks, Dongjoon!
Bests,
Takeshi
On Wed, Jan 15,
+1 Thanks.
Sincerely,
DB Tsai
--
Web: https://www.dbtsai.com
PGP Key ID: 42E5B25A8F7A82C1
On Tue, Jan 14, 2020 at 11:08 AM Sean Owen wrote:
>
> Yeah it's something about the env I spun up, but I don't know what. It
> happens frequently whe
Yeah it's something about the env I spun up, but I don't know what. It
happens frequently when I test, but not on Jenkins.
The Kafka error comes up every now and then and a clean rebuild fixes
it, but not in my case. I don't know why.
But if nobody else sees it, I'm pretty sure it's just an artifac
Thank you, Sean.
First of all, the `Ubuntu` job on Amplab Jenkins farm is green.
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-branch-2.4-test-sbt-hadoop-2.7-ubuntu-testing/
For the failures,
1. Yes, the `HiveExternalCatalogVersionsSuite` flakiness is a known one.
+1 from me. I checked sigs/licenses, and built/tested from source on
Java 8 + Ubuntu 18.04 with " -Pyarn -Phive -Phive-thriftserver
-Phadoop-2.7 -Pmesos -Pkubernetes -Psparkr -Pkinesis-asl". I do get
test failures, but, these are some I have always seen on Ubuntu, and I
do not know why they happen.
+1.
I verified with GPG and tested RC1 with the followings.
- Profile: -Pyarn -Phadoop-2.7 -Pkubernetes -Pkinesis-asl -Phive
-Phive-thriftserver
- Java: OpenJDK 1.8.0_232
- OS: CentOS (7.5.1804)
- All Scala/Java UTs and JDBC IT passed.
- Test with Amazon EKS
Client Version: v1.17.0
Please vote on releasing the following candidate as Apache Spark version
2.4.5.
The vote is open until January 16th 5AM PST and passes if a majority +1 PMC
votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 2.4.5
[ ] -1 Do not release this package because ..
+1, the late response... :(
Anyway, happy new year, all!
Bests,
Takeshi
On Tue, Jan 7, 2020 at 2:50 AM Dongjoon Hyun
wrote:
> Thank you all.
>
> I'll start to check and prepare the 2.4.5 release.
>
> Bests,
> Dongjoon.
>
> On Sun, Jan 5, 2020 at 22:51 Xiao Li wrote:
>
>> +1
>>
>> Xiao
>>
>> On
Thank you all.
I'll start to check and prepare the 2.4.5 release.
Bests,
Dongjoon.
On Sun, Jan 5, 2020 at 22:51 Xiao Li wrote:
> +1
>
> Xiao
>
> On Sun, Jan 5, 2020 at 9:50 PM Holden Karau wrote:
>
>> +1
>>
>> On Sun, Jan 5, 2020 at 9:40 PM Wenchen Fan wrote:
>>
>>> +1
>>>
>>> On Mon, Jan 6,
+1
Xiao
On Sun, Jan 5, 2020 at 9:50 PM Holden Karau wrote:
> +1
>
> On Sun, Jan 5, 2020 at 9:40 PM Wenchen Fan wrote:
>
>> +1
>>
>> On Mon, Jan 6, 2020 at 12:02 PM Jungtaek Lim <
>> kabhwan.opensou...@gmail.com> wrote:
>>
>>> +1 to have another Spark 2.4 release, as Spark 2.4.4 was released in
+1
On Sun, Jan 5, 2020 at 9:40 PM Wenchen Fan wrote:
> +1
>
> On Mon, Jan 6, 2020 at 12:02 PM Jungtaek Lim
> wrote:
>
>> +1 to have another Spark 2.4 release, as Spark 2.4.4 was released in 4
>> months old and there's release window for this.
>>
>> On Mon, Jan 6, 2020 at 12:38 PM Hyukjin Kwon
+1
On Mon, Jan 6, 2020 at 12:02 PM Jungtaek Lim
wrote:
> +1 to have another Spark 2.4 release, as Spark 2.4.4 was released in 4
> months old and there's release window for this.
>
> On Mon, Jan 6, 2020 at 12:38 PM Hyukjin Kwon wrote:
>
>> Yeah, I think it's nice to have another maintenance rele
+1 to have another Spark 2.4 release, as Spark 2.4.4 was released in 4
months old and there's release window for this.
On Mon, Jan 6, 2020 at 12:38 PM Hyukjin Kwon wrote:
> Yeah, I think it's nice to have another maintenance release given Spark
> 3.0 timeline.
>
> 2020년 1월 6일 (월) 오전 7:58, Dongjo
Yeah, I think it's nice to have another maintenance release given Spark 3.0
timeline.
2020년 1월 6일 (월) 오전 7:58, Dongjoon Hyun 님이 작성:
> Hi, All.
>
> Happy New Year (2020)!
>
> Although we slightly missed the timeline for 3.0 branch cut last month,
> it seems that we keep 2.4.x timeline on track.
>
Hi, All.
Happy New Year (2020)!
Although we slightly missed the timeline for 3.0 branch cut last month,
it seems that we keep 2.4.x timeline on track.
https://spark.apache.org/versioning-policy.html
As of today, `branch-2.4` has 154 patches since v2.4.4.
$ git log --oneline v2.4.4..HEA
Thank you all. I'll make a PR to Apache Spark website.
Bests,
Dongjoon.
On Tue, Dec 10, 2019 at 11:43 PM Wenchen Fan wrote:
> Sounds good. Thanks for bringing this up!
>
> On Wed, Dec 11, 2019 at 3:18 PM Takeshi Yamamuro
> wrote:
>
>> That looks nice, thanks!
>> I checked the previous v2.4.4 r
Sounds good. Thanks for bringing this up!
On Wed, Dec 11, 2019 at 3:18 PM Takeshi Yamamuro
wrote:
> That looks nice, thanks!
> I checked the previous v2.4.4 release; it has around 130 commits (from
> 2.4.3 to 2.4.4), so
> I think branch-2.4 already has enough commits for the next release.
>
> A
That looks nice, thanks!
I checked the previous v2.4.4 release; it has around 130 commits (from
2.4.3 to 2.4.4), so
I think branch-2.4 already has enough commits for the next release.
A commit list from 2.4.3 to 2.4.4;
https://github.com/apache/spark/compare/5ac2014e6c118fbeb1fe8e5c8064c4a8ee9d182
Sure, seems fine. The release cadence slows down in a branch over time
as there is probably less to fix, so Jan-Feb 2020 for 2.4.5 and
something like middle or Q3 2020 for 2.4.6 is a reasonable
expectation. It might plausibly be the last 2.4.x release but who
knows.
On Mon, Dec 9, 2019 at 12:29 PM
Hi, All.
Along with the discussion on 3.0.0, I'd like to discuss about the next
releases on `branch-2.4`.
As we know, `branch-2.4` is our LTS branch and also there exists some
questions on the release plans. More releases are important not only for
the latest K8s version support, but also for del
36 matches
Mail list logo
