I did review it and solving this problem makes sense. I will comment in the
JIRA.
From: Jungtaek Lim
Sent: Sunday, August 25, 2019 3:34:22 PM
To: dev
Subject: Design review of SPARK-28594
Hi devs,
I have been working on designing SPARK-28594 [1] (though I've s
hi,
i am struggling to understand if concurrent writes to same basedir but
different partitions are save with file sources such as parquet.
i tested this in spark 2.4 and spark 3.0.0-SNAPSHOT with real concurrent
jobs on hdfs and it seemed to work fine. admittedly this was a rather
limited test.
a
We are happy to announce the availability of Spark 2.4.4!
Spark 2.4.4 is a maintenance release containing stability fixes. This
release is based on the branch-2.4 maintenance branch of Spark. We strongly
recommend all 2.4 users to upgrade to this stable release.
To download Spark 2.4.4, head over
Great, thanks for reviewing, Felix!
On Mon, Sep 2, 2019 at 2:16 AM Felix Cheung
wrote:
> I did review it and solving this problem makes sense. I will comment in
> the JIRA.
>
> --
> *From:* Jungtaek Lim
> *Sent:* Sunday, August 25, 2019 3:34:22 PM
> *To:* dev
> *Sub
So I'm not a lawyer, but looking at the JDK8 License FAQ (
https://www.oracle.com/technetwork/java/javase/overview/oracle-jdk-faqs.html )
makes me feel like we should support folks who are on the 2.x branch and
want to migrate away from JDK8. Although the Redhat post about OpenJDK8 is
reassuring (
That's just the Oracle 'commercial' JDK though. OpenJDK is still OSS
and 8 is updated there too. Oracle JDK is still free too, just not for
commercial use and commercial support for 8 (IIUC). I think that's the
'audience' here, and I don't know how large or small that is.
That said a key first que
https://issues.apache.org/jira/browse/SPARK-28945
On Sun, Sep 1, 2019 at 2:53 PM Koert Kuipers wrote:
> hi,
> i am struggling to understand if concurrent writes to same basedir but
> different partitions are save with file sources such as parquet.
>
> i tested this in spark 2.4 and spark 3.0.0-S
Great! Thanks!
On Mon, Sep 2, 2019 at 5:55 AM Dongjoon Hyun
wrote:
> We are happy to announce the availability of Spark 2.4.4!
>
> Spark 2.4.4 is a maintenance release containing stability fixes. This
> release is based on the branch-2.4 maintenance branch of Spark. We strongly
> recommend all 2
YaY!
2019년 9월 2일 (월) 오후 1:27, Wenchen Fan 님이 작성:
> Great! Thanks!
>
> On Mon, Sep 2, 2019 at 5:55 AM Dongjoon Hyun
> wrote:
>
>> We are happy to announce the availability of Spark 2.4.4!
>>
>> Spark 2.4.4 is a maintenance release containing stability fixes. This
>> release is based on the branch
