Hi, all, happy new year!
+1 on the release of 2.2.3/2.3.3.
I checked there is no ongoing issue targeting on 2.3.3, too.
On Thu, Jan 3, 2019 at 8:50 AM Felix Cheung
wrote:
> +1 on 2.2.3 of course
>
>
> --
> *From:* Dongjoon Hyun
> *Sent:* Wednesday, January 2, 2019 1
Hi Matt, i'm a developer of SparkRDMA shuffle manager:
https://github.com/Mellanox/SparkRDMA
Thanks for your effort on improving Spark Shuffle API. We are very
interested in participating in this. Have for now several comments:
1. Went through these 4 documents:
https://docs.google.com/document/d/
Unsubscribe me, please.
Thank you so much
Unsub me 2 pls.
On Thu, 3 Jan 2019 at 15:22, marco rocchi <
rocchi.1407...@studenti.uniroma1.it> wrote:
> Unsubscribe me, please.
>
> Thank you so much
>
Thank you for additional support for 2.2.3, Felix and Takeshi!
The following is the update for Apache Spark 2.2.3 release.
For correctness issues, two more patches landed on `branch-2.2`.
SPARK-22951 fix aggregation after dropDuplicates on empty dataframes
SPARK-25591 Avoid overwrit
Yes, that one's not going to be back-ported to 2.3. I think it's fine to
proceed with a 2.2 release with what's there now and call it done.
Note that Spark 2.3 would be EOL around September of this year.
On Thu, Jan 3, 2019 at 2:31 PM Dongjoon Hyun
wrote:
> Thank you for additional support for 2
Just wondering if there is a good reason to keep around the
pre-tungsten on-heap memory mode for Spark 3, and make
spark.memory.offHeap.enabled always true? It would simplify the code
somewhat, but I don't feel I'm so aware of the tradeoffs.
I know we didn't deprecate it, but it's been off by defa
Thank you, Sean!
Bests,
Dongjoon.
On Thu, Jan 3, 2019 at 2:50 PM Sean Owen wrote:
> Yes, that one's not going to be back-ported to 2.3. I think it's fine to
> proceed with a 2.2 release with what's there now and call it done.
> Note that Spark 2.3 would be EOL around September of this year.
>
The issue with the offheap mode is it is a pretty big behavior change and does
require additional setup (also for users that run with UDFs that allocate a lot
of heap memory, it might not be as good).
I can see us removing the legacy mode since it's been legacy for a long time
and perhaps very
