Thank you for additional support for 2.2.3, Felix and Takeshi!
The following is the update for Apache Spark 2.2.3 release.
For correctness issues, two more patches landed on `branch-2.2`.
SPARK-22951 fix aggregation after dropDuplicates on empty dataframes
SPARK-25591 Avoid overwriting deserialized accumulator
Currently, if we use the following JIRA search query, there exist one JIRA
issue; SPARK-25206.
Query: project = SPARK AND fixVersion in (2.3.0, 2.3.1, 2.3.2, 2.3.3,
2.4.0, 2.4.1, 3.0.0) AND fixVersion not in (2.2.0, 2.2.1, 2.2.2, 2.2.3) AND
affectedVersion in (2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.2.0, 2.2.1, 2.2.2, 2.2.3)
AND labels in (Correctness, correctness)
SPARK-25206 ( https://issues.apache.org/jira/browse/SPARK-25206 ) has
Affected Version: 2.2.2, 2.3.1
Target Versions: 2.3.2, 2.4.0
Fixed Version: 2.4.0
Although SPARK-25206 is labeled as a correctness issue, 2.3.2 already
missed it due to the technical difficulties and risks. Instead, it's marked
as a known issue. As we see, it's not targeted to 2.3.3, too.
I know the correctness issue policy on new releases. However, for me, Spark
2.2.3 is a little bit exceptional release since it's a farewell release and
branch-2.2 is already EOL and too far from the active branch master.
So, I'd like to put SPARK-25206 out of the scope of the farewell release
and recommend the users to use the other latest release. For example, Spark
2.4.0 for SPARK-25206.
How do you think about that?
Bests,
Dongjoon.
On Thu, Jan 3, 2019 at 12:02 AM Takeshi Yamamuro <linguin....@gmail.com>
wrote:
> Hi, all, happy new year!
>
> +1 on the release of 2.2.3/2.3.3.
> I checked there is no ongoing issue targeting on 2.3.3, too.
>
> On Thu, Jan 3, 2019 at 8:50 AM Felix Cheung <felixcheun...@hotmail.com>
> wrote:
>
>> +1 on 2.2.3 of course
>>
>>
>> ------------------------------
>> *From:* Dongjoon Hyun <dongjoon.h...@gmail.com>
>> *Sent:* Wednesday, January 2, 2019 12:21 PM
>> *To:* Saisai Shao
>> *Cc:* Xiao Li; Felix Cheung; Sean Owen; dev
>> *Subject:* Re: Apache Spark 2.2.3 ?
>>
>> Thank you for swift feedbacks and Happy New Year. :)
>> For 2.2.3 release on next week, I see two positive opinions (including
>> mine)
>> and don't see any direct objections.
>>
>> Apache Spark has a mature, resourceful, and fast-growing community.
>> One of the important characteristic of the mature community is
>> the expectable behavior where the users are able to depend on.
>> For instance, we have a nice tradition to cut the branch as a sign of
>> feature freeze.
>> The *final* release of a branch is not only good for the end users, but
>> also a good sign of the EOL of the branch for all.
>>
>> As a junior committer of the community, I want to contribute to deliver
>> the final 2.2.3 release to the community and to finalize `branch-2.2`.
>>
>> * For Apache Spark JIRA, I checked that there is no on-going issues
>> targeting on `2.2.3`.
>> * For commits, I reviewed the newly landed commits after `2.2.2` tag and
>> updated a few missing JIRA issues accordingly.
>> * Apparently, we can release 2.2.3 next week.
>>
>> BTW, I'm +1 for the next 2.3/2.4 and have been expecting those releases
>> before Spark+AI Summit (April) because we did like that usually.
>> Please send another email to `dev` mailing list because it's worth to
>> receive more attentions and requests.
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Tue, Jan 1, 2019 at 9:35 PM Saisai Shao <sai.sai.s...@gmail.com>
>> wrote:
>>
>>> Agreed to have a new branch-2.3 release, as we already accumulated
>>> several fixes.
>>>
>>> Thanks
>>> Saisai
>>>
>>> Xiao Li <lix...@databricks.com> 于2019年1月2日周三 下午1:32写道：
>>>
>>>> Based on the commit history,
>>>> https://gitbox.apache.org/repos/asf?p=spark.git;a=shortlog;h=refs/heads/branch-2.3
>>>> contains more critical fixes. Maybe the priority is higher?
>>>>
>>>> On Tue, Jan 1, 2019 at 9:22 PM Felix Cheung <felixcheun...@hotmail.com>
>>>> wrote:
>>>>
>>>>> Speaking of, it’s been 3 months since 2.3.2... (Sept 2018)
>>>>>
>>>>> And 2 months since 2.4.0 (Nov 2018) - does the community feel 2.4
>>>>> branch is stabilizing?
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> *From:* Sean Owen <sro...@gmail.com>
>>>>> *Sent:* Tuesday, January 1, 2019 8:30 PM
>>>>> *To:* Dongjoon Hyun
>>>>> *Cc:* dev
>>>>> *Subject:* Re: Apache Spark 2.2.3 ?
>>>>>
>>>>> I agree with that logic, and if you're volunteering to do the legwork,
>>>>> I don't see a reason not to cut a final 2.2 release.
>>>>>
>>>>> On Tue, Jan 1, 2019 at 9:19 PM Dongjoon Hyun <dongjoon.h...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > Hi, All.
>>>>> >
>>>>> > Apache Spark community has a policy maintaining the feature branch
>>>>> for 18 months. I think it's time for the 2.2.3 release since 2.2.0 is
>>>>> released on July 2017.
>>>>> >
>>>>> > http://spark.apache.org/versioning-policy.html
>>>>> >
>>>>> > After 2.2.2 (July 2018), `branch-2.2` has 40 patches (including
>>>>> security patches).
>>>>> >
>>>>> >
>>>>> https://gitbox.apache.org/repos/asf?p=spark.git;a=shortlog;h=refs/heads/branch-2.2
>>>>> >
>>>>> > If it's okay and there is no further plan on `branch-2.2`, I want to
>>>>> volunteer to prepare the first RC (early next week?).
>>>>> >
>>>>> > Please let me know your opinions about this.
>>>>> >
>>>>> > Bests,
>>>>> > Dongjoon.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
>>>>>
>>>>>
>>>>
>>>> --
>>>> [image:
>>>> https://databricks.com/sparkaisummit/north-america?utm_source=email&utm_medium=signature]
>>>>
>>>
>
> --
> ---
> Takeshi Yamamuro
>
