Re: Welcome Jose Torres as a Spark committer

[Gengliang Wang]

Congrats Jose!


> 在 2019年1月31日，上午6:51，Bryan Cutler  写道：
> 
> Congrats Jose!
> 
> On Tue, Jan 29, 2019, 10:48 AM Shixiong Zhu   wrote:
> Hi all,
> 
> The Apache Spark PMC recently added Jose Torres as a committer on the 
> project. Jose has been a major contributor to Structured Streaming. Please 
> join me in welcoming him!
> 
> Best Regards,
> 
> Shixiong Zhu
> 

Re: CVE-2018-11760: Apache Spark local privilege escalation vulnerability

[Imran Rashid]

I received some questions about what the exact change was which fixed the
issue, and the PMC decided to post info in jira to make it easier for the
community to track.  The relevant details are all on

https://issues.apache.org/jira/browse/SPARK-26802

On Mon, Jan 28, 2019 at 1:08 PM Imran Rashid  wrote:

> Severity: Important
>
> Vendor: The Apache Software Foundation
>
> Versions affected:
> All Spark 1.x, Spark 2.0.x, and Spark 2.1.x versions
> Spark 2.2.0 to 2.2.2
> Spark 2.3.0 to 2.3.1
>
> Description:
> When using PySpark , it's possible for a different local user to connect
> to the Spark application and impersonate the user running the Spark
> application.  This affects versions 1.x, 2.0.x, 2.1.x, 2.2.0 to 2.2.2, and
> 2.3.0 to 2.3.1.
>
> Mitigation:
> 1.x, 2.0.x, 2.1.x, and 2.2.x users should upgrade to 2.2.3 or newer
> 2.3.x users should upgrade to 2.3.2 or newer
> Otherwise, affected users should avoid using PySpark in multi-user
> environments.
>
> Credit:
> This issue was reported by Luca Canali and Jose Carlos Luna Duran from
> CERN.
>
> References:
> https://spark.apache.org/security.html
>

Re: [VOTE] Release Apache Spark 2.3.3 (RC1)

[Takeshi Yamamuro]

Thanks for letting me know, Jungtaek!
OK, I'll start to test branch-2.3 by myself and prepare the RC2 package
this weekend.
I'll open the next vote then.

Best,
Takeshi

On Thu, Jan 31, 2019 at 8:54 AM Jungtaek Lim  wrote:

> Please proceed without SPARK-26154 given that it is unlikely expected to
> get merged in one week. The patch needs some more work, and we still
> haven't reached consensus on the approach.
>
> Btw, could one of committer justify and modify the priority and
> correctness label on SPARK-26154? I mentioned some committers before and
> got no response.
>
> 2019년 1월 29일 (화) 오전 10:23, Takeshi Yamamuro 님이 작성:
>
>> If there is no objection in following responses, I'll wait one more week
>> while watching that PR progress.
>> Once that PR merged, I'll start to prepare the next vote.
>>
>>
>>
>> On Tue, Jan 29, 2019 at 4:57 AM Jungtaek Lim  wrote:
>>
>>> Regarding PR 23634, it is waiting for getting consensus on the approach
>>> for the fix, as well as it also needs to have some time to clean up some
>>> code and move focus to concern backward compatibility. I'm postponing these
>>> works since I haven't reached consensus on the approach.
>>>
>>> So it may take some days or even some weeks to get PR 23634 merged (if
>>> consensus will not be made in time).
>>>
>>> 2019년 1월 29일 (화) 오전 2:18, Sean Owen 님이 작성:
>>>
 More analysis at https://github.com/apache/spark/pull/23634
 It's not a regression, though it does relate to correctness, although
 somewhat niche.
 TD, Jose et al, is this a Blocker? and is the fix probably reliable
 enough to commit now?

 On Mon, Jan 28, 2019 at 10:59 AM Sandeep Katta
  wrote:
 >
 > I feel this https://issues.apache.org/jira/browse/SPARK-26154 bug
 should be fixed in this release as it is related to data correctness
 >
 > On Mon, 28 Jan 2019 at 17:55, Takeshi Yamamuro 
 wrote:
 >>
 >> Hi, all
 >>
 >> I checked the two issues below had been resolved and there is no
 blocker for branch-2.3 now, so I'll start prepare RC2 tomorrow.
 >> https://issues.apache.org/jira/browse/SPARK-26682
 >> https://issues.apache.org/jira/browse/SPARK-26709
 >>
 >> If there are some blockers and critical issues in branch-2.3, please
 let me know.
 >>
 >> Best,
 >> Takeshi
 >>
 >> On Thu, Jan 24, 2019 at 10:06 AM Takeshi Yamamuro <
 linguin@gmail.com> wrote:
 >>>
 >>> Thanks, all.
 >>>
 >>> I'll start a new vote as rc2 after the two issues above resolved.
 >>>
 >>> Best,
 >>> Takeshi
 >>>
 >>>
 >>> On Thu, Jan 24, 2019 at 7:59 AM Xiao Li 
 wrote:
 
  -1
 
  https://issues.apache.org/jira/browse/SPARK-26709 is another
 blocker ticket that returns incorrect results.
 
 
  Marcelo Vanzin  于2019年1月23日周三
 下午12:01写道：
 >
 > -1 too.
 >
 > I just upgraded https://issues.apache.org/jira/browse/SPARK-26682
 to
 > blocker. It's a small fix and we should make it in 2.3.3.
 >
 > On Thu, Jan 17, 2019 at 6:49 PM Takeshi Yamamuro <
 linguin@gmail.com> wrote:
 > >
 > > Please vote on releasing the following candidate as Apache
 Spark version 2.3.3.
 > >
 > > The vote is open until January 20 8:00PM (PST) and passes if a
 majority +1 PMC votes are cast, with
 > > a minimum of 3 +1 votes.
 > >
 > > [ ] +1 Release this package as Apache Spark 2.3.3
 > > [ ] -1 Do not release this package because ...
 > >
 > > To learn more about Apache Spark, please see
 http://spark.apache.org/
 > >
 > > The tag to be voted on is v2.3.3-rc1 (commit
 b5ea9330e3072e99841270b10dc1d2248127064b):
 > > https://github.com/apache/spark/tree/v2.3.3-rc1
 > >
 > > The release files, including signatures, digests, etc. can be
 found at:
 > > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-bin/
 > >
 > > Signatures used for Spark RCs can be found in this file:
 > > https://dist.apache.org/repos/dist/dev/spark/KEYS
 > >
 > > The staging repository for this release can be found at:
 > >
 https://repository.apache.org/content/repositories/orgapachespark-1297
 > >
 > > The documentation corresponding to this release can be found at:
 > > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-docs/
 > >
 > > The list of bug fixes going into 2.3.3 can be found at the
 following URL:
 > > https://issues.apache.org/jira/projects/SPARK/versions/12343759
 > >
 > > FAQ
 > >
 > > =
 > > How can I help test this release?
 > > =
 > >
 > > If you are a Spark user, you can help us test this releas