Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
Thanks for the check, Felix! Yea, I'll wait for the new test report. But, it never took so much time to run tests in branch-2.3 before, ... network issue? On Wed, Jan 23, 2019 at 2:19 AM Felix Cheung wrote: > I’ve tried a couple of times. The latest test run took 12 hr+ > > 1 aborted suite: > 00:53:25.769 WARN > org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to > download Spark 2.3.2 from > http://mirrors.koehn.com/apache//spark/spark-2.3.2/spark-2.3.2-bin-hadoop2.7.tgz: > Error writing to server > 00:53:25.812 WARN > org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to > download Spark 2.3.2 from > http://mirror.cc.columbia.edu/pub/software/apache//spark/spark-2.3.2/spark-2.3.2-bin-hadoop2.7.tgz: > Error writing to server > 00:53:25.838 WARN > org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to > download Spark 2.3.2 from > https://archive.apache.org/dist/spark/spark-2.3.2/spark-2.3.2-bin-hadoop2.7.tgz: > Socket closed > > org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite *** ABORTED *** > Exception encountered when invoking run on a nested suite - Unable to > download Spark 2.3.2 (HiveExternalCatalogVersionsSuite.scala:97) > > And then it stopped. I checked this morning the archive link should be > valid. Try to see if I can try again/resume from it. > > > -- > *From:* Takeshi Yamamuro > *Sent:* Sunday, January 20, 2019 6:45 PM > *To:* Sean Owen > *Cc:* Spark dev list > *Subject:* Re: [VOTE] Release Apache Spark 2.3.3 (RC1) > > Oh, sorry for that and I misunderstood the Apache release policy. > Yea, its ok to keep the RC1 voting. > > Best, > Takeshi > > On Mon, Jan 21, 2019 at 11:07 AM Sean Owen wrote: > >> OK, if it passes tests, I'm +1 on the release. >> Can anyone else verify the tests pass? >> >> What is the reason for a new RC? I didn't see any other issues reported. >> >> On Sun, Jan 20, 2019 at 8:03 PM Takeshi Yamamuro >> wrote: >> > >> > Hi, all >> > >> > Thanks for the checks, Sean and Felix. >> > I'll start the next vote as RC2 this Tuesday noon (PST). >> > >> > > Sean >> > I re-run JavaTfIdfSuite on my env and it passed. >> > I used `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos >> -Psparkr` and >> > run the tests on a EC2 instance below (I launched the new instance for >> the tests); >> > >> > $ cat /etc/os-release >> > NAME="Amazon Linux" >> > VERSION="2" >> > ID="amzn" >> > ID_LIKE="centos rhel fedora" >> > VERSION_ID="2" >> > PRETTY_NAME="Amazon Linux 2" >> > ANSI_COLOR="0;33" >> > CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" >> > HOME_URL="https://amazonlinux.com/"; >> > $ java -version >> > openjdk version "1.8.0_191" >> > OpenJDK Runtime Environment (build 1.8.0_191-b12) >> > OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode) >> > >> > >> > >> > >> > On Mon, Jan 21, 2019 at 9:53 AM Felix Cheung >> wrote: >> >> >> >> +1 >> >> >> >> My focus is on R (sorry couldn’t cross validate what’s Sean is seeing) >> >> >> >> tested: >> >> reviewed doc >> >> R package test >> >> win-builder, r-hub >> >> Tarball/package signature >> >> >> >> >> >> >> >> >> >> From: Takeshi Yamamuro >> >> Sent: Thursday, January 17, 2019 6:49 PM >> >> To: Spark dev list >> >> Subject: [VOTE] Release Apache Spark 2.3.3 (RC1) >> >> >> >> Please vote on releasing the following candidate as Apache Spark >> version 2.3.3. >> >> >> >> The vote is open until January 20 8:00PM (PST) and passes if a >> majority +1 PMC votes are cast, with >> >> a minimum of 3 +1 votes. >> >> >> >> [ ] +1 Release this package as Apache Spark 2.3.3 >> >> [ ] -1 Do not release this package because ... >> >> >> >> To learn more about Apache Spark, please see http://spark.apache.org/ >> >> >> >> The tag to be voted on is v2.3.3-rc1 (commit >> b5ea9330e3072e99841270b10dc1d2248127064b): >> >> https://github.com/apache/spark/tree/v2.3.3-rc1 >> >> >> >> The release files, including signatures, digests, etc. can be found at: >> >> https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-bin/ >> >> >> >> Signatures used for Spark RCs can be found in this file: >> >> https://dist.apache.org/repos/dist/dev/spark/KEYS >> >> >> >> The staging repository for this release can be found at: >> >> https://repository.apache.org/content/repositories/orgapachespark-1297 >> >> >> >> The documentation corresponding to this release can be found at: >> >> https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-docs/ >> >> >> >> The list of bug fixes going into 2.3.3 can be found at the following >> URL: >> >> https://issues.apache.org/jira/projects/SPARK/versions/12343759 >> >> >> >> FAQ >> >> >> >> = >> >> How can I help test this release? >> >> = >> >> >> >> If you are a Spark user, you can help us test this release by taking >> >> an existing Spark workload and running on this release candidate, then >> >> reporting any regressions. >> >> >> >> If you're working in PySpark
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
Recently, I came across this bug: https://issues.apache.org/jira/browse/SPARK-26706. It seems appropriate to include it in 2.3.3, doesn't it? Thanks, Anton ср, 23 янв. 2019 г. в 13:08, Takeshi Yamamuro : > Thanks for the check, Felix! > > Yea, I'll wait for the new test report. > But, it never took so much time to run tests in branch-2.3 before, ... > network issue? > > On Wed, Jan 23, 2019 at 2:19 AM Felix Cheung > wrote: > >> I’ve tried a couple of times. The latest test run took 12 hr+ >> >> 1 aborted suite: >> 00:53:25.769 WARN >> org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to >> download Spark 2.3.2 from >> http://mirrors.koehn.com/apache//spark/spark-2.3.2/spark-2.3.2-bin-hadoop2.7.tgz: >> Error writing to server >> 00:53:25.812 WARN >> org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to >> download Spark 2.3.2 from >> http://mirror.cc.columbia.edu/pub/software/apache//spark/spark-2.3.2/spark-2.3.2-bin-hadoop2.7.tgz: >> Error writing to server >> 00:53:25.838 WARN >> org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite: Failed to >> download Spark 2.3.2 from >> https://archive.apache.org/dist/spark/spark-2.3.2/spark-2.3.2-bin-hadoop2.7.tgz: >> Socket closed >> >> org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite *** ABORTED *** >> Exception encountered when invoking run on a nested suite - Unable to >> download Spark 2.3.2 (HiveExternalCatalogVersionsSuite.scala:97) >> >> And then it stopped. I checked this morning the archive link should be >> valid. Try to see if I can try again/resume from it. >> >> >> -- >> *From:* Takeshi Yamamuro >> *Sent:* Sunday, January 20, 2019 6:45 PM >> *To:* Sean Owen >> *Cc:* Spark dev list >> *Subject:* Re: [VOTE] Release Apache Spark 2.3.3 (RC1) >> >> Oh, sorry for that and I misunderstood the Apache release policy. >> Yea, its ok to keep the RC1 voting. >> >> Best, >> Takeshi >> >> On Mon, Jan 21, 2019 at 11:07 AM Sean Owen wrote: >> >>> OK, if it passes tests, I'm +1 on the release. >>> Can anyone else verify the tests pass? >>> >>> What is the reason for a new RC? I didn't see any other issues reported. >>> >>> On Sun, Jan 20, 2019 at 8:03 PM Takeshi Yamamuro >>> wrote: >>> > >>> > Hi, all >>> > >>> > Thanks for the checks, Sean and Felix. >>> > I'll start the next vote as RC2 this Tuesday noon (PST). >>> > >>> > > Sean >>> > I re-run JavaTfIdfSuite on my env and it passed. >>> > I used `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos >>> -Psparkr` and >>> > run the tests on a EC2 instance below (I launched the new instance for >>> the tests); >>> > >>> > $ cat /etc/os-release >>> > NAME="Amazon Linux" >>> > VERSION="2" >>> > ID="amzn" >>> > ID_LIKE="centos rhel fedora" >>> > VERSION_ID="2" >>> > PRETTY_NAME="Amazon Linux 2" >>> > ANSI_COLOR="0;33" >>> > CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" >>> > HOME_URL="https://amazonlinux.com/"; >>> > $ java -version >>> > openjdk version "1.8.0_191" >>> > OpenJDK Runtime Environment (build 1.8.0_191-b12) >>> > OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode) >>> > >>> > >>> > >>> > >>> > On Mon, Jan 21, 2019 at 9:53 AM Felix Cheung < >>> felixcheun...@hotmail.com> wrote: >>> >> >>> >> +1 >>> >> >>> >> My focus is on R (sorry couldn’t cross validate what’s Sean is seeing) >>> >> >>> >> tested: >>> >> reviewed doc >>> >> R package test >>> >> win-builder, r-hub >>> >> Tarball/package signature >>> >> >>> >> >>> >> >>> >> >>> >> From: Takeshi Yamamuro >>> >> Sent: Thursday, January 17, 2019 6:49 PM >>> >> To: Spark dev list >>> >> Subject: [VOTE] Release Apache Spark 2.3.3 (RC1) >>> >> >>> >> Please vote on releasing the following candidate as Apache Spark >>> version 2.3.3. >>> >> >>> >> The vote is open until January 20 8:00PM (PST) and passes if a >>> majority +1 PMC votes are cast, with >>> >> a minimum of 3 +1 votes. >>> >> >>> >> [ ] +1 Release this package as Apache Spark 2.3.3 >>> >> [ ] -1 Do not release this package because ... >>> >> >>> >> To learn more about Apache Spark, please see http://spark.apache.org/ >>> >> >>> >> The tag to be voted on is v2.3.3-rc1 (commit >>> b5ea9330e3072e99841270b10dc1d2248127064b): >>> >> https://github.com/apache/spark/tree/v2.3.3-rc1 >>> >> >>> >> The release files, including signatures, digests, etc. can be found >>> at: >>> >> https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-bin/ >>> >> >>> >> Signatures used for Spark RCs can be found in this file: >>> >> https://dist.apache.org/repos/dist/dev/spark/KEYS >>> >> >>> >> The staging repository for this release can be found at: >>> >> >>> https://repository.apache.org/content/repositories/orgapachespark-1297 >>> >> >>> >> The documentation corresponding to this release can be found at: >>> >> https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-docs/ >>> >> >>> >> The list of bug fixes going into 2.3.3 can be found at the following >>> URL: >>> >> https://issues.apache.org/jira/
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
I'm not clear if it's a correctness bug from that description, and if it's not a regression, no it does not need to go into 2.3.3. If it's a real bug, sure it can be merged to 2.3.x. On Wed, Jan 23, 2019 at 7:54 AM Anton Okolnychyi wrote: > > Recently, I came across this bug: > https://issues.apache.org/jira/browse/SPARK-26706. > > It seems appropriate to include it in 2.3.3, doesn't it? > > Thanks, > Anton > - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Custom Window Function
Hi, I want to write custom window functions in spark which are also optimisable for catalyst. Can you provide some hints where to start? Also posting to DEVLIST as I believe this is a rather exotic topic. Best, Georg
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
It is a correctness bug. I have updated the description with an example. It has been there for a while, so I am not sure about the priority. ср, 23 янв. 2019 г. в 14:48, Sean Owen : > I'm not clear if it's a correctness bug from that description, and if > it's not a regression, no it does not need to go into 2.3.3. If it's a > real bug, sure it can be merged to 2.3.x. > > On Wed, Jan 23, 2019 at 7:54 AM Anton Okolnychyi > wrote: > > > > Recently, I came across this bug: > https://issues.apache.org/jira/browse/SPARK-26706. > > > > It seems appropriate to include it in 2.3.3, doesn't it? > > > > Thanks, > > Anton > > >
Re: Custom Window Function
Hi Herman, Thanks a lot. So far I only found most of the documentation about UDAF. Could you point me anywhere (besides just reading spark's source code) which explains how to work with custom AggregateFunctions? Best, Georg Am Mi., 23. Jan. 2019 um 16:02 Uhr schrieb Herman van Hovell < her...@databricks.com>: > Hi Georg, > > In most cases you want to implement an aggregate function. You can either > define a UDAF, or an AggregateFunction. The latter is an internal > expression, so it is faster but we might break it across versions of spark. > > Hope this helps. > > Cheers, > Herman > > On Wed, Jan 23, 2019 at 3:55 PM Georg Heiler > wrote: > >> Hi, >> >> I want to write custom window functions in spark which are also >> optimisable for catalyst. >> Can you provide some hints where to start? >> >> Also posting to DEVLIST as I believe this is a rather exotic topic. >> >> Best, >> Georg >> >
Re: Removing old HiveMetastore(0.12~0.14) from Spark 3.0.0?
Got it. Thank you for sharing that, Reynold. So, you mean they will use `Apache Spark 3.0.0` on the old clusters with Hive 0.x, right? If that happens actually, no problem to keep them. Bests, Dongjoon. On Tue, Jan 22, 2019 at 11:49 PM Xiao Li wrote: > Based on my experience in development of Spark SQL, the maintenance cost > is very small for supporting different versions of Hive metastore. Feel > free to ping me if we hit any issue about it. > > Cheers, > > Xiao > > Reynold Xin 于2019年1月22日周二 下午11:18写道： > >> Actually a non trivial fraction of users / customers I interact with >> still use very old Hive metastores. Because it’s very difficult to upgrade >> Hive metastore wholesale (it’d require all the production jobs that access >> the same metastore be upgraded at once). This is even harder than JVM >> upgrade which can be done on a per job basis, or OS upgrade that can be >> done on a per machine basis. >> >> Is there high maintenance cost with keeping these? My understanding is >> that Michael did a good job initially with classloader isolation and >> modular design that they are very easy to maintain. >> >> On Jan 22, 2019, at 11:13 PM, Hyukjin Kwon wrote: >> >> Yea, I was thinking about that too. They are too old to keep. +1 for >> removing them out. >> >> 2019년 1월 23일 (수) 오전 11:30, Dongjoon Hyun 님이 작성: >> >>> Hi, All. >>> >>> Currently, Apache Spark supports Hive Metastore(HMS) 0.12 ~ 2.3. >>> Among them, HMS 0.x releases look very old since we are in 2019. >>> If these are not used in the production any more, can we drop HMS 0.x >>> supports in 3.0.0? >>> >>> hive-0.12.0 2013-10-10 >>> hive-0.13.0 2014-04-15 >>> hive-0.13.1 2014-11-16 >>> hive-0.14.0 2014-11-16 >>> ( https://archive.apache.org/dist/hive/ ) >>> >>> In addition, if there is someone who is still using these HMS versions >>> and has a plan to install and use Spark 3.0.0 with these HMS versions, >>> could you reply this email thread? If there is a reason, that would be very >>> helpful for me. >>> >>> Thanks, >>> Dongjoon. >>> >>
Re: Removing old HiveMetastore(0.12~0.14) from Spark 3.0.0?
It is not even an old “cluster”. It is a central metastore shares by multiple clusters. On Wed, Jan 23, 2019 at 10:04 AM Dongjoon Hyun wrote: > Got it. Thank you for sharing that, Reynold. > > So, you mean they will use `Apache Spark 3.0.0` on the old clusters with > Hive 0.x, right? > > If that happens actually, no problem to keep them. > > Bests, > Dongjoon. > > > On Tue, Jan 22, 2019 at 11:49 PM Xiao Li wrote: > >> Based on my experience in development of Spark SQL, the maintenance cost >> is very small for supporting different versions of Hive metastore. Feel >> free to ping me if we hit any issue about it. >> >> Cheers, >> >> Xiao >> >> Reynold Xin 于2019年1月22日周二 下午11:18写道： >> >>> Actually a non trivial fraction of users / customers I interact with >>> still use very old Hive metastores. Because it’s very difficult to upgrade >>> Hive metastore wholesale (it’d require all the production jobs that access >>> the same metastore be upgraded at once). This is even harder than JVM >>> upgrade which can be done on a per job basis, or OS upgrade that can be >>> done on a per machine basis. >>> >>> Is there high maintenance cost with keeping these? My understanding is >>> that Michael did a good job initially with classloader isolation and >>> modular design that they are very easy to maintain. >>> >>> On Jan 22, 2019, at 11:13 PM, Hyukjin Kwon wrote: >>> >>> Yea, I was thinking about that too. They are too old to keep. +1 for >>> removing them out. >>> >>> 2019년 1월 23일 (수) 오전 11:30, Dongjoon Hyun 님이 작성: >>> Hi, All. Currently, Apache Spark supports Hive Metastore(HMS) 0.12 ~ 2.3. Among them, HMS 0.x releases look very old since we are in 2019. If these are not used in the production any more, can we drop HMS 0.x supports in 3.0.0? hive-0.12.0 2013-10-10 hive-0.13.0 2014-04-15 hive-0.13.1 2014-11-16 hive-0.14.0 2014-11-16 ( https://archive.apache.org/dist/hive/ ) In addition, if there is someone who is still using these HMS versions and has a plan to install and use Spark 3.0.0 with these HMS versions, could you reply this email thread? If there is a reason, that would be very helpful for me. Thanks, Dongjoon. >>>
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
-1 Agreed with Anton that this bug will potentially corrupt the data silently. As he is ready to submit a PR, I'll suggest to wait to include the fix. Thanks! Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 0x5CED8B896A6BDFA0 On Wed, Jan 23, 2019 at 7:10 AM Anton Okolnychyi wrote: > > It is a correctness bug. I have updated the description with an example. It > has been there for a while, so I am not sure about the priority. > > ср, 23 янв. 2019 г. в 14:48, Sean Owen : >> >> I'm not clear if it's a correctness bug from that description, and if >> it's not a regression, no it does not need to go into 2.3.3. If it's a >> real bug, sure it can be merged to 2.3.x. >> >> On Wed, Jan 23, 2019 at 7:54 AM Anton Okolnychyi >> wrote: >> > >> > Recently, I came across this bug: >> > https://issues.apache.org/jira/browse/SPARK-26706. >> > >> > It seems appropriate to include it in 2.3.3, doesn't it? >> > >> > Thanks, >> > Anton >> > - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
-1 too. I just upgraded https://issues.apache.org/jira/browse/SPARK-26682 to blocker. It's a small fix and we should make it in 2.3.3. On Thu, Jan 17, 2019 at 6:49 PM Takeshi Yamamuro wrote: > > Please vote on releasing the following candidate as Apache Spark version > 2.3.3. > > The vote is open until January 20 8:00PM (PST) and passes if a majority +1 > PMC votes are cast, with > a minimum of 3 +1 votes. > > [ ] +1 Release this package as Apache Spark 2.3.3 > [ ] -1 Do not release this package because ... > > To learn more about Apache Spark, please see http://spark.apache.org/ > > The tag to be voted on is v2.3.3-rc1 (commit > b5ea9330e3072e99841270b10dc1d2248127064b): > https://github.com/apache/spark/tree/v2.3.3-rc1 > > The release files, including signatures, digests, etc. can be found at: > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-bin/ > > Signatures used for Spark RCs can be found in this file: > https://dist.apache.org/repos/dist/dev/spark/KEYS > > The staging repository for this release can be found at: > https://repository.apache.org/content/repositories/orgapachespark-1297 > > The documentation corresponding to this release can be found at: > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-docs/ > > The list of bug fixes going into 2.3.3 can be found at the following URL: > https://issues.apache.org/jira/projects/SPARK/versions/12343759 > > FAQ > > = > How can I help test this release? > = > > If you are a Spark user, you can help us test this release by taking > an existing Spark workload and running on this release candidate, then > reporting any regressions. > > If you're working in PySpark you can set up a virtual env and install > the current RC and see if anything important breaks, in the Java/Scala > you can add the staging repository to your projects resolvers and test > with the RC (make sure to clean up the artifact cache before/after so > you don't end up building with a out of date RC going forward). > > === > What should happen to JIRA tickets still targeting 2.3.3? > === > > The current list of open tickets targeted at 2.3.3 can be found at: > https://issues.apache.org/jira/projects/SPARK and search for "Target > Version/s" = 2.3.3 > > Committers should look at those and triage. Extremely important bug > fixes, documentation, and API tweaks that impact compatibility should > be worked on immediately. Everything else please retarget to an > appropriate release. > > == > But my bug isn't fixed? > == > > In order to make timely releases, we will typically not hold the > release unless the bug in question is a regression from the previous > release. That being said, if there is something which is a regression > that has not been correctly targeted please ping me or a committer to > help target the issue. > > -- > --- > Takeshi Yamamuro -- Marcelo - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
-1 https://issues.apache.org/jira/browse/SPARK-26709 is another blocker ticket that returns incorrect results. Marcelo Vanzin 于2019年1月23日周三 下午12:01写道： > -1 too. > > I just upgraded https://issues.apache.org/jira/browse/SPARK-26682 to > blocker. It's a small fix and we should make it in 2.3.3. > > On Thu, Jan 17, 2019 at 6:49 PM Takeshi Yamamuro > wrote: > > > > Please vote on releasing the following candidate as Apache Spark version > 2.3.3. > > > > The vote is open until January 20 8:00PM (PST) and passes if a majority > +1 PMC votes are cast, with > > a minimum of 3 +1 votes. > > > > [ ] +1 Release this package as Apache Spark 2.3.3 > > [ ] -1 Do not release this package because ... > > > > To learn more about Apache Spark, please see http://spark.apache.org/ > > > > The tag to be voted on is v2.3.3-rc1 (commit > b5ea9330e3072e99841270b10dc1d2248127064b): > > https://github.com/apache/spark/tree/v2.3.3-rc1 > > > > The release files, including signatures, digests, etc. can be found at: > > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-bin/ > > > > Signatures used for Spark RCs can be found in this file: > > https://dist.apache.org/repos/dist/dev/spark/KEYS > > > > The staging repository for this release can be found at: > > https://repository.apache.org/content/repositories/orgapachespark-1297 > > > > The documentation corresponding to this release can be found at: > > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-docs/ > > > > The list of bug fixes going into 2.3.3 can be found at the following URL: > > https://issues.apache.org/jira/projects/SPARK/versions/12343759 > > > > FAQ > > > > = > > How can I help test this release? > > = > > > > If you are a Spark user, you can help us test this release by taking > > an existing Spark workload and running on this release candidate, then > > reporting any regressions. > > > > If you're working in PySpark you can set up a virtual env and install > > the current RC and see if anything important breaks, in the Java/Scala > > you can add the staging repository to your projects resolvers and test > > with the RC (make sure to clean up the artifact cache before/after so > > you don't end up building with a out of date RC going forward). > > > > === > > What should happen to JIRA tickets still targeting 2.3.3? > > === > > > > The current list of open tickets targeted at 2.3.3 can be found at: > > https://issues.apache.org/jira/projects/SPARK and search for "Target > Version/s" = 2.3.3 > > > > Committers should look at those and triage. Extremely important bug > > fixes, documentation, and API tweaks that impact compatibility should > > be worked on immediately. Everything else please retarget to an > > appropriate release. > > > > == > > But my bug isn't fixed? > > == > > > > In order to make timely releases, we will typically not hold the > > release unless the bug in question is a regression from the previous > > release. That being said, if there is something which is a regression > > that has not been correctly targeted please ping me or a committer to > > help target the issue. > > > > -- > > --- > > Takeshi Yamamuro > > > > -- > Marcelo > > - > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
Re: [VOTE] Release Apache Spark 2.3.3 (RC1)
Thanks, all. I'll start a new vote as rc2 after the two issues above resolved. Best, Takeshi On Thu, Jan 24, 2019 at 7:59 AM Xiao Li wrote: > -1 > > https://issues.apache.org/jira/browse/SPARK-26709 is another blocker > ticket that returns incorrect results. > > > Marcelo Vanzin 于2019年1月23日周三 下午12:01写道： > >> -1 too. >> >> I just upgraded https://issues.apache.org/jira/browse/SPARK-26682 to >> blocker. It's a small fix and we should make it in 2.3.3. >> >> On Thu, Jan 17, 2019 at 6:49 PM Takeshi Yamamuro >> wrote: >> > >> > Please vote on releasing the following candidate as Apache Spark >> version 2.3.3. >> > >> > The vote is open until January 20 8:00PM (PST) and passes if a majority >> +1 PMC votes are cast, with >> > a minimum of 3 +1 votes. >> > >> > [ ] +1 Release this package as Apache Spark 2.3.3 >> > [ ] -1 Do not release this package because ... >> > >> > To learn more about Apache Spark, please see http://spark.apache.org/ >> > >> > The tag to be voted on is v2.3.3-rc1 (commit >> b5ea9330e3072e99841270b10dc1d2248127064b): >> > https://github.com/apache/spark/tree/v2.3.3-rc1 >> > >> > The release files, including signatures, digests, etc. can be found at: >> > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-bin/ >> > >> > Signatures used for Spark RCs can be found in this file: >> > https://dist.apache.org/repos/dist/dev/spark/KEYS >> > >> > The staging repository for this release can be found at: >> > https://repository.apache.org/content/repositories/orgapachespark-1297 >> > >> > The documentation corresponding to this release can be found at: >> > https://dist.apache.org/repos/dist/dev/spark/v2.3.3-rc1-docs/ >> > >> > The list of bug fixes going into 2.3.3 can be found at the following >> URL: >> > https://issues.apache.org/jira/projects/SPARK/versions/12343759 >> > >> > FAQ >> > >> > = >> > How can I help test this release? >> > = >> > >> > If you are a Spark user, you can help us test this release by taking >> > an existing Spark workload and running on this release candidate, then >> > reporting any regressions. >> > >> > If you're working in PySpark you can set up a virtual env and install >> > the current RC and see if anything important breaks, in the Java/Scala >> > you can add the staging repository to your projects resolvers and test >> > with the RC (make sure to clean up the artifact cache before/after so >> > you don't end up building with a out of date RC going forward). >> > >> > === >> > What should happen to JIRA tickets still targeting 2.3.3? >> > === >> > >> > The current list of open tickets targeted at 2.3.3 can be found at: >> > https://issues.apache.org/jira/projects/SPARK and search for "Target >> Version/s" = 2.3.3 >> > >> > Committers should look at those and triage. Extremely important bug >> > fixes, documentation, and API tweaks that impact compatibility should >> > be worked on immediately. Everything else please retarget to an >> > appropriate release. >> > >> > == >> > But my bug isn't fixed? >> > == >> > >> > In order to make timely releases, we will typically not hold the >> > release unless the bug in question is a regression from the previous >> > release. That being said, if there is something which is a regression >> > that has not been correctly targeted please ping me or a committer to >> > help target the issue. >> > >> > -- >> > --- >> > Takeshi Yamamuro >> >> >> >> -- >> Marcelo >> >> - >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >> -- --- Takeshi Yamamuro
