Signature, hashes, LICENSE/NOTICE, source tarball looks OK. I built
for Hadoop 2.6 (-Pyarn -Phive -Phadoop-2.6) on Ubuntu from source and
tests pass. The release looks OK except that I'd like to resolve the
Blockers before giving a +1.
I'm seeing some test failures, and wanted to cross-check with others.
They're all in Hive. Some I think are due to Java 8 differences and
are just test issues; they expect an exact output from a query plan
and some HashSet ordering differences make it trivially different. If
so, I've seen this in the past and we could ignore it for now, but
would be good to get a second set of eyes. The trace is big so it's at
the end.
When rerunning with Java 7 I get a different error due to Hive version support:
- success sanity check *** FAILED ***
java.lang.RuntimeException: [download failed:
org.jboss.netty#netty;3.2.2.Final!netty.jar(bundle), download failed:
commons-net#commons-net;3.1!commons-net.jar]
at
org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:972)
at
org.apache.spark.sql.hive.client.IsolatedClientLoader$$anonfun$3.apply(IsolatedClientLoader.scala:62)
...
Hive / possible Java 8 test issue:
- windowing.q -- 20. testSTATs *** FAILED ***
Results do not match for windowing.q -- 20. testSTATs:
== Parsed Logical Plan ==
'WithWindowDefinition Map(w1 -> WindowSpecDefinition ROWS BETWEEN 2
PRECEDING AND 2 FOLLOWING)
'Project ['p_mfgr,'p_name,'p_size,UnresolvedWindowExpression
WindowSpecReference(w1)
UnresolvedWindowFunction stddev
UnresolvedAttribute [p_retailprice]
AS sdev#159481,UnresolvedWindowExpression WindowSpecReference(w1)
UnresolvedWindowFunction stddev_pop
UnresolvedAttribute [p_retailprice]
AS sdev_pop#159482,UnresolvedWindowExpression WindowSpecReference(w1)
UnresolvedWindowFunction collect_set
UnresolvedAttribute [p_size]
AS uniq_size#159483,UnresolvedWindowExpression WindowSpecReference(w1)
UnresolvedWindowFunction variance
UnresolvedAttribute [p_retailprice]
AS var#159484,UnresolvedWindowExpression WindowSpecReference(w1)
UnresolvedWindowFunction corr
UnresolvedAttribute [p_size]
UnresolvedAttribute [p_retailprice]
AS cor#159485,UnresolvedWindowExpression WindowSpecReference(w1)
UnresolvedWindowFunction covar_pop
UnresolvedAttribute [p_size]
UnresolvedAttribute [p_retailprice]
AS covarp#159486]
'UnresolvedRelation [part], None
== Analyzed Logical Plan ==
p_mfgr: string, p_name: string, p_size: int, sdev: double, sdev_pop:
double, uniq_size: array<int>, var: double, cor: double, covarp:
double
Project
[p_mfgr#159489,p_name#159488,p_size#159492,sdev#159481,sdev_pop#159482,uniq_size#159483,var#159484,cor#159485,covarp#159486]
Window [p_mfgr#159489,p_name#159488,p_size#159492,p_retailprice#159494],
[HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStd(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
sdev#159481,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStd(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
sdev_pop#159482,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCollectSet(p_size#159492)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
uniq_size#159483,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFVariance(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
var#159484,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCorrelation(p_size#159492,p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
cor#159485,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCovariance(p_size#159492,p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
covarp#159486], WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2
FOLLOWING
Project [p_mfgr#159489,p_name#159488,p_size#159492,p_retailprice#159494]
MetastoreRelation default, part, None
== Optimized Logical Plan ==
Project
[p_mfgr#159489,p_name#159488,p_size#159492,sdev#159481,sdev_pop#159482,uniq_size#159483,var#159484,cor#159485,covarp#159486]
Window [p_mfgr#159489,p_name#159488,p_size#159492,p_retailprice#159494],
[HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStd(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
sdev#159481,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStd(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
sdev_pop#159482,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCollectSet(p_size#159492)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
uniq_size#159483,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFVariance(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
var#159484,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCorrelation(p_size#159492,p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
cor#159485,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCovariance(p_size#159492,p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
covarp#159486], WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2
FOLLOWING
Project [p_mfgr#159489,p_name#159488,p_size#159492,p_retailprice#159494]
MetastoreRelation default, part, None
== Physical Plan ==
Project
[p_mfgr#159489,p_name#159488,p_size#159492,sdev#159481,sdev_pop#159482,uniq_size#159483,var#159484,cor#159485,covarp#159486]
Window [p_mfgr#159489,p_name#159488,p_size#159492,p_retailprice#159494],
[HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStd(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
sdev#159481,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFStd(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
sdev_pop#159482,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCollectSet(p_size#159492)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
uniq_size#159483,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFVariance(p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
var#159484,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCorrelation(p_size#159492,p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
cor#159485,HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCovariance(p_size#159492,p_retailprice#159494)
WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2 FOLLOWING AS
covarp#159486], WindowSpecDefinition ROWS BETWEEN 2 PRECEDING AND 2
FOLLOWING
Sort [p_mfgr#159489 ASC,p_mfgr#159489 ASC,p_name#159488 ASC], false
Exchange (HashPartitioning 2)
HiveTableScan
[p_mfgr#159489,p_name#159488,p_size#159492,p_retailprice#159494],
(MetastoreRelation default, part, None), None
...
On Tue, May 19, 2015 at 5:10 PM, Patrick Wendell <pwend...@gmail.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.4.0!
>
> The tag to be voted on is v1.4.0-rc1 (commit 777a081):
> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=777a08166f1fb144146ba32581d4632c3466541e
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-1.4.0-rc1/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1092/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-1.4.0-rc1-docs/
>
> Please vote on releasing this package as Apache Spark 1.4.0!
>
> The vote is open until Friday, May 22, at 17:03 UTC and passes
> if a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.4.0
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
> http://spark.apache.org/
>
> == How can I help test this release? ==
> If you are a Spark user, you can help us test this release by
> taking a Spark 1.3 workload and running on this release candidate,
> then reporting any regressions.
>
> == What justifies a -1 vote for this release? ==
> This vote is happening towards the end of the 1.4 QA period,
> so -1 votes should only occur for significant regressions from 1.3.1.
> Bugs already present in 1.3.X, minor regressions, or bugs related
> to new features will not block this release.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
> For additional commands, e-mail: dev-h...@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org
