GitHub user djvulee opened a pull request:
https://github.com/apache/spark/pull/3566
Branch 1.1 typo error in HistoryServer
There is a typo error in the 167 & 168 line in HistoryServer.scala file.
The "./sbin/spark-history-server.sh " should be
"./sbin/start-history-server.sh "
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/spark branch-1.1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3566.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3566
----
commit 9a62cf3655dcab49b5c0f94ad094603eaf288251
Author: Michael Armbrust <[email protected]>
Date: 2014-08-27T22:14:08Z
[SPARK-3235][SQL] Ensure in-memory tables don't always broadcast.
Author: Michael Armbrust <[email protected]>
Closes #2147 from marmbrus/inMemDefaultSize and squashes the following
commits:
5390360 [Michael Armbrust] Merge remote-tracking branch 'origin/master'
into inMemDefaultSize
14204d3 [Michael Armbrust] Set the context before creating
SparkLogicalPlans.
8da4414 [Michael Armbrust] Make sure we throw errors when leaf nodes fail
to provide statistcs
18ce029 [Michael Armbrust] Ensure in-memory tables don't always broadcast.
(cherry picked from commit 7d2a7a91f263bb9fbf24dc4dbffde8fe5e2c7442)
Signed-off-by: Michael Armbrust <[email protected]>
commit 0c03fb621e5b080f24863cfc17032bd828b65b99
Author: Patrick Wendell <[email protected]>
Date: 2014-08-27T22:48:00Z
Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 9af3fb7385d1f9f221962f1d2d725ff79bd82033.
commit 0b17c7d4f2176f0c0e8aaab95e034be54467ff30
Author: Patrick Wendell <[email protected]>
Date: 2014-08-27T22:48:13Z
Revert "[maven-release-plugin] prepare release v1.1.0-snapshot2"
This reverts commit e1535ad3c6f7400f2b7915ea91da9c60510557ba.
commit d4cf7a068da099f0f07f04a834d7edf6b743ceb3
Author: Matthew Farrellee <[email protected]>
Date: 2014-08-27T22:50:30Z
Add line continuation for script to work w/ py2.7.5
Error was -
$ SPARK_HOME=$PWD/dist ./dev/create-release/generate-changelist.py
File "./dev/create-release/generate-changelist.py", line 128
if day < SPARK_REPO_CHANGE_DATE1 or
^
SyntaxError: invalid syntax
Author: Matthew Farrellee <[email protected]>
Closes #2139 from mattf/master-fix-generate-changelist.py-0 and squashes
the following commits:
6b3a900 [Matthew Farrellee] Add line continuation for script to work w/
py2.7.5
(cherry picked from commit 64d8ecbbe94c47236ff2d8c94d7401636ba6fca4)
Signed-off-by: Patrick Wendell <[email protected]>
commit 8597e9cf356b0d8e17600a49efc4c4a0356ecb5d
Author: Patrick Wendell <[email protected]>
Date: 2014-08-27T22:55:59Z
BUILD: Updating CHANGES.txt for Spark 1.1
commit 58b0be6a29eab817d350729710345e9f39e4c506
Author: Patrick Wendell <[email protected]>
Date: 2014-08-27T23:28:08Z
[maven-release-plugin] prepare release v1.1.0-rc1
commit 78e3c036eee7113b2ed144eec5061e070b479e56
Author: Patrick Wendell <[email protected]>
Date: 2014-08-27T23:28:27Z
[maven-release-plugin] prepare for next development iteration
commit 54ccd93e621c1bc4afc709a208b609232ab701d1
Author: Andrew Or <[email protected]>
Date: 2014-08-28T06:03:46Z
[HOTFIX] Wait for EOF only for the PySpark shell
In `SparkSubmitDriverBootstrapper`, we wait for the parent process to send
us an `EOF` before finishing the application. This is applicable for the
PySpark shell because we terminate the application the same way. However if we
run a python application, for instance, the JVM actually never exits unless it
receives a manual EOF from the user. This is causing a few tests to timeout.
We only need to do this for the PySpark shell because Spark submit runs as
a python subprocess only in this case. Thus, the normal Spark shell doesn't
need to go through this case even though it is also a REPL.
Thanks davies for reporting this.
Author: Andrew Or <[email protected]>
Closes #2170 from andrewor14/bootstrap-hotfix and squashes the following
commits:
42963f5 [Andrew Or] Do not wait for EOF unless this is the pyspark shell
(cherry picked from commit dafe343499bbc688e266106e4bb897f9e619834e)
Signed-off-by: Patrick Wendell <[email protected]>
commit 233c283e3d946bdcbf418375122c5763559c0119
Author: Michael Armbrust <[email protected]>
Date: 2014-08-28T06:05:34Z
[HOTFIX][SQL] Remove cleaning of UDFs
It is not safe to run the closure cleaner on slaves. #2153 introduced this
which broke all UDF execution on slaves. Will re-add cleaning of UDF closures
in a follow-up PR.
Author: Michael Armbrust <[email protected]>
Closes #2174 from marmbrus/fixUdfs and squashes the following commits:
55406de [Michael Armbrust] [HOTFIX] Remove cleaning of UDFs
(cherry picked from commit 024178c57419f915d26414e1b91ea0019c3650db)
Signed-off-by: Patrick Wendell <[email protected]>
commit ad0fab23d810f0f62e71813edfc3101fcec5e40b
Author: Patrick Wendell <[email protected]>
Date: 2014-08-27T22:40:40Z
HOTFIX: Don't build with YARN support for Mapr3
commit 79e86ef3e1a3ee03a7e3b166a5c7dee11c6d60d7
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T06:45:54Z
[maven-release-plugin] prepare release v1.1.0-rc1
commit a118ea5c59d653f5a3feda21455ba60bc722b3b1
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T06:46:02Z
[maven-release-plugin] prepare for next development iteration
commit c0e3bc17ede69f792cd96556bb3d8acd98b239ce
Author: Cheng Lian <[email protected]>
Date: 2014-08-28T07:08:09Z
[SQL] Fixed 2 comment typos in SQLConf
Author: Cheng Lian <[email protected]>
Closes #2172 from liancheng/sqlconf-typo and squashes the following commits:
115cc71 [Cheng Lian] Fixed 2 comment typos in SQLConf
(cherry picked from commit 68f75dcdfe7e8ab229b73824692c4b3d4c39946c)
Signed-off-by: Michael Armbrust <[email protected]>
commit 2e8ad9976c10a5a7fdefba9b612fc61e48f2b6cc
Author: Michael Armbrust <[email protected]>
Date: 2014-08-28T07:15:23Z
[SPARK-3230][SQL] Fix udfs that return structs
We need to convert the case classes into Rows.
Author: Michael Armbrust <[email protected]>
Closes #2133 from marmbrus/structUdfs and squashes the following commits:
189722f [Michael Armbrust] Merge remote-tracking branch 'origin/master'
into structUdfs
8e29b1c [Michael Armbrust] Use existing function
d8d0b76 [Michael Armbrust] Fix udfs that return structs
(cherry picked from commit 76e3ba4264c4a0bc2c33ae6ac862fc40bc302d83)
Signed-off-by: Michael Armbrust <[email protected]>
commit 71ec0140f7e121bdba3d19e8219e91a5e9d1e320
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T07:15:26Z
Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit a118ea5c59d653f5a3feda21455ba60bc722b3b1.
commit 56070f12f455bae645cba887a74c72b12f1085f8
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T07:15:30Z
Revert "[maven-release-plugin] prepare release v1.1.0-rc1"
This reverts commit 79e86ef3e1a3ee03a7e3b166a5c7dee11c6d60d7.
commit a9df703081449c41c32cc982781fe1922eaf91ad
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T07:19:03Z
Additional CHANGES.txt
commit da4b94c86c9dd0d624b3040aa4b9449be9f60fc3
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T07:50:32Z
[maven-release-plugin] prepare release v1.1.0-rc1
commit 96926c5a42c5970ed74c50db5bd9c68cacf92207
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T07:50:43Z
[maven-release-plugin] prepare for next development iteration
commit 473b02dfdd8e616717d9d10e4d88e3274e8c8e03
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:24Z
Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 96926c5a42c5970ed74c50db5bd9c68cacf92207.
commit ecdbeeff8d62c6e0cdd4f998f76628e5692eb0c3
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:31Z
Revert "[maven-release-plugin] prepare release v1.1.0-rc1"
This reverts commit da4b94c86c9dd0d624b3040aa4b9449be9f60fc3.
commit 4186c45f5bc3b72c87d45bdd791983d41968dc34
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:33Z
Revert "Revert "[maven-release-plugin] prepare release v1.1.0-rc1""
This reverts commit 56070f12f455bae645cba887a74c72b12f1085f8.
commit df6194403a8431048521db1883f9b6feb9ec9983
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:36Z
Revert "Revert "[maven-release-plugin] prepare for next development
iteration""
This reverts commit 71ec0140f7e121bdba3d19e8219e91a5e9d1e320.
commit d01b3faf896c7e5e26bd1a8d1f9f125097e4dd6f
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:41Z
Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit a118ea5c59d653f5a3feda21455ba60bc722b3b1.
commit c818b2b0408c42bf7a9ce933e83bcd5e35c42817
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:44Z
Revert "[maven-release-plugin] prepare release v1.1.0-rc1"
This reverts commit 79e86ef3e1a3ee03a7e3b166a5c7dee11c6d60d7.
commit c0bacc161a2c6e252a62ff3369e6644bb9ef9da6
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:46Z
Revert "[maven-release-plugin] prepare for next development iteration"
This reverts commit 78e3c036eee7113b2ed144eec5061e070b479e56.
commit 1d03330ec40f3818f4fb226f79c17111f17a1b34
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T08:55:48Z
Revert "[maven-release-plugin] prepare release v1.1.0-rc1"
This reverts commit 58b0be6a29eab817d350729710345e9f39e4c506.
commit f07183249b74dd857069028bf7d570b35f265585
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T09:29:24Z
[maven-release-plugin] prepare release v1.1.0-rc1
commit f8f7a0c9dce764ece8acdc41d35bbf448dba7e92
Author: Patrick Wendell <[email protected]>
Date: 2014-08-28T09:29:30Z
[maven-release-plugin] prepare for next development iteration
commit fd98020a9a9880a09147f01d8480cab204e1c5d7
Author: Tatiana Borisova <[email protected]>
Date: 2014-08-28T17:36:36Z
[SPARK-3150] Fix NullPointerException in in Spark recovery: Add
initializing default values in DriverInfo.init()
The issue happens when Spark is run standalone on a cluster.
When master and driver fall simultaneously on one node in a cluster, master
tries to recover its state and restart spark driver.
While restarting driver, it falls with NPE exception (stacktrace is below).
After falling, it restarts and tries to recover its state and restart Spark
driver again. It happens over and over in an infinite cycle.
Namely, Spark tries to read DriverInfo state from zookeeper, but after
reading it happens to be null in DriverInfo.worker.
https://issues.apache.org/jira/browse/SPARK-3150
Author: Tatiana Borisova <[email protected]>
Closes #2062 from tanyatik/spark-3150 and squashes the following commits:
9936043 [Tatiana Borisova] Add initializing default values in
DriverInfo.init()
(cherry picked from commit 70d814665baa8b8ca868d3126452105ecfa5cbff)
Signed-off-by: Josh Rosen <[email protected]>
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
