[jira] [Resolved] (SPARK-21941) Stop storing unused attemptId in SQLTaskMetrics

2017-09-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21941. - Resolution: Fixed Assignee: Andrew Ash Fix Version/s: 2.3.0 > Stop storing unused attempt

[jira] [Commented] (SPARK-21902) BlockManager.doPut will hide actually exception when exception thrown in finally block

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159623#comment-16159623 ] Apache Spark commented on SPARK-21902: -- User 'caneGuy' has created a pull request fo

[jira] [Created] (SPARK-21962) Distributed Tracing in Spark

2017-09-08 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-21962: -- Summary: Distributed Tracing in Spark Key: SPARK-21962 URL: https://issues.apache.org/jira/browse/SPARK-21962 Project: Spark Issue Type: New Feature Co

[jira] [Assigned] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21961: Assignee: Apache Spark > Filter out BlockStatuses Accumulators during replaying history lo

[jira] [Commented] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159515#comment-16159515 ] Apache Spark commented on SPARK-21961: -- User 'zhouyejoe' has created a pull request

[jira] [Assigned] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21961: Assignee: (was: Apache Spark) > Filter out BlockStatuses Accumulators during replaying

[jira] [Commented] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159514#comment-16159514 ] Ye Zhou commented on SPARK-21961: - Pull Request Added: https://github.com/apache/spark/pu

[jira] [Updated] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21961: Description: As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of memory in Driver

[jira] [Updated] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21961: Description: As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of memory in Driver

[jira] [Updated] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21961: Description: As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of memory in Driver

[jira] [Updated] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21961: Description: As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of memory in Driver

[jira] [Updated] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21961: Description: As described in SPARK-20923, TaskMetrics._updatedBlockStatuses uses a lot of memory in Driver

[jira] [Updated] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Zhou updated SPARK-21961: Attachment: Objects_Count_in_Heap.png One_Thread_Took_24GB.png > Filter out BlockStatuses A

[jira] [Created] (SPARK-21961) Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server

2017-09-08 Thread Ye Zhou (JIRA)
Ye Zhou created SPARK-21961: --- Summary: Filter out BlockStatuses Accumulators during replaying history logs in Spark History Server Key: SPARK-21961 URL: https://issues.apache.org/jira/browse/SPARK-21961 Pro

[jira] [Created] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

2017-09-08 Thread Karthik Palaniappan (JIRA)
Karthik Palaniappan created SPARK-21960: --- Summary: Spark Streaming Dynamic Allocation should respect spark.executor.instances Key: SPARK-21960 URL: https://issues.apache.org/jira/browse/SPARK-21960

[jira] [Resolved] (SPARK-19866) Add local version of Word2Vec findSynonyms for spark.ml: Python API

2017-09-08 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-19866. - Resolution: Fixed Fix Version/s: 2.3.0 > Add local version of Word2Vec findSynonyms for spark.ml:

[jira] [Resolved] (SPARK-15243) Binarizer.explainParam(u"...") raises ValueError

2017-09-08 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-15243. - Resolution: Fixed > Binarizer.explainParam(u"...") raises ValueError > --

[jira] [Updated] (SPARK-15243) Binarizer.explainParam(u"...") raises ValueError

2017-09-08 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-15243: Fix Version/s: 2.3.0 > Binarizer.explainParam(u"...") raises ValueError > -

[jira] [Assigned] (SPARK-15243) Binarizer.explainParam(u"...") raises ValueError

2017-09-08 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-15243: --- Assignee: Hyukjin Kwon (was: Seth Hendrickson) > Binarizer.explainParam(u"...") raises ValueError >

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-09-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158997#comment-16158997 ] Marcelo Vanzin commented on SPARK-18085: [~jincheng] that is caused by SPARK-1770

[jira] [Commented] (SPARK-21907) NullPointerException in UnsafeExternalSorter.spill()

2017-09-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158995#comment-16158995 ] Kazuaki Ishizaki commented on SPARK-21907: -- If you cannot provide a repro, could

[jira] [Resolved] (SPARK-21959) Python RDD goes into never ending garbage collection service when spark submit is triggered in oozie

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21959. --- Resolution: Invalid There's no detail on the job, and no indication that this is a problem in Spark.

[jira] [Closed] (SPARK-21959) Python RDD goes into never ending garbage collection service when spark submit is triggered in oozie

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-21959. - > Python RDD goes into never ending garbage collection service when spark > submit is triggered in oozie > -

[jira] [Created] (SPARK-21959) Python RDD goes into never ending garbage collection service when spark submit is triggered in oozie

2017-09-08 Thread Vega Paleri (JIRA)
Vega Paleri created SPARK-21959: --- Summary: Python RDD goes into never ending garbage collection service when spark submit is triggered in oozie Key: SPARK-21959 URL: https://issues.apache.org/jira/browse/SPARK-21959

[jira] [Updated] (SPARK-21893) Put Kafka 0.8 behind a profile

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21893: -- Description: Kafka does not support 0.8.x for Scala 2.12. This code will have to, at least, be optiona

[jira] [Commented] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Kevin Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158948#comment-16158948 ] Kevin Zhang commented on SPARK-21944: - [~mgaido] Do you mean the following way by say

[jira] [Commented] (SPARK-20958) Roll back parquet-mr 1.8.2 to parquet-1.8.1

2017-09-08 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158924#comment-16158924 ] Ryan Blue commented on SPARK-20958: --- [~spiricalsalsaz], you need to only pin parquet-av

[jira] [Updated] (SPARK-21128) Running R tests multiple times failed due to pre-exiting "spark-warehouse" / "metastore_db"

2017-09-08 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21128: - Target Version/s: 2.2.1, 2.3.0 (was: 2.3.0) Fix Version/s: 2.2.1 > Running R tests multip

[jira] [Resolved] (SPARK-21946) Flaky test: InMemoryCatalogedDDLSuite.`alter table: rename cached table`

2017-09-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-21946. - Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 > Flaky test: InMemoryCatalogedDDL

[jira] [Assigned] (SPARK-21946) Flaky test: InMemoryCatalogedDDLSuite.`alter table: rename cached table`

2017-09-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-21946: --- Assignee: Kazuaki Ishizaki > Flaky test: InMemoryCatalogedDDLSuite.`alter table: rename cached table

[jira] [Updated] (SPARK-21936) backward compatibility test framework for HiveExternalCatalog

2017-09-08 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21936: Fix Version/s: 2.2.1 > backward compatibility test framework for HiveExternalCatalog >

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-09-08 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158865#comment-16158865 ] Marcelo Vanzin commented on SPARK-18085: Do you really mean fixed, as in you're n

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-09-08 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158845#comment-16158845 ] Li Jin commented on SPARK-21190: To [~bryanc]'s point, PR [#18659|https://github.com/apac

[jira] [Created] (SPARK-21958) Attempting to save large Word2Vec model hangs driver in constant GC.

2017-09-08 Thread Travis Hegner (JIRA)
Travis Hegner created SPARK-21958: - Summary: Attempting to save large Word2Vec model hangs driver in constant GC. Key: SPARK-21958 URL: https://issues.apache.org/jira/browse/SPARK-21958 Project: Spark

[jira] [Commented] (SPARK-21067) Thrift Server - CTAS fail with Unable to move source

2017-09-08 Thread Dominic Ricard (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158802#comment-16158802 ] Dominic Ricard commented on SPARK-21067: [~zhangxin0112zx] Our solution was to mi

[jira] [Assigned] (SPARK-21957) Add current_user function

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21957: Assignee: Apache Spark > Add current_user function > - > >

[jira] [Commented] (SPARK-21957) Add current_user function

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158745#comment-16158745 ] Apache Spark commented on SPARK-21957: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-21957) Add current_user function

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21957: Assignee: (was: Apache Spark) > Add current_user function > -

[jira] [Created] (SPARK-21957) Add current_user function

2017-09-08 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-21957: --- Summary: Add current_user function Key: SPARK-21957 URL: https://issues.apache.org/jira/browse/SPARK-21957 Project: Spark Issue Type: New Feature Com

[jira] [Comment Edited] (SPARK-20958) Roll back parquet-mr 1.8.2 to parquet-1.8.1

2017-09-08 Thread Anthony Dotterer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155387#comment-16155387 ] Anthony Dotterer edited comment on SPARK-20958 at 9/8/17 2:27 PM: -

[jira] [Assigned] (SPARK-21956) Fetch up to max bytes when buf really released

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21956: Assignee: Apache Spark > Fetch up to max bytes when buf really released >

[jira] [Assigned] (SPARK-21956) Fetch up to max bytes when buf really released

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21956: Assignee: (was: Apache Spark) > Fetch up to max bytes when buf really released > -

[jira] [Commented] (SPARK-21956) Fetch up to max bytes when buf really released

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158548#comment-16158548 ] Apache Spark commented on SPARK-21956: -- User 'caneGuy' has created a pull request fo

[jira] [Created] (SPARK-21956) Fetch up to max bytes when buf really released

2017-09-08 Thread zhoukang (JIRA)
zhoukang created SPARK-21956: Summary: Fetch up to max bytes when buf really released Key: SPARK-21956 URL: https://issues.apache.org/jira/browse/SPARK-21956 Project: Spark Issue Type: Improvemen

[jira] [Updated] (SPARK-21942) DiskBlockManager crashing when a root local folder has been externally deleted by OS

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21942: -- Affects Version/s: (was: 2.2.1) (was: 2.3.0) (

[jira] [Commented] (SPARK-21905) ClassCastException when call sqlContext.sql on temp table

2017-09-08 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158496#comment-16158496 ] Kazuaki Ishizaki commented on SPARK-21905: -- While I ran the following code (I do

[jira] [Reopened] (SPARK-21951) Unable to add the new column and writing into the Hive using spark

2017-09-08 Thread jalendhar Baddam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jalendhar Baddam reopened SPARK-21951: -- This is still exist. And it's getting the AnalysisException. > Unable to add the new colum

[jira] [Reopened] (SPARK-21952) Unable to load the csv file into Dataset using Spark with java

2017-09-08 Thread jalendhar Baddam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jalendhar Baddam reopened SPARK-21952: -- This is still exist ,Please re check > Unable to load the csv file into Dataset using Sp

[jira] [Commented] (SPARK-4502) Spark SQL reads unneccesary nested fields from Parquet

2017-09-08 Thread Andriy Kushnir (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158444#comment-16158444 ] Andriy Kushnir commented on SPARK-4502: --- Just tried this patch on Spark 2.2.0 There

[jira] [Comment Edited] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158406#comment-16158406 ] Marco Gaido edited comment on SPARK-21944 at 9/8/17 10:31 AM: -

[jira] [Comment Edited] (SPARK-19090) Dynamic Resource Allocation not respecting spark.executor.cores

2017-09-08 Thread Carlos Vicenti (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157932#comment-16157932 ] Carlos Vicenti edited comment on SPARK-19090 at 9/8/17 10:19 AM: --

[jira] [Comment Edited] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158406#comment-16158406 ] Marco Gaido edited comment on SPARK-21944 at 9/8/17 9:57 AM: -

[jira] [Commented] (SPARK-21944) Watermark on window column is wrong

2017-09-08 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158406#comment-16158406 ] Marco Gaido commented on SPARK-21944: - [~kevinzhang] you should define the watermark

[jira] [Commented] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158402#comment-16158402 ] Sean Owen commented on SPARK-21955: --- You might be on to something, but this is poorly d

[jira] [Created] (SPARK-21955) OneForOneStreamManager may leak memory when network is poor

2017-09-08 Thread poseidon (JIRA)
poseidon created SPARK-21955: Summary: OneForOneStreamManager may leak memory when network is poor Key: SPARK-21955 URL: https://issues.apache.org/jira/browse/SPARK-21955 Project: Spark Issue Ty

[jira] [Commented] (SPARK-21942) DiskBlockManager crashing when a root local folder has been externally deleted by OS

2017-09-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158320#comment-16158320 ] Saisai Shao commented on SPARK-21942: - Personally I would like to fail fast if such t

[jira] [Commented] (SPARK-21942) DiskBlockManager crashing when a root local folder has been externally deleted by OS

2017-09-08 Thread Ruslan Shestopalyuk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158304#comment-16158304 ] Ruslan Shestopalyuk commented on SPARK-21942: - [~jerryshao] I believe the onl

[jira] [Commented] (SPARK-21954) JacksonUtils should verify MapType's value type instead of key type

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158300#comment-16158300 ] Apache Spark commented on SPARK-21954: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-21954) JacksonUtils should verify MapType's value type instead of key type

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21954: Assignee: Apache Spark > JacksonUtils should verify MapType's value type instead of key ty

[jira] [Assigned] (SPARK-21954) JacksonUtils should verify MapType's value type instead of key type

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21954: Assignee: (was: Apache Spark) > JacksonUtils should verify MapType's value type instea

[jira] [Created] (SPARK-21954) JacksonUtils should verify MapType's value type instead of key type

2017-09-08 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-21954: --- Summary: JacksonUtils should verify MapType's value type instead of key type Key: SPARK-21954 URL: https://issues.apache.org/jira/browse/SPARK-21954 Project: Sp

[jira] [Commented] (SPARK-21128) Running R tests multiple times failed due to pre-exiting "spark-warehouse" / "metastore_db"

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158276#comment-16158276 ] Apache Spark commented on SPARK-21128: -- User 'felixcheung' has created a pull reques

[jira] [Commented] (SPARK-21942) DiskBlockManager crashing when a root local folder has been externally deleted by OS

2017-09-08 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158271#comment-16158271 ] Saisai Shao commented on SPARK-21942: - {quote} https://github.com/search?utf8=%E2%9C%

[jira] [Commented] (SPARK-21953) Show both memory and disk bytes spilled if either is present

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158263#comment-16158263 ] Apache Spark commented on SPARK-21953: -- User 'ash211' has created a pull request for

[jira] [Assigned] (SPARK-21953) Show both memory and disk bytes spilled if either is present

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21953: Assignee: Apache Spark > Show both memory and disk bytes spilled if either is present > --

[jira] [Assigned] (SPARK-21953) Show both memory and disk bytes spilled if either is present

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21953: Assignee: (was: Apache Spark) > Show both memory and disk bytes spilled if either is p

[jira] [Created] (SPARK-21953) Show both memory and disk bytes spilled if either is present

2017-09-08 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-21953: -- Summary: Show both memory and disk bytes spilled if either is present Key: SPARK-21953 URL: https://issues.apache.org/jira/browse/SPARK-21953 Project: Spark Iss

[jira] [Commented] (SPARK-18085) SPIP: Better History Server scalability for many / large applications

2017-09-08 Thread jincheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158247#comment-16158247 ] jincheng commented on SPARK-18085: -- {code:java} com.fasterxml.jackson.databind.exc.Unre

[jira] [Commented] (SPARK-21936) backward compatibility test framework for HiveExternalCatalog

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158241#comment-16158241 ] Apache Spark commented on SPARK-21936: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-21726) Check for structural integrity of the plan in QO in test mode

2017-09-08 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16158224#comment-16158224 ] Apache Spark commented on SPARK-21726: -- User 'viirya' has created a pull request for

[jira] [Resolved] (SPARK-21931) add LNNVL function

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21931. --- Resolution: Won't Fix > add LNNVL function > -- > > Key: SPARK-21931

[jira] [Resolved] (SPARK-21915) Model 1 and Model 2 ParamMaps Missing

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21915. --- Resolution: Fixed Fix Version/s: 2.2.1 Issue resolved by pull request 19152 [https://github.co

[jira] [Resolved] (SPARK-21951) Unable to add the new column and writing into the Hive using spark

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21951. --- Resolution: Invalid This doesn't express a problem, and questions should go to the mailing list > Un

[jira] [Resolved] (SPARK-21952) Unable to load the csv file into Dataset using Spark with java

2017-09-08 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21952. --- Resolution: Invalid Spam > Unable to load the csv file into Dataset using Spark with java > ---

[jira] [Created] (SPARK-21952) Unable to load the csv file into Dataset using Spark with java

2017-09-08 Thread jalendhar Baddam (JIRA)
jalendhar Baddam created SPARK-21952: Summary: Unable to load the csv file into Dataset using Spark with java Key: SPARK-21952 URL: https://issues.apache.org/jira/browse/SPARK-21952 Project: Spar