[jira] [Updated] (SPARK-20965) Support PREPARE/EXECUTE/DECLARE/FETCH statements

2017-06-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-20965: - Summary: Support PREPARE/EXECUTE/DECLARE/FETCH statements (was: Support PREPARE and EXEC

[jira] [Updated] (SPARK-20965) Support PREPARE/EXECUTE/DECLARE/FETCH statements

2017-06-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-20965: - Description: It might help to implement PREPARE/EXECUTE/DECLARE/FETCH statements by refe

[jira] [Updated] (SPARK-20965) Support PREPARE and EXECUTE statements

2017-06-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-20965: - Description: https://github.com/apache/spark/compare/master...maropu:SPARK-20965#diff-06e

[jira] [Updated] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21221: -- Shepherd: Joseph K. Bradley > CrossValidator and TrainValidationSplit Persist Nested Es

[jira] [Commented] (SPARK-21222) Move elimination of Distinct clause from analyzer to optimizer

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064229#comment-16064229 ] Apache Spark commented on SPARK-21222: -- User 'gengliangwang' has created a pull requ

[jira] [Assigned] (SPARK-21222) Move elimination of Distinct clause from analyzer to optimizer

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21222: Assignee: Apache Spark > Move elimination of Distinct clause from analyzer to optimizer >

[jira] [Assigned] (SPARK-21222) Move elimination of Distinct clause from analyzer to optimizer

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21222: Assignee: (was: Apache Spark) > Move elimination of Distinct clause from analyzer to o

[jira] [Created] (SPARK-21222) Move elimination of Distinct clause from analyzer to optimizer

2017-06-26 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-21222: -- Summary: Move elimination of Distinct clause from analyzer to optimizer Key: SPARK-21222 URL: https://issues.apache.org/jira/browse/SPARK-21222 Project: Spark

[jira] [Comment Edited] (SPARK-6635) DataFrame.withColumn can create columns with identical names

2017-06-26 Thread Yichuan Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064095#comment-16064095 ] Yichuan Wang edited comment on SPARK-6635 at 6/27/17 1:43 AM: --

[jira] [Commented] (SPARK-6635) DataFrame.withColumn can create columns with identical names

2017-06-26 Thread Yichuan Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064095#comment-16064095 ] Yichuan Wang commented on SPARK-6635: - withColumn have this strange behavior with join

[jira] [Commented] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064068#comment-16064068 ] Ajay Saini commented on SPARK-21221: Note: In order for python persistence of OneVsRe

[jira] [Issue Comment Deleted] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Saini updated SPARK-21221: --- Comment: was deleted (was: Pull Request Here: https://github.com/apache/spark/pull/18428) > CrossVal

[jira] [Commented] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064065#comment-16064065 ] Apache Spark commented on SPARK-21221: -- User 'ajaysaini725' has created a pull reque

[jira] [Assigned] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21221: Assignee: (was: Apache Spark) > CrossValidator and TrainValidationSplit Persist Nested

[jira] [Assigned] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21221: Assignee: Apache Spark > CrossValidator and TrainValidationSplit Persist Nested Estimators

[jira] [Commented] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Ajay Saini (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064064#comment-16064064 ] Ajay Saini commented on SPARK-21221: Pull Request Here: https://github.com/apache/spa

[jira] [Commented] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

2017-06-26 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064062#comment-16064062 ] Zhenhua Wang commented on SPARK-17129: -- [~mbasmanova] Thanks for working on it~ I'll

[jira] [Created] (SPARK-21221) CrossValidator and TrainValidationSplit Persist Nested Estimators

2017-06-26 Thread Ajay Saini (JIRA)
Ajay Saini created SPARK-21221: -- Summary: CrossValidator and TrainValidationSplit Persist Nested Estimators Key: SPARK-21221 URL: https://issues.apache.org/jira/browse/SPARK-21221 Project: Spark

[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

2017-06-26 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064007#comment-16064007 ] Yuming Wang commented on SPARK-21063: - [~pbykov], I just verified it. It can get the

[jira] [Closed] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle closed SPARK-21212. - Resolution: Not A Problem > Can't use Count(*) with Order Clause > --

[jira] [Commented] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064005#comment-16064005 ] Shawn Lavelle commented on SPARK-21212: --- I think you're right. I know my users (not

[jira] [Created] (SPARK-21220) Use outputPartitioning's bucketing if possible on write

2017-06-26 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-21220: -- Summary: Use outputPartitioning's bucketing if possible on write Key: SPARK-21220 URL: https://issues.apache.org/jira/browse/SPARK-21220 Project: Spark Issue Typ

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2017-06-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063984#comment-16063984 ] Reynold Xin commented on SPARK-14220: - If all those issues have been released than it

[jira] [Commented] (SPARK-21218) Convert IN predicate to equivalent Parquet filter

2017-06-26 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063981#comment-16063981 ] Andrew Duffy commented on SPARK-21218: -- Good catch, looks like a dupe. [~hyukjin.kwo

[jira] [Resolved] (SPARK-17091) ParquetFilters rewrite IN to OR of Eq

2017-06-26 Thread Andrew Duffy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Duffy resolved SPARK-17091. -- Resolution: Won't Fix Should've closed this last year, but at the time based on Hyukjin Kwon's

[jira] [Comment Edited] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063952#comment-16063952 ] Shawn Lavelle edited comment on SPARK-21212 at 6/26/17 11:26 PM: --

[jira] [Commented] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063968#comment-16063968 ] Sean Owen commented on SPARK-21212: --- Yes but you are not selecting the thing you order

[jira] [Updated] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Lavelle updated SPARK-21212: -- Description: I don't think this should fail the query: _Notes: VALUE is a column of table TABL

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2017-06-26 Thread Flavio Brasil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063957#comment-16063957 ] Flavio Brasil commented on SPARK-14220: --- [~rxin] Could you expand your last comment

[jira] [Comment Edited] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063952#comment-16063952 ] Shawn Lavelle edited comment on SPARK-21212 at 6/26/17 11:17 PM: --

[jira] [Comment Edited] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063952#comment-16063952 ] Shawn Lavelle edited comment on SPARK-21212 at 6/26/17 11:17 PM: --

[jira] [Commented] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063952#comment-16063952 ] Shawn Lavelle commented on SPARK-21212: --- [~srowen], I can assure you that value is

[jira] [Commented] (SPARK-21218) Convert IN predicate to equivalent Parquet filter

2017-06-26 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063925#comment-16063925 ] Hyukjin Kwon commented on SPARK-21218: -- I believe it is a duplicate of SPARK-17091.

[jira] [Commented] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063915#comment-16063915 ] Sean Owen commented on SPARK-21212: --- Yes, but this doesn't work unless you say what 'va

[jira] [Commented] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063909#comment-16063909 ] Apache Spark commented on SPARK-21219: -- User 'ericvandenbergfb' has created a pull r

[jira] [Assigned] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21219: Assignee: (was: Apache Spark) > Task retry occurs on same executor due to race conditi

[jira] [Assigned] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21219: Assignee: Apache Spark > Task retry occurs on same executor due to race condition with bla

[jira] [Commented] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Shawn Lavelle (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063903#comment-16063903 ] Shawn Lavelle commented on SPARK-21212: --- I redacted most of the information to prot

[jira] [Comment Edited] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-26 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063873#comment-16063873 ] Li Jin edited comment on SPARK-21190 at 6/26/17 10:02 PM: -- [~r..

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-26 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063873#comment-16063873 ] Li Jin commented on SPARK-21190: [~r...@databricks.com], The use case of seeing entire p

[jira] [Updated] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Eric Vandenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21219: Description: When a task fails it is (1) added into the pending task list and then (2) cor

[jira] [Updated] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Eric Vandenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21219: Description: When a task fails it is added into the pending task list and corresponding bl

[jira] [Updated] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Eric Vandenberg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Vandenberg updated SPARK-21219: Attachment: spark_executor.log.anon spark_driver.log.anon > Task retry occu

[jira] [Created] (SPARK-21219) Task retry occurs on same executor due to race condition with blacklisting

2017-06-26 Thread Eric Vandenberg (JIRA)
Eric Vandenberg created SPARK-21219: --- Summary: Task retry occurs on same executor due to race condition with blacklisting Key: SPARK-21219 URL: https://issues.apache.org/jira/browse/SPARK-21219 Proj

[jira] [Assigned] (SPARK-21216) Streaming DataFrames fail to join with Hive tables

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21216: Assignee: Apache Spark (was: Burak Yavuz) > Streaming DataFrames fail to join with Hive t

[jira] [Commented] (SPARK-21216) Streaming DataFrames fail to join with Hive tables

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063662#comment-16063662 ] Apache Spark commented on SPARK-21216: -- User 'brkyvz' has created a pull request for

[jira] [Assigned] (SPARK-21216) Streaming DataFrames fail to join with Hive tables

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21216: Assignee: Burak Yavuz (was: Apache Spark) > Streaming DataFrames fail to join with Hive t

[jira] [Commented] (SPARK-21217) Support ColumnVector.Array.toArray()

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063631#comment-16063631 ] Apache Spark commented on SPARK-21217: -- User 'kiszk' has created a pull request for

[jira] [Assigned] (SPARK-21217) Support ColumnVector.Array.toArray()

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21217: Assignee: (was: Apache Spark) > Support ColumnVector.Array.toArray() > ---

[jira] [Assigned] (SPARK-21217) Support ColumnVector.Array.toArray()

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21217: Assignee: Apache Spark > Support ColumnVector.Array.toArray() > --

[jira] [Assigned] (SPARK-21218) Convert IN predicate to equivalent Parquet filter

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21218: Assignee: (was: Apache Spark) > Convert IN predicate to equivalent Parquet filter > --

[jira] [Assigned] (SPARK-21218) Convert IN predicate to equivalent Parquet filter

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21218: Assignee: Apache Spark > Convert IN predicate to equivalent Parquet filter > -

[jira] [Commented] (SPARK-21218) Convert IN predicate to equivalent Parquet filter

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063620#comment-16063620 ] Apache Spark commented on SPARK-21218: -- User 'ptkool' has created a pull request for

[jira] [Created] (SPARK-21218) Convert IN predicate to equivalent Parquet filter

2017-06-26 Thread Michael Styles (JIRA)
Michael Styles created SPARK-21218: -- Summary: Convert IN predicate to equivalent Parquet filter Key: SPARK-21218 URL: https://issues.apache.org/jira/browse/SPARK-21218 Project: Spark Issue T

[jira] [Created] (SPARK-21217) Support ColumnVector.Array.toArray()

2017-06-26 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-21217: Summary: Support ColumnVector.Array.toArray() Key: SPARK-21217 URL: https://issues.apache.org/jira/browse/SPARK-21217 Project: Spark Issue Type: Impr

[jira] [Created] (SPARK-21216) Streaming DataFrames fail to join with Hive tables

2017-06-26 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-21216: --- Summary: Streaming DataFrames fail to join with Hive tables Key: SPARK-21216 URL: https://issues.apache.org/jira/browse/SPARK-21216 Project: Spark Issue Type:

[jira] [Commented] (SPARK-21158) SparkSQL function SparkSession.Catalog.ListTables() does not handle spark setting for case-sensitivity

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063561#comment-16063561 ] Apache Spark commented on SPARK-21158: -- User 'cammachusa' has created a pull request

[jira] [Assigned] (SPARK-21158) SparkSQL function SparkSession.Catalog.ListTables() does not handle spark setting for case-sensitivity

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21158: Assignee: (was: Apache Spark) > SparkSQL function SparkSession.Catalog.ListTables() do

[jira] [Assigned] (SPARK-21158) SparkSQL function SparkSession.Catalog.ListTables() does not handle spark setting for case-sensitivity

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21158: Assignee: Apache Spark > SparkSQL function SparkSession.Catalog.ListTables() does not hand

[jira] [Commented] (SPARK-20889) SparkR grouped documentation for Column methods

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063552#comment-16063552 ] Apache Spark commented on SPARK-20889: -- User 'actuaryzhang' has created a pull reque

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063515#comment-16063515 ] Reynold Xin commented on SPARK-21190: - [~icexelloss] Thanks. Your proposal brings up

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063406#comment-16063406 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:56 PM: -

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063406#comment-16063406 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:55 PM: -

[jira] [Commented] (SPARK-21214) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve '

2017-06-26 Thread Michael Kunkel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063462#comment-16063462 ] Michael Kunkel commented on SPARK-21214: Greetings, Forget my last email. BR M

[jira] [Commented] (SPARK-21214) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve '

2017-06-26 Thread Michael Kunkel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063461#comment-16063461 ] Michael Kunkel commented on SPARK-21214: Greetings, Would you please inform me o

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063431#comment-16063431 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:24 PM: -

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063431#comment-16063431 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:23 PM: -

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063431#comment-16063431 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:22 PM: -

[jira] [Commented] (SPARK-21215) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve

2017-06-26 Thread Michael Kunkel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063453#comment-16063453 ] Michael Kunkel commented on SPARK-21215: [~srowen] I am new to this board, so I d

[jira] [Commented] (SPARK-21183) Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value t

2017-06-26 Thread Matthew Walton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063448#comment-16063448 ] Matthew Walton commented on SPARK-21183: Hi Sean, I think the issue is Spark is

[jira] [Comment Edited] (SPARK-21199) Its not possible to impute Vector types

2017-06-26 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063416#comment-16063416 ] Franklyn Dsouza edited comment on SPARK-21199 at 6/26/17 5:16 PM: -

[jira] [Comment Edited] (SPARK-21199) Its not possible to impute Vector types

2017-06-26 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063416#comment-16063416 ] Franklyn Dsouza edited comment on SPARK-21199 at 6/26/17 5:16 PM: -

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063431#comment-16063431 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:15 PM: -

[jira] [Comment Edited] (SPARK-21199) Its not possible to impute Vector types

2017-06-26 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063416#comment-16063416 ] Franklyn Dsouza edited comment on SPARK-21199 at 6/26/17 5:15 PM: -

[jira] [Resolved] (SPARK-21215) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21215. --- Resolution: Duplicate > Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot >

[jira] [Resolved] (SPARK-21214) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve '

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21214. --- Resolution: Invalid This should be a question on the mailing list. > Exception in thread "main" org

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063431#comment-16063431 ] Saif Addin commented on SPARK-21198: Regarding listtables, here is the code used ins

[jira] [Commented] (SPARK-21183) Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value t

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063428#comment-16063428 ] Sean Owen commented on SPARK-21183: --- I'm not sure that follows, but I don't know either

[jira] [Created] (SPARK-21215) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve

2017-06-26 Thread Michael Kunkel (JIRA)
Michael Kunkel created SPARK-21215: -- Summary: Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve Key: SPARK-21215 URL: https://issues.apache.org/jira/browse/SPARK-21215

[jira] [Created] (SPARK-21214) Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve '

2017-06-26 Thread Michael Kunkel (JIRA)
Michael Kunkel created SPARK-21214: -- Summary: Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve ' Key: SPARK-21214 URL: https://issues.apache.org/jira/browse/SPARK-21214

[jira] [Commented] (SPARK-21183) Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value t

2017-06-26 Thread Matthew Walton (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063421#comment-16063421 ] Matthew Walton commented on SPARK-21183: Well, if only the SQuirreL Client tool d

[jira] [Commented] (SPARK-21199) Its not possible to impute Vector types

2017-06-26 Thread Franklyn Dsouza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063416#comment-16063416 ] Franklyn Dsouza commented on SPARK-21199: - For this particular scenario I have a

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063406#comment-16063406 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:03 PM: -

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063406#comment-16063406 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:02 PM: -

[jira] [Commented] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063406#comment-16063406 ] Saif Addin commented on SPARK-21198: Okay, I think there is something odd somewhere i

[jira] [Comment Edited] (SPARK-21198) SparkSession catalog is terribly slow

2017-06-26 Thread Saif Addin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063406#comment-16063406 ] Saif Addin edited comment on SPARK-21198 at 6/26/17 5:01 PM: -

[jira] [Commented] (SPARK-21183) Unable to return Google BigQuery INTEGER data type into Spark via google BigQuery JDBC driver: java.sql.SQLDataException: [Simba][JDBC](10140) Error converting value t

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063387#comment-16063387 ] Sean Owen commented on SPARK-21183: --- This looks like an error from simba, not Spark? >

[jira] [Commented] (SPARK-21212) Can't use Count(*) with Order Clause

2017-06-26 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063376#comment-16063376 ] Sean Owen commented on SPARK-21212: --- You don't define 'value' anywhere, as it says. >

[jira] [Commented] (SPARK-17129) Support statistics collection and cardinality estimation for partitioned tables

2017-06-26 Thread Maria (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063342#comment-16063342 ] Maria commented on SPARK-17129: --- [~ZenWzh], I opened SPARK-21213 and submitted PR https://

[jira] [Resolved] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

2017-06-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-13669. --- Resolution: Fixed Fix Version/s: 2.3.0 > Job will always fail in the external shuffle

[jira] [Assigned] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

2017-06-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-13669: - Assignee: Saisai Shao > Job will always fail in the external shuffle service unavailable

[jira] [Assigned] (SPARK-20898) spark.blacklist.killBlacklistedExecutors doesn't work in YARN

2017-06-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-20898: - Assignee: Saisai Shao > spark.blacklist.killBlacklistedExecutors doesn't work in YARN >

[jira] [Resolved] (SPARK-20898) spark.blacklist.killBlacklistedExecutors doesn't work in YARN

2017-06-26 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-20898. --- Resolution: Fixed Fix Version/s: 2.3.0 > spark.blacklist.killBlacklistedExecutors does

[jira] [Commented] (SPARK-21213) Support collecting partition-level statistics: rowCount and sizeInBytes

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063338#comment-16063338 ] Apache Spark commented on SPARK-21213: -- User 'mbasmanova' has created a pull request

[jira] [Assigned] (SPARK-21213) Support collecting partition-level statistics: rowCount and sizeInBytes

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21213: Assignee: (was: Apache Spark) > Support collecting partition-level statistics: rowCoun

[jira] [Assigned] (SPARK-21213) Support collecting partition-level statistics: rowCount and sizeInBytes

2017-06-26 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21213: Assignee: Apache Spark > Support collecting partition-level statistics: rowCount and sizeI

[jira] [Updated] (SPARK-21213) Support collecting partition-level statistics: rowCount and sizeInBytes

2017-06-26 Thread Maria (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria updated SPARK-21213: -- Summary: Support collecting partition-level statistics: rowCount and sizeInBytes (was: Support collecting part

[jira] [Commented] (SPARK-21063) Spark return an empty result from remote hadoop cluster

2017-06-26 Thread Peter Bykov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063289#comment-16063289 ] Peter Bykov commented on SPARK-21063: - [~q79969786] I tried this solution, but same r

[jira] [Updated] (SPARK-21213) Support collecting partition-level statistics

2017-06-26 Thread Maria (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maria updated SPARK-21213: -- Issue Type: Sub-task (was: New Feature) Parent: SPARK-17129 > Support collecting partition-level stati

[jira] [Created] (SPARK-21213) Support collecting partition-level statistics

2017-06-26 Thread Maria (JIRA)
Maria created SPARK-21213: - Summary: Support collecting partition-level statistics Key: SPARK-21213 URL: https://issues.apache.org/jira/browse/SPARK-21213 Project: Spark Issue Type: New Feature

  1   2   >