[jira] [Commented] (SPARK-9273) Add Convolutional Neural network to Spark MLlib

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638328#comment-14638328 ] Sean Owen commented on SPARK-9273: -- I think this is duplicated / subsumed many times by o

[jira] [Commented] (SPARK-9270) spark.app.name is not honored by pyspark

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638325#comment-14638325 ] Sean Owen commented on SPARK-9270: -- Yeah there's probably an additional change to be made

[jira] [Commented] (SPARK-9270) spark.app.name is not honored by pyspark

2015-07-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638319#comment-14638319 ] Cheolsoo Park commented on SPARK-9270: -- [~sowen], I just tested SPARK-9180 now. even

[jira] [Updated] (SPARK-8695) TreeAggregation shouldn't be triggered when it doesn't save wall-clock time

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-8695: - Assignee: Xiangrui Meng > TreeAggregation shouldn't be triggered when it doesn't save wall-clock time > --

[jira] [Resolved] (SPARK-8695) TreeAggregation shouldn't be triggered when it doesn't save wall-clock time

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-8695. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7397 [https://github.com/ap

[jira] [Commented] (SPARK-9256) Message delay causes Master crash upon registering application

2015-07-22 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638311#comment-14638311 ] Liang-Chi Hsieh commented on SPARK-9256: Since the FileSystemPersistenceEngine per

[jira] [Updated] (SPARK-9270) spark.app.name is not honored by pyspark

2015-07-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated SPARK-9270: - Description: Currently, the app name is hardcoded in pyspark as "PySparkShell", and the {{spark.a

[jira] [Created] (SPARK-9273) Add Convolutional Neural network to Spark MLlib

2015-07-22 Thread yuhao yang (JIRA)
yuhao yang created SPARK-9273: - Summary: Add Convolutional Neural network to Spark MLlib Key: SPARK-9273 URL: https://issues.apache.org/jira/browse/SPARK-9273 Project: Spark Issue Type: New Featu

[jira] [Resolved] (SPARK-8935) Implement code generation for all casts

2015-07-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8935. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7365 [https://github.com/

[jira] [Updated] (SPARK-9243) Update crosstab doc for pairs that have no occurrences

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-9243: - Priority: Trivial (was: Major) > Update crosstab doc for pairs that have no occurrences > ---

[jira] [Commented] (SPARK-9243) Update crosstab doc for pairs that have no occurrences

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638302#comment-14638302 ] Apache Spark commented on SPARK-9243: - User 'mengxr' has created a pull request for th

[jira] [Assigned] (SPARK-9243) Update crosstab doc for pairs that have no occurrences

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-9243: Assignee: Xiangrui Meng > Update crosstab doc for pairs that have no occurrences >

[jira] [Commented] (SPARK-9239) HiveUDAF support for AggregateFunction2

2015-07-22 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638276#comment-14638276 ] Cheng Hao commented on SPARK-9239: -- [~yhuai] are you working on this now? Or I can take i

[jira] [Updated] (SPARK-7254) Extend PIC to handle Graphs directly

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7254: - Target Version/s: 1.5.0 > Extend PIC to handle Graphs directly > -

[jira] [Updated] (SPARK-7254) Extend PIC to handle Graphs directly

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-7254: - Assignee: Liang-Chi Hsieh > Extend PIC to handle Graphs directly > ---

[jira] [Resolved] (SPARK-7254) Extend PIC to handle Graphs directly

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-7254. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 6054 [https://githu

[jira] [Resolved] (SPARK-9268) Params.setDefault should not keep varargs annotation

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9268. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7604 [https://githu

[jira] [Updated] (SPARK-9270) spark.app.name is not honored by pyspark

2015-07-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated SPARK-9270: - Description: Currently, the app name is hardcoded in pyspark as "PySparkShell", and the {{spark.a

[jira] [Created] (SPARK-9272) Persist information of individual partitions when persisting partitioned data source tables to metastore

2015-07-22 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-9272: - Summary: Persist information of individual partitions when persisting partitioned data source tables to metastore Key: SPARK-9272 URL: https://issues.apache.org/jira/browse/SPARK-9272

[jira] [Created] (SPARK-9271) Concurrency bug triggered by partition predicate push-down

2015-07-22 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-9271: - Summary: Concurrency bug triggered by partition predicate push-down Key: SPARK-9271 URL: https://issues.apache.org/jira/browse/SPARK-9271 Project: Spark Issue Type

[jira] [Comment Edited] (SPARK-9270) spark.app.name is not honored by spark-shell and pyspark

2015-07-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638187#comment-14638187 ] Cheolsoo Park edited comment on SPARK-9270 at 7/23/15 5:43 AM: -

[jira] [Commented] (SPARK-9270) spark.app.name is not honored by spark-shell and pyspark

2015-07-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638187#comment-14638187 ] Cheolsoo Park commented on SPARK-9270: -- Oh it is. I missed it. So spark-shell is fixe

[jira] [Commented] (SPARK-9270) spark.app.name is not honored by spark-shell and pyspark

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638185#comment-14638185 ] Sean Owen commented on SPARK-9270: -- Seems very related to, or almost a duplicate of SPARK

[jira] [Created] (SPARK-9270) spark.app.name is not honored by spark-shell and pyspark

2015-07-22 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created SPARK-9270: Summary: spark.app.name is not honored by spark-shell and pyspark Key: SPARK-9270 URL: https://issues.apache.org/jira/browse/SPARK-9270 Project: Spark Issue

[jira] [Assigned] (SPARK-9069) Remove unlimited DecimalType

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9069: --- Assignee: Davies Liu (was: Apache Spark) > Remove unlimited DecimalType > --

[jira] [Assigned] (SPARK-9069) Remove unlimited DecimalType

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9069: --- Assignee: Apache Spark (was: Davies Liu) > Remove unlimited DecimalType > --

[jira] [Commented] (SPARK-9069) Remove unlimited DecimalType

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638165#comment-14638165 ] Apache Spark commented on SPARK-9069: - User 'davies' has created a pull request for th

[jira] [Commented] (SPARK-9036) SparkListenerExecutorMetricsUpdate messages not included in JsonProtocol

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638160#comment-14638160 ] Sean Owen commented on SPARK-9036: -- Precisely, in this case I figure credit, such as it i

[jira] [Resolved] (SPARK-8364) Add crosstab to SparkR DataFrames

2015-07-22 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-8364. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 731

[jira] [Updated] (SPARK-9180) Accept --name option in spark-submit

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9180: - Assignee: Kenichi Maehashi > Accept --name option in spark-submit > >

[jira] [Commented] (SPARK-9192) add initialization phase for nondeterministic expression

2015-07-22 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638140#comment-14638140 ] Cheng Lian commented on SPARK-9192: --- Yeah, it makes sense. Thanks! > add initialization

[jira] [Resolved] (SPARK-9262) Treat Scala compiler warnings as errors

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9262. Resolution: Fixed Fix Version/s: 1.5.0 > Treat Scala compiler warnings as errors > --

[jira] [Assigned] (SPARK-8668) expr function to convert SQL expression into a Column

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8668: --- Assignee: (was: Apache Spark) > expr function to convert SQL expression into a Column > -

[jira] [Commented] (SPARK-8668) expr function to convert SQL expression into a Column

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638126#comment-14638126 ] Apache Spark commented on SPARK-8668: - User 'JDrit' has created a pull request for thi

[jira] [Assigned] (SPARK-8668) expr function to convert SQL expression into a Column

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8668: --- Assignee: Apache Spark > expr function to convert SQL expression into a Column >

[jira] [Updated] (SPARK-9144) Remove DAGScheduler.runLocallyWithinThread and spark.localExecution.enabled

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9144: --- Labels: releasenotes (was: ) > Remove DAGScheduler.runLocallyWithinThread and spark.localExecution.en

[jira] [Resolved] (SPARK-9144) Remove DAGScheduler.runLocallyWithinThread and spark.localExecution.enabled

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-9144. Resolution: Fixed Fix Version/s: 1.5.0 > Remove DAGScheduler.runLocallyWithinThread and spark

[jira] [Assigned] (SPARK-9264) When dealing with Union/Intersect/Except, we cast FloatType/DoubleType to the wrong DecimalType

2015-07-22 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9264: - Assignee: Davies Liu > When dealing with Union/Intersect/Except, we cast FloatType/DoubleType to

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

2015-07-22 Thread K S Sreenivasa Raghavan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638078#comment-14638078 ] K S Sreenivasa Raghavan commented on SPARK-6192: I am interested in python

[jira] [Updated] (SPARK-9016) Make random forest classifier extend Classifier abstraction

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9016: - Assignee: holdenk > Make random forest classifier extend Classifier abstraction >

[jira] [Updated] (SPARK-9016) Make random forest classifier extend Classifier abstraction

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9016: - Shepherd: Joseph K. Bradley > Make random forest classifier extend Classifier abstraction

[jira] [Updated] (SPARK-9016) Make random forest classifier extend Classifier abstraction

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9016: - Target Version/s: 1.5.0 > Make random forest classifier extend Classifier abstraction > --

[jira] [Created] (SPARK-9269) Add Set to the matching type in ArrayConverter

2015-07-22 Thread Alex Liu (JIRA)
Alex Liu created SPARK-9269: --- Summary: Add Set to the matching type in ArrayConverter Key: SPARK-9269 URL: https://issues.apache.org/jira/browse/SPARK-9269 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-9220) Streaming K-means implementation exception while processing windowed stream

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637983#comment-14637983 ] Joseph K. Bradley commented on SPARK-9220: -- [~freeman-lab] Do you have time to lo

[jira] [Updated] (SPARK-9222) Make class instantiation variables in DistributedLDAModel [private] clustering

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9222: - Assignee: Manoj Kumar > Make class instantiation variables in DistributedLDAModel [private

[jira] [Updated] (SPARK-9222) Make class instantiation variables in DistributedLDAModel [private] clustering

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9222: - Shepherd: Joseph K. Bradley > Make class instantiation variables in DistributedLDAModel [p

[jira] [Resolved] (SPARK-8484) Add TrainValidationSplit to ml.tuning

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-8484. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7337 [https://githu

[jira] [Updated] (SPARK-9149) Add an example of spark.ml KMeans

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9149: - Assignee: Yu Ishikawa > Add an example of spark.ml KMeans > --

[jira] [Updated] (SPARK-9149) Add an example of spark.ml KMeans

2015-07-22 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-9149: - Issue Type: Documentation (was: Sub-task) Parent: (was: SPARK-7879) > Add an

[jira] [Assigned] (SPARK-9268) Params.setDefault should not keep varargs annotation

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9268: --- Assignee: Joseph K. Bradley (was: Apache Spark) > Params.setDefault should not keep varargs

[jira] [Commented] (SPARK-9268) Params.setDefault should not keep varargs annotation

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637973#comment-14637973 ] Apache Spark commented on SPARK-9268: - User 'jkbradley' has created a pull request for

[jira] [Assigned] (SPARK-9268) Params.setDefault should not keep varargs annotation

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9268: --- Assignee: Apache Spark (was: Joseph K. Bradley) > Params.setDefault should not keep varargs

[jira] [Updated] (SPARK-9223) Support model save/load in Python's LDA

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-9223: - Assignee: Manoj Kumar > Support model save/load in Python's LDA >

[jira] [Updated] (SPARK-9223) Support model save/load in Python's LDA

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-9223: - Target Version/s: 1.5.0 > Support model save/load in Python's LDA > --

[jira] [Resolved] (SPARK-9223) Support model save/load in Python's LDA

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-9223. -- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7587 [https://githu

[jira] [Created] (SPARK-9268) Params.setDefault should not keep varargs annotation

2015-07-22 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-9268: Summary: Params.setDefault should not keep varargs annotation Key: SPARK-9268 URL: https://issues.apache.org/jira/browse/SPARK-9268 Project: Spark Is

[jira] [Commented] (SPARK-746) Automatically Use Avro Serialization for Avro Objects

2015-07-22 Thread Joseph Batchik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637963#comment-14637963 ] Joseph Batchik commented on SPARK-746: -- I did some benchmarks for both the current imp

[jira] [Assigned] (SPARK-9247) Use BytesToBytesMap in unsafe broadcast join

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9247: --- Assignee: Apache Spark (was: Davies Liu) > Use BytesToBytesMap in unsafe broadcast join > --

[jira] [Assigned] (SPARK-9247) Use BytesToBytesMap in unsafe broadcast join

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9247: --- Assignee: Davies Liu (was: Apache Spark) > Use BytesToBytesMap in unsafe broadcast join > --

[jira] [Commented] (SPARK-9247) Use BytesToBytesMap in unsafe broadcast join

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637951#comment-14637951 ] Apache Spark commented on SPARK-9247: - User 'davies' has created a pull request for th

[jira] [Updated] (SPARK-9256) Message delay causes Master crash upon registering application

2015-07-22 Thread Colin Scott (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Scott updated SPARK-9256: --- Priority: Major (was: Minor) > Message delay causes Master crash upon registering application > -

[jira] [Updated] (SPARK-9131) UDFs change data values

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9131: --- Target Version/s: 1.4.2, 1.5.0 > UDFs change data values > --- > >

[jira] [Updated] (SPARK-9131) UDFs change data values

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9131: --- Priority: Blocker (was: Critical) > UDFs change data values > --- > >

[jira] [Updated] (SPARK-9230) SparkR RFormula should support StringType features

2015-07-22 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-9230: - Shepherd: Xiangrui Meng > SparkR RFormula should support StringType features > ---

[jira] [Resolved] (SPARK-9180) Accept --name option in spark-submit

2015-07-22 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-9180. --- Resolution: Fixed Fix Version/s: 1.5.0 > Accept --name option in spark-submit > ---

[jira] [Assigned] (SPARK-9266) Prevent "managed memory leak detected" exception from masking original exception

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9266: --- Assignee: Apache Spark (was: Josh Rosen) > Prevent "managed memory leak detected" exception

[jira] [Assigned] (SPARK-9266) Prevent "managed memory leak detected" exception from masking original exception

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9266: --- Assignee: Josh Rosen (was: Apache Spark) > Prevent "managed memory leak detected" exception

[jira] [Commented] (SPARK-9266) Prevent "managed memory leak detected" exception from masking original exception

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637893#comment-14637893 ] Apache Spark commented on SPARK-9266: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-9267) Remove highly unnecessary accumulators stringify methods

2015-07-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9267: - Description: {code} def stringifyPartialValue(partialValue: Any): String = "%s".format(partialValue) def s

[jira] [Updated] (SPARK-9267) Remove highly unnecessary accumulators stringify methods

2015-07-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9267: - Assignee: (was: Andrew Or) > Remove highly unnecessary accumulators stringify methods > --

[jira] [Updated] (SPARK-9267) Remove highly unnecessary accumulators stringify methods

2015-07-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9267: - Description: {code} def stringifyPartialValue(partialValue: Any): String = "%s".format(partialValue) def s

[jira] [Updated] (SPARK-9267) Remove highly unnecessary accumulators stringify methods

2015-07-22 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-9267: - Description: {code} def stringifyPartialValue(partialValue: Any): String = "%s".format(partialValue) def s

[jira] [Created] (SPARK-9267) Remove highly unnecessary accumulators stringify methods

2015-07-22 Thread Andrew Or (JIRA)
Andrew Or created SPARK-9267: Summary: Remove highly unnecessary accumulators stringify methods Key: SPARK-9267 URL: https://issues.apache.org/jira/browse/SPARK-9267 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9265: --- Priority: Critical (was: Major) > Dataframe.limit joined with another dataframe can be non-determinis

[jira] [Resolved] (SPARK-8975) Implement a mechanism to send a new rate from the driver to the block generator

2015-07-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-8975. -- Resolution: Fixed Fix Version/s: 1.5.0 > Implement a mechanism to send a new rate from th

[jira] [Commented] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637876#comment-14637876 ] Reynold Xin commented on SPARK-9265: Updating the title since it is not exactly a data

[jira] [Created] (SPARK-9266) Prevent "managed memory leak detected" exception from masking original exception

2015-07-22 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-9266: - Summary: Prevent "managed memory leak detected" exception from masking original exception Key: SPARK-9266 URL: https://issues.apache.org/jira/browse/SPARK-9266 Project: Spa

[jira] [Assigned] (SPARK-8889) showDagViz will cause java.lang.OutOfMemoryError: Java heap space

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8889: --- Assignee: Apache Spark > showDagViz will cause java.lang.OutOfMemoryError: Java heap space >

[jira] [Commented] (SPARK-8889) showDagViz will cause java.lang.OutOfMemoryError: Java heap space

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637874#comment-14637874 ] Apache Spark commented on SPARK-8889: - User 'rekhajoshm' has created a pull request fo

[jira] [Assigned] (SPARK-8889) showDagViz will cause java.lang.OutOfMemoryError: Java heap space

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-8889: --- Assignee: (was: Apache Spark) > showDagViz will cause java.lang.OutOfMemoryError: Java he

[jira] [Commented] (SPARK-9237) Added Top N Column Values for DataFrames

2015-07-22 Thread Ted Malaska (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637871#comment-14637871 ] Ted Malaska commented on SPARK-9237: It is very common in banks, to do data quality ch

[jira] [Updated] (SPARK-9264) When dealing with Union/Intersect/Except, we cast FloatType/DoubleType to the wrong DecimalType

2015-07-22 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-9264: Description: The problem is https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apa

[jira] [Updated] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-07-22 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-9265: --- Assignee: (was: Reynold Xin) > Dataframe.limit joined with another dataframe can be non-determinis

[jira] [Commented] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-07-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637786#comment-14637786 ] Tathagata Das commented on SPARK-9265: -- cc [~andrewor14], who helped be generate this

[jira] [Updated] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-07-22 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-9265: - Description: {code} import org.apache.spark.sql._ import org.apache.spark.sql.functions._ val re

[jira] [Created] (SPARK-9265) Dataframe.limit joined with another dataframe can be non-deterministic

2015-07-22 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-9265: Summary: Dataframe.limit joined with another dataframe can be non-deterministic Key: SPARK-9265 URL: https://issues.apache.org/jira/browse/SPARK-9265 Project: Spark

[jira] [Resolved] (SPARK-9244) Increase some default memory limits

2015-07-22 Thread Matei Zaharia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matei Zaharia resolved SPARK-9244. -- Resolution: Fixed Fix Version/s: 1.5.0 > Increase some default memory limits > --

[jira] [Commented] (SPARK-9036) SparkListenerExecutorMetricsUpdate messages not included in JsonProtocol

2015-07-22 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637768#comment-14637768 ] Ryan Williams commented on SPARK-9036: -- [~srowen] did you just assign this to me sinc

[jira] [Assigned] (SPARK-9260) Standalone scheduling can overflow a worker with cores

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9260: --- Assignee: Nishkam Ravi (was: Apache Spark) > Standalone scheduling can overflow a worker wit

[jira] [Created] (SPARK-9264) When dealing with Union/Intersect/Except, we cast FloatType/DoubleType to the wrong DecimalType

2015-07-22 Thread Yin Huai (JIRA)
Yin Huai created SPARK-9264: --- Summary: When dealing with Union/Intersect/Except, we cast FloatType/DoubleType to the wrong DecimalType Key: SPARK-9264 URL: https://issues.apache.org/jira/browse/SPARK-9264 P

[jira] [Assigned] (SPARK-9260) Standalone scheduling can overflow a worker with cores

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-9260: --- Assignee: Apache Spark (was: Nishkam Ravi) > Standalone scheduling can overflow a worker wit

[jira] [Commented] (SPARK-9260) Standalone scheduling can overflow a worker with cores

2015-07-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637762#comment-14637762 ] Apache Spark commented on SPARK-9260: - User 'nishkamravi2' has created a pull request

[jira] [Updated] (SPARK-9046) Decimal type support improvement and bug fix

2015-07-22 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-9046: Summary: Decimal type support improvement and bug fix (was: Decimal type support improvement) > Decimal ty

[jira] [Updated] (SPARK-9206) ClassCastException using HiveContext with GoogleHadoopFileSystem as fs.defaultFS

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9206: - Assignee: Dennis Huo > ClassCastException using HiveContext with GoogleHadoopFileSystem as > fs.defaultFS

[jira] [Updated] (SPARK-9204) Add default params test to linear regression

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9204: - Assignee: holdenk > Add default params test to linear regression > ---

[jira] [Updated] (SPARK-9081) fillna/dropna should also fill/drop NaN values in addition to null values

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9081: - Assignee: Yijie Shen > fillna/dropna should also fill/drop NaN values in addition to null values > ---

[jira] [Updated] (SPARK-9157) Implement code generation for Substring

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9157: - Assignee: Tarek Auel > Implement code generation for Substring > --- >

[jira] [Updated] (SPARK-9036) SparkListenerExecutorMetricsUpdate messages not included in JsonProtocol

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9036: - Assignee: Ryan Williams > SparkListenerExecutorMetricsUpdate messages not included in JsonProtocol > -

[jira] [Updated] (SPARK-4234) Always do paritial aggregation

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4234: - Assignee: Cheng Hao > Always do paritial aggregation > --- > >

[jira] [Updated] (SPARK-9237) Added Top N Column Values for DataFrames

2015-07-22 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-9237: - Component/s: SQL I suppose my question is how commonly used this would be, to justify actually adding it

  1   2   3   >