[jira] [Commented] (SPARK-6491) Spark will put the current working dir to the CLASSPATH

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377407#comment-14377407 ] Apache Spark commented on SPARK-6491: - User 'marsishandsome' has created a pull reques

[jira] [Updated] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

2015-03-23 Thread Zhang JiaJin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang JiaJin updated SPARK-6487: Description: [~mengxr] [~zhangyouhua] Sequential pattern mining is an important branch in the patter

[jira] [Created] (SPARK-6491) Spark will put the current working dir to the CLASSPATH

2015-03-23 Thread Liangliang Gu (JIRA)
Liangliang Gu created SPARK-6491: Summary: Spark will put the current working dir to the CLASSPATH Key: SPARK-6491 URL: https://issues.apache.org/jira/browse/SPARK-6491 Project: Spark Issue T

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-23 Thread Henry Saputra (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377386#comment-14377386 ] Henry Saputra commented on SPARK-6479: -- What do you mean by "migrating Tachyon to new

[jira] [Created] (SPARK-6490) Deprecate configurations for "askWithReply" and use new configuration names

2015-03-23 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6490: --- Summary: Deprecate configurations for "askWithReply" and use new configuration names Key: SPARK-6490 URL: https://issues.apache.org/jira/browse/SPARK-6490 Project: Spar

[jira] [Commented] (SPARK-6375) Bad formatting in analysis errors

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377366#comment-14377366 ] Apache Spark commented on SPARK-6375: - User 'marmbrus' has created a pull request for

[jira] [Assigned] (SPARK-6375) Bad formatting in analysis errors

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6375: --- Assignee: Michael Armbrust > Bad formatting in analysis errors >

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-03-23 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377357#comment-14377357 ] Debasish Das edited comment on SPARK-2426 at 3/24/15 6:13 AM: --

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-03-23 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377357#comment-14377357 ] Debasish Das edited comment on SPARK-2426 at 3/24/15 6:11 AM: --

[jira] [Comment Edited] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-03-23 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377357#comment-14377357 ] Debasish Das edited comment on SPARK-2426 at 3/24/15 6:11 AM: --

[jira] [Commented] (SPARK-2426) Quadratic Minimization for MLlib ALS

2015-03-23 Thread Debasish Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377357#comment-14377357 ] Debasish Das commented on SPARK-2426: - [~acopich] From your comment before "Anyway, l2

[jira] [Commented] (SPARK-3306) Addition of external resource dependency in executors

2015-03-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377343#comment-14377343 ] Reynold Xin commented on SPARK-3306: Can you elaborate on why this needs to be Spark a

[jira] [Commented] (SPARK-6483) Spark SQL udf(ScalaUdf) is very slow

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377338#comment-14377338 ] Apache Spark commented on SPARK-6483: - User 'zzcclp' has created a pull request for th

[jira] [Updated] (SPARK-5692) Model import/export for Word2Vec

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-5692: - Assignee: Manoj Kumar (was: ANUPAM MEDIRATTA) > Model import/export for Word2Vec > --

[jira] [Commented] (SPARK-5692) Model import/export for Word2Vec

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377266#comment-14377266 ] Xiangrui Meng commented on SPARK-5692: -- [~anupamme] You should get familiar with Scal

[jira] [Commented] (SPARK-6352) Supporting non-default OutputCommitter when using saveAsParquetFile

2015-03-23 Thread Pei-Lun Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377264#comment-14377264 ] Pei-Lun Lee commented on SPARK-6352: The above PR adds a new hadoop config value "spa

[jira] [Resolved] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-23 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Williams resolved SPARK-6449. -- Resolution: Implemented Fix Version/s: 1.3.0 > Driver OOM results in reported application

[jira] [Commented] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-23 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377251#comment-14377251 ] Ryan Williams commented on SPARK-6449: -- Seems like this was fixed as of [SPARK-6018|

[jira] [Commented] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-23 Thread Ryan Williams (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377249#comment-14377249 ] Ryan Williams commented on SPARK-6449: -- It doesn't look like it; [here is a gist|htt

[jira] [Updated] (SPARK-6489) Optimize lateral view with explode to not read unnecessary columns

2015-03-23 Thread Konstantin Shaposhnikov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shaposhnikov updated SPARK-6489: --- Description: Currently a query with "lateral view explode(...)" results in an

[jira] [Commented] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377234#comment-14377234 ] Xiangrui Meng commented on SPARK-6487: -- [~Zhang JiaJin] I'm not very familiar with pa

[jira] [Created] (SPARK-6489) Optimize lateral view with explode to not read unnecessary columns

2015-03-23 Thread Konstantin Shaposhnikov (JIRA)
Konstantin Shaposhnikov created SPARK-6489: -- Summary: Optimize lateral view with explode to not read unnecessary columns Key: SPARK-6489 URL: https://issues.apache.org/jira/browse/SPARK-6489

[jira] [Commented] (SPARK-4036) Add Conditional Random Fields (CRF) algorithm to Spark MLlib

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377223#comment-14377223 ] Xiangrui Meng commented on SPARK-4036: -- You don't have to use or change the Optimizer

[jira] [Updated] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

2015-03-23 Thread Zhang JiaJin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhang JiaJin updated SPARK-6487: Description: [~mengxr] [~zhangyouhua] Sequential pattern mining is an important branch in the patter

[jira] [Created] (SPARK-6488) Support addition/multiplication in PySpark's BlockMatrix

2015-03-23 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6488: Summary: Support addition/multiplication in PySpark's BlockMatrix Key: SPARK-6488 URL: https://issues.apache.org/jira/browse/SPARK-6488 Project: Spark Issue

[jira] [Created] (SPARK-6487) Add sequential pattern mining algorithm to Spark MLlib

2015-03-23 Thread Zhang JiaJin (JIRA)
Zhang JiaJin created SPARK-6487: --- Summary: Add sequential pattern mining algorithm to Spark MLlib Key: SPARK-6487 URL: https://issues.apache.org/jira/browse/SPARK-6487 Project: Spark Issue Type

[jira] [Updated] (SPARK-6486) Add BlockMatrix in PySpark

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng updated SPARK-6486: - Description: We should add BlockMatrix to PySpark. Internally, we can use DataFrames and MatrixUDT

[jira] [Created] (SPARK-6486) Add BlockMatrix in PySpark

2015-03-23 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6486: Summary: Add BlockMatrix in PySpark Key: SPARK-6486 URL: https://issues.apache.org/jira/browse/SPARK-6486 Project: Spark Issue Type: Sub-task Compo

[jira] [Created] (SPARK-6485) Add CoordinateMatrix/RowMatrix/IndexedRowMatrix in PySpark

2015-03-23 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6485: Summary: Add CoordinateMatrix/RowMatrix/IndexedRowMatrix in PySpark Key: SPARK-6485 URL: https://issues.apache.org/jira/browse/SPARK-6485 Project: Spark Issu

[jira] [Commented] (SPARK-6100) Distributed linear algebra in PySpark/MLlib

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377208#comment-14377208 ] Xiangrui Meng commented on SPARK-6100: -- We don't have APIs for distributed matrices i

[jira] [Commented] (SPARK-3278) Isotonic regression

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377205#comment-14377205 ] Xiangrui Meng commented on SPARK-3278: -- Did you try truncating the digits of x to red

[jira] [Commented] (SPARK-6464) Add a new transformation of rdd named processCoalesce which was particularly to deal with the small and cached rdd

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377206#comment-14377206 ] Apache Spark commented on SPARK-6464: - User 'SaintBacchus' has created a pull request

[jira] [Resolved] (SPARK-6334) spark-local dir not getting cleared during ALS

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6334. -- Resolution: Duplicate SPARK-5955 was merged. So if you can use the latest master, you can set c

[jira] [Commented] (SPARK-3735) Sending the factor directly or AtA based on the cost in ALS

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377200#comment-14377200 ] Xiangrui Meng commented on SPARK-3735: -- The proposal is actually something different.

[jira] [Commented] (SPARK-6192) Enhance MLlib's Python API (GSoC 2015)

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377197#comment-14377197 ] Xiangrui Meng commented on SPARK-6192: -- Thanks for the update! The current version lo

[jira] [Commented] (SPARK-6361) Support adding a column with metadata in DataFrames

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377187#comment-14377187 ] Apache Spark commented on SPARK-6361: - User 'mengxr' has created a pull request for th

[jira] [Commented] (SPARK-3720) support ORC in spark sql

2015-03-23 Thread iward (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377155#comment-14377155 ] iward commented on SPARK-3720: -- [~zhanzhang], I see. since the patch is delayed, so we can't

[jira] [Commented] (SPARK-1006) MLlib ALS gets stack overflow with too many iterations

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377152#comment-14377152 ] Xiangrui Meng commented on SPARK-1006: -- This is fixed as part of SPARK-5955, where we

[jira] [Commented] (SPARK-6430) Cannot resolve column correctlly when using left semi join

2015-03-23 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377139#comment-14377139 ] zzc commented on SPARK-6430: what's wrong with this? > Cannot resolve column correctlly when

[jira] [Updated] (SPARK-1684) Merge script should standardize SPARK-XXX prefix

2015-03-23 Thread Michelle Casbon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michelle Casbon updated SPARK-1684: --- Attachment: spark_pulls_before_after.txt Test data (spark_pulls_before_after.txt): titles from

[jira] [Commented] (SPARK-1684) Merge script should standardize SPARK-XXX prefix

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377108#comment-14377108 ] Apache Spark commented on SPARK-1684: - User 'texasmichelle' has created a pull request

[jira] [Commented] (SPARK-5368) Spark should support NAT (via akka improvements)

2015-03-23 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377106#comment-14377106 ] jay vyas commented on SPARK-5368: - looks like this is subsumed maybe by the work going on

[jira] [Commented] (SPARK-6229) Support SASL encryption in network/common module

2015-03-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377102#comment-14377102 ] Marcelo Vanzin commented on SPARK-6229: --- Hi, me again. So I finally got back to actu

[jira] [Comment Edited] (SPARK-5368) Spark should support NAT (via akka improvements)

2015-03-23 Thread jay vyas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375163#comment-14375163 ] jay vyas edited comment on SPARK-5368 at 3/24/15 1:59 AM: -- Okay,

[jira] [Commented] (SPARK-5368) Spark should support NAT (via akka improvements)

2015-03-23 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377096#comment-14377096 ] Matthew Farrellee commented on SPARK-5368: -- [~jayunit100] the relevant config is

[jira] [Comment Edited] (SPARK-5368) Spark should support NAT (via akka improvements)

2015-03-23 Thread Matthew Farrellee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377096#comment-14377096 ] Matthew Farrellee edited comment on SPARK-5368 at 3/24/15 1:58 AM: -

[jira] [Commented] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

2015-03-23 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377080#comment-14377080 ] Josh Rosen commented on SPARK-6484: --- To provide some extra context for this JIRA, I thin

[jira] [Updated] (SPARK-5941) `def table` is not using the unresolved logical plan `UnresolvedRelation`

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5941: Component/s: (was: DataFrame) SQL > `def table` is not using the unreso

[jira] [Updated] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6465: Component/s: (was: DataFrame) SQL > GenericRowWithSchema: KryoException

[jira] [Updated] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6475: Component/s: (was: DataFrame) > DataFrame should support array types when creating DFs f

[jira] [Updated] (SPARK-6189) Pandas to DataFrame conversion should check field names for periods

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6189: Component/s: (was: DataFrame) > Pandas to DataFrame conversion should check field names

[jira] [Updated] (SPARK-5919) Enable broadcast joins for Parquet files

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5919: Component/s: (was: DataFrame) SQL > Enable broadcast joins for Parquet

[jira] [Created] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

2015-03-23 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-6484: --- Summary: Ganglia metrics xml reporter doesn't escape correctly Key: SPARK-6484 URL: https://issues.apache.org/jira/browse/SPARK-6484 Project: Spark Iss

[jira] [Created] (SPARK-6483) Spark SQL udf(ScalaUdf) is very slow

2015-03-23 Thread zzc (JIRA)
zzc created SPARK-6483: -- Summary: Spark SQL udf(ScalaUdf) is very slow Key: SPARK-6483 URL: https://issues.apache.org/jira/browse/SPARK-6483 Project: Spark Issue Type: Improvement Components:

[jira] [Created] (SPARK-6482) Remove synchronization of Hive Native commands

2015-03-23 Thread David Ross (JIRA)
David Ross created SPARK-6482: - Summary: Remove synchronization of Hive Native commands Key: SPARK-6482 URL: https://issues.apache.org/jira/browse/SPARK-6482 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-6481) Set "In Progress" when a PR is opened for an issue

2015-03-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377034#comment-14377034 ] Nicholas Chammas commented on SPARK-6481: - I'm guessing this will be done via [gi

[jira] [Created] (SPARK-6481) Set "In Progress" when a PR is opened for an issue

2015-03-23 Thread Michael Armbrust (JIRA)
Michael Armbrust created SPARK-6481: --- Summary: Set "In Progress" when a PR is opened for an issue Key: SPARK-6481 URL: https://issues.apache.org/jira/browse/SPARK-6481 Project: Spark Issue

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-23 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376978#comment-14376978 ] Zhan Zhang commented on SPARK-6479: --- The current API may not be good enough as it has so

[jira] [Resolved] (SPARK-6124) Support jdbc connection properties in OPTIONS part of the query

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6124. - Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by p

[jira] [Commented] (SPARK-6373) Add SSL/TLS for the Netty based BlockTransferService

2015-03-23 Thread Jeffrey Turpin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376907#comment-14376907 ] Jeffrey Turpin commented on SPARK-6373: --- Hey Aaron, Thanks for the feedback! I defi

[jira] [Updated] (SPARK-6369) InsertIntoHiveTable should use logic from SparkHadoopWriter

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6369: Assignee: Cheng Lian > InsertIntoHiveTable should use logic from SparkHadoopWriter > ---

[jira] [Updated] (SPARK-6437) SQL ExternalSort should use CompletionIterator to clean up temp files

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6437: Assignee: Michael Armbrust > SQL ExternalSort should use CompletionIterator to clean up temp

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5508: Priority: Critical (was: Major) > Arrays and Maps stored with Hive Parquet Serde may not be

[jira] [Updated] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-5508: Assignee: Cheng Lian > Arrays and Maps stored with Hive Parquet Serde may not be able to rea

[jira] [Commented] (SPARK-6480) histogram() bucket function is wrong in some simple edge cases

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376894#comment-14376894 ] Apache Spark commented on SPARK-6480: - User 'srowen' has created a pull request for th

[jira] [Updated] (SPARK-4925) Publish Spark SQL hive-thriftserver maven artifact

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-4925: Assignee: Patrick Wendell > Publish Spark SQL hive-thriftserver maven artifact > --

[jira] [Updated] (SPARK-2973) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2973: Priority: Blocker (was: Critical) > Use LocalRelation for all ExecutedCommands, avoid job f

[jira] [Updated] (SPARK-2973) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2973: Summary: Use LocalRelation for all ExecutedCommands, avoid job for take/collect() (was: Add

[jira] [Updated] (SPARK-2973) Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-2973: Target Version/s: 1.4.0 (was: 1.3.1) > Use LocalRelation for all ExecutedCommands, avoid jo

[jira] [Updated] (SPARK-6450) Native Parquet reader does not assign table name as qualifier

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6450: Summary: Native Parquet reader does not assign table name as qualifier (was: Self joining q

[jira] [Assigned] (SPARK-6451) Support CombineSum in Code Gen

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6451: --- Assignee: Michael Armbrust > Support CombineSum in Code Gen > ---

[jira] [Assigned] (SPARK-6054) SQL UDF returning object of case class; regression from 1.2.0

2015-03-23 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6054: --- Assignee: Michael Armbrust > SQL UDF returning object of case class; regression from

[jira] [Created] (SPARK-6480) histogram() bucket function is wrong in some simple edge cases

2015-03-23 Thread Sean Owen (JIRA)
Sean Owen created SPARK-6480: Summary: histogram() bucket function is wrong in some simple edge cases Key: SPARK-6480 URL: https://issues.apache.org/jira/browse/SPARK-6480 Project: Spark Issue T

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-23 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6479: -- Attachment: SparkOffheapsupportbyHDFS.pdf The design doc also includes stuff from SPARK-6112 > Create o

[jira] [Updated] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-6479: --- Summary: Create off-heap block storage API (internal) (was: Create off-heap block storage API) > Cre

[jira] [Commented] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

2015-03-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376849#comment-14376849 ] Reynold Xin commented on SPARK-6112: [~zhanzhang] I created https://issues.apache.org/

[jira] [Comment Edited] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

2015-03-23 Thread Allan Douglas R. de Oliveira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376833#comment-14376833 ] Allan Douglas R. de Oliveira edited comment on SPARK-5928 at 3/23/15 10:54 PM: -

[jira] [Created] (SPARK-6479) Create off-heap block storage API

2015-03-23 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-6479: -- Summary: Create off-heap block storage API Key: SPARK-6479 URL: https://issues.apache.org/jira/browse/SPARK-6479 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

2015-03-23 Thread Allan Douglas R. de Oliveira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376833#comment-14376833 ] Allan Douglas R. de Oliveira commented on SPARK-5928: - I will answer w

[jira] [Commented] (SPARK-6478) new RDD.pipeWithPartition method

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376827#comment-14376827 ] Apache Spark commented on SPARK-6478: - User 'redbaron' has created a pull request for

[jira] [Created] (SPARK-6478) new RDD.pipeWithPartition method

2015-03-23 Thread Maxim Ivanov (JIRA)
Maxim Ivanov created SPARK-6478: --- Summary: new RDD.pipeWithPartition method Key: SPARK-6478 URL: https://issues.apache.org/jira/browse/SPARK-6478 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-6112) Provide OffHeap support through HDFS RAM_DISK

2015-03-23 Thread Zhan Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-6112: -- Attachment: SparkOffheapsupportbyHDFS.pdf Design doc for hdfs offheap support > Provide OffHeap support

[jira] [Commented] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-23 Thread Calvin Jia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376813#comment-14376813 ] Calvin Jia commented on SPARK-6122: --- [~pwendell] Are you referring to the issues here:

[jira] [Assigned] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng reassigned SPARK-6475: Assignee: Xiangrui Meng > DataFrame should support array types when creating DFs from JavaB

[jira] [Commented] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376780#comment-14376780 ] Apache Spark commented on SPARK-6475: - User 'mengxr' has created a pull request for th

[jira] [Reopened] (SPARK-6122) Upgrade Tachyon dependency to 0.6.0

2015-03-23 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell reopened SPARK-6122: I reverted this because it looks like it was responsible for some testing failures due to the d

[jira] [Commented] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376740#comment-14376740 ] Apache Spark commented on SPARK-6477: - User 'brennonyork' has created a pull request f

[jira] [Updated] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-23 Thread Brennon York (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brennon York updated SPARK-6477: Issue Type: Improvement (was: Bug) > Run MIMA tests before the Spark test suite > -

[jira] [Commented] (SPARK-5338) Support cluster mode with Mesos

2015-03-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376731#comment-14376731 ] Apache Spark commented on SPARK-5338: - User 'tnachen' has created a pull request for t

[jira] [Created] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-23 Thread Brennon York (JIRA)
Brennon York created SPARK-6477: --- Summary: Run MIMA tests before the Spark test suite Key: SPARK-6477 URL: https://issues.apache.org/jira/browse/SPARK-6477 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-6476) Spark fileserver not started on same IP as using spark.driver.host

2015-03-23 Thread Rares Vernica (JIRA)
Rares Vernica created SPARK-6476: Summary: Spark fileserver not started on same IP as using spark.driver.host Key: SPARK-6476 URL: https://issues.apache.org/jira/browse/SPARK-6476 Project: Spark

[jira] [Created] (SPARK-6475) DataFrame should support array types when creating DFs from JavaBeans.

2015-03-23 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6475: Summary: DataFrame should support array types when creating DFs from JavaBeans. Key: SPARK-6475 URL: https://issues.apache.org/jira/browse/SPARK-6475 Project: Spark

[jira] [Commented] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376584#comment-14376584 ] Nicholas Chammas commented on SPARK-6474: - This change also fits the pattern of [

[jira] [Commented] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Andrew Drozdov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376579#comment-14376579 ] Andrew Drozdov commented on SPARK-6474: --- Great, and thanks. Taking a look now. > Re

[jira] [Updated] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-6474: Issue Type: Improvement (was: Bug) > Replace image.run with connection.run_instances in spa

[jira] [Resolved] (SPARK-6308) VectorUDT is displayed as `vecto` in dtypes

2015-03-23 Thread Xiangrui Meng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangrui Meng resolved SPARK-6308. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5118 [https://githu

[jira] [Comment Edited] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376572#comment-14376572 ] Nicholas Chammas edited comment on SPARK-6474 at 3/23/15 8:29 PM: --

[jira] [Updated] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-6474: Priority: Minor (was: Major) > Replace image.run with connection.run_instances in spark_ec2

[jira] [Commented] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376572#comment-14376572 ] Nicholas Chammas commented on SPARK-6474: - LGTM. > Replace image.run with connect

[jira] [Updated] (SPARK-6474) Replace image.run with connection.run_instances in spark_ec2.py

2015-03-23 Thread Andrew Drozdov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Drozdov updated SPARK-6474: -- Summary: Replace image.run with connection.run_instances in spark_ec2.py (was: Replace image.ru

  1   2   3   >