[jira] [Created] (SPARK-21622) Support Offset in SparkR

Wayne Zhang created SPARK-21622: --- Summary: Support Offset in SparkR Key: SPARK-21622 URL: https://issues.apache.org/jira/browse/SPARK-21622 Project: Spark Issue Type: Improvement Comp

[jira] [Commented] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112274#comment-16112274 ] Kevin Zhang commented on SPARK-21590: - [~brkyvz] I don' t know the test you illustrat

[jira] [Commented] (SPARK-21034) Filter not getting pushed down the groupBy clause when first() or last() aggregate function is used

[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112239#comment-16112239 ] Abhijit Bhole commented on SPARK-21034: --- Is there any example where a filter on a c

[jira] [Created] (SPARK-21621) Reset numRecordsWritten after DiskBlockObjectWriter.commitAndGet called

Xianyang Liu created SPARK-21621: Summary: Reset numRecordsWritten after DiskBlockObjectWriter.commitAndGet called Key: SPARK-21621 URL: https://issues.apache.org/jira/browse/SPARK-21621 Project: Spar

[jira] [Commented] (SPARK-21546) dropDuplicates with watermark yields RuntimeException due to binding failure

[ https://issues.apache.org/jira/browse/SPARK-21546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112146#comment-16112146 ] Kevin Zhang commented on SPARK-21546: - [~zsxwing] in my case I hope to use a watermar

[jira] [Updated] (SPARK-21620) Add metrics url in spark web ui.

[ https://issues.apache.org/jira/browse/SPARK-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21620: --- Description: Add metrics url in spark web ui. Big data system several other components of the

[jira] [Updated] (SPARK-21620) Add metrics url in spark web ui.

[ https://issues.apache.org/jira/browse/SPARK-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-21620: --- Summary: Add metrics url in spark web ui. (was: Add metrics in web ui.) > Add metrics url in

[jira] [Created] (SPARK-21620) Add metrics in web ui.

guoxiaolongzte created SPARK-21620: -- Summary: Add metrics in web ui. Key: SPARK-21620 URL: https://issues.apache.org/jira/browse/SPARK-21620 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-21619) Fail the execution of canonicalized plans explicitly

Reynold Xin created SPARK-21619: --- Summary: Fail the execution of canonicalized plans explicitly Key: SPARK-21619 URL: https://issues.apache.org/jira/browse/SPARK-21619 Project: Spark Issue Type

[jira] [Updated] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-12717: - Fix Version/s: 2.2.1 2.1.2 > pyspark broadcast fails when using multiple threa

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112036#comment-16112036 ] Saisai Shao commented on SPARK-21570: - Sorry I'm not familiar with NFS/Lustre FS, doe

[jira] [Commented] (SPARK-21615) Fix broken redirect in collaborative filtering docs to databricks training repo

[ https://issues.apache.org/jira/browse/SPARK-21615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112026#comment-16112026 ] Hyukjin Kwon commented on SPARK-21615: -- It should have been done automatically but t

[jira] [Created] (SPARK-21618) http(s) not accepted in spark-submit jar uri

Ben Mayne created SPARK-21618: - Summary: http(s) not accepted in spark-submit jar uri Key: SPARK-21618 URL: https://issues.apache.org/jira/browse/SPARK-21618 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-21615) Fix broken redirect in collaborative filtering docs to databricks training repo

[ https://issues.apache.org/jira/browse/SPARK-21615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112022#comment-16112022 ] Ayush Singh commented on SPARK-21615: - [~hyukjin.kwon] That's me, is there any way to

[jira] [Commented] (SPARK-14712) spark.ml LogisticRegressionModel.toString should summarize model

[ https://issues.apache.org/jira/browse/SPARK-14712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112003#comment-16112003 ] Bravo Zhang commented on SPARK-14712: - User 'bravo-zhang' has created a pull request

[jira] [Commented] (SPARK-20086) issue with pyspark 2.1.0 window function

[ https://issues.apache.org/jira/browse/SPARK-20086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111838#comment-16111838 ] Jacky Shen commented on SPARK-20086: will there be a fix for 2.1.0? Regards, Jacky

[jira] [Created] (SPARK-21617) ALTER TABLE...ADD COLUMNS creates invalid metadata in Hive metastore for DS tables

Marcelo Vanzin created SPARK-21617: -- Summary: ALTER TABLE...ADD COLUMNS creates invalid metadata in Hive metastore for DS tables Key: SPARK-21617 URL: https://issues.apache.org/jira/browse/SPARK-21617

[jira] [Updated] (SPARK-21542) Helper functions for custom Python Persistence

[ https://issues.apache.org/jira/browse/SPARK-21542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-21542: -- Shepherd: Joseph K. Bradley > Helper functions for custom Python Persistence >

[jira] [Resolved] (SPARK-21546) dropDuplicates with watermark yields RuntimeException due to binding failure

[ https://issues.apache.org/jira/browse/SPARK-21546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21546. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.3.0

[jira] [Commented] (SPARK-21570) File __spark_libs__XXX.zip does not exist on networked file system w/ yarn

[ https://issues.apache.org/jira/browse/SPARK-21570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111701#comment-16111701 ] Albert Chu commented on SPARK-21570: My setup is unique. The primary unique part is

[jira] [Updated] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anirudh Ramanathan updated SPARK-18278: --- Summary: SPIP: Support native submission of spark jobs to a kubernetes cluster (was:

[jira] [Commented] (SPARK-21612) Allow unicode strings in __getitem__ of StructType

[ https://issues.apache.org/jira/browse/SPARK-21612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111601#comment-16111601 ] Hyukjin Kwon commented on SPARK-21612: -- User 'rik-coenders' has created a pull reque

[jira] [Commented] (SPARK-21615) Fix broken redirect in collaborative filtering docs to databricks training repo

[ https://issues.apache.org/jira/browse/SPARK-21615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111606#comment-16111606 ] Hyukjin Kwon commented on SPARK-21615: -- User 'singhay' has created a pull request fo

[jira] [Commented] (SPARK-21110) Structs should be usable in inequality filters

[ https://issues.apache.org/jira/browse/SPARK-21110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111603#comment-16111603 ] Hyukjin Kwon commented on SPARK-21110: -- User 'aray' has created a pull request for t

[jira] [Commented] (SPARK-14932) Allow DataFrame.replace() to replace values with None

[ https://issues.apache.org/jira/browse/SPARK-14932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111605#comment-16111605 ] Hyukjin Kwon commented on SPARK-14932: -- User 'bravo-zhang' has created a pull reques

[jira] [Commented] (SPARK-20713) Speculative task that got CommitDenied exception shows up as failed

[ https://issues.apache.org/jira/browse/SPARK-20713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111604#comment-16111604 ] Hyukjin Kwon commented on SPARK-20713: -- User 'nlyu' has created a pull request for t

[jira] [Commented] (SPARK-21546) dropDuplicates with watermark yields RuntimeException due to binding failure

[ https://issues.apache.org/jira/browse/SPARK-21546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111607#comment-16111607 ] Hyukjin Kwon commented on SPARK-21546: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-21599) Collecting column statistics for datasource tables may fail with java.util.NoSuchElementException

[ https://issues.apache.org/jira/browse/SPARK-21599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111592#comment-16111592 ] Hyukjin Kwon commented on SPARK-21599: -- User 'dilipbiswal' has created a pull reques

[jira] [Commented] (SPARK-21603) The wholestage codegen will be much slower then wholestage codegen is closed when the function is too long

[ https://issues.apache.org/jira/browse/SPARK-21603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111596#comment-16111596 ] Hyukjin Kwon commented on SPARK-21603: -- User 'eatoncys' has created a pull request f

[jira] [Commented] (SPARK-21608) Window rangeBetween() API should allow literal boundary

[ https://issues.apache.org/jira/browse/SPARK-21608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111599#comment-16111599 ] Hyukjin Kwon commented on SPARK-21608: -- User 'jiangxb1987' has created a pull reques

[jira] [Commented] (SPARK-9221) Support IntervalType in Range Frame

[ https://issues.apache.org/jira/browse/SPARK-9221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111600#comment-16111600 ] Hyukjin Kwon commented on SPARK-9221: - User 'jiangxb1987' has created a pull request f

[jira] [Commented] (SPARK-21567) Dataset with Tuple of type alias throws error

[ https://issues.apache.org/jira/browse/SPARK-21567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111597#comment-16111597 ] Hyukjin Kwon commented on SPARK-21567: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-19112) add codec for ZStandard

[ https://issues.apache.org/jira/browse/SPARK-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111593#comment-16111593 ] Hyukjin Kwon commented on SPARK-19112: -- User 'sitalkedia' has created a pull request

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111585#comment-16111585 ] Hyukjin Kwon commented on SPARK-19634: -- User 'WeichenXu123' has created a pull reque

[jira] [Commented] (SPARK-21330) Bad partitioning does not allow to read a JDBC table with extreme values on the partition column

[ https://issues.apache.org/jira/browse/SPARK-21330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111587#comment-16111587 ] Hyukjin Kwon commented on SPARK-21330: -- User 'aray' has created a pull request for t

[jira] [Commented] (SPARK-18535) Redact sensitive information from Spark logs and UI

[ https://issues.apache.org/jira/browse/SPARK-18535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111589#comment-16111589 ] Hyukjin Kwon commented on SPARK-18535: -- User 'dmvieira' has created a pull request f

[jira] [Commented] (SPARK-21587) Filter pushdown for EventTime Watermark Operator

[ https://issues.apache.org/jira/browse/SPARK-21587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111582#comment-16111582 ] Hyukjin Kwon commented on SPARK-21587: -- User 'joseph-torres' has created a pull requ

[jira] [Commented] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

[ https://issues.apache.org/jira/browse/SPARK-21571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111584#comment-16111584 ] Hyukjin Kwon commented on SPARK-21571: -- User 'ericvandenbergfb' has created a pull r

[jira] [Commented] (SPARK-21596) Audit the places calling HDFSMetadataLog.get

[ https://issues.apache.org/jira/browse/SPARK-21596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111586#comment-16111586 ] Hyukjin Kwon commented on SPARK-21596: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-19720) Redact sensitive information from SparkSubmit console output

[ https://issues.apache.org/jira/browse/SPARK-19720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111591#comment-16111591 ] Hyukjin Kwon commented on SPARK-19720: -- User 'dmvieira' has created a pull request f

[jira] [Commented] (SPARK-10878) Race condition when resolving Maven coordinates via Ivy

[ https://issues.apache.org/jira/browse/SPARK-10878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111588#comment-16111588 ] Hyukjin Kwon commented on SPARK-10878: -- User 'Victsm' has created a pull request for

[jira] [Commented] (SPARK-21580) There's a bug with `Group by ordinal`

[ https://issues.apache.org/jira/browse/SPARK-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111575#comment-16111575 ] Hyukjin Kwon commented on SPARK-21580: -- User '10110346' has created a pull request f

[jira] [Commented] (SPARK-21559) Remove Mesos fine-grained mode

[ https://issues.apache.org/jira/browse/SPARK-21559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111577#comment-16111577 ] Hyukjin Kwon commented on SPARK-21559: -- User 'skonto' has created a pull request for

[jira] [Commented] (SPARK-21584) Update R method for summary to call new implementation

[ https://issues.apache.org/jira/browse/SPARK-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111578#comment-16111578 ] Hyukjin Kwon commented on SPARK-21584: -- User 'aray' has created a pull request for t

[jira] [Commented] (SPARK-8288) ScalaReflection should also try apply methods defined in companion objects when inferring schema from a Product type

[ https://issues.apache.org/jira/browse/SPARK-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111572#comment-16111572 ] Hyukjin Kwon commented on SPARK-8288: - User 'drewrobb' has created a pull request for

[jira] [Commented] (SPARK-20963) Support column aliases for aliased relation in FROM clause

[ https://issues.apache.org/jira/browse/SPARK-20963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111574#comment-16111574 ] Hyukjin Kwon commented on SPARK-20963: -- User 'maropu' has created a pull request for

[jira] [Commented] (SPARK-21574) set hive.exec.max.dynamic.partitions lose effect

[ https://issues.apache.org/jira/browse/SPARK-21574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111573#comment-16111573 ] Hyukjin Kwon commented on SPARK-21574: -- User 'wangyum' has created a pull request fo

[jira] [Commented] (SPARK-21583) Create a ColumnarBatch with ArrowColumnVectors for row based iteration

[ https://issues.apache.org/jira/browse/SPARK-21583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111579#comment-16111579 ] Hyukjin Kwon commented on SPARK-21583: -- User 'BryanCutler' has created a pull reques

[jira] [Commented] (SPARK-21254) History UI: Taking over 1 minute for initial page display

[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111576#comment-16111576 ] Hyukjin Kwon commented on SPARK-21254: -- User '2ooom' has created a pull request for

[jira] [Commented] (SPARK-20433) Update jackson-databind to 2.6.7.1

[ https://issues.apache.org/jira/browse/SPARK-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111580#comment-16111580 ] Hyukjin Kwon commented on SPARK-20433: -- User 'ash211' has created a pull request for

[jira] [Commented] (SPARK-21552) Add decimal type support to ArrowWriter.

[ https://issues.apache.org/jira/browse/SPARK-21552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111565#comment-16111565 ] Hyukjin Kwon commented on SPARK-21552: -- User 'ueshin' has created a pull request for

[jira] [Commented] (SPARK-21306) OneVsRest Conceals Columns That May Be Relevant To Underlying Classifier

[ https://issues.apache.org/jira/browse/SPARK-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111570#comment-16111570 ] Hyukjin Kwon commented on SPARK-21306: -- User 'facaiy' has created a pull request for

[jira] [Commented] (SPARK-19720) Redact sensitive information from SparkSubmit console output

[ https://issues.apache.org/jira/browse/SPARK-19720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111571#comment-16111571 ] Hyukjin Kwon commented on SPARK-19720: -- User 'dmvieira' has created a pull request f

[jira] [Commented] (SPARK-21560) Add hold mode for the LiveListenerBus

[ https://issues.apache.org/jira/browse/SPARK-21560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111566#comment-16111566 ] Hyukjin Kwon commented on SPARK-21560: -- User 'xuanyuanking' has created a pull reque

[jira] [Commented] (SPARK-21551) pyspark's collect fails when getaddrinfo is too slow

[ https://issues.apache.org/jira/browse/SPARK-21551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111564#comment-16111564 ] Hyukjin Kwon commented on SPARK-21551: -- User 'peay' has created a pull request for t

[jira] [Commented] (SPARK-21485) API Documentation for Spark SQL functions

[ https://issues.apache.org/jira/browse/SPARK-21485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111563#comment-16111563 ] Hyukjin Kwon commented on SPARK-21485: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-21566) Python method for summary

[ https://issues.apache.org/jira/browse/SPARK-21566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111567#comment-16111567 ] Hyukjin Kwon commented on SPARK-21566: -- User 'aray' has created a pull request for t

[jira] [Commented] (SPARK-21306) OneVsRest Conceals Columns That May Be Relevant To Underlying Classifier

[ https://issues.apache.org/jira/browse/SPARK-21306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111569#comment-16111569 ] Hyukjin Kwon commented on SPARK-21306: -- User 'facaiy' has created a pull request for

[jira] [Commented] (SPARK-20679) Let ML ALS recommend for a subset of users/items

[ https://issues.apache.org/jira/browse/SPARK-20679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111562#comment-16111562 ] Hyukjin Kwon commented on SPARK-20679: -- User 'MLnick' has created a pull request for

[jira] [Commented] (SPARK-20990) Multi-line support for JSON

[ https://issues.apache.org/jira/browse/SPARK-20990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111553#comment-16111553 ] Hyukjin Kwon commented on SPARK-20990: -- User 'mgaido91' has created a pull request f

[jira] [Commented] (SPARK-21544) Test jar of some module should not install or deploy twice

[ https://issues.apache.org/jira/browse/SPARK-21544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111560#comment-16111560 ] Hyukjin Kwon commented on SPARK-21544: -- User 'caneGuy' has created a pull request fo

[jira] [Commented] (SPARK-20822) Generate code to get value from CachedBatchColumnVector in ColumnarBatch

[ https://issues.apache.org/jira/browse/SPARK-20822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111561#comment-16111561 ] Hyukjin Kwon commented on SPARK-20822: -- User 'kiszk' has created a pull request for

[jira] [Commented] (SPARK-21535) Reduce memory requirement for CrossValidator and TrainValidationSplit

[ https://issues.apache.org/jira/browse/SPARK-21535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111556#comment-16111556 ] Hyukjin Kwon commented on SPARK-21535: -- User 'hhbyyh' has created a pull request for

[jira] [Commented] (SPARK-21070) Pick up cloudpickle upgrades from cloudpickle python module

[ https://issues.apache.org/jira/browse/SPARK-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111558#comment-16111558 ] Hyukjin Kwon commented on SPARK-21070: -- User 'holdenk' has created a pull request fo

[jira] [Commented] (SPARK-21481) Add indexOf method in ml.feature.HashingTF similar to mllib.feature.HashingTF

[ https://issues.apache.org/jira/browse/SPARK-21481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111559#comment-16111559 ] Hyukjin Kwon commented on SPARK-21481: -- User 'facaiy' has created a pull request for

[jira] [Commented] (SPARK-20396) groupBy().apply() with pandas udf in pyspark

[ https://issues.apache.org/jira/browse/SPARK-20396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111554#comment-16111554 ] Hyukjin Kwon commented on SPARK-20396: -- User 'icexelloss' has created a pull request

[jira] [Commented] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111548#comment-16111548 ] Imran Rashid commented on SPARK-13669: -- Since I dragged my feet about the usefulness

[jira] [Comment Edited] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111546#comment-16111546 ] Anirudh Ramanathan edited comment on SPARK-18278 at 8/2/17 7:11 PM: ---

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111546#comment-16111546 ] Anirudh Ramanathan commented on SPARK-18278: FYI: sandflee's question was add

[jira] [Resolved] (SPARK-21490) SparkLauncher may fail to redirect streams

[ https://issues.apache.org/jira/browse/SPARK-21490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21490. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.3.0 > SparkLaun

[jira] [Assigned] (SPARK-21584) Update R method for summary to call new implementation

[ https://issues.apache.org/jira/browse/SPARK-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-21584: Assignee: Andrew Ray > Update R method for summary to call new implementation > --

[jira] [Commented] (SPARK-21616) SparkR 2.3.0 migration guide, release note

[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111483#comment-16111483 ] Felix Cheung commented on SPARK-21616: -- SPARK-21584 > SparkR 2.3.0 migration guide,

[jira] [Commented] (SPARK-21584) Update R method for summary to call new implementation

[ https://issues.apache.org/jira/browse/SPARK-21584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111485#comment-16111485 ] Felix Cheung commented on SPARK-21584: -- https://github.com/apache/spark/pull/18786

[jira] [Updated] (SPARK-21616) SparkR 2.3.0 migration guide, release note

[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21616: - Fix Version/s: (was: 2.2.0) > SparkR 2.3.0 migration guide, release note > --

[jira] [Updated] (SPARK-21616) SparkR 2.3.0 migration guide, release note

[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21616: - Description: >From looking at changes since 2.2.0, this/these should be documented in the >migra

[jira] [Updated] (SPARK-21616) SparkR 2.3.0 migration guide, release note

[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21616: - Affects Version/s: (was: 2.2.0) 2.3.0 > SparkR 2.3.0 migration guide,

[jira] [Updated] (SPARK-21616) SparkR 2.3.0 migration guide, release note

[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21616: - Target Version/s: 2.3.0 (was: 2.2.0, 2.3.0) > SparkR 2.3.0 migration guide, release note > -

[jira] [Updated] (SPARK-21616) SparkR 2.3.0 migration guide, release note

[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-21616: - Summary: SparkR 2.3.0 migration guide, release note (was: CLONE - SparkR 2.2.0 migration guide,

[jira] [Created] (SPARK-21616) CLONE - SparkR 2.2.0 migration guide, release note

Felix Cheung created SPARK-21616: Summary: CLONE - SparkR 2.2.0 migration guide, release note Key: SPARK-21616 URL: https://issues.apache.org/jira/browse/SPARK-21616 Project: Spark Issue Type

[jira] [Commented] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111467#comment-16111467 ] Shixiong Zhu commented on SPARK-21590: -- [~brkyvz] Yeah, some people may process data

[jira] [Commented] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111460#comment-16111460 ] Burak Yavuz commented on SPARK-21590: - There are tests to make sure it supports "nega

[jira] [Commented] (SPARK-21590) Structured Streaming window start time should support negative values to adjust time zone

[ https://issues.apache.org/jira/browse/SPARK-21590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111458#comment-16111458 ] Burak Yavuz commented on SPARK-21590: - Does it output incorrect results if you provid

[jira] [Commented] (SPARK-21034) Filter not getting pushed down the groupBy clause when first() or last() aggregate function is used

[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111454#comment-16111454 ] Andrew Ray commented on SPARK-21034: Yes a=1 is the filter to be pushed down. It is n

[jira] [Commented] (SPARK-21565) aggregate query fails with watermark on eventTime but works with watermark on timestamp column generated by current_timestamp

[ https://issues.apache.org/jira/browse/SPARK-21565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111451#comment-16111451 ] Shixiong Zhu commented on SPARK-21565: -- Thanks for reporting it. I can reproduce the

[jira] [Commented] (SPARK-21597) Avg event time calculated in progress may be wrong

[ https://issues.apache.org/jira/browse/SPARK-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111441#comment-16111441 ] Shixiong Zhu commented on SPARK-21597: -- Resolved by https://github.com/apache/spark/

[jira] [Commented] (SPARK-21034) Filter not getting pushed down the groupBy clause when first() or last() aggregate function is used

[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111443#comment-16111443 ] Vish Persaud commented on SPARK-21034: -- [~a1ray] In the example provided, seems the

[jira] [Resolved] (SPARK-21597) Avg event time calculated in progress may be wrong

[ https://issues.apache.org/jira/browse/SPARK-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21597. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.3.0

[jira] [Updated] (SPARK-21597) Avg event time calculated in progress may be wrong

[ https://issues.apache.org/jira/browse/SPARK-21597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-21597: - Priority: Minor (was: Major) > Avg event time calculated in progress may be wrong >

[jira] [Commented] (SPARK-21615) Fix broken redirect in collaborative filtering docs to databricks training repo

[ https://issues.apache.org/jira/browse/SPARK-21615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111403#comment-16111403 ] Ayush Singh commented on SPARK-21615: - [~srowen] Hi, I opened a [PR|https://github.co

[jira] [Commented] (SPARK-21615) Fix broken redirect in collaborative filtering docs to databricks training repo

[ https://issues.apache.org/jira/browse/SPARK-21615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111367#comment-16111367 ] Sean Owen commented on SPARK-21615: --- Sure, just open a PR. If this is old content and t

[jira] [Created] (SPARK-21615) Fix broken redirect in collaborative filtering docs to databricks training repo

Ayush Singh created SPARK-21615: --- Summary: Fix broken redirect in collaborative filtering docs to databricks training repo Key: SPARK-21615 URL: https://issues.apache.org/jira/browse/SPARK-21615 Project

[jira] [Commented] (SPARK-12717) pyspark broadcast fails when using multiple threads

[ https://issues.apache.org/jira/browse/SPARK-12717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111304#comment-16111304 ] Bryan Cutler commented on SPARK-12717: -- Sure, I'll open a PR for 2.2 and ping you.

[jira] [Commented] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111232#comment-16111232 ] Joseph Wang commented on SPARK-21594: - Thanks. I would read about guideline. Joseph

[jira] [Commented] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111218#comment-16111218 ] Sean Owen commented on SPARK-21594: --- http://spark.apache.org/contributing.html This is

[jira] [Commented] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111203#comment-16111203 ] Joseph Wang commented on SPARK-21594: - The place you suggests seem to be a question a

[jira] [Resolved] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21594. --- Resolution: Invalid [~Monday0927!] do not reopen this -- wrong place. Please read my message and the

[jira] [Closed] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen closed SPARK-21594. - > Missing probability output from MutilayerPerceptronClassifier > ---

[jira] [Reopened] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Wang reopened SPARK-21594: - > Missing probability output from MutilayerPerceptronClassifier > ---

[jira] [Updated] (SPARK-21594) Missing probability output from MutilayerPerceptronClassifier

[ https://issues.apache.org/jira/browse/SPARK-21594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Wang updated SPARK-21594: Remaining Estimate: 168h Original Estimate: 168h Description: The semi-supervised l

[jira] [Commented] (SPARK-21034) Filter not getting pushed down the groupBy clause when first() or last() aggregate function is used

[ https://issues.apache.org/jira/browse/SPARK-21034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1631#comment-1631 ] Andrew Ray commented on SPARK-21034: {{first}} is not a deterministic function and th

  1   2   >