Hi,
Got a question here around! I am setting a setJobGroup to be cancelled within a
Future.
Thing is, such future is affected by several map, flatMap, for comprehensions
and even andThen side effects. I am being unable to cancel the job properly
within a group, and I am unsure whether this is
Hi all, I'm very new to spark and looking forward to get deep into the topic.
Right now I am trying to inherit my own transformer, by what I am reading so
far, it is not very public that we can apply to this practice as "users".
I am defining my transformer based on the Binarizer, but simply fail
Thank you, I will do as you suggested.
Ps: I read that in this random user archive I found:
http://mail-archives.us.apache.org/mod_mbox/spark-user/201506.mbox/%3c55709f7b.2090...@gmail.com%3E
Saif
From: Feynman Liang [mailto:fli...@databricks.com]
Sent: Monday, September 14, 2015 4:08 PM
To: Ell
Hi all,
wondering if any could make the new 1.5.0 stallSkinnyQR to work.
Follows my output, which is a big loop of the same errors until the shell dies.
I am curious since im failing to load any implementations from BLAS, LAPACK,
etc.
scala> mat.tallSkinnyQR(false)
15/09/22 10:18:11 WARN LAPACK:
Hi all,
I have a process where local mode takes only 40 seconds. While the same on
stand-alone mode, being the same node used for local mode the only available
node, is taking up for ever. rdd actions hang up.
I could only "sort this out" by turning speculation on, so the same task
hanging is
When running stand-alone cluster mode job, the process hangs up randomly during
a DataFrame flatMap or explode operation, in HiveContext:
-->> df.flatMap(r => for (n <- 1 to r.getInt(ind)) yield r)
This does not happen either with SQLContext in cluster, or Hive/SQL in local
mode, where it works
Hi all, would this be a bug??
val ws = Window.
partitionBy("clrty_id").
orderBy("filemonth_dtt")
val nm = "repeatMe"
df.select(df.col("*"), rowNumber().over(ws).cast("int").as(nm))
stacked_data.filter(stacked_data("repeatMe").isNotNull).or
Hi,
I have figured this only happens in cluster mode. working properly in local[32]
From: saif.a.ell...@wellsfargo.com [mailto:saif.a.ell...@wellsfargo.com]
Sent: Thursday, October 08, 2015 10:23 AM
To: dev@spark.apache.org
Subject: RowNumber in HiveContext returns null, negative numbers or huge
