2016 at 4:08 PM, Michael Armbrust
wrote:
> Looks like you found a bug. I've filed them here:
>
> SPARK-12987 - Drop fails when columns contain dots
> SPARK-12988 - Can't drop columns that contain dots
>
> On Fri, Jan 22, 2016 at 3:18 PM, Joshua TAYLOR
> wrote:
>>
Frame.org$apache$spark$sql$DataFrame$$withPlan(DataFrame.scala:2165)
at org.apache.spark.sql.DataFrame.select(DataFrame.scala:751)
at org.apache.spark.sql.DataFrame.drop(DataFrame.scala:1286)
at SparkExample.lambda$0(SparkExample.java:45)
at SparkExample.ma
://www.inside-r.org/packages/cran/arules/docs/discretize
>
> R code for example :
>
> ### equal frequency
> table(discretize(data$some_column, "frequency", categories=10))
>
>
> #k-means
> table(discretize(data$some_column, "cluster", categories=10))
>
> Thanks a lot !
>
--
Joshua Taylor, http://www.cs.rpi.edu/~tayloj/
arkExample.lambda$0(SparkExample.java:45)
at SparkExample.main(SparkExample.java:54)
Case 1
+---+---+
|a_b|a.c|
+---+---+
| t| 2|
| u| 4|
+---+---+
Case 2
+---+
|a.c|
+---+
| 2|
| 4|
+---+
Case 3
+---+
|a.c|
+---+
| 2|
| 4|
+---+
Thanks in advance,
Joshua
--
Joshua Taylor, http://www.cs.rpi.edu/~tayloj/
.DataFrame.select(DataFrame.scala:751)
at org.apache.spark.sql.DataFrame.drop(DataFrame.scala:1286)
at SparkExample.lambda$0(SparkExample.java:45)
at SparkExample.main(SparkExample.java:54)
Case 1
+---+---+
|a_b|a.c|
+---+---+
| t| 2|
| u| 4|
+---+---+
Case 2
+---+
|a.c|
+---+
| 2|
| 4|
+---+
Case 3
+---+
|a.c|
+---+
| 2|
| 4|
+---+
Thanks in advance,
Joshua
--
Joshua Taylor, http://www.cs.rpi.edu/~tayloj/
arkExample.lambda$0(SparkExample.java:45)
at SparkExample.main(SparkExample.java:54)
Case 1
+---+---+
|a_b|a.c|
+---+---+
| t| 2|
| u| 4|
+---+---+
Case 2
+---+
|a.c|
+---+
| 2|
| 4|
+---+
Case 3
+---+
|a.c|
+---+
| 2|
| 4|
+---+
Thanks in advance,
Joshua
--
Joshua Taylor, http://www.cs.rpi.edu/~tayloj/
