yeah that could work, since i should know (or be able to find out) all the
input columns
On Thu, May 26, 2016 at 11:30 PM, Takeshi Yamamuro
wrote:
> You couldn't do like this?
>
> --
> val func = udf((i: Int) => Tuple2(i, i))
> val df = Seq((1, ..., 0), (2, ..., 5)).toDF("input", "c0", "c1", ...
You couldn't do like this?
--
val func = udf((i: Int) => Tuple2(i, i))
val df = Seq((1, ..., 0), (2, ..., 5)).toDF("input", "c0", "c1", other
needed columns, "cX")
df.select(func($"a").as("r"), $"c0", $"c1", $"cX").select($"r._1",
$"r._2", $"c0", $"c1", $"cX")
// maropu
On Fri,
yes, but i also need all the columns (plus of course the 2 new ones) in my
output. your select operation drops all the input columns.
best, koert
On Thu, May 26, 2016 at 11:02 PM, Takeshi Yamamuro
wrote:
> Couldn't you include all the needed columns in your input dataframe?
>
> // maropu
>
> On
Couldn't you include all the needed columns in your input dataframe?
// maropu
On Fri, May 27, 2016 at 1:46 AM, Koert Kuipers wrote:
> that is nice and compact, but it does not add the columns to an existing
> dataframe
>
> On Wed, May 25, 2016 at 11:39 PM, Takeshi Yamamuro
> wrote:
>
>> Hi,
>
that is nice and compact, but it does not add the columns to an existing
dataframe
On Wed, May 25, 2016 at 11:39 PM, Takeshi Yamamuro
wrote:
> Hi,
>
> How about this?
> --
> val func = udf((i: Int) => Tuple2(i, i))
> val df = Seq((1, 0), (2, 5)).toDF("a", "b")
> df.select(func($"a").as("r")).sel
Hi,
How about this?
--
val func = udf((i: Int) => Tuple2(i, i))
val df = Seq((1, 0), (2, 5)).toDF("a", "b")
df.select(func($"a").as("r")).select($"r._1", $"r._2")
// maropu
On Thu, May 26, 2016 at 5:11 AM, Koert Kuipers wrote:
> hello all,
>
> i have a single udf that creates 2 outputs (so a
hello all,
i have a single udf that creates 2 outputs (so a tuple 2). i would like to
add these 2 columns to my dataframe.
my current solution is along these lines:
df
.withColumn("_temp_", udf(inputColumns))
.withColumn("x", col("_temp_)("_1"))
.withColumn("y", col("_temp_")("_2"))
.drop
