Re: join two DataFrames, same column name

> > You can include * and a column alias in the same select clause > var df1 = sqlContext.sql("select *, column_id AS table1_id from table1") FYI, this does not ultimately work as the * still includes column_id and you cannot have two columns of that name in the joined DataFrame. So I ended up a

Re: join two DataFrames, same column name

Michael, thank you for the workaround and for letting me know of the upcoming enhancements, both of which sound appealing. On Sun, Mar 22, 2015 at 1:25 PM, Michael Armbrust wrote: > You can include * and a column alias in the same select clause > var df1 = sqlContext.sql("select *, column_id AS

Re: join two DataFrames, same column name

You can include * and a column alias in the same select clause var df1 = sqlContext.sql("select *, column_id AS table1_id from table1") I'm also hoping to resolve SPARK-6376 before Spark 1.3.1 which will let you do something like: var df1 = sqlCo

join two DataFrames, same column name

I have a couple of data frames that I pulled from SparkSQL and the primary key of one is a foreign key of the same name in the other. I'd rather not have to specify each column in the SELECT statement just so that I can rename this single column. When I try to join the data frames, I get an excep