Thanks Vadim. But I am looking for an API either in DataSet, DataFrame, or
DataFrameWriter etc. The way you suggested can be done via a query like
spark.sql(""" ALTER TABLE `table` ADD PARTITION (partcol=1) LOCATION
'/path/to/your/dataset' """), and before that I write it to a specified
location fi
This should work:
```
ALTER TABLE `table` ADD PARTITION (partcol=1) LOCATION
'/path/to/your/dataset'
```
On Wed, Jul 19, 2017 at 6:13 PM, ctang wrote:
> I wonder if there are any easy ways (or APIs) to insert a dataframe (or
> DataSet), which does not contain the partition columns, as a static
>
Not sure about the writer api, but you could always register a temp table
for that dataframe and execute insert hql.
On Thu, Jul 20, 2017 at 6:13 AM, ctang wrote:
> I wonder if there are any easy ways (or APIs) to insert a dataframe (or
> DataSet), which does not contain the partition columns, a
