Hello,
Is there any API to insert data into a single partition of a table?
Let's say I have a table with 2 columns (col_a, col_b) and a partition by
date.
After doing some computation for a specific date, I have a DataFrame with 2
columns (col_a, col_b) which I would like to insert into a specific date
partition. What is the best way to achieve this?
It seems that if I add a date column to my DataFrame, and turn on dynamic
partitioning, I can do:
df.write.partitionBy("date").insertInto("my_table")
But it seems overkill to use dynamic partitioning function for such a case.
Thanks for any pointers!
Isabelle
