The partitionedBy typo in the doc is already fixed in the master branch of
the Iceberg repo.
I filed a PR to add `using("iceberg")` to the `writeTo` examples for
creating a table (if you want to create an *Iceberg* table).
On Fri, May 27, 2022 at 12:58 PM Wing Yew Poon <wyp...@cloudera.com> wrote:
> One other note:
> When creating the table, you need `using("iceberg")`. The example should
> read
>
> data.writeTo("prod.db.table")
> .using("iceberg")
> .tableProperty("write.format.default", "orc")
> .partitionedBy($"level", days($"ts"))
> .createOrReplace()
>
> - Wing Yew
>
>
> On Fri, May 27, 2022 at 11:29 AM Wing Yew Poon <wyp...@cloudera.com>
> wrote:
>
>> That is a typo in the sample code. The doc itself (
>> https://iceberg.apache.org/docs/latest/spark-writes/#creating-tables)
>> says:
>> "Create and replace operations support table configuration methods, like
>> partitionedBy and tableProperty"
>> You could also have looked up the API in Spark documentation:
>>
>> https://spark.apache.org/docs/latest/api/scala/org/apache/spark/sql/DataFrameWriterV2.html
>> There you would have found that the method is partitionedBy, not
>> partitionBy.
>>
>> - Wing Yew
>>
>>
>> On Fri, May 27, 2022 at 4:32 AM Saulius Pakalka
>> <saulius.paka...@oxylabs.io.invalid> wrote:
>>
>>> Hi,
>>>
>>> I am trying to create partitioned iceberg table using scala code below
>>> based on example in docs.
>>>
>>> df_c.writeTo(output_table)
>>> .partitionBy(days(col("last_updated")))
>>> .createOrReplace()
>>>
>>> However, this code does not compile and throws two errors:
>>>
>>> value partitionBy is not a member of
>>> org.apache.spark.sql.DataFrameWriterV2[org.apache.spark.sql.Row]
>>> [error] possible cause: maybe a semicolon is missing before `value
>>> partitionBy'?
>>> [error] .partitionBy(days(col("last_updated")))
>>> [error] ^
>>> [error] not found: value days
>>> [error] .partitionBy(days(col("last_updated")))
>>> [error] ^
>>> [error] two errors found
>>>
>>> Not sure where to look for a problem. Any advice appreciated.
>>>
>>> Best regards,
>>>
>>> Saulius Pakalka
>>>
>>>
