Hi Experts,

I have a question regarding the supported syntax for bucketed table sort
order.

By looking at the documentation for Spark 3.5.3, CREATE DATASOURCE TABLE
<https://spark.apache.org/docs/3.5.3/sql-ref-syntax-ddl-create-table-datasource.html?fbclid=IwZXh0bgNhZW0CMTEAAR7gpuktLD9zUK8iQlhf8J0UJv9TIxm0f2TqVI_ybTb3_X0Xw4CjaJHfXBmJuA_aem_CLNLqubpQRJjSCpQImVYuA#syntax>
supports the following query:

CREATE TABLE table_identifier
    USING data_source
    [ CLUSTERED BY ( col_name3, col_name4, ... )
        [ SORTED BY ( col_name [ ASC | DESC ], ... ) ]
        INTO num_buckets BUCKETS ]

However, when I tried executing like

CREATE TABLE test_table
    USING data_source
    CLUSTERED BY ( col_name )
        SORTED BY ( col_name DESC )
        INTO num_buckets BUCKETS

it throws
*Operation not allowed: Column ordering must be ASC, was 'DESC'.*

I also found the source code
<https://github.com/apache/spark/blob/8eb9e340d12fa0bd98dea6b87e2a675cb757d3b0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala#L3210-L3226>,
and it looks like it is not supported yet.

Am I missing something? Thanks for any insights.

Best,
Junping Luo

Reply via email to