Hi Sandeep,
One way would be to register your dataframe, you loaded, as a temporary
view. See
https://spark.apache.org/docs/latest/sql-getting-started.html#global-temporary-view
-R
On Fri, Mar 29, 2019 at 5:12 PM Sandeep Sagar <sandeep.sa...@meltwater.com>
wrote:
> Hi,
>
> Exploring Iceberg here. Trying a hello-world in java.
>
>
>
> Am able to read and write to a table(“company”) on disk with Spark.
> However, when I tried –
>
>
>
> Dataset<Row> updated= spark.sql("SELECT * FROM iceberg.company where
> date_updated != 0");
>
>
>
> Got the following exception -
>
> Caused by: org.apache.spark.sql.AnalysisException: Unsupported data source
> type for direct query on files: iceberg;
>
> at
> org.apache.spark.sql.execution.datasources.ResolveSQLOnFile$$anonfun$apply$4.applyOrElse(rules.scala:56)
>
> ... 51 more
>
>
>
> Is it not supported yet or I am doing it incorrectly?
>
> I have been able to read the dataset and then filtering on it by using the
> following (where Company is my POJO)
>
>
>
>
>
> spark.read.format(“iceberg”).load(company.location()).as(Company.getEncoder());
>
>
>
> Appreciate the help.
>
> -Sandeep
> The information contained in this email may be confidential. It has been
> sent for the sole use of the intended recipient(s). If the reader of this
> email is not an intended recipient, you are hereby notified that any
> unauthorized review, use, disclosure, dissemination, distribution, or
> copying of this message is strictly prohibited. If you have received this
> email in error, please notify the sender immediately and destroy all copies
> of the message.
>
