I think for the current requirement, substrait is something, which I'd like
to give it a try.
Thanks,
Surya
On Thu, Aug 29, 2024 at 11:59 AM Kevin Liu wrote:
> If you're using open table formats, Delta Lake has the "generated column"
> feature which supports specifying a formula using other ta
If you're using open table formats, Delta Lake has the "generated column"
feature which supports specifying a formula using other table columns.
https://docs.databricks.com/en/delta/generated-columns.html
https://delta.io/blog/2023-04-12-delta-lake-generated-columns/
Cheers,
Kevin
On Thu, Aug 29
Hi!
Another option would be converting to an arrow-backed pandas table and
using a dataframe query method. Other libraries like DuckDB most
likely offer similar options.
BR
J
czw., 29 sie 2024 o 02:54 Felipe Oliveira Carvalho
napisał(a):
>
> You can build `compure::Expression` instances [1] an
You can build `compure::Expression` instances [1] and use them in different
contexts like scanning datasets [2] and producing Substrait plans [3] that
you can execute.
But you have to write your own parser and define the scope and semantics of
the operations you would support.
[1]
https://github.
