Jiashu-Hu commented on code in PR #15353:
URL: https://github.com/apache/datafusion/pull/15353#discussion_r2010728423
##########
docs/source/library-user-guide/adding-udfs.md:
##########
@@ -1160,6 +1160,89 @@ async fn main() -> Result<()> {
// +---+
```
+## Custom Expression Planning
+
+DataFusion provides native support for a limited set of SQL operators by
default. For operators not natively defined, developers can extend DataFusion's
functionality by implementing custom expression planning. This extensibility is
a core feature of DataFusion, allowing it to be customized for particular
workloads and requirements.
+
+### Implementing Custom Expression Planning
+
+To extend DataFusion with support for custom operators not natively available,
you need to:
+
+1. Implement the `ExprPlanner` trait: This allows you to define custom logic
for planning expressions that DataFusion doesn't natively recognize. The trait
provides the necessary interface to translate logical expressions into physical
execution plans.
+
+ For a detailed documentation please see: [Trait
ExprPlanner](https://docs.rs/datafusion/latest/datafusion/logical_expr/planner/trait.ExprPlanner.html)
+
+2. Register your custom planner: Integrate your implementation with
DataFusion's `SessionContext` to ensure your custom planning logic is invoked
during the query optimization and execution planning phase.
+
+ For a detailed documentation please see: [fn
register_expr_planner](https://docs.rs/datafusion/latest/datafusion/execution/trait.FunctionRegistry.html#method.register_expr_planner)
+
+See example below:
+
+```rust
+# use arrow::array::RecordBatch;
+# use std::sync::Arc;
+
+# use datafusion::common::{assert_batches_eq, DFSchema};
+# use datafusion::error::Result;
+# use datafusion::execution::FunctionRegistry;
+# use datafusion::logical_expr::Operator;
+# use datafusion::prelude::*;
+# use datafusion::sql::sqlparser::ast::BinaryOperator;
+# use datafusion_common::ScalarValue;
+# use datafusion_expr::expr::Alias;
+# use datafusion_expr::planner::{ExprPlanner, PlannerResult, RawBinaryExpr};
+# use datafusion_expr::BinaryExpr;
+
+# #[derive(Debug)]
+# // Define the custom planner
+# struct MyCustomPlanner;
+
+// Implement ExprPlanner for cutom operator logic
+impl ExprPlanner for MyCustomPlanner {
+ fn plan_binary_op(
+ &self,
+ expr: RawBinaryExpr,
+ _schema: &DFSchema,
+ ) -> Result<PlannerResult<RawBinaryExpr>> {
+ match &expr.op {
+ // Map `->` to string concatenation
+ BinaryOperator::Arrow => {
+ // Rewrite `->` as a string concatenation operation
+ // - `left` and `right` are the operands (e.g., 'hello' and
'world')
+ // - `Operator::StringConcat` tells DataFusion to concatenate
them
+ Ok(PlannerResult::Planned(Expr::BinaryExpr(BinaryExpr {
+ left: Box::new(expr.left.clone()),
+ right: Box::new(expr.right.clone()),
+ op: Operator::StringConcat,
+ })))
+ }
+ _ => Ok(PlannerResult::Original(expr)),
+ }
+ }
+}
+
+use datafusion::execution::context::SessionContext;
+use datafusion::arrow::util::pretty;
+
+#[tokio::main]
+async fn main() -> Result<()> {
+ let config = SessionConfig::new().set_str("datafusion.sql_parser.dialect",
"postgres");
+ let mut ctx = SessionContext::new_with_config(config);
+ ctx.register_expr_planner(Arc::new(MyCustomPlanner))?;
+ let results = ctx.sql("select 'foo'->'bar';").await?.collect().await?;
+
+ pretty::print_batches(&results)?;
Review Comment:
Hi alamb, Thank you very much for your review! Yes, absolutely, I've updated
it with an `assert_batches_eq!` in the latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
