alamb commented on code in PR #14837:
URL: https://github.com/apache/datafusion/pull/14837#discussion_r2162340524
##########
datafusion/core/src/physical_planner.rs:
##########
@@ -775,12 +776,44 @@ impl DefaultPhysicalPlanner {
let runtime_expr =
self.create_physical_expr(predicate, input_dfschema,
session_state)?;
+
+ let filter = match self.try_plan_async_exprs(
Review Comment:
I agree it feels unnatural
The downside in my mind of trying to put it in PhysicalExpr is then it
complicates implementing PhysicalExpr even when most `PhysicalExpr`s don't need
to worry about it
Thus I think treating async udfs specially while not ideal will make it
easier to understand how different they are
##########
datafusion-examples/examples/async_udf.rs:
##########
@@ -0,0 +1,256 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use arrow::array::{ArrayIter, ArrayRef, AsArray, Int64Array, RecordBatch,
StringArray};
+use arrow::compute::kernels::cmp::eq;
+use arrow_schema::{DataType, Field, Schema};
+use async_trait::async_trait;
+use datafusion::common::error::Result;
+use datafusion::common::internal_err;
+use datafusion::common::types::{logical_int64, logical_string};
+use datafusion::common::utils::take_function_args;
+use datafusion::config::ConfigOptions;
+use datafusion::execution::{FunctionRegistry, SessionStateBuilder};
+use datafusion::logical_expr::async_udf::{
+ AsyncScalarFunctionArgs, AsyncScalarUDF, AsyncScalarUDFImpl,
+};
+use datafusion::logical_expr::{
+ ColumnarValue, Signature, TypeSignature, TypeSignatureClass, Volatility,
+};
+use datafusion::logical_expr_common::signature::Coercion;
+use datafusion::physical_expr_common::datum::apply_cmp;
+use datafusion::prelude::SessionContext;
+use log::trace;
+use std::any::Any;
+use std::sync::Arc;
+
+#[tokio::main]
Review Comment:
tracking with
- https://github.com/apache/datafusion/issues/16521
##########
datafusion/expr/src/async_udf.rs:
##########
@@ -0,0 +1,135 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use crate::{ScalarFunctionArgs, ScalarUDF, ScalarUDFImpl};
+use arrow::array::ArrayRef;
+use arrow::datatypes::{DataType, SchemaRef};
+use async_trait::async_trait;
+use datafusion_common::config::ConfigOptions;
+use datafusion_common::error::Result;
+use datafusion_common::internal_err;
+use datafusion_expr_common::columnar_value::ColumnarValue;
+use datafusion_expr_common::signature::Signature;
+use std::any::Any;
+use std::fmt::{Debug, Display};
+use std::sync::Arc;
+
+/// A scalar UDF that can invoke using async methods
+///
+/// Note this is less efficient than the ScalarUDFImpl, but it can be used
+/// to register remote functions in the context.
+///
+/// The name is chosen to mirror ScalarUDFImpl
+#[async_trait]
+pub trait AsyncScalarUDFImpl: Debug + Send + Sync {
+ /// the function cast as any
+ fn as_any(&self) -> &dyn Any;
+
+ /// The name of the function
+ fn name(&self) -> &str;
+
+ /// The signature of the function
+ fn signature(&self) -> &Signature;
+
+ /// The return type of the function
+ fn return_type(&self, _arg_types: &[DataType]) -> Result<DataType>;
Review Comment:
Since we originally made this PR, the udfs have been changed again (to use
Fields/FieldRef rather than DataType)...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
