jayzhan211 commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2796973499
https://github.com/apache/datafusion/pull/15685
@adriangb, the `snapshot` is different but I think the overall idea should
be the same, while we avoid remapping each time we
berkaysynnada commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2796511240
I couldn't fully follow but, do we have remaining issues known with
`DynamicFilterPhysicalExpr` internals?
--
This is an automated message from the Apache Git Service.
To res
alamb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2796543649
I think some of the concerns will become more clear as we connect this code
all up together and see it in action. As it is currently not yet hooked up I
think it is hard to reason abou
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2796541790
I think @jayzhan211 still has concerns but I'm having trouble understanding
what they translate to in practice
--
This is an automated message from the Apache Git Service.
To res
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038776161
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038773841
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038773841
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038768921
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038641608
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038636475
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038636475
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038641608
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038636475
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038636475
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038642878
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038641608
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038641608
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2038636475
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037178254
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
berkaysynnada commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037187944
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or mo
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2786603792
Okay nope this PR is set up correctly: `dynamic_filter` is a private module
in `datafusion-physical-plan`, so we can use it from within `SortExec`/`TopK`,
`HashJoinExec`, etc.
alamb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037015006
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contr
alamb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2794732220
SInce this is an internal API that is not yet hooked up, let's merge merge
it in and keep iterating on main (so we can hook it up and start testing it
asap). If we can find simpler wa
alamb merged PR #15568:
URL: https://github.com/apache/datafusion/pull/15568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusi
alamb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2794732353
Thanks everyone!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037392525
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037240441
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037378369
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2793681651
@berkaysynnada @jayzhan211 I added a test in f59577c26 that shows how this
interacts with `ParquetSource` and should help with confusion in
https://github.com/apache/datafusion/pull
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037417258
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037417258
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037348021
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037334037
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037246933
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037242880
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037235927
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037235439
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037178254
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037178254
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037185406
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037178254
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
jayzhan211 commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037178254
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,380 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more
berkaysynnada commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037022463
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,442 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or mo
alamb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2037010293
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
///
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2036221582
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035755891
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,55 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2036022198
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,442 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035863537
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,442 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035755891
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,55 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035754530
##
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##
@@ -0,0 +1,442 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more co
berkaysynnada commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035353055
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,55 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
alamb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2035073646
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
///
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2787176334
Thank you for the review, I'll address comments this afternoon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2787225434
> > My view for DynamicFilterSource is not that it's a public API in
DataFusion. It's a private implementation detail / helper that internal
implementations of dynamic PhysicalExpr
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2787229822
> I think it is exactly the API someone would need if they wanted to
implement dynamic filters for distributed joins -- for example 'snapshot' would
make / create a bloom filter to
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2033710338
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
alamb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2787162573
> My view for DynamicFilterSource is not that it's a public API in
DataFusion. It's a private implementation detail / helper that internal
implementations of dynamic PhysicalExpr can u
alamb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2033687058
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
///
adriangb commented on PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#issuecomment-2786429909
My view for `DynamicFilterSource` is not that it's a public API in
DataFusion. It's a private implementation detail / helper that internal
implementations of dynamic `PhysicalExpr`
adriangb commented on code in PR #15568:
URL: https://github.com/apache/datafusion/pull/15568#discussion_r2030171299
##
datafusion/physical-expr-common/src/physical_expr.rs:
##
@@ -283,6 +284,51 @@ pub trait PhysicalExpr: Send + Sync + Display + Debug +
DynEq + DynHash {
/
60 matches
Mail list logo
