Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-10 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2031035548 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-10 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783526836 > The only thing I think is needed for this PR is to either > > 1. remove the bechmark > 2. update the benchmark ti actually test `BinaryExpr` evaluation time done

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
berkaysynnada commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2034007815 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit conditi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
berkaysynnada commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2034007019 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit conditi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2787055441 the extended tests appear to be failing after this PR: - https://github.com/apache/datafusion/issues/15641 -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
alamb commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2033602903 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition +///

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2033205911 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
berkaysynnada commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2033185234 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit conditi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2033160515 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
berkaysynnada commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2033101460 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit conditi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
alamb commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2032964400 ## datafusion/physical-expr/benches/binary_op.rs: ## @@ -0,0 +1,373 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2786029908 Filed https://github.com/apache/datafusion/issues/15631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
alamb merged PR #15462: URL: https://github.com/apache/datafusion/pull/15462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-08 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2786018330 Thanks again for the wonderful work @acking-you -- I ran the benchmarks locally and indeed they show a non trivial amount of time being spent in count_ones. ![Screenshot 202

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2784190060 > I think the likely part of it getting slower is short-circuiting whenever a `true` is observed (having a branch for each item to check). > > For this case it might be inte

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783230355 > > Security audit failure is not related to this PR > > How can this be resolved? - it is tracked by https://github.com/apache/datafusion/issues/15571 There are ide

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
Dandandan commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783121877 > > The related results are as follows. I’m not sure what you think about these results — should we continue using true_count/false_count, or look for a better solution? @alamb @Da

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2031069949 ## datafusion/physical-expr/benches/boolean_op.rs: ## @@ -0,0 +1,187 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor l

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783033518 > I don't fully understand your performance results given that the two functions seem very similar -- maybe it has to do with option hanlding messing auto vectorization or somethi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
alamb commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2031057246 ## datafusion/physical-expr/benches/boolean_op.rs: ## @@ -0,0 +1,187 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor licens

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783035350 > Security audit failure is not related to this PR How can this be resolved? -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2782983563 I sincerely apologize for the delay in updating this PR. I have now designed a detailed comparative test for the `bool_or/bool_and` issue described by @Dandandan. The related resu

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783021714 > The related results are as follows. I’m not sure what you think about these results — should we continue using true_count/false_count, or look for a better solution? @alamb @Dandanda

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-07 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2783022496 Security audit is not related to this RP -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-06 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2030375948 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-06 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2030375948 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-06 Thread via GitHub
berkaysynnada commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2030260718 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit conditi

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-05 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020615756 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-05 Thread via GitHub
Dandandan commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020558418 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition +

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-03 Thread via GitHub
alamb commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2026912500 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition +///

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-03 Thread via GitHub
alamb commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2775475696 > > I looked for a binary expression and/or micro benchmark and could not find one: https://github.com/apache/datafusion/tree/c929a1cd133590e4944bc2c7900611220450335a/datafusion/physic

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-02 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2774487207 > I looked for a binary expression and/or micro benchmark and could not find one: https://github.com/apache/datafusion/tree/c929a1cd133590e4944bc2c7900611220450335a/datafusion/phy

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-01 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2023116930 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-04-01 Thread via GitHub
ctsk commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2022894121 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition +/// 1

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-31 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020615756 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-31 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020726805 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-31 Thread via GitHub
Dandandan commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020695065 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition +

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-31 Thread via GitHub
Dandandan commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020695065 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -805,6 +811,47 @@ impl BinaryExpr { } } +/// Check if it meets the short-circuit condition +

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-30 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2764998480 > Also, could you please add the new Q6 benchmark in a separate PR so I can more easily run my benchmark scripts before/after your code change? I have successfully split Q6

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-30 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2020360286 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -358,7 +358,50 @@ impl PhysicalExpr for BinaryExpr { fn evaluate(&self, batch: &RecordBatch) -

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-29 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2019944906 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -358,7 +358,50 @@ impl PhysicalExpr for BinaryExpr { fn evaluate(&self, batch: &RecordBatch) -

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-29 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2019945905 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -358,7 +358,50 @@ impl PhysicalExpr for BinaryExpr { fn evaluate(&self, batch: &RecordBatch) -

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-29 Thread via GitHub
acking-you commented on code in PR #15462: URL: https://github.com/apache/datafusion/pull/15462#discussion_r2019944906 ## datafusion/physical-expr/src/expressions/binary.rs: ## @@ -358,7 +358,50 @@ impl PhysicalExpr for BinaryExpr { fn evaluate(&self, batch: &RecordBatch) -

Re: [PR] Add short circuit evaluation for `AND` and `OR` [datafusion]

2025-03-29 Thread via GitHub
acking-you commented on PR #15462: URL: https://github.com/apache/datafusion/pull/15462#issuecomment-2764093707 > Also, could you please add the new Q6 benchmark in a separate PR so I can more easily run my benchmark scripts before/after your code change? Okey,I got it.Do you mean tha