advancedxy commented on issue #12955:
URL: https://github.com/apache/datafusion/issues/12955#issuecomment-2443027052
I noticed this bug too.
We can have a logical plan to represent the Intersect/Except operators. But
in the essential, the `intersect all` could be rewrote by primitive operations,
such as: (quoted from Spark's RewriteIntersectAll, ExceptAll could be rewrote
in a similar way)
```
Replaces logical Intersect operator using a combination of Union, Aggregate
and Generate operator.
Input Query :
SELECT c1 FROM ut1 INTERSECT ALL SELECT c1 FROM ut2
Rewritten Query:
SELECT c1
FROM (
SELECT replicate_row(min_count, c1)
FROM (
SELECT c1, If (vcol1_cnt > vcol2_cnt, vcol2_cnt, vcol1_cnt) AS
min_count
FROM (
SELECT c1, count(vcol1) as vcol1_cnt, count(vcol2) as
vcol2_cnt
FROM (
SELECT true as vcol1, null as , c1 FROM ut1
UNION ALL
SELECT null as vcol1, true as vcol2, c1 FROM ut2
) AS union_all
GROUP BY c1
HAVING vcol1_cnt >= 1 AND vcol2_cnt >= 1
)
)
)
```
I think we could has the `intersect w/wo all` operator and rewrite the
operator in a primitive one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
