alamb commented on PR #15591:
URL: https://github.com/apache/datafusion/pull/15591#issuecomment-2890589301
> I think still need some time to evaluate what benefits can this brings
I think I am somewhat lost with the current state. Your comment on
https://github.com/apache/datafusion/pull/15591#issuecomment-2823308363 states
> I add a query in extened.sql, the blocked approach can get a obvious
improvement as expected.
>
> I have confidence it can even improve more in some other queries according
to poc https://github.com/apache/datafusion/pull/11943
And the most recent benchmark run
https://github.com/apache/datafusion/pull/15591#issuecomment-2890539606 seems
to confirm this finding:
>
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query ┃ HEAD ┃ intermeidate-result-blocked-approach ┃
Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0 │ 1916.94ms │ 1866.52ms │ no
change │
│ QQuery 1 │ 709.88ms │ 673.73ms │ +1.05x
faster │
│ QQuery 2 │ 1482.93ms │ 1391.57ms │ +1.07x
faster │
│ QQuery 3 │ 701.88ms │ 721.01ms │ no
change │
│ QQuery 4 │ 1494.91ms │ 1494.55ms │ no
change │
│ QQuery 5 │ 15627.54ms │ 15418.84ms │ no
change │
│ QQuery 6 │ 2070.08ms │ 2042.39ms │ no
change │
│ QQuery 7 │ 2108.54ms │ 1952.66ms │ +1.08x
faster │
└──────────────┴────────────┴──────────────────────────────────────┴───────────────┘
However, some of the results on shorter running queries (few groups) shows
perhaps a slowdown:
https://github.com/apache/datafusion/pull/15591#issuecomment-2890527702
From those numbers, is it a fair assessment that the blocked approach
improves performance when there is a large number of intermediate groups, but
does not when there is a small number of groups?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
