zhuqi-lucas commented on PR #14877: URL: https://github.com/apache/datafusion/pull/14877#issuecomment-2683862205
> Thank you @zhuqi-lucas -- this is really cool! > > I have a suggestion that I think would make the code better, but we could do it as a follow on PR or never in my opinion. > > In addition to the very nice tests you have in this PR I tested it manaully (notes below) and it worked great > > Testing without limit: > > ```shell > cargo run -p datafusion-cli -- --maxrows inf -c "select * from 'benchmarks/data/hits.parquet'" > ``` > > And output was produced immediately and streamed out > > I also tested with > > ``` > cargo run -p datafusion-cli -- -c "select * from 'benchmarks/data/hits.parquet'" > ``` > > Which printed out the first 40 rows and then continued running but did not print anything, as expected Thank you @alamb for review, i created a follow-up to do the code clean: https://github.com/apache/datafusion/issues/14886 And we can do some code clean work including: 1. Add streaming state struct and clean up print batch logic 2. Remove older logics about max-rows 3. Add more docs and remove confusing docs 4. Etc -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
