alamb commented on issue #9016: URL: https://github.com/apache/datafusion/issues/9016#issuecomment-5266418336
Thanks @goutamadwant that will be amazing. -- I think the key here will be to break the work down into manageable pieces (so that we can review it incrementally) I also suspect that in writing this you will uncover areas in need of additional features / cleanu, so we will have to be careful not to try and boil the ocean What I would suggest doing first, is creating an example in datafusion-examples of the simplest possible streaming system -- basically a program that shows how to feed rows into a streaming query (perhaps something like a filter) and shows how rows come out of it as you push rows in. As I mentioned, I suspect this won't work great at first and well need to update some things (like the FilterExec) but starting with the example will help motivate that With that example, we cna then start writing a "Using datafusion for streaming" section in the library user guide https://datafusion.apache.org/library-user-guide/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
