alamb commented on issue #27089: URL: https://github.com/apache/datafusion/issues/27089#issuecomment-6632612504
I haven't had time to read this issue in detail as it is so verbose and describes a lot of internal implementation detail I am not familar with > Case 1, one shredded file and one unshredded file. Rows 4-6 come from the unshredded file. The adapter gives them typed_value = NULL while value holds the whole 18-byte object: It isn't clear to me if this is good or bad or what the expected output is. Could you please rewrite the ticket to explain clearly: 1. What the input file / layout is (I realize you provided a bunch of code but please make it easier to undersrtand quickly for a human) 2. What query you are running (already there) 3. What output is returned (already there) 4. What the expected output is (not there) Basically, I want to get to the point where we can write reproducers in terms of end to end datafusion APIs (sql or dataframe) that show the problem. Then we will be in a much better place to work on a solution I would like to suggest we start with making it easier to create reproducers so it is clearer what is going on. I see the rust reproducer here, but perhaps we can start with something like this: - https://github.com/datafusion-contrib/datafusion-variant/issues/104 Then we can more easily express what the -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
