zeroshade commented on PR #94: URL: https://github.com/apache/parquet-testing/pull/94#issuecomment-3198517351
It was simply quicker to read in the files generated by Java as Arrow and then write them back out to Parquet (properly marking Variant types etc.) than to put together something to generate the test cases from scratch. Go has https://github.com/apache/arrow-go/blob/main/arrow/extensions/variant.go#L126 which allows creating your own shredded Variant array which can get written to Parquet with tests added by https://github.com/apache/arrow-go/commit/2cf2b297672f37895c83a42bfaa1ac08352efc16. > (my main concern are safe-guards in place if Arrow representation fed in is not correct)? Tests are added by https://github.com/apache/arrow-go/pull/455 which performs validation when constructing the Arrow representation and writing to Parquet. This is also why arrow-go can't generate the test cases which aren't valid, the incorrect constructions cause errors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
