andygrove commented on issue #1182: URL: https://github.com/apache/datafusion-ballista/issues/1182#issuecomment-2661606344
The root cause is somehow related to reading Utf8View from Parquet and then writing via IPC ``` col type Decimal128(11, 2) size 65632 col type Utf8View size 22167034 col type Utf8View size 7300778758 col type Int64 size 32864 col type Utf8View size 21978646 col type Utf8View size 14447222990 col type Utf8View size 14463162232 col type Utf8View size 24259892 ``` If I disable Utf8View then the problem is resolved: ``` config.options_mut().execution.parquet.schema_force_view_types = false; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
