Le 21/11/2021 à 19:48, Ian Joiner a écrit :
I see. Now the question is what we should do about such columns in the ORC writer as well as maybe some other writers since the Null type, as opposed to all Null columns of a numeric or binary type, doesn’t exist in such formats.
We could perhaps add an option to silently turn them into another type, but they wouldn't roundtrip properly unless we also serialize the Arrow schema as we do in Parquet.
For now, people will have to detect such columns and cast them manually, I think.
Regards Antoine.
