chenkovsky commented on PR #14362: URL: https://github.com/apache/datafusion/pull/14362#issuecomment-2625969618
> > > So you're worried that if a Parquet file has this special metadata on a field we will wrongly interpret it as a system column? Or are you saying that's a good thing / the goal? > > > > > > I'm worried about it. > > You're worried that someone by accident (eg because it was written by another system) would add field metadata with the key `datafusion.system_column` and the value `true` and that their intention would not be for DataFusion to interpret that as a system column? event not by accident. sometimes we select _rowid and save it in parquet. the parquet will contain magic metadata. then we read this parquet. we get two system columns _rowid. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
