Jackie-Jiang commented on issue #16317: URL: https://github.com/apache/pinot/issues/16317#issuecomment-3071174212
> Imo, we should actually enforce that users add columns to the schema which are being used in aggregation or other methods. This might not always be possible. One example would be: the original value is number; the aggregated value is bytes (e.g. HyperLogLog). We cannot store the aggregated value as number, and we cannot keep the original value because it needs to be aggregated at ingestion time. I do agree ideally user should config all columns referenced during ingestion in schema. We can consider adding a new field type, which is only used for data type conversion, but not for column definition in the table post ingestion. Similarly (out of the scope of this issue), we can add a field type for deleted column. See #15931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
