Re: Non string type partitions

I guess that it has to do with indexing and partitioning data to nodes. Have a look at data partitioning system design concept and key range partitions

Re: Non string type partitions

bumping this up again for suggestions?.. Is the official recommendation to not have *int* or *date* typed partition columns? On Wed, 12 Apr 2023 at 10:44, Charles vinodh wrote: > There are other distributed execution engines (like hive, trino) that do > support non-string data types for partiti

Re: Non string type partitions

There are other distributed execution engines (like hive, trino) that do support non-string data types for partition columns such as date and integer. Any idea why this restriction exists in Spark? .. On Tue, 11 Apr 2023 at 20:34, Chitral Verma wrote: > Because the name of the directory cannot

Re: Non string type partitions

Because the name of the directory cannot be an object, it has to be a string to create partitioned dirs like "date=2023-04-10" On Tue, 11 Apr, 2023, 8:27 pm Charles vinodh, wrote: > > Hi Team, > > We are running into the below error when we are trying to run a simple > query a partitioned table

Non string type partitions

Hi Team, We are running into the below error when we are trying to run a simple query a partitioned table in Spark. *MetaException(message:Filtering is supported only on partition keys of type string) * Our the partition column has been to type *date *instead of string and query is a very simpl