Re: Parquet schema per bucket in Streaming File Sink

2021-12-03 Thread Zack Loebel
Unfortunately this does not solve my use case. Because I want to be able to create and change the various outputs at runtime (the partition keys would be dynamic) and as such the sql/extraction would have to change during execution. Which I did not believe to be supported. I'm also operating at the

Re: Parquet schema per bucket in Streaming File Sink

2021-11-30 Thread Francesco Guardiani
Hi Zack, > I want to customize this job to "explode" the map as column names and values You can do this in a select statement extracting manually the map values using the map access built-in