Re:Re:Re: Could you please give me a hand about json object in flink sql

2022-04-03 Thread lixiongfeng
May be you can get some inspiration from JsonDeserializationSchema an JSONKeyValueDeserializationSchema. At 2022-04-02 14:47:08, "wang" <24248...@163.com> wrote: Hi, Thanks so much for your support! But sorry to say I'm still confused about it. No matter what the udf looks

Source with S3 bucket with millions ( billions ) of object ( small files )

2022-04-03 Thread Vishal Santoshi
Folks, I am doing a simple batch job that uses readFile() with "s3a://[bucket_name]" as the path with setNestedFileEnumeration(true). I am a little curious about a few things. In batch mode which I think is turned on by FileProcessingMode.PROCESS_ONCE mode does the source list all the S3 o