Re: How to create schema for flexible json data in Flink SQL

2020-06-01 Thread Guodong Wang
ple, SchemaRegistryCatalog, > once a new avro schema (say schema id = 100) is added to the registry, > users can use this new schema with table "mytopic-100". > > > Best, > Jark > > > On Fri, 29 May 2020 at 22:05, Guodong Wang wrote: > >> Benchao, >

Re: How to create schema for flexible json data in Flink SQL

2020-05-29 Thread Guodong Wang
the > data types at compile stage. > > Maybe I've misunderstood your question at the very beginning. I thought > your case is #2. If your case is #1, then schema inference is a good > choice. > > Guodong Wang 于2020年5月28日周四 下午11:39写道: > >> Yes. Setting the value typ

Re: How to create schema for flexible json data in Flink SQL

2020-05-28 Thread Guodong Wang
Hi Guodong, > > Does the RAW type meet your requirements? For example, you can specify > map type, and the value for the map is the raw JsonNode > parsed from Jackson. > This is not supported yet, however IMO this could be supported. > > Guodong Wang 于2020年5月28日周四 下午9:43写道: >

Re: How to create schema for flexible json data in Flink SQL

2020-05-28 Thread Guodong Wang
ng non-existed > fields to be null. > > For 1, I think maybe we can support it in the future, and I've created > jira[1] to track this. > > [1] https://issues.apache.org/jira/browse/FLINK-18002 > > Guodong Wang 于2020年5月28日周四 下午6:32写道: > >> Hi ! >

How to create schema for flexible json data in Flink SQL

2020-05-28 Thread Guodong Wang
Hi ! I want to use Flink SQL to process some json events. It is quite challenging to define a schema for the Flink SQL table. My data source's format is some json like this { "top_level_key1": "some value", "nested_object": { "nested_key1": "abc", "nested_key2": 123, "nested_key3": ["element1", "