Re: Filter Null in Array in SQL Connector

2020-12-01 Thread Rex Fenley
d": "ts_ms" }, >>>>>> { >>>>>> "type": "string", >>>>>> "optional": true, >>>>>> "name": "io.debezium.data.Enum",

Re: Filter Null in Array in SQL Connector

2020-12-01 Thread Danny Chan
rameters": { "allowed": "true,last,false" }, >>>>> "default": "false", >>>>> "field": "snapshot" >>>>> }, >>>>> { "type": &q

Re: Filter Null in Array in SQL Connector

2020-11-30 Thread Rex Fenley
ield": "snapshot" >>>> }, >>>> { "type": "string", "optional": false, "field": "db" }, >>>> { "type": "string", "optional": false, "field&q

Re: Filter Null in Array in SQL Connector

2020-11-20 Thread Rex Fenley
"schema" }, >>> { "type": "string", "optional": false, "field": "table" }, >>> { "type": "int64", "optional": true, "field": "txId" }, >>>

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Rex Fenley
uot;field": "xmin" } >> ], >> "optional": false, >> "name": "io.debezium.connector.postgresql.Source", >> "field": "source" >> }, >> { "type": "st

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Danny Chan
quot;, > "fields": [ > { "type": "string", "optional": false, "field": "version" }, > { "type": "string", "optional": false, "field": "connector" }, &g

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Jark Wu
ot;: "string", "optional": false, "field": "op" }, > { "type": "int64", "optional": true, "field": "ts_ms" }, > { > "type": "struct", > "fields&q

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Rex Fenley
{ "version": "1.3.0.Final", "connector": "postgresql", "name": "db", "ts_ms": 1605739197360, "snapshot": "true", "db": "db", "schema": "publi

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Dylan Forciea
Thursday, November 19, 2020 at 2:24 AM To: Rex Fenley mailto:r...@remind101.com>> Cc: Flink ML mailto:user@flink.apache.org>> Subject: Re: Filter Null in Array in SQL Connector Hi, Fenley ~ You are right, parsing nulls of ARRAY field is not supported now, i have logged an issue [1]

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Jark Wu
> *Date: *Thursday, November 19, 2020 at 2:24 AM > *To: *Rex Fenley > *Cc: *Flink ML > *Subject: *Re: Filter Null in Array in SQL Connector > > > > Hi, Fenley ~ > > > > You are right, parsing nulls of ARRAY field is not supported now, i have > logged an issue [

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Dylan Forciea
Chan Date: Thursday, November 19, 2020 at 2:24 AM To: Rex Fenley Cc: Flink ML Subject: Re: Filter Null in Array in SQL Connector Hi, Fenley ~ You are right, parsing nulls of ARRAY field is not supported now, i have logged an issue [1] and would fix it soon ~ [1] https://issues.apache.org/jira

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Danny Chan
Can you also share your problematic json string here ? So that we can decide the specific error case cause. Rex Fenley 于2020年11月19日周四 下午2:51写道: > Hi, > > I recently discovered some of our data has NULL values arriving in an > ARRAY column. This column is being consumed by Flink via the Kafka > c

Re: Filter Null in Array in SQL Connector

2020-11-19 Thread Danny Chan
Hi, Fenley ~ You are right, parsing nulls of ARRAY field is not supported now, i have logged an issue [1] and would fix it soon ~ [1] https://issues.apache.org/jira/browse/FLINK-20234 Rex Fenley 于2020年11月19日周四 下午2:51写道: > Hi, > > I recently discovered some of our data has NULL values arriving

Filter Null in Array in SQL Connector

2020-11-18 Thread Rex Fenley
Hi, I recently discovered some of our data has NULL values arriving in an ARRAY column. This column is being consumed by Flink via the Kafka connector Debezium format. We seem to be receiving NullPointerExceptions for when these NULL values in the arrays arrive which restarts the source operator i