Hi,
good question. Without this method, I guess the children would always be
mandatory. So for nullable parent it is making all the children optional. For
our case we want to be able to honour the nullable specified in the SQL for
each child, so the existing approach is not sufficient,
kind regards, David.
On 2024/09/10 15:15:14 Julian Hyde wrote:
> I haven’t read the code in a while. What is fixupNullabilty trying to
> achieve?
>
> > On Sep 10, 2024, at 3:40 AM, David Radley <david_rad...@uk.ibm.com> wrote:
> >
> >
> > Hello,
> > I am using flink and have a table definition like this.
> > CREATE TABLE source_1
> > (
> > `order_id` STRING NOT NULL,
> > `order_time` STRING NOT NULL,
> > `buyer` ROW<
> > `first_name` STRING,
> > `last_name` STRING NOT NULL,
> > `title` STRING NOT NULL
> > >
> > )
> >
> > WITH (
> > 'connector' = 'kafka',
> > 'topic' = 'mytopic',
> > 'properties.bootstrap.servers' = 'localhost:9092',
> > 'value.format' = 'avro',
> > 'value.fields-include' = 'ALL',
> > 'scan.startup.mode' = 'earliest-offset'
> > );
> >
> > This errors as code in Calcite does not create the correct schema. It
> > `last_name` STRING NOT NULL, `title` STRING NOT NULL to both be
> > nullable.
> >
> > The cause of this is in the table planner when we convert SqlDataTypeSpec
> > to the RelDataType. We push the nullable of buyer onto all of its fields,
> > losing the children’s nullable status .
> >
> > In the debugger I see that it is taking the nullable true (from buyer) and
> > putting this on all the children in fixupNullability. I can see that the
> > SqlDataTypeSpec has this information but it is not used.
> >
> > In terms of fixing this, I wonder if the type could be created with the
> > correct nullable so it doesn’t need to be fixed up or whether we should
> > pass down the SqlDataTypeSpec so it has the right information to set the
> > nullable status. I am new with the cee base so would appreciate your
> > thoughts around this – I am happy to implement the change if we get
> > consensus. WDYT?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Unless otherwise stated above:
> >
> > IBM United Kingdom Limited
> > Registered in England and Wales with number 741598
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
>
