Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

; properties. Similar to the createTemporaryView(DataStream stream). >>>>>>> >>>>>>> As for the ConnectTableDescriptor I agree this is very similar to >>>>>>> CatalogTable. I am not sure though if we should get rid of it. I

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

;>>> > >>>>> the discussions around FLIP-57. Please take a look. I would like to > >>>>> start a vote on this FLIP as soon as the vote on FLIP-57 goes > through. > >>>>> > >>>>> Best, > >>>>> >

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

quivalent interfaces on TableSource/TableSink: getPartitions, getPartitionFieldNames Maybe it's not a good idea to add these interfaces to TableSource/TableSink. What do you think? Best, Jingsong Lee ------------------------------ From:Kurt Young Send T

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

>> > >>> And it seems only one thing left, how do we deal with > >>> ConnectTableDescriptor? > >>> It's actually very similar with serializable CatalogTable, both carry > >>> > >>> some > >>> > >>> text

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

standing. +1 to registering a source/sink does not fit into the SQL world. I am OK to have a deprecated registerTemporarySource/Sink to compatible with old ways. Best, Jingsong Lee ------------------------------ From:Timo Walther Send Time:2019年9月17日

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

d some catalog equivalent > > > > interfaces > > > > on TableSource/TableSink: getPartitions, getPartitionFieldNames > > Maybe it's not a good idea to add these interfaces to > > TableSource/TableSink. What do you think? > > > > Best, > > Jingsong Lee > > > > > > ---

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

ted e.g. datastream. Therefore we must > > have > > a > > separate methods for that. In the end the temporary tables are held > > in > > memory as CatalogTables. > Best, > Dawid > > On Thu, 19 Sep 2019, 10:08 JingsongLee, > .invalid> > > wrote: > > > Hi dawid: > Can temporary tables achieve the same capabiliti

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

t; >>>>>>>> Thanks Dawid for the design doc. >>>>>>>> >>>>>>>> In general, I’m +1 to the FLIP. >>>>>>>> >>>>>>>> >>>>>>>> +1 to the single-string and parse way to express object path. >>>>>>

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

+1 to deprecate registerTableSink & registerTableSource. > >>>>>> But I would suggest to provide an easy way to register a custom > >>>>>> source/sink before we drop them (this is another story). > >>>>>> Currently, it’s not easy to implement a custom connector descriptor. > >>>&

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Kurt Young Send Time:2019年9月18日(星期三) 17:54 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, Sorry to join this party late. Big +1 to this flip, especially for the dropping "registerTableSink & registerTableSource" part. These are i

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

understanding they can. Underneath they will be > CatalogTables. > >>>> The > >>>>> difference is the lifetime of the tables. Plus some of the user > facing > >>>>> interfaces cannot be persisted e.g. datastream. Therefore we must > have > >> a > >>>>>

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

hink? Best, Jingsong Lee -- From:Kurt Young Send Time:2019年9月18日(星期三) 17:54 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, Sorry to join this party late. Big +1 to this flip, especially for the dropping "registerTableSink & registerTableSource&qu

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

; >>>> and we should try to unify them through CatalogTable after we > introduce > >>>> the concept of Catalog. > >>>> > >>>> From my understanding, what we can registered should all be metadata, > >>>> TableSource/Tabl

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

gt;>> to either use connect() or DDL API which can be validated and stored in >>>>> catalog. >>>>> >>>>> Also from a concept perspective, registering a source/sink does not fit >>>>> into the SQL world. SQL does not know about source/sinks but only about >>>>> tables. If the responsibility

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

g equivalent interfaces > >> on TableSource/TableSink: getPartitions, getPartitionFieldNames > >> Maybe it's not a good idea to add these interfaces to > >> TableSource/TableSink. What do you think? > >> > >> Best, > >> Jingsong Lee &

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

ce >>>> and string parsing in the code, which seems to an anti-pattern to OOP. >>> Have >>>> we considered using ObjectIdenitifier with optional catalog and db >> parts, >>>> esp. if we are worried about arguments of variable length or method &g

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

think? > > Best, > Jingsong Lee > > > -- > From:Kurt Young > Send Time:2019年9月18日(星期三) 17:54 > To:dev > Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table > module > > Hi all, > > Sorry to join this party late. Big +1 to this flip, e

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

ways. > > Best, > Jingsong Lee > > > -- > From:Timo Walther > Send Time:2019年9月17日(星期二) 08:00 > To:dev > Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table > module > > Hi Dawid, > > thanks fo

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

ays. > > Best, > Jingsong Lee > > > -- > From:Timo Walther > Send Time:2019年9月17日(星期二) 08:00 > To:dev > Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table > module > > Hi Dawid, > > thanks for the design document. It

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

-- From:Timo Walther Send Time:2019年9月17日(星期二) 08:00 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi Dawid, thanks for the design document. It fixes big concept gaps due to historical reasons with proper support for serializability and catalog

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

-- From:Dawid Wysakowicz Send Time:2019年9月6日(星期五) 16:21 To:dev Subject:Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, @Jingsong Could you elaborate a bit more what do you mean by "some Connectors are difficult to convert all states to properties

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Thanks david for pushing this forward. I have one concern about temporary objects and non-persistent catalog(e.g., GenericInMemoryCatalog). In SQL, temporary objects exist at the session level. They are only visible to the session in which they were created and are automatically dropped when that s

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

n contain java state. But can't use TableSource-related features, like project & filter push down, partition support, etc.. Any idea about this? Best, Jingsong Lee ------------------ From:Dawid Wysakowicz Send Time:2019年9月4日(星期三) 22:20 To:d

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

rtition support, etc.. >> >> Any idea about this? >> >> Best, >> Jingsong Lee >> >> >> -- >> From:Dawid Wysakowicz >> Send Time:2019年9月4日(星期三) 22:20 >> To:dev >> Su

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

h down, partition support, etc.. > > Any idea about this? > > Best, > Jingsong Lee > > > -- > From:Dawid Wysakowicz > Send Time:2019年9月4日(星期三) 22:20 > To:dev > Subject:[DISCUSS] FLIP-64: Support for Tem

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

id Wysakowicz Send Time:2019年9月4日(星期三) 22:20 To:dev Subject:[DISCUSS] FLIP-64: Support for Temporary Objects in Table module Hi all, As part of FLIP-30 a Catalog API was introduced that enables storing table meta objects permanently. At the same time the majority of current APIs create temp

[DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Hi all, As part of FLIP-30 a Catalog API was introduced that enables storing table meta objects permanently. At the same time the majority of current APIs create temporary objects that cannot be serialized. We shoul