Thank you all for the discussion. I added a note that we might revisit
the signature with an instance.
I started a separate voting thread for the current state of the FLIP. I
will appreciate your votes.
Best,
Dawid
On 11/10/2019 13:21, Jark Wu wrote:
> Hi Timo,
>
> Sounds good to me.
>
> Thanks
Hi Timo,
Sounds good to me.
Thanks,
Jark
On Fri, 11 Oct 2019 at 17:37, Timo Walther wrote:
> Hi Jark,
>
> great to see that there is consensus on something like `CREATE FUNCTION
> bf1 WITH (precision=0.001)`.
>
> In the near future, there should be no difference between a catalog
> function an
Hi Jark,
great to see that there is consensus on something like `CREATE FUNCTION
bf1 WITH (precision=0.001)`.
In the near future, there should be no difference between a catalog
function and a temporary function. This should simplify the stack a lot.
How about we postpone this topic after F
Hi Timo,
The reason to put the parameter in constructor instead of `eval` is that
1) the initialization for the parameters may take long
2) there might be a lot of parameters, it's verbose to specify them in
every where it's used.
Having some syntax like `CREATE FUNCTION bf1 WITH (precision=0.001
Thanks for your feedback Kurt and Jark.
I'm wondering why we allow setting `val bf1 = new BloomFilter(0.001)`,
`val bf2 = new BloomFilter(0.005)`. This is not very SQL-like.
Parameters should be passed when calling the function instead.
Users can use the query itself to parametize the functio
Hi, Timo
Regarding to createTemporaryFunction, I agree with Kurt, we should
encourage the class way, but should keep the instance way.
The reason is that this is an important feature to support parameterize
function as you said, but this can't be addressed by global job parameters
and open() metho
Regarding to createTemporaryFunction, could we just keep both choices? I
agree
we should encourage users to use the class, and it's the only choice when
user
using DDL. But for some Table API & SQL programs, I think it's actually
more nature
to use an object instead of use the classes. But this jus
Thanks for the comments.
@Timo I think your suggestion makes sense. I changed the signature to
void createTemporaryFunction(String path, Class
functionClass);
This will mean that after we make QueryOperations string serializable
and we expose the type inference in functions we will have only a
Thanks for the clarification Timo, that's sounds good to me. Let's continue
to discuss other things.
Best,
Kurt
On Thu, Oct 10, 2019 at 4:14 PM Timo Walther wrote:
> The purpose of ConnectTableDescriptor was to have a programmatic way of
> expressing `CREATE TABLE` with JavaDocs and IDE suppor
The purpose of ConnectTableDescriptor was to have a programmatic way of
expressing `CREATE TABLE` with JavaDocs and IDE support. But I agree
that it needs some rework in the future, once we have finalized the DDL
to ensure that both concepts are in sync.
Regards,
Timo
On 10.10.19 16:08, Kurt
Regarding to ConnectTableDescriptor, if in the end it becomes a builder
of CatalogTable, I would be ok with that. But it doesn't look like a builder
now, it's more like another form of TableSource/TableSink.
Best,
Kurt
On Thu, Oct 10, 2019 at 3:44 PM Timo Walther wrote:
> Hi everyone,
>
> than
Hi everyone,
thanks for the great discussion with a good outcome.
I have one last comment about:
void createTemporaryFunction(String path, UserDefinedFunction function);
We should encourage users to register a class instead of an instance. We
should enforce a default constructor in the future
Hi Dawid,
+1 for proposed changes
On Wed, Oct 9, 2019 at 12:15 PM Dawid Wysakowicz
wrote:
> Sorry for a very delayed response.
>
> @Kurt Yes, this is the goal to have a function created like new
> Function(...) also be wrapped into CatalogFunction. This would have to
> be though a temporary fun
Sorry for a very delayed response.
@Kurt Yes, this is the goal to have a function created like new
Function(...) also be wrapped into CatalogFunction. This would have to
be though a temporary function as we cannot represent that as a set of
properties. Similar to the createTemporaryView(DataStream
IIUC it's good to see that both serializable (tables description from DDL)
and unserializable (tables with DataStream underneath) tables are treated
unify with CatalogTable.
Can I also assume functions that either come from a function class (from
DDL)
or function objects (newed by user) will also
Thanks Dawid for the design doc.
In general, I’m +1 to the FLIP.
+1 to the single-string and parse way to express object path.
+1 to deprecate registerTableSink & registerTableSource.
But I would suggest to provide an easy way to register a custom source/sink
before we drop them (this is an
Hi JingsongLee,
>From my understanding they can. Underneath they will be CatalogTables. The
difference is the lifetime of the tables. Plus some of the user facing
interfaces cannot be persisted e.g. datastream. Therefore we must have a
separate methods for that. In the end the temporary tables are
Hi dawid:
Can temporary tables achieve the same capabilities as catalog table?
like statistics: CatalogTableStatistics, CatalogColumnStatistics,
PartitionStatistics
like partition support: we have added some catalog equivalent interfaces on
TableSource/TableSink: getPartitions, getPartitionFieldN
Hi all,
Sorry to join this party late. Big +1 to this flip, especially for the
dropping
"registerTableSink & registerTableSource" part. These are indeed legacy
and we should try to unify them through CatalogTable after we introduce
the concept of Catalog.
>From my understanding, what we can regis
After some development and thinking, I have a general understanding.
+1 to registering a source/sink does not fit into the SQL world.
I am OK to have a deprecated registerTemporarySource/Sink to compatible with
old ways.
Best,
Jingsong Lee
---
Hi Dawid,
thanks for the design document. It fixes big concept gaps due to
historical reasons with proper support for serializability and catalog
support in mind.
I would not mind a registerTemporarySource/Sink, but the problem that I
see is that many people think that this is the recommende
Thanks david for pushing this forward.
I have one concern about temporary objects and non-persistent catalog(e.g.,
GenericInMemoryCatalog).
In SQL, temporary objects exist at the session level. They are only visible
to the session in which they were created and are automatically dropped
when that s
Hi dawid:
It is difficult to describe specific examples.
Sometimes users will generate some java converters through some
Java code, or generate some Java classes through third-party
libraries. Of course, these can be best done through properties.
But this requires additional work from users.My
Hi all,
@Jingsong Could you elaborate a bit more what do you mean by
/"//some Connectors//are difficult to convert all states to properties"/
All the Flink provided connectors will definitely be expressible with
properties (In the end you should be able to use them from DDL). I think
if a TableS
Thanks to Dawid for starting the discussion and writeup. It looks pretty
good to me except that I'm a little concerned about the object reference
and string parsing in the code, which seems to an anti-pattern to OOP. Have
we considered using ObjectIdenitifier with optional catalog and db parts,
esp
Thanks dawid, +1 for this approach.
One concern is the removal of registerTableSink & registerTableSource
in TableEnvironment. It has two alternatives:
1.the properties approach (DDL, descriptor).
2.from/toDataStream.
#1 can only be properties, not java states, and some Connectors
are difficult
26 matches
Mail list logo
