[
https://issues.apache.org/jira/browse/FLINK-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208238#comment-16208238
]
Fabian Hueske commented on FLINK-7548:
--------------------------------------
Thanks for the comments [~twalthr] and [~xccui].
@ Timo: I would not add {{TimestampExtractor}} and {{WatermarkAssigner}} to
{{TableSchema}}. {{TableSchema}} is also used at different places and timestamp
extractors & watermarks assigners are not relevant in these contexts. IMO,
{{TimestampExtractor}} and {{WatermarkAssigner}} belong to a {{TableSource}}
but not to the schema. The schema could encode that these fields are
non-nullable and watermarked (or however, we want to call this property). I
think the issue of defining fields twice is rather an issue of a concrete
{{TableSource}} implementation but not so much of the internal API. Given the
current API, it is possible to have a {{TableSource}} that requires to define a
field just once (in whatever way) and generate an appropriate TableSchema and
proctime field name.
@ Xingcan:
1. With the current design it shouldn't be too hard to define a
{{TimestampExtractor}} that parses a string field. I would not add it to the
{{ExistingField}} extractor because it would need an additional parameter that
specifies the formatting of the timestamp string.
2. Multiple rowtime attributes might not be possible yet but might be later.
With this API we have the option to add this feature later without breaking the
API.
3. It's also possible to add a {{TimestampExtractor}} that assigns the current
timestamp as event timestamp, i.e., ingestion time.
4. Yes, that was my initial motivation to use RexNode instead of Table API
Expressions. I was thinking that it would be easier to parse SQL expressions
from a CREATE TABLE statement into RexNodes (using some Calcite code) than to
Expressions. OTOH, I think it makes sense to be consistent and not expose
Calcite API. I'd be happy to hear what others think about the RexNode vs.
Expression choice.
> Support watermark generation for TableSource
> --------------------------------------------
>
> Key: FLINK-7548
> URL: https://issues.apache.org/jira/browse/FLINK-7548
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Reporter: Jark Wu
> Assignee: Fabian Hueske
> Priority: Blocker
> Fix For: 1.4.0
>
>
> As discussed in FLINK-7446, currently the TableSource only support to define
> rowtime field, but not support to extract watermarks from the rowtime field.
> We can provide a new interface called {{DefinedWatermark}}, which has two
> methods {{getRowtimeAttribute}} (can only be an existing field) and
> {{getWatermarkGenerator}}. The {{DefinedRowtimeAttribute}} will be marked
> deprecated.
> How to support periodic and punctuated watermarks and support some built-in
> strategies needs further discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
