Hi Ashwin,
if you quickly want to make this work you can look into
org.apache.flink.table.descriptors.RowtimeValidator#getRowtimeComponents.
This is the component that converts the string property into a
org.apache.flink.table.sources.tsextractors.TimestampExtractor. You can
implement your custom timestamp extractor that performs some logic.
Regards,
Timo
Am 17.07.18 um 14:37 schrieb Ashwin Sinha:
Hi Timo,
We want to add this functionality in a forked branch. Can you guide us
with steps to quickly apply a patch/fix for the same?
On Mon, Jul 16, 2018 at 9:06 PM Ashwin Sinha <ashwin.si...@go-mmt.com
<mailto:ashwin.si...@go-mmt.com>> wrote:
Thanks Timo for the clarification, but our processing actually
involves aggregations on huge past data also, which won't be
served by processing time.
Is this a WIP feature?
On Mon, Jul 16, 2018 at 7:29 PM Timo Walther <twal...@apache.org
<mailto:twal...@apache.org>> wrote:
Hi Ashwin,
the SQL Client is in an early development stage right now and
has some limitations. Your problem is one of them. I files an
issue for this: https://issues.apache.org/jira/browse/FLINK-9864
There is no easy solution to fix this problem. Maybe you can
use processing-time for your windows?
Regards,
Timo
Am 16.07.18 um 14:25 schrieb Ashwin Sinha:
Hi Users,
In Flink1.5 SQL CLient
<https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/table/sqlClient.html>,
we are trying to define rowTime from a nested JSON element,
but struggling with syntax.
JSON data format: https://pastebin.com/ByCLhEnF
YML table config: https://pastebin.com/cgEtQPDQ
Now, in above config, we want to access
*payload.after.modifiedon *as rowTime column. We tried SQL
query <https://pastebin.com/yCU4WWhK> with aggregation on
'payload.after.modifiedon' as time but get this
<https://pastebin.com/bTbushYN> error.
Is there anyway where we can register nested timestamp field
as rowTime for the source table?
--
*Ashwin Sinha *| Data Engineer
ashwin.si...@go-mmt.com <mailto:shivam.sha...@go-mmt.com> |
9452075361
<https://www.makemytrip.com/><https://www.goibibo.com/>
<https://www.redbus.in/>
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
This message is intended only for the use of the addressee
and may contain information that is privileged, confidential
and exempt from disclosure under applicable law. If the
reader of this message is not the intended recipient, or the
employee or agent responsible for delivering the message to
the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication
is strictly prohibited. If you have received this e-mail in
error, please notify us immediately by return e-mail and
delete this e-mail and all attachments from your system.
--
*Ashwin Sinha *| Data Engineer
ashwin.si...@go-mmt.com <mailto:shivam.sha...@go-mmt.com> | 9452075361
<https://www.makemytrip.com/><https://www.goibibo.com/>
<https://www.redbus.in/>
--
*Ashwin Sinha *| Data Engineer
ashwin.si...@go-mmt.com <mailto:shivam.sha...@go-mmt.com> | 9452075361
<https://www.makemytrip.com/><https://www.goibibo.com/>
<https://www.redbus.in/>
::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------
This message is intended only for the use of the addressee and may
contain information that is privileged, confidential and exempt from
disclosure under applicable law. If the reader of this message is not
the intended recipient, or the employee or agent responsible for
delivering the message to the intended recipient, you are hereby
notified that any dissemination, distribution or copying of this
communication is strictly prohibited. If you have received this e-mail
in error, please notify us immediately by return e-mail and delete
this e-mail and all attachments from your system.
