Hi,
I dig meanwhile more through this and I think I found a bug actually.
The scenario that I was trying to describe was something like
1. You have a generic datastream with Tuple (alternatively I could move
to row I guess) and you get the data from whatever stream (e.g., kafka, network
socket…)
2. In the map/flat map function you parse and instantiate the tuple
generically
3. In the “returns()” function of the map you enforce the types
DataStream<Tuple> = env.socketTextStream(…)
.map(new mapFunction(){
Public Tuple map(String value){
Tuple out =
Tuple.getTupleClass(#)
…
out.setField(SqlTimeTypeInfo.TIMESTAMP,0)
…
}}) .returns(“Tuple#<java.sql.TIMESTAMP,…>”);
The problem is that if you rely on the type extraction mechanism called after
the returns to recognize TIMESTAMP of type SqlTimeTypeInfo it will not happen
but instead a GenericType<TIMESTAMP> will be created.
It looks like the type parsers were not extended to consider this types
Dr. Radu Tudoran
Senior Research Engineer - Big Data Expert
IT R&D Division
[cid:image007.jpg@01CD52EB.AD060EE0]
HUAWEI TECHNOLOGIES Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München
E-mail: radu.tudo...@huawei.com<mailto:radu.tudo...@huawei.com>
Mobile: +49 15209084330
Telephone: +49 891588344173
HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany,
www.huawei.com<http://www.huawei.com/>
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
This e-mail and its attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed above.
Any use of the information contained herein in any way (including, but not
limited to, total or partial disclosure, reproduction, or dissemination) by
persons other than the intended recipient(s) is prohibited. If you receive this
e-mail in error, please notify the sender by phone or email immediately and
delete it!
From: Fabian Hueske [mailto:fhue...@gmail.com]
Sent: Wednesday, October 26, 2016 10:11 PM
To: user@flink.apache.org
Subject: Re: TIMESTAMP TypeInformation
Hi Radu,
I might not have complete understood your problem, but if you do
val env = StreamExecutionEnvironment.getExecutionEnvironment
val tEnv = TableEnvironment.getTableEnvironment(env)
val ds = env.fromElements( (1, 1L, new Time(1,2,3)) )
val t = ds.toTable(tEnv, 'a, 'b, 'c)
val results = t
.select('c + 10.seconds)
then field 'c will be of type SqlTimeTypeInfo and handled as such.
Hope this helps,
Fabian
2016-10-25 17:32 GMT+02:00 Radu Tudoran
<radu.tudo...@huawei.com<mailto:radu.tudo...@huawei.com>>:
Re-hi,
I actually realized that the problem comes from the fact that the datastream
that I am registering does not create properly the types.
I am using something like
DataStream<Tuple> … .returns(“TupleX<,….,java.sql.Timestamp,
java.sql.Time>”)…and I was expecting that these will be converted to
SqlTimeTypeInfo…but it is converted to GenericType. Anythoughts how I could
force the type to be recognize as a SqlTimeType?
From: Radu Tudoran
Sent: Tuesday, October 25, 2016 4:46 PM
To: 'user@flink.apache.org<mailto:user@flink.apache.org>'
Subject: TIMESTAMP TypeInformation
Hi,
I would like to create a TIMESTAMP type from the data schema. I would need this
to match against the FlinkTypeFactory (toTypeInfo())
def toTypeInfo(relDataType: RelDataType): TypeInformation[_] =
relDataType.getSqlTypeName match {
case BOOLEAN => BOOLEAN_TYPE_INFO
case TINYINT => BYTE_TYPE_INFO
case SMALLINT => SHORT_TYPE_INFO
case INTEGER => INT_TYPE_INFO
case BIGINT => LONG_TYPE_INFO
case FLOAT => FLOAT_TYPE_INFO
case DOUBLE => DOUBLE_TYPE_INFO
case VARCHAR | CHAR => STRING_TYPE_INFO
case DECIMAL => BIG_DEC_TYPE_INFO
// date/time types
case DATE => SqlTimeTypeInfo.DATE
case TIME => SqlTimeTypeInfo.TIME
case TIMESTAMP => SqlTimeTypeInfo.TIMESTAMP
I tried to use create the TypeInformation by calling directly
SqlTimeTypeInfo.TIMESTAMP . However, it seems that relDataType.getSqlTypeName
match is of type ANY instead of being of type TIMESTAMP.
Any thoughts of how to create the proper TIMESTAMP typeinformation?
