Wouldn't that be orthogonal to adding it to the TypeInfoParser?
2016-10-27 15:22 GMT+02:00 Greg Hogan <c...@greghogan.com>:
> Fabian,
>
> Should we instead add this as a registered TypeInfoFactory?
>
> Greg
>
> On Thu, Oct 27, 2016 at 3:55 AM, Fabian Hueske <fhue...@gmail.com> wrote:
>
>> Yes, I think you are right.
>> TypeInfoParser needs to be extended to parse the java.sql.* types into
>> the corresponding TypeInfos.
>>
>> Can you open a JIRA for that?
>>
>> Thanks, Fabian
>>
>> 2016-10-27 9:31 GMT+02:00 Radu Tudoran <radu.tudo...@huawei.com>:
>>
>>> Hi,
>>>
>>>
>>>
>>> I dig meanwhile more through this and I think I found a bug actually.
>>>
>>>
>>>
>>> The scenario that I was trying to describe was something like
>>>
>>> 1. You have a generic datastream with Tuple (alternatively I
>>> could move to row I guess) and you get the data from whatever stream (e.g.,
>>> kafka, network socket…)
>>>
>>> 2. In the map/flat map function you parse and instantiate the
>>> tuple generically
>>>
>>> 3. In the “returns()” function of the map you enforce the types
>>>
>>>
>>>
>>> DataStream<Tuple> = env.socketTextStream(…)
>>>
>>> .map(new mapFunction(){
>>>
>>> Public Tuple map(String value){
>>>
>>> Tuple out =
>>> Tuple.getTupleClass(#)
>>>
>>> …
>>>
>>> out.setField(SqlTimeTypeInfo.TIMESTAMP,0)
>>>
>>> …
>>>
>>>
>>>
>>> }}) .returns(“Tuple#<java.sql.TIMESTAMP,…>”);
>>>
>>>
>>>
>>>
>>>
>>> The problem is that if you rely on the type extraction mechanism called
>>> after the returns to recognize TIMESTAMP of type SqlTimeTypeInfo it will
>>> not happen but instead a GenericType<TIMESTAMP> will be created.
>>>
>>> It looks like the type parsers were not extended to consider this types
>>>
>>>
>>>
>>> Dr. Radu Tudoran
>>>
>>> Senior Research Engineer - Big Data Expert
>>>
>>> IT R&D Division
>>>
>>>
>>>
>>> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>>>
>>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>>
>>> European Research Center
>>>
>>> Riesstrasse 25, 80992 München
>>>
>>>
>>>
>>> E-mail: *radu.tudo...@huawei.com <radu.tudo...@huawei.com>*
>>>
>>> Mobile: +49 15209084330
>>>
>>> Telephone: +49 891588344173
>>>
>>>
>>>
>>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
>>> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
>>> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
>>> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
>>> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>>>
>>> This e-mail and its attachments contain confidential information from
>>> HUAWEI, which is intended only for the person or entity whose address is
>>> listed above. Any use of the information contained herein in any way
>>> (including, but not limited to, total or partial disclosure, reproduction,
>>> or dissemination) by persons other than the intended recipient(s) is
>>> prohibited. If you receive this e-mail in error, please notify the sender
>>> by phone or email immediately and delete it!
>>>
>>>
>>>
>>> *From:* Fabian Hueske [mailto:fhue...@gmail.com]
>>> *Sent:* Wednesday, October 26, 2016 10:11 PM
>>> *To:* user@flink.apache.org
>>> *Subject:* Re: TIMESTAMP TypeInformation
>>>
>>>
>>>
>>> Hi Radu,
>>>
>>> I might not have complete understood your problem, but if you do
>>>
>>> val env = StreamExecutionEnvironment.getExecutionEnvironment
>>> val tEnv = TableEnvironment.getTableEnvironment(env)
>>>
>>> val ds = env.fromElements( (1, 1L, new Time(1,2,3)) )
>>> val t = ds.toTable(tEnv, 'a, 'b, 'c)
>>>
>>> val results = t
>>> .select('c + 10.seconds)
>>>
>>> then field 'c will be of type SqlTimeTypeInfo and handled as such.
>>>
>>> Hope this helps,
>>>
>>> Fabian
>>>
>>>
>>>
>>> 2016-10-25 17:32 GMT+02:00 Radu Tudoran <radu.tudo...@huawei.com>:
>>>
>>> Re-hi,
>>>
>>>
>>>
>>> I actually realized that the problem comes from the fact that the
>>> datastream that I am registering does not create properly the types.
>>>
>>>
>>>
>>> I am using something like
>>>
>>>
>>>
>>> DataStream<Tuple> … .returns(“TupleX<,….,java.sql.Timestamp,
>>> java.sql.Time>”)…and I was expecting that these will be converted to
>>> SqlTimeTypeInfo…but it is converted to GenericType. Anythoughts how I could
>>> force the type to be recognize as a SqlTimeType?
>>>
>>>
>>>
>>>
>>>
>>> *From:* Radu Tudoran
>>> *Sent:* Tuesday, October 25, 2016 4:46 PM
>>> *To:* 'user@flink.apache.org'
>>> *Subject:* TIMESTAMP TypeInformation
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I would like to create a TIMESTAMP type from the data schema. I would
>>> need this to match against the FlinkTypeFactory (toTypeInfo())
>>>
>>>
>>>
>>> *def* toTypeInfo(relDataType: RelDataType): TypeInformation[_] =
>>> relDataType.getSqlTypeName *match* {
>>>
>>> *case* BOOLEAN => BOOLEAN_TYPE_INFO
>>>
>>> *case* TINYINT => BYTE_TYPE_INFO
>>>
>>> *case* SMALLINT => SHORT_TYPE_INFO
>>>
>>> *case* INTEGER => INT_TYPE_INFO
>>>
>>> *case* BIGINT => LONG_TYPE_INFO
>>>
>>> *case* FLOAT => FLOAT_TYPE_INFO
>>>
>>> *case* DOUBLE => DOUBLE_TYPE_INFO
>>>
>>> *case* VARCHAR | CHAR => STRING_TYPE_INFO
>>>
>>> *case* DECIMAL => BIG_DEC_TYPE_INFO
>>>
>>>
>>>
>>> // date/time types
>>>
>>> *case* DATE => SqlTimeTypeInfo.DATE
>>>
>>> *case* TIME => SqlTimeTypeInfo.TIME
>>>
>>> *case* *TIMESTAMP** => SqlTimeTypeInfo.**TIMESTAMP*
>>>
>>>
>>>
>>> I tried to use create the TypeInformation by calling directly
>>> SqlTimeTypeInfo.TIMESTAMP . However, it seems that
>>> relDataType.getSqlTypeName match is of type ANY instead of being of type
>>> TIMESTAMP.
>>>
>>>
>>>
>>> Any thoughts of how to create the proper TIMESTAMP typeinformation?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>
