On 23.03.2016 10:38, Chesnay Schepler wrote:
On 23.03.2016 10:04, Stefano Bortoli wrote:
I had a look at the JDBC input format, and it does indeed interpret
BIGDECIMAL and NUMERIC values as double.
This sounds more like a bug actually. Feel free to open a JIRA for this.
Actually, this was done so that every field is a primitive or String.
The status of the JDBCInputFormat is not adequate for real world use
case, as for example does not deal with NULL values.
This was already reported in FLINK-3471. To clarify, for NULL fields
the format fails only if the type is either DECIMAL, NUMERIC, DATE,
TIME, TIMESTAMP, or SQLXML. Other types should default to 0, empty
string or false; which actually isn't intended behavior, but caused by
JDBC itself.
Defaulting to some value seems the only way to deal with this issue,
since we can't store null in a Tuple.
I wasn't sure what value DATE, TIME, TIMESTAMP and SQLXML should
default to, as such i didn't change them yet. I also just dislike the
fact that a straight copy from A to B will not produce the same table.
However, with little effort we fixed few stuff and now we are getting
to something usable. We are actually trying to do something a-la
sqoop, therefore given a boundary query, we create the splits, and
then assign it to the input format to read the database with
configurable parallelism. We are still working on it. If we get to
something stable and working, we'll gladly share it.
saluti,
Stefano
2016-03-22 15:46 GMT+01:00 Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>>:
The JDBC formats don't make any assumption as to what DB backend
is used.
A JDBC float in general is returned as a double, since that was
the recommended mapping i found when i wrote the formats.
Is the INT returned as a double as well?
Note: The (runtime) output type is in no way connected to the
TypeInfo you pass when constructing the format.
On 21.03.2016 14 <tel:21.03.2016%2014>:16, Stefano Bortoli wrote:
Hi squirrels,
I working on a flink job connecting to a Oracle DB. I started
from the JDBC example for Derby, and used the TupleTypeInfo
to configure the fields of the tuple as it is read.
The record of the example has 2 INT, 1 FLOAT and 2 VARCHAR.
Apparently, using Oracle, all the numbers are read as Double,
causing a ClassCast exception. Of course I can fix it by
changing the TupleTypeInfo, but I wonder whether there is
some assumption for Oracle and Numbers.
Thanks a lot for your support!
saluti,
Stefano
