Sure.
So the DriverManager has a static variable called "registeredDrivers".
When DriverManager.getConnection() is called, the method is looking up if
an registered driver for that connection (in this case "mysql") is
available.
For drivers to be in that list, they have to register themselves using
the DriverManager.registerDriver() method.
Drivers can register themselves with a static constructor (which is
executed when Java loads a class):
public class PoolingDriver implements Driver {
/** Register myself with the {@link DriverManager}. */
static {
try {
DriverManager.registerDriver(new PoolingDriver());
} catch(Exception e) {
}
}
To execute that driver registration, you need to do:
Class.forName("org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver");
because then Java is loading the class and executing the static constructor
which is registering the driver at the connection manager.
When executing Flink locally, you are using only one JVM. By calling the
MySQL driver manually in the main() method of your flink job, you are
registering the MySQL driver at the DriverManager of that JVM.
However, when you run Flink in a distributed cluster, at the TaskManager
JVMs, the MySQL driver is not loaded at the DriverManager there.
Therefore, you have to make sure that Class.forName("
org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver"); (this is not
the correct class for the MySQL driver)
has been called.
One approach to do that is to call Class.forName() in the open() method of
your function.
Best,
Robert
On Fri, Jun 5, 2015 at 6:54 PM, Flavio Pompermaier <pomperma...@okkam.it>
wrote:
> HI Robert,
> In the main method I connect to a mysql table that acts as a data-source
> repository that I use to know which dataset I need to load. All mysql
> classes are present in the shaded jar.
> Could you explain a little bit more in detail the solution to fix this
> problem please? Sorry but I didn't understand it :(
>
> Thanks,
> Flavio
> On 5 Jun 2015 18:33, "Robert Metzger" <rmetz...@apache.org> wrote:
>
>> Hi Stefano,
>>
>> I doubt that there are conflicting dependencies because Flink does not
>> contain MySQL dependencies.
>> Are you using Flink's JDBCInputFormat or custom code?
>>
>> For drivers to register at java.sql's DriverManager, their classes need
>> to be loaded first. To load a class, you need to call
>> Class.forName("<classname>");
>>
>> Maybe you are loading the class in the application's main() method (thats
>> why it is working from eclipse) but not on the cluster instances which are
>> supposed to read the data.
>>
>> On Fri, Jun 5, 2015 at 5:16 PM, Stefano Bortoli <s.bort...@gmail.com>
>> wrote:
>>
>>> Hi Robert,
>>>
>>> I answer on behalf of Flavio. He told me the driver jar was included.
>>> Smells lik class-loading issue due to 'conflicting' dependencies. Is it
>>> possible?
>>>
>>> Saluti,
>>> Stefano
>>>
>>> 2015-06-05 16:24 GMT+02:00 Robert Metzger <rmetz...@apache.org>:
>>>
>>>> Hi,
>>>>
>>>> is the MySQL driver part of the Jar file that you've build?
>>>>
>>>> On Fri, Jun 5, 2015 at 4:11 PM, Flavio Pompermaier <
>>>> pomperma...@okkam.it> wrote:
>>>>
>>>>> Hi to all,
>>>>>
>>>>> I'm using a fresh build of flink-0.9-SNAPSHOT and in my flink job I
>>>>> set up a mysql connection.
>>>>> When I run the job from Eclipse everything is fine,
>>>>> while when running the job from the Web UI I get the following
>>>>> exception:
>>>>>
>>>>> java.sql.SQLException: No suitable driver found for
>>>>> jdbc:mysql:/localhost:3306/mydb?autoReconnect=true
>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:215)
>>>>>
>>>>> How can I fix that?
>>>>>
>>>>> Best,
>>>>> Flavio
>>>>>
>>>>
>>>>
>>>
>>
