In which way is the driver in the classpath?
- fat jar?
- in the nested /out folder in the slim jar?
On Fri, Jun 5, 2015 at 7:23 PM, Flavio Pompermaier <pomperma...@okkam.it>
wrote:
> Actually I just need to load it in the main method (job manager) before
> calling any flink operation, I retrieve the records in a mysql table
> because they contain the path of files I'll need to read. Nothing more
> nothing less
> On 5 Jun 2015 19:06, "Robert Metzger" <rmetz...@apache.org> wrote:
>
>> Sure.
>>
>> So the DriverManager has a static variable called "registeredDrivers".
>> When DriverManager.getConnection() is called, the method is looking up
>> if an registered driver for that connection (in this case "mysql") is
>> available.
>>
>> For drivers to be in that list, they have to register themselves using
>> the DriverManager.registerDriver() method.
>>
>> Drivers can register themselves with a static constructor (which is
>> executed when Java loads a class):
>>
>> public class PoolingDriver implements Driver {
>> /** Register myself with the {@link DriverManager}. */
>> static {
>> try {
>> DriverManager.registerDriver(new PoolingDriver());
>> } catch(Exception e) {
>> }
>> }
>>
>> To execute that driver registration, you need to do:
>> Class.forName("org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver
>> ");
>> because then Java is loading the class and executing the static
>> constructor which is registering the driver at the connection manager.
>>
>> When executing Flink locally, you are using only one JVM. By calling the
>> MySQL driver manually in the main() method of your flink job, you are
>> registering the MySQL driver at the DriverManager of that JVM.
>>
>> However, when you run Flink in a distributed cluster, at the TaskManager
>> JVMs, the MySQL driver is not loaded at the DriverManager there.
>> Therefore, you have to make sure that Class.forName("
>> org.datanucleus.store.rdbms.datasource.dbcp.PoolingDriver"); (this is
>> not the correct class for the MySQL driver)
>> has been called.
>> One approach to do that is to call Class.forName() in the open() method
>> of your function.
>>
>> Best,
>> Robert
>>
>>
>> On Fri, Jun 5, 2015 at 6:54 PM, Flavio Pompermaier <pomperma...@okkam.it>
>> wrote:
>>
>>> HI Robert,
>>> In the main method I connect to a mysql table that acts as a data-source
>>> repository that I use to know which dataset I need to load. All mysql
>>> classes are present in the shaded jar.
>>> Could you explain a little bit more in detail the solution to fix this
>>> problem please? Sorry but I didn't understand it :(
>>>
>>> Thanks,
>>> Flavio
>>> On 5 Jun 2015 18:33, "Robert Metzger" <rmetz...@apache.org> wrote:
>>>
>>>> Hi Stefano,
>>>>
>>>> I doubt that there are conflicting dependencies because Flink does not
>>>> contain MySQL dependencies.
>>>> Are you using Flink's JDBCInputFormat or custom code?
>>>>
>>>> For drivers to register at java.sql's DriverManager, their classes need
>>>> to be loaded first. To load a class, you need to call
>>>> Class.forName("<classname>");
>>>>
>>>> Maybe you are loading the class in the application's main() method
>>>> (thats why it is working from eclipse) but not on the cluster instances
>>>> which are supposed to read the data.
>>>>
>>>> On Fri, Jun 5, 2015 at 5:16 PM, Stefano Bortoli <s.bort...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Robert,
>>>>>
>>>>> I answer on behalf of Flavio. He told me the driver jar was included.
>>>>> Smells lik class-loading issue due to 'conflicting' dependencies. Is it
>>>>> possible?
>>>>>
>>>>> Saluti,
>>>>> Stefano
>>>>>
>>>>> 2015-06-05 16:24 GMT+02:00 Robert Metzger <rmetz...@apache.org>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> is the MySQL driver part of the Jar file that you've build?
>>>>>>
>>>>>> On Fri, Jun 5, 2015 at 4:11 PM, Flavio Pompermaier <
>>>>>> pomperma...@okkam.it> wrote:
>>>>>>
>>>>>>> Hi to all,
>>>>>>>
>>>>>>> I'm using a fresh build of flink-0.9-SNAPSHOT and in my flink job I
>>>>>>> set up a mysql connection.
>>>>>>> When I run the job from Eclipse everything is fine,
>>>>>>> while when running the job from the Web UI I get the following
>>>>>>> exception:
>>>>>>>
>>>>>>> java.sql.SQLException: No suitable driver found for
>>>>>>> jdbc:mysql:/localhost:3306/mydb?autoReconnect=true
>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:596)
>>>>>>> at java.sql.DriverManager.getConnection(DriverManager.java:215)
>>>>>>>
>>>>>>> How can I fix that?
>>>>>>>
>>>>>>> Best,
>>>>>>> Flavio
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>
