Hi Robert, I am using 0.10.1.
> Am 19.01.2016 um 17:42 schrieb Robert Metzger <rmetz...@apache.org>: > > Hi Max, > > which version of Flink are you using? > > On Tue, Jan 19, 2016 at 5:35 PM, Maximilian Bode <maximilian.b...@tngtech.com > <mailto:maximilian.b...@tngtech.com>> wrote: > Hi everyone, > > I am facing a problem using the JDBCInputFormat which occurred in a larger > Flink job. As a minimal example I can reproduce it when just writing data > into a csv after having read it from a database, i.e. > > DataSet<Tuple1<String>> existingData = env.createInput( > JDBCInputFormat.buildJDBCInputFormat() > .setDrivername("oracle.jdbc.driver.OracleDriver") > .setUsername(…) > .setPassword(…) > .setDBUrl(…) > .setQuery("select DATA from TABLENAME") > .finish(), > new TupleTypeInfo<>(Tuple1.class, BasicTypeInfo.STRING_TYPE_INFO)); > existingData.writeAsCsv(…); > > where DATA is a column containing strings of length ~25 characters and > TABLENAME contains 20 million rows. > > After starting the job on a YARN cluster (using -tm 3072 and leaving the > other memory settings at default values), Flink happily goes along at first > but then fails after something like three million records have been sent by > the JDBCInputFormat. The Exception reads "The slot in which the task was > executed has been released. Probably loss of TaskManager …". The local > taskmanager.log in the affected container reads > "java.lang.OutOfMemoryError: GC overhead limit exceeded > at > java.util.Collections$UnmodifiableCollection.iterator(Collections.java:1063) > at > org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:119) > at > org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83) > at > org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) > at > org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744)" > > Any ideas what is going wrong here? > > Cheers, > Max > > — > Maximilian Bode * Junior Consultant * maximilian.b...@tngtech.com > <mailto:maximilian.b...@tngtech.com> > TNG Technology Consulting GmbH, Betastr. 13a, 85774 Unterföhring > Geschäftsführer: Henrik Klagges, Christoph Stock, Dr. Robert Dahlke > Sitz: Unterföhring * Amtsgericht München * HRB 135082 > >
signature.asc
Description: Message signed with OpenPGP using GPGMail
