[jira] [Created] (FLINK-3270) Add example for reading and writing to Kafka

Robert Metzger created FLINK-3270: - Summary: Add example for reading and writing to Kafka Key: FLINK-3270 URL: https://issues.apache.org/jira/browse/FLINK-3270 Project: Flink Issue Type: Impr

Stackoverflow in Serialization

Hi, I am using flink to train a ML model. There, I need to broadcast huge broadcast variables in each iteration. During the serialization I get a Stackoverflow exception. If I increase the Stack size then it works well. The Stackoverflow could be avoided If you do not use recursion in serializ

Re: Flink distributed cache in scalding

Hi Milosz, The RuntimeContext of a Flink operator is a member of FlinkFlowProcess. You could try to add a getRuntimeContext() method to FlinkFlowProcess and cast the FlowProcess parameter in your user code to FlinkFlowProcess. Of course, this change would tie the program to Flink as the execution

Re: Stackoverflow in Serialization

Hi Hilmi, I've never seen such a problem. Could you please provide a bit more background information. Which types are you serializing? Which serializer are you using? What do you mean with recursion? Maybe you can provide a simple example program which reproduces the problem. The thing is that if

Re: Stackoverflow in Serialization

Hi, I have a complex object with a MutableList and a Map containing thousands of objects. Here is a snippet of the exception. The actual exception is much longer. There, you can see that the exception is caused by too many recursive calls. For each recursive call the stack size increases becau

Re: Stackoverflow in Serialization

As you can see from the namespace and the object names, it's not our code which causes the stack overflow. It's Kryo code which you're calling and thus we have not control over it. And in fact, I doubt that the problem is related to the number of elements in your map but rather to the structure of

Deep Learning on Flink

Hey all, Does anyone know of any specific deep learning implementations on Flink. I searched the mailing list and saw there was a discussion last year, but that is about all that came up. I need to do some deep learning for work, and learn about it my self - if nothing out there exists, I'll wri

Re: Deep Learning on Flink

Hi Trevor, at the moment we don't have a deep learning library implemented on top of Flink. As far as I know, Suneel (smar...@apache.org) tinkered with the idea to implement deep learning with Flink. Maybe he can tell you more about his progress so far. Supporting deep learning would be awesome :

Re: Deep Learning on Flink

Hi Trevor, There's been an initiative that's just started out on to add Flink as another execution engine on Deeplearning4J (http://deeplearning4j.org). We could move this conversation there. On Thu, Jan 21, 2016 at 9:40 AM, Trevor Grant wrote: > Hey all, > > Does anyone know of any specific

Re: Flink distributed cache in scalding

Fabian, It's true cascading doesn't have native distributed cache support, but scalding does. https://github.com/twitter/scalding/blob/develop/scalding-core/src/main/scala/com/twitter/scalding/filecache/DistributedCacheFile.scala It already abstracts local / Hadoop mode. I'm trying to add Flink di

Re: Scala 2.10/2.11 Maven dependencies

https://issues.apache.org/jira/browse/FLINK-2940 There is now a pending pull request: https://github.com/apache/flink/pull/1529 As I was working on the changes, I discovered we have some more modules which have a Scala dependency which could be avoided. Namely these are "flink-java8", and "flink-

Feedback welcome: reworking the examples with ParameterTools

Hello dev, I'm currently working on [FLINK-2021] Rework examples to use ParameterTool and we're trying to improve the readability of the K-Means example until we reach a satisfactory starting point to rework the other examples as well. I think Rob

[jira] [Created] (FLINK-3271) Using webhdfs in a flink topology throws classnotfound exception

Abhishek Agarwal created FLINK-3271: --- Summary: Using webhdfs in a flink topology throws classnotfound exception Key: FLINK-3271 URL: https://issues.apache.org/jira/browse/FLINK-3271 Project: Flink

Re: Scala 2.10/2.11 Maven dependencies

> On 21 Jan 2016, at 17:51, Maximilian Michels wrote: > > https://issues.apache.org/jira/browse/FLINK-2940 > > There is now a pending pull request: https://github.com/apache/flink/pull/1529 > > As I was working on the changes, I discovered we have some more > modules which have a Scala depende

Re: What is the topic & offset used for in KeyedDeserializationSchema.deserialize()?

Hi Robert, No problem, thanks for the notice! -- View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/What-is-the-topic-offset-used-for-in-KeyedDeserializationSchema-deserialize-tp9911p9980.html Sent from the Apache Flink Mailing List archive. mailing li