Re: Working with storm compatibility layer

2016-01-14 Thread Matthias J. Sax
Just saw your email after my answer... Have a look here about task slots. https://ci.apache.org/projects/flink/flink-docs-release-0.10/setup/config.html#configuring-taskmanager-processing-slots Also have a look here (starting from 18:30): https://www.youtube.com/watch?v=UEkjRN8jRx4 -Matthias O

Re: Working with storm compatibility layer

2016-01-14 Thread Matthias J. Sax
Hi, the logs shows: > org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: > Resources available to scheduler: Number of instances=1, total number of > slots=1, available slots=0 You need to increase your task slots in conf/flink-conf.yaml. Look for parameter "taskmanager

Re: Working with storm compatibility layer

2016-01-14 Thread Shinhyung Yang
Dear Matthias, Thank you very much again. I changed the value of taskmanager.numberOfTaskSlots from 1 to 128 in flink-0.10.1/conf/flink-conf.yaml, stopped the local cluster and started local cluster again. And it works fine and well. (It is still running and I can check it clear on the webfrontend

Re: Working with storm compatibility layer

2016-01-14 Thread Shinhyung Yang
Dear Matthias, Thank you for a quick reply. It failed again, however I was able to access to its WebFrontend and it gave me some logs. I wanted to show logs immediately before digging down into it. 19:48:18,011 INFO org.apache.flink.runtime.jobmanager.JobManager - Submitting job 6f5228

Re: Working with storm compatibility layer

2016-01-14 Thread Matthias J. Sax
Hi, I can submit the topology without any problems. Your code is fine. If your program "exits silently" I would actually assume, that you submitted the topology successfully. Can you see the topology in JobManager WebFrontend? If not, do you see any errors in the log files? -Matthias On 01/14/2

Re: Working with storm compatibility layer

2016-01-13 Thread Shinhyung Yang
Dear Matthias, Thank you for the reply! I am so sorry to respond late on the matter. > I just double checked the Flink code and during translation from Storm > to Flink declareOuputFields() is called twice. You are right that is > does the same job twice, but that is actually not a problem. The F

Re: Working with storm compatibility layer

2016-01-09 Thread Matthias J. Sax
Hi, I just double checked the Flink code and during translation from Storm to Flink declareOuputFields() is called twice. You are right that is does the same job twice, but that is actually not a problem. The Flink code is cleaner this way to I guess we will not change it. About lifecyle: If you

Re: Working with storm compatibility layer

2016-01-09 Thread Shinhyung Yang
Dear Matthias, Thank you for replying! that sounds weird and should not happen -- Spout.open() should get > called exactly once. That's what I thought too. I'm new to both Storm and Flink so it's quite complicated for me to handle both yet; would it be helpful for me if I know storm's lifecyle

Re: Working with storm compatibility layer

2016-01-09 Thread Matthias J. Sax
Hello Shinhyung, that sounds weird and should not happen -- Spout.open() should get called exactly once. I am not sure about multiple calls to declareOuputFields though -- if might be called multiple times -- would need to double check the code. However, the call to declareOuputFields should be i

Working with storm compatibility layer

2016-01-08 Thread Shinhyung Yang
Howdies to everyone, I'm trying to use the storm compatibility layer on Flink 0.10.1. The original storm topology works fine on Storm 0.9.5 and I have incorporated FlinkLocalCluster, FlinkTopologyBuilder, and FlinkTopology classes according to the programming guide (https://ci.apache.org/projects/