An issue with low-throughput on Flink 1.8.3 running Yahoo streaming benchmarks

2019-12-29 Thread Shinhyung Yang
ns: 5 process.hosts: 1 process.cores: 16 # flink-conf.yaml jobmanager.heap.mb: 1024 taskmanager.heap.mb: 15360 taskmanager.numberOfTaskSlots: 16 taskmanager.memory.preallocate: false parallelism.default: 1 taskmanager.network.numberOfBuffers: 6432 Thank you and with best regards, Shinhyung Yang [1]:

Re: scaling a flink streaming application on a single node

2016-04-04 Thread Shinhyung Yang
anager.heap.mb: 5000 > taskmanager.heap.mb: 10 > > So, about 100 GB of RAM for the TaskManager. Definitely I will try this! The result will be really interesting for sure. In this case, am I still good to go with 64 task slots with a single task manager? Thank you. With best regards, Shinhyung Yang.

Re: scaling a flink streaming application on a single node

2016-03-31 Thread Shinhyung Yang
mum before moving on to run the experiment on multiple machines. Thank you again! With best regards, Shinhyung Yang

scaling a flink streaming application on a single node

2016-03-31 Thread Shinhyung Yang
st that might be worth trying. Thank you. With best regards, Shinhyung Yang #== # parallelism.default: 1 #== Source: Read Text File Source -> Flat M

Re: Working with storm compatibility layer

2016-01-14 Thread Shinhyung Yang
webfrontend) Although I'm not sure whether it would be safe to keep the value like this or not. Thank you. Best regards, Shinhyung On Thu, Jan 14, 2016 at 7:53 PM, Shinhyung Yang wrote: > Dear Matthias, > > Thank you for a quick reply. It failed again, however I was able to > access to it

Re: Working with storm compatibility layer

2016-01-14 Thread Shinhyung Yang
blems. Your code is fine. > > If your program "exits silently" I would actually assume, that you > submitted the topology successfully. Can you see the topology in > JobManager WebFrontend? If not, do you see any errors in the log files? > > -Matthias > > On 01/14/2016

Re: Working with storm compatibility layer

2016-01-13 Thread Shinhyung Yang
ncrease the log level to get more > information: > https://ci.apache.org/projects/flink/flink-docs-release-0.10/internals/logging.html I tried to inject `log4j.properties' file that I got from a sample flink-quickstart-java application created from `mvn archetype:generate' to a ./targe

Re: Working with storm compatibility layer

2016-01-09 Thread Shinhyung Yang
he > latest version from Flink master branch. I should work with 0.10.1 > without problems. It was vey tedious for me to deal with a pom.xml file and .m2 repository. So I preferred to use maven central. But I should try with the master branch if I have to. I will quickly check if I could s

Working with storm compatibility layer

2016-01-08 Thread Shinhyung Yang
Spout class but declareOutputFields method is called for multiple times and the program exits silently. Do you guys have any idea what's going on here or have any suggestions? If needed, then please ask me for more information. Thank you for reading. With best regards, Shinhyung Yang