Sounds good. Is there a basic example somewhere I can have a look at? Niels
On Thu, Aug 25, 2016 at 2:55 PM, Maximilian Michels <m...@apache.org> wrote: > Hi Niels, > > If you're using 1.1.1, then you can instantiate the > YarnClusterDescriptor and supply it with the Flink jar and > configuration and subsequently call `deploy()` on it to receive a > ClusterClient for Yarn which you can submit programs using the > `run(PackagedProgram program, String args)` method. You can also > cancel jobs or shutdown the cluster from the ClusterClient. > > Cheers, > Max > > On Thu, Aug 25, 2016 at 10:24 AM, Niels Basjes <ni...@basjes.nl> wrote: > > Hi, > > > > We have a situation where we need to start a flink batch job on a yarn > > cluster the moment an event arrives over a queue. > > These events occur at a very low rate (like once or twice a week). > > > > The idea we have is to run an application that listens to the queue and > > executes the batch when it receives a message. > > > > We found that if we start this using 'flink run -m yarn-cluster ..." the > > moment we run this the jobmanager in yarn is started and the resources > for > > these batches is claimed immediately. > > > > What is the recommended way to only claim these resources when we > actually > > have a job to run? > > Can we 'manually' start and stop the jobmanager in yarn in some way from > our > > java code? > > > > -- > > Best regards / Met vriendelijke groeten, > > > > Niels Basjes > -- Best regards / Met vriendelijke groeten, Niels Basjes
