Hi, I’m not sure if I understand your problem and your context, but spawning a batch job every 45 seconds doesn’t sound as a that bad idea (as long as the job is short).
Another idea would be to incorporate this batch job inside your streaming job, for example by reading from Cassandra using an AsyncIO operator: https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/operators/asyncio.html Quick google search revealed for example this: https://stackoverflow.com/questions/43067681/read-data-from-cassandra-for-processing-in-flink Piotrek > On 23 Nov 2018, at 10:33, eric hoffmann <sigfrid.hoffm...@gmail.com> wrote: > > Hi > Is it possible to call batch job on a streaming context? > what i want to do is: > for a given input event, fetch cassandra elements based on event data, apply > transformation on them and apply a ranking when all elements fetched by > cassandra are processed. > If i do this in batch mode i would have to submit a job on each events and i > can have an event every 45 seconds. > Is there any alternative? can i start a batch job that will receive some > external request, process it and wait for another request? > thx > Eric
