Enriching events with data from external http resources

[Maciek Próchniak]

Hi,

Our data streams do some filtering based on data from external http 
resources (not maintained by us, they're really fast with redis as storage).

So far we did that by just invoking synchronously some http client in 
map/flatMap operations. It works without errors but it seems somehow 
inefficient to have to use degree of paralellism just to wait for 
blocking http call - especially when you think about all recent 
developments of fast async clients and so on. I was wondering if there 
is some way of invoking http (or other external) service in non-blocking 
way.

I know it's not really desired way of using flink and that it would be 
better to keep data as state inside stream and have it updated by some 
join operator, but for us it's a bit of overkill - what's more, we have 
many (not so large) streams, it would be not really feasible to keep all 
of state (which is the same) in each of them.

Are there any patterns/ways - existing or planned of dealing with such 
situation?

thanks,

maciek