Hello Farhon, I think your idea about KStream-KTable join is a good approach with some tweaks, more specifically:
1. Model your rider request as a normal record stream with the combo key of (latitude, longitude). 2. Model your driver location as an ever-updating table with the combo key of (latitude, longitude), and value as the driver information. Whenever a driver's location gets updated, it is captured as two updates: a delete with the old location key and an insert with the new location key. 3. Then upon receiving a new record from the rider stream, issue the range query based on its key with the range on the materialized table. Does this work for you? Guozhang On Mon, Aug 29, 2016 at 11:36 AM, Michael Noll <mich...@confluent.io> wrote: > Quick reply only, since I am on my mobile. Not an exact answer to your > problem but still somewhat related: > http://www.infolace.com/blog/2016/07/14/simple-spatial- > windowing-with-kafka-streams/ > (perhaps you have seen this already). > > -Michael > > > > On Sun, Aug 28, 2016 at 4:55 AM, Farhon Zaharia <fzahari...@gmail.com> > wrote: > > > Hello friends, > > > > I am designing a new streaming component and am looking at how to use > Kafka > > Streams. > > > > I need some guidance with the appropriate flow. > > > > *Problem to solve:* > > The problem I am working on is I have a large group of riders and > drivers. > > I would like to match available riders to nearby drivers. (For > simplicity > > think of Uber or Lyft) > > > > The goal is to have the drivers within a certain delta of gps coordinates > > be notified of a potential rider. > > > > For example, a rider requests to be picked up at location: > > 37.788517, -122.406812 > > > > I would like to select the nearby drivers to send notifications of an > > available match by selecting nearby drivers within a range > > > > latitude < 37.798517 && latitude > 37.778517 && longitude < -122.4106812 > && > > longitude > -122.3906812 > > > > *Note this is a POC and would prefer to select the most nearby drivers, > > then after lookup the address and use my own graph for streets and > > calculate the shortest path on my own. > > > > I would like to have 3 initial topics: riders, drivers, and > paired-onride > > > > What is the best way to do this with Kafka Streams? > > > > *What I have tried or considered:* > > I was considering storing drivers in a Ktable and having riders in a > > KStream and joining them. But I don't think this will work because the > > join is dependent on the key, which in this case I was looking more for a > > select statement to look for a range of gps coordinates as noted above. > > The drivers location will be updated periodically. > > > > I was also thinking of filtering the KStream based on the gps range and > > making a smaller subselection of available drivers within a certain > > distance to a rider. > > > > At this point I am seeking some guidance and if this is not an ideal > > use-case that is also ok. Thanks for any information or direction you > can > > provide. > > > > > > -Farhon > > > -- -- Guozhang
