Agreeing with Philip! Our use of Kafka is with a relatively small number of high volume producers and consumers. I'm pretty sure Kafka wasn't designed for large numbers of either Producers or Consumers, as would be necessary for any significant number of mobile clients.
Also, while an Android device is certainly capable of a persistent connection, it's generally not desirable from an application design perspective, or from a power management perspective. An exception to this might be an embedded Android deployment with relatively few clients, of course, so YMMV. Steve On Wed, Jul 16, 2014 at 9:35 PM, Philip O'Toole < philip.oto...@yahoo.com.invalid> wrote: > FWIW, I happen to know Subodh -- we worked together many years back. We > discussed this a little off-the-list, but perhaps my thoughts might be of > wider interest. > > Kafka, in my experience, works best when Producers have a persistent TCP > connection to the Broker(s) (and possibly Zookeeper). I guess that it isn't > really practical for a mobile device. Also, Kafka messages are a specific > protocol over TCP, which will need to be coded for the Android platform. I > don't know how difficult this is, but doesn't seem worth it. > > Putting a lightweight HTTP server in front of Kafka is definitely the way > to go. This could be written in Go, Python, Ruby, node.js -- almost > anything. These servers will be stateless, and you can run many of them > behind a load-balancer or, perhaps AWS Route 53, for redundancy. All > standard stuff, and it's pretty easy nowadays to build scalable web > servers. What happens to the data afterwards is much harder. :-) > > Furthermore, I'm sure it's much easier and quicker to write code on > Android to perform an arbitrary POST to a URL in the Cloud. Mocking out an > HTTP endpoint is very easy too, for the purposes of testing the client > software. > > And the HTTP server can actually be HTTPS, giving the software at the edge > confidence it is talking to the real endpoint. Perhaps run the HTTP servers > behind nginx, and let it do the SSL termination. Secure transport is not > built-into Kafka, last time I checked. > > So while strictly speaking it is more software, the system is > significantly more manageable and robust. > > Philip > > On Wednesday, July 16, 2014 6:33 PM, Steve Robenalt < > sroben...@highwire.org> wrote: > > > > Hi Subodh, > > I would think you'd be better off having an app server of some kind as an > intermediary that accepts messages from your android app and posts them to > Kafka for you, rather than having your app be a Kafka Producer on its own. > > Steve > > > > On Wed, Jul 16, 2014 at 1:22 PM, Subodh Nijsure < > subodh.nijs...@sigsensetech.com> wrote: > > > Hello, > > > > I am trying to make my Android application to be the producer of messages > > that get sent to Kafka server. > > > > I am trying to compile my android code using following libraries - > > kafka_2.9.2-0.8.1.1.jar, scala-library-2.9.2.jar > > > > However I am running into issue where it produces too many methods. > > > > So the question: > > > > Has anyone been successful in sending messages form Android application > to > > server. If I want to trim the libraries should I be just compiling source > > myself or is there and/or any other alternative I should look at to get > my > > android app to send message to Kafka server. > > > > -Subodh > > >
