Thanks Philip, I will certainly take a look. I had not considered the fact that Kafka doesn't have any security/authentication mechanism yet, so using some kind of secure gateway certainly makes sense.
-Subodh On Wed, Jul 16, 2014 at 10:03 PM, Philip O'Toole < philip_o_to...@yahoo.com.invalid> wrote: > You should find code here that will help you get a HTTP app server > together, that writes to Kafka on the back-end. > > https://cwiki.apache.org/confluence/display/KAFKA/Clients > https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem > > > > > On Wednesday, July 16, 2014 9:36 PM, Philip O'Toole > <philip.oto...@yahoo.com.INVALID> wrote: > > > > FWIW, I happen to know Subodh -- we worked together many years back. We > discussed this a little off-the-list, but perhaps my thoughts might be of > wider interest. > > Kafka, in my experience, works best when Producers have a persistent TCP > connection to the Broker(s) (and possibly Zookeeper). I guess that it isn't > really practical for a mobile device. Also, Kafka messages are a specific > protocol over TCP, which will need to be coded for the Android platform. I > don't know how difficult this is, but doesn't seem worth it. > > Putting a lightweight HTTP server in front of Kafka is definitely the way > to go. This could be written in Go, Python, Ruby, node.js -- almost > anything. These servers will be stateless, and you can run many of them > behind a load-balancer or, perhaps AWS Route 53, for redundancy. All > standard stuff, and it's pretty easy nowadays to build scalable web > servers. What happens to the data afterwards is much harder. :-) > > Furthermore, I'm sure it's much easier and quicker to write code on > Android to perform an arbitrary POST to a URL in the Cloud. Mocking out an > HTTP endpoint is very easy too, for the purposes of testing the client > software. > > And the HTTP server can actually be HTTPS, giving the software at the edge > confidence it is talking to the real endpoint. Perhaps run the HTTP servers > behind nginx, and let it do the SSL termination. Secure transport is not > built-into Kafka, last time I checked. > > So while strictly speaking it is more software, the system is > significantly more manageable and robust. > > Philip > > > On Wednesday, July 16, 2014 6:33 PM, Steve Robenalt < > sroben...@highwire.org> wrote: > > > > Hi Subodh, > > I would think you'd be better off having an app server of some kind as an > intermediary that accepts messages from your android app and posts them to > Kafka for you, rather than having your app be a Kafka Producer on its own. > > Steve > > > > On Wed, Jul 16, 2014 at 1:22 PM, Subodh Nijsure < > subodh.nijs...@sigsensetech.com> wrote: > > > Hello, > > > > I am trying to make my Android application to be the producer of messages > > that get sent to Kafka server. > > > > I am trying to compile my android code using following libraries - > > kafka_2.9.2-0.8.1.1.jar, scala-library-2.9.2.jar > > > > However I am running into issue where it produces too many methods. > > > > So the question: > > > > Has anyone been successful in sending messages form Android application > to > > server. If I want to trim the libraries should I be just compiling source > > myself or is there and/or any other alternative I should look at to get > my > > android app to send message to Kafka server. > > > > -Subodh > > >
