Smarth, I think integrating into the Aurora Scheduler is a great idea.
If we do that I think it may be a good idea to integrate with the twitter.commons.application ( see an example for a REST application https://github.com/twitter/commons/tree/master/src/java/com/twitter/common/examples/pingpong ) for consistency with logging, etc. I can work with you to get that set up this week. Bill, Does that seem like an acceptable plan? //jeff ________________________________ From: Bill Farner [wfar...@apache.org] Sent: Wednesday, April 23, 2014 10:28 AM To: Madan, Smarth Cc: dev@aurora.incubator.apache.org; madan.sma...@gmail.com; Rose, Jeff; Jolly, Gaurav Subject: Re: Aurora Client over HTTP/JSON Firstly, i think it's _awesome_ that you're hacking with our code! Thanks for sharing a pointer to your fork! How would you feel about integrating this kind of thing into the scheduler rather than a separate webapp? I'd be happy to help acquaint you with the scheduler code if that presents a hurdle. -=Bill On Tue, Apr 22, 2014 at 2:04 PM, Madan, Smarth <sma...@paypal.com<mailto:sma...@paypal.com>> wrote: Hi Bill, I created a REST client for scheduling jobs on Aurora. It uses the Thrift generated java and connects to the Aurora Scheduler server skipping the Python client. I also built a simple UI on top of it. Currently I am testing the UI. My code is in my repository forked from the original Aurura repo : https://github.com/smarth-madan/incubator-aurora . I think it would be a useful tool to schedule jobs using a simple UI and let people try it out easily. I could raise a pull request once I am done testing for the same. Please let me know your suggestions. Thanks Smarth Madan From: Bill Farner <wfar...@apache.org<mailto:wfar...@apache.org>> Date: Thursday, March 27, 2014 at 12:34 PM To: "dev@aurora.incubator.apache.org<mailto:dev@aurora.incubator.apache.org>" <dev@aurora.incubator.apache.org<mailto:dev@aurora.incubator.apache.org>> Cc: "Madan, Smarth" <sma...@paypal.com<mailto:sma...@paypal.com>> Subject: Re: Aurora Client over HTTP/JSON Hi Smarth! This is out of scope of AURORA-20. That ticket tracks creating a guide for new developers on the client. However, i'm not opposed to the feature you propose. The client currently speaks HTTP/JSON. If you invoke ./gradlew run, you might notice these API calls when you view some of the pages on the scheduler web interface. Chrome developer tools makes it easy to convert these calls into curl commands. Here's one stripped down: $ curl 'http://localhost:8081/api' --data-binary '[1,"getRoleSummary",1,0,{}]' [1,"getRoleSummary",2,0,{"0":{"rec":{"1":{"i32":1},"3":{"rec":{"17":{"rec":{"1":{"set":["rec",1,{"1":{"str":"mesos"},"2":{"i32":40},"3":{"i32":20}}]}}}}},"4":{"rec":{"1":{"i32":3}}},"5":{"rec":{"1":{"str":"local"},"2":{"i32":3}}}}}}] This is invoking methods on the thrift API [1], and results are encoded using TJSONProtocol (configured in SchedulerAPIServlet [2]). You'll notice from the output above that TJSONProtocol encodes fields with their numeric field IDs, as specified in api.thrift. This is a fundamental characteristic of IDLs like thrift and protobuf - field IDs rather than names are transmitted over the wire. Obviously, this is somewhat at odds with the traditional HTTP/JSON approach. I see three obvious paths forward, some of which might not suit your needs: - use a part of our client as a python API from your tooling - use thrift-compiled code in your tooling - augment the scheduler to have separate endpoints that expose fields by name rather than numeric ID. there may be prior art for this, but otherwise would look something like SchedulerAPIServlet along with a different TProtocolFactory [3]. Hope that helps! Cheers! -=Bill [1] https://github.com/apache/incubator-aurora/blob/master/src/main/thrift/org/apache/aurora/gen/api.thrift#L496-531 [2] https://github.com/apache/incubator-aurora/blob/master/src/main/java/org/apache/aurora/scheduler/thrift/SchedulerAPIServlet.java [3] http://people.apache.org/~thejas/thrift-0.9/javadoc/org/apache/thrift/protocol/TProtocolFactory.html
