Ok, here's code for the pure json apibeta bench. I realized it was a bit unfair because I was making concurrent requests (max 3). Stripping out all concurrency, it's coming in at 17.2sec.
drop in src/aurora/, no dependencies required. go build o build -o dist/aurora2 aurora/client2/bench dist/aurora2 -url="http://<aurora_master>:8081/apibeta" On Mon, Dec 29, 2014 at 3:36 PM, Isaac Councill <is...@hioscar.com> wrote: > Here's source from go thrift (warning: very ugly). I had to make a few > modifications to the ttypes and client libraries to get it working. It > requires git.apache.org/thrift.git from the 0.9.2 tree (0.9.1 generates > code that is much farther from correct). > > after putting it in src/aurora/client: > go build -o dist/aurora aurora/client/thrift/read_only_scheduler-remote > > then: > time dist/aurora -u="http://<aurora_master>:8081/api" -http=true -P=json > doBench > > Perhaps my general unhappiness with thrift right now has to due with > immature go support. Just take a look at that generated code, and it didn't > actually work, at all, without some messing around. > > Re: Kevin's question about differences between getTaskStatus requests, the > contents I was filling in are the same (role, environment, jobName), but > inspecting network traffic, it did appear that the taskStatusRequest from > go thrift is adding empty values for nil list fields, which could impact > the results. > > As for the apibeta client, it will take me a little extra time to rip that > out into a sharable form. > > On Mon, Dec 29, 2014 at 2:17 PM, Jake Farrell <jfarr...@apache.org> wrote: > >> Hey Isaac >> Would love to hear your pain points with Thrift and also can you share >> your >> source for the test clients >> >> -Jake >> >> On Mon, Dec 29, 2014 at 1:27 AM, Isaac Councill <is...@hioscar.com> >> wrote: >> >> > tl;dr; >> > apibeta seems way faster (and arguably better) than thrift api. What are >> > the long term objectives for apibeta? >> > >> > >> > Hi, >> > >> > I've been working on some aurora integrations, primarily a blackbox >> > monitoring tool at present, and was looking for the best way to >> communicate >> > with the scheduler. >> > >> > For a large read-only example, I wanted to dump the latest scheduler >> status >> > info for all our prod jobs, basically: >> > >> > for all roles: >> > for all jobs in role: >> > get scheduler status >> > >> > We have about 120 prod jobs in aurora right now (growing fast). I >> > benchmarked 3 strategies against our prod cluster (mean of 5 tries each >> > from remote vpn, variance was small in each case): >> > >> > 1) aurora2 client: ./aurora2 job status cluster/<role>/prod > /dev/null >> > 126.0sec >> > >> > 2) golang thrift API >> > 584.3sec (I might be able make a better task query, but still... this is >> > for only ~120 calls) >> > >> > 3) Pure json apibeta client in golang >> > 13.4sec (again, might be able to optimize query strategy) >> > >> > As a side note, getting the golang thrift client to work was a very >> painful >> > and illuminating experience. >> > >> > I'm inclined to stick with apibeta. It's fast and the documentation is >> > great. If api changes become a concern, well after today I'd honestly >> > prefer rolling my own binding generator. >> > >> > Are there plans for /apibeta wrt /api? >> > >> > >
golang_json_client.tgz
Description: GNU Zip compressed data
