On 17 April 2013 17:10, Prasanna Santhanam <t...@apache.org> wrote:
> On Wed, Apr 17, 2013 at 11:52:08AM +0530, Prasanna Santhanam wrote:
>> On Wed, Apr 17, 2013 at 11:10:01AM +0530, Rohit Yadav wrote:
>> >
>> >
>> > If it does not cost much, let's introduce a new dependency for requesting
>> > stuff (get or post etc.) using "requests" [1] for both marvin and
>> > cloudmonkey. Vijay, you can write your own requester, as it's not much
>> > effort required to implement your own requester once you how signature is
>> > calculated.
>> >
>> > IMO, cloudstackConnection could be rewritten using latest ways and better
>> > libraries as it's not much code and is totally do-able with much less
>> > effort. A lot of flows for auth or non-auth are inter-dependent, the
>> > json->obj and obj->json parts and requesting logic is not straight forward
>> > (each method does *only* one thing). We need to make it generic by
>> > providing both raw and processed outputs, for example the result expects to
>> > return an obj and not json (maybe I need json and not an obj), one reason
>> > why I chose to write my own for cloudmonkey after I tried to reuse it.
>> > Lastly, host marvin on pypi so it's easier for any developer to get it off
>> > the shelf and start hacking some cool clients.
>> >
>> > Cheers.
>> >
>> > [1] http://docs.python-requests.org/en/latest/
>> >
>> >
>>
>> Yeah - we should simplify cloudstackConnection and make it easily
>> exportable for both cloudmonkey,marvin and any other tools that might
>> require to make connections to CS mgmt. requests certainly looks
>> elegant. I'm trying to experiment with it to see if I can cook up
>> something for marvin.
>>
>> At one point I moved from httplib to urllib2 to overcome an annoying
>> BadStatusLine issue from httplib. I still find it hard to throttle
>> multiple tests through a single connection and urllib3 seems to handle
>> that. requests appears to be based on urllib3. So it would make sense
>> to move to it.
>>
>> The json decode,encode is not done within cloudstackConnection. That
>> part is annoying and I explored json and simplejson and there doesn't
>> seem to be an easy way to accomplish marvin's case. Esp. with nested
>> json objects I find it easier to use a recursive json decoder as in
>> jsonHelper. I've switched to simplejson and find that it performs
>> better for large jsons like our listXxx calls. jsonHelper right now
>> makes plain python objects which can be improved to make it a response
>> object as returned by the API. So a createPhysicalNetworkResponse
>> looks like the physicalNetworkResponse and not a plain python dict.
>>
>
> I refactored the cloudstackconnection and pushed a wip branch
> requestsformarvin. Just need to iron out the POST data passing for
> cmds and the key signing. I think we can add the login mechanism based
> on session-based auth that the UI does with requests as well. That way
> we have a single module for all forms of auth done by CS.
>
Fixed in master and tested using the bvt. I added some basic support
for being able to handle POST but it's not yet clear to me which of
the APIs will handle POST. May be if API discovery exposes the
mechanism GET/POST for an API we can handle the POST data elegantly
for marvin and cloudmonkey. What say?
I'm trying to bring in further changes to marvin and add DSL support
after which I'll look into putting it up on pypi.
commit 167781ec0184f1332c0b45ca599f460d7d7b5c92
Author: Prasanna Santhanam <t...@apache.org>
Date: Wed Apr 17 17:06:15 2013 +0530
refactor marvin to use requests instead of urllib2
Use python-requests [1] for Marvin. Requests enables graceful handling
of http connections. Marvin's cloudstackConnection has been refactored,
cleaned up to act as a single module for all kinds of cloudstack API
requesting.
TODO:
1. session based login mechanism of the UI should work from
cloudstackConnection
2. cloudmonkey can also reuse /import marvin.cloudstackConnection
3. More graceful handling of POST requests
[1] http://docs.python-requests.org/en/latest/
Signed-off-by: Prasanna Santhanam <t...@apache.org>
