Greg,
You were correct, its doing a lookup on initialization(something I'll have
to deal with next). Here is a traceback(cut off at app code) when a bad
port is used:
File "/home/mike/django/riak-python-client/riak/bucket.py", line 260, in
get
return obj.reload(r)
File "/home/mike/django/riak-python-client/riak/riak_object.py", line
373, in reload
Result = t.get(self, r, vtag)
File "/home/mike/django/riak-python-client/riak/transports/http.py", line
108, in get
response = self.http_request('GET', url)
File "/home/mike/django/riak-python-client/riak/transports/http.py", line
467, in http_request
conn.request(method, uri, body, headers)
File "/usr/lib/python2.7/httplib.py", line 955, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 989, in _send_request
self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 951, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 811, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 773, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 754, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 111] Connection refused
So it works as expected, which is great.
Here is the fork with the hostports hack,
https://github.com/glassresistor/riak-python-client/tree/feature/pooled_servers
**just noticed some whitespace diffs, will rebase soon
Once I run the tests etc. I'll consider a pull request, its not impressive
but I doubt it would hurt.
Next step make the transport remove a connection and retry on failure.
Obviously how this aspect works is going to be really app dependent, so it
should be easily overridden I think subclassing a transport is completely
reasonable. I don't know the best default but personally I prefer to
shuffle the list order at the cm level, connect to one, and use the
connections it returns as the new connection list.
I think one key thing is that it might be best for the connection manager
to keep track of previously failed connections but only connect to them
when a) data is indirectly returned from the previously failed node b) all
other connections have failed. The goals are two fold 1) decrease the
connection time to that of a single node when possible, 2) increase the
liklyhood of a connection will succeed(since failures waste time).
On Tue, Jan 24, 2012 at 9:51 AM, Greg Stein <gst...@gmail.com> wrote:
> On Tue, Jan 24, 2012 at 12:34, Michael Clemmons <glassresis...@gmail.com>
> wrote:
> > Greg,
> > Your amazing thanks. In my application its failing on the start of the
> > application, I do not believe while trying to do a request but its
> possible
> > let me grok and get back to you with some trace backs.
>
> Sounds good. I just looked at the code and it "should" lazy-connect.
> You shouldn't see any failures at setup time. Only when you make a
> request and it tries to establish te connection.
>
> > As far as Im aware to define more than one hostport with the client you
> > still have to hack the client. Adding an optional hostports or servers
> > parameter would be simple.
>
> Yes. I was focusing on the lower-level code, and didn't hack
> RiakClient's API. I'm thinking two things: add hostports, and a
> start_monitor parameter (and code up the latter, of course).
>
> > Being able to define the connection manager as a kwarg might be a good
> > option. If the intent is to define the conextmanager by subclassing the
> > transport, things make more sense.
>
> There are a couple paths to take, at least. The current code says
> "subclass the transport and override the default_cm classvar". That is
> sufficient, but maybe there is a better/clearer approach.
>
> > I think for multiple nodes round robin
> > might be the most sane default for longterm or short term connections.
>
> It does imply that you'll distribute your request load across the
> ring. If N clients are running, then this is important (otherwise, all
> clients would just hit the first node in the config). There are
> certainly more sophisticated algorithms possible, but the round-robin
> used right now should work for most users.
>
> Note that if a particular request takes a while, the connection is NOT
> in the ConnectionManager. That will ensure that you don't back up a
> bunch of future requests behind a single, slow request. Only when the
> (slow) request completes will the connection be returned to the CM for
> usage by another request.
>
> > Thanks again for replying, I'll see what happens when I try this with
> > multiple live nodes, and get back with more thoughts.
>
> Excellent!
>
> Cheers,
> -g
>
--
-Michael
_______________________________________________
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
