Yup, agreed. I can certainly see use-cases where a client may have particular knowledge about data/server locality, and wants to open connections to *just* those servers. The transports that I am advocating open connections to servers described by one layer higher. That layer could say "all Riak servers in the cluster", or it could say "these three servers holding copies of <this> data".
An auto-discovery layer (to add and remove) in-between the client and the transport would be handy. I can envision that, and my proposed code makes it possible, but (at my current stage) I don't have a use for it. I'd hope that others can build on my connection pooling work to see that through. Cheers, -g On Thu, Sep 8, 2011 at 07:49, Phil Stanhope <stanh...@gmail.com> wrote: > I like the idea of RiakTransport as you describe it. It opens the door to > other potential underlying transports and isolating a client from knowledge > of those (websockets, tcp, zeromq messaging come to mind). I'm not > suggesting that RIAK will ever support these transports by this comment, > however. > I also agree with the need to have RiakRingAwareTransport as an additional > layer that *might* be used by a client. There may be valid reasons why a > client might want to force particular traffic onto a subset of the ring > (e.g. M/R config, Search Config, forcing read/write traffic onto different > nodes, etc). Again, I'm not suggesting that using a subset of the ring for > particular operations is the best practice. But it may be necessary to do so > in order to validate and do certain types of testing to prove or disprove > certain access patterns. > -phil > > On Thu, Sep 8, 2011 at 7:31 AM, Greg Stein <gst...@gmail.com> wrote: >> >> Hey, all... >> >> After a couple comments on my recent work, and some archaeology on the >> Python/Riak work over the past month... I've realized that I might >> have a very different view of RiakTransport compared to what I'm >> seeing in the current work. I figured it best to bring that to the >> forefront and discuss: >> >> In my view, RiakTransport is used by RiakClient (and others) to "talk >> to the Riak server". >> >> Some of the current work, and some proposed pull requests, seem to >> take the position that a RiakTransport is "one connection to a server, >> and the client should manage those". >> >> Needless to say, I'm in favor of my own position :-) ... I think it is >> best to transfer *all* responsibility for talking to the server(s) to >> the transport layer. I really don't think the client/bucket/object >> layers should know anything about talking to the server(s). I'd like >> to see the transport layer be told about all server(s) available, and >> then it Just Works. >> >> I'm still a newbie with all this code, and need to keep plugging away >> at the higher levels of functionality and compensation for problems. >> I'd like to build up some code that contacts *one* given server, asks >> for all of the ring servers, and then opens connections to those >> servers. And then, it should (automatically) maintain client >> connections based on what is happening with the Riak cluster. The >> current (proposed) code manages connections to N servers, but has no >> automatic add/remove based on changes in the cluster status. I think >> this happens at a layer *just* above the actual transport. ie. >> something tracks the changes in the ring status and its servers, and >> transmits those changes into the transport layer, which alters its >> communication with that cluster (regardless of whether that >> communication is via HTTP or protobuf). >> >> >> Okee doke. That's the end of my brain dump and future thoughts on the >> transport and communication layer. I'd really like some feedback, >> review, and thoughts. >> >> Thanks! >> -g >> >> _______________________________________________ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
