Re: Location-aware replication based on objects' access pattern

2011-04-06 Thread Sasha Dolgy
I use the global ring to provide quick reference data. I don't see it as a bottle neck because there isn't a lot of data floating around it. Just enough to satisfy specific use cases we have. If the end user needs to access the data in the US ring while in Asia via the web, I would opt to transf

Re: Location-aware replication based on objects' access pattern

2011-04-06 Thread Yudong Gao
On Wed, Apr 6, 2011 at 3:55 AM, Sasha Dolgy wrote: > I had been asked this question from a strategy point of view, and > referenced how linkedin.com appears to handle this. > > > Specific region data is stored on a ring in that region.  While based > in the middle east, my linkedin.com profile wa

Re: Location-aware replication based on objects' access pattern

2011-04-06 Thread Sasha Dolgy
I had been asked this question from a strategy point of view, and referenced how linkedin.com appears to handle this. Specific region data is stored on a ring in that region. While based in the middle east, my linkedin.com profile was kept on the middle east part of linkedin.com ... when I moved

Re: Location-aware replication based on objects' access pattern

2011-04-06 Thread Jonathan Colby
good to see a discussion on this. This also has practical use for business continuity where you can control that the clients in a given data center first write replicas to its own data center, then to the other data center for backup. If I understand correctly, a write takes the token into ac

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Jonathan Ellis
On Tue, Apr 5, 2011 at 10:45 PM, Yudong Gao wrote: >> A better solution would be to just push the DecoratedKey into the >> ReplicationStrategy so it can make its decision before information is >> thrown away. > > I agree. So in this case, I guess the hashed based token ring is still > preserved to

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Yudong Gao
On Tue, Apr 5, 2011 at 9:59 PM, Jonathan Ellis wrote: > On Tue, Apr 5, 2011 at 8:37 PM, Yudong Gao wrote: >> One thing I am worrying about is how to maintain the location >> information for each row. The current partitioner maps a key to MD5 >> hash, and it is almost impossible to control the has

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Jonathan Ellis
On Tue, Apr 5, 2011 at 8:37 PM, Yudong Gao wrote: > One thing I am worrying about is how to maintain the location > information for each row. The current partitioner maps a key to MD5 > hash, and it is almost impossible to control the hashed token by > manipulating the value of the key. Also, main

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Yudong Gao
Thanks for the reply, Jonathan! This per-row control is exactly what I need. I will be happy to help tackle it in the long term. Is there some further information or plan for this issues? One thing I am worrying about is how to maintain the location information for each row. The current partition

Re: Location-aware replication based on objects' access pattern

2011-04-05 Thread Jonathan Ellis
You'd really want https://issues.apache.org/jira/browse/CASSANDRA-2369 to control per-row. Let me know if you'd like to help tackle that. On Tue, Apr 5, 2011 at 5:05 PM, Yudong Gao wrote: > > Hi, > > I am thinking about using Cassandra for our research project, and we > are thinking about one int

Location-aware replication based on objects' access pattern

2011-04-05 Thread Yudong Gao
Hi, I am thinking about using Cassandra for our research project, and we are thinking about one interesting feature. Our setup has multiple datacenters located in different geography locations. Data is accessed with predictable patterns. Think of something like Craigslist, data objects correspon