One more question: does each node keep an index of their own values, or is the index global?
Alexander > Thank you very much, this is the information I was looking for. I started > adding secondary index functionality to Cassandra myself, and it turns out > I am doing almost exactly the same thing. I will try to change my code to > use your implementation as well to compare results. > > Alexander > >> Alexander: >> >> The secondary indexes in 0.7.0 (type KEYS) are stored internally in a >> column >> family, and are kept synchronized with the base data via locking on a >> local >> node, meaning they are always consistent on the local node. Eventual >> consistency still applies between nodes, but a returned result will >> always >> match your query. >> >> This index column family stores a mapping from index values to a sorted >> list >> of matching row keys. When you query for rows between x and y matching a >> value z (via the get_indexed_slices call), Cassandra performs a lookup >> to >> the index column family for the slice of columns in row z between x and >> y. >> If any matches are found in the index, they are row keys that match the >> index clause, and we query the base data to return you those rows. >> >> Iterating through all of the rows matching an index clause on your >> cluster >> is guaranteed to touch N/RF of the nodes in your cluster, because each >> node >> only knows about data that is indexed locally. >> >> Some portions of the indexing implementation are not fully baked yet: >> for >> instance, although the API allows you to specify multiple columns, only >> one >> index will actually be used per query, and the rest of the clauses will >> be >> brute forced. >> >> A second secondary index implementation has been on the back burner for >> a >> while: it provides an identical API, but does not use a column family to >> store the index, and should be more efficient for append only data. See >> https://issues.apache.org/jira/browse/CASSANDRA-1472 >> >> Thanks, >> Stu >> >> On Wed, Feb 9, 2011 at 2:35 AM, <alta...@ceid.upatras.gr> wrote: >> >>> Thank you for the links, I did read a bit in the comments of the >>> ticket, >>> but I couldn't get much out of it. >>> >>> I am mainly interested in how the index is stored and partitioned, not >>> how >>> it is used. I think the people in the dev list will probably be better >>> qualified to answer that. My questions always seem to get moved to the >>> user list, and usually with good cause, but I think this time it should >>> be >>> in the dev list :) Please move it back, if you can. >>> >>> Alexander >>> >>> > AFAIK this was the ticket the original work was done under >>> > https://issues.apache.org/jira/browse/CASSANDRA-1415 >>> > >>> > also http://www.datastax.com/docs/0.7/data_model/secondary_indexes >>> > and http://pycassa.githubcom/pycassa/tutorial.html#indexes may help >>> > >>> > (sorry on reflection the email prob did not need to be moved from >>> dev, >>> my >>> > bad) >>> > Aaron >>> > >>> > On 09 Feb, 2011,at 09:16 AM, Aaron Morton <aa...@thelastpickle.com> >>> wrote: >>> > >>> > Moving to the user group. >>> > >>> > >>> > >>> > On 08 Feb, 2011,at 11:39 PM, alta...@ceid.upatras.gr wrote: >>> > >>> > Hello, >>> > >>> > I'd like some information about how secondary indices work under the >>> hood. >>> > >>> > 1) Is data stored in some external data structure, or is it stored in >>> an >>> > actual Cassandra table, as columns within column families? >>> > 2) Is data stored sorted or not? How is it partitioned? >>> > 3) How can I access index data? >>> > >>> > Thanks in a advance, >>> > >>> > Alexander Altanis >>> > >>> >> > >
