Re: Cassandra lucene secondary indexes

2018-12-13 Thread dinesh.jo...@yahoo.com.INVALID
Providing logs or more technical information might be helpful. If it is cassandra-lucene related issue, perhaps it'll be better to open a issue in their github repo? Dinesh On Wednesday, December 12, 2018, 11:17:06 PM GMT+5:30, Brian Spindler wrote: Hi all, we recently started using

Cassandra lucene secondary indexes

2018-12-12 Thread Brian Spindler
Hi all, we recently started using the cassandra-lucene secondary index support that Instaclustr recently assumed ownership of, thank you btw! We are experiencing a strange issue where adding/removing nodes fails and the joining node is left hung with a compaction "Secondary index build" and it jus

RE: Secondary Indexes C* 3.0

2018-02-23 Thread Jacques-Henri Berthemet
A very interesting and detailed article, thank you DuyHai. I think this should be part of general Cassandra documentation. -- Jacques-Henri Berthemet From: DuyHai Doan [mailto:doanduy...@gmail.com] Sent: Thursday, February 22, 2018 7:04 PM To: user Subject: Re: Secondary Indexes C* 3.0 Read

Re: Secondary Indexes C* 3.0

2018-02-22 Thread DuyHai Doan
2018 at 9:35 AM, Akash Gangil > wrote: > >> Hi, >> >> I was wondering if there are recommendations around the cardinality of >> secondary indexes. >> >> As I understand an index on a column with many distinct values will be >> inefficient. Is it becaus

Re: Secondary Indexes C* 3.0

2018-02-22 Thread Akash Gangil
To provide more context, I was going through this https://docs.datastax.com/en/cql/3.3/cql/cql_using/useWhenIndex.html#useWhenIndex__highCardCol On Thu, Feb 22, 2018 at 9:35 AM, Akash Gangil wrote: > Hi, > > I was wondering if there are recommendations around the cardinality of &g

Secondary Indexes C* 3.0

2018-02-22 Thread Akash Gangil
Hi, I was wondering if there are recommendations around the cardinality of secondary indexes. As I understand an index on a column with many distinct values will be inefficient. Is it because the index would only direct me to the specfic sstable, but then it sequentially searches for the target

Questions about maintaining secondary indexes.

2017-11-03 Thread Razi Khaja
Hello, I have a few questions about secondary indexes. 1st Question: Quoting this FAQ: https://wiki.apache.org/cassandra/SecondaryIndexes Q: When you write a new row, when/how does the index get updated? What I > would like to know is the atomicity of the operation--is the "ind

Re: Datastax Driver Mapper & Secondary Indexes

2017-09-26 Thread Nicolas Henneaux
uesday, 26 September 2017 at 15:17 To: user Subject: Re: Datastax Driver Mapper & Secondary Indexes If you're looking for schema generation from Bean annotations: https://github.com/doanduyhai/Achilles/wiki/DDL-Scripts-Generation On Tue, Sep 26, 2017 at 2:50 PM, Daniel Hölbling-Inzko mail

Re: Datastax Driver Mapper & Secondary Indexes

2017-09-26 Thread DuyHai Doan
gt;> as part of the where clause. Note that you should really understand the >> drawbacks of secondary indexes before using them, as they might not be >> incredibly efficient depending on what you need them for. >> http://www.wentnet.com/blog/?p=77 and https://pantheon.io/blog/ >

Re: Datastax Driver Mapper & Secondary Indexes

2017-09-26 Thread Daniel Hölbling-Inzko
fying it > as part of the where clause. Note that you should really understand the > drawbacks of secondary indexes before using them, as they might not be > incredibly efficient depending on what you need them for. > http://www.wentnet.com/blog/?p=77 and > https://pantheon.io/

Re: Datastax Driver Mapper & Secondary Indexes

2017-09-26 Thread kurt greaves
If you've created a secondary index you simply query it by specifying it as part of the where clause. Note that you should really understand the drawbacks of secondary indexes before using them, as they might not be incredibly efficient depending on what you need them for. http://www.wentne

Datastax Driver Mapper & Secondary Indexes

2017-09-26 Thread Daniel Hölbling-Inzko
Hi, I am currently moving an application from SQL to Cassandra using Java. I successfully got the DataStax driver and the mapper up and running, but can't seem to figure out how to set secondary indexes through the mapper. I also can't seem to find anything related to indexes in the mapp

Secondary indexes don't return expected results in 2.2.3

2017-03-03 Thread Enrique Bautista
Hello everyone, I run a 2.2.3 cluster and I've found that sometimes queries by secondary indexes don't return the expected results, even though the data is in the table. I'm sure the data is in there because queries by partition key or without WHERE clause do return the expected

Restricting secondary indexes

2016-04-11 Thread Thanigai Vellore
Hello, In a multi-DC setup (where one DC serves real-time traffic and the other DC serves up analytical loads), is it possible to setup and restrict secondary indexes only to the analytics DC? The intent is to not create the overhead of the secondary index on the DC where real-time traffic is

Feedback on Secondary Indexes

2015-09-02 Thread Edouard COLE
Hello, I don't know if this is a good place to talk about that, but I think this might help some people running into the same issue, so I will simply give some feedback here about what I was running in the last few months I've been using secondary index (yes, this is bad), but the side effects

Re: Secondary indexes not working properly since 2.1.0-rc2 ?

2014-08-14 Thread Fabrice Larcher
2.1.0-rc2, 2.1.0-rc3 and I currently use >> 2.1.0-rc5. Since 2.1.0-rc2, it appears that the secondary indexes are not >> always working. Just after the INSERT of a row, the index seems to be >> there. But after a while (I do not know when or why), SELECT statements >> based on a

Re: Secondary indexes not working properly since 2.1.0-rc2 ?

2014-08-13 Thread DuyHai Doan
wrote: > Hello, > > I have used C* 2.1.0-rc1, 2.1.0-rc2, 2.1.0-rc3 and I currently use > 2.1.0-rc5. Since 2.1.0-rc2, it appears that the secondary indexes are not > always working. Just after the INSERT of a row, the index seems to be > there. But after a while (I do not know when o

Secondary indexes not working properly since 2.1.0-rc2 ?

2014-08-13 Thread Fabrice Larcher
Hello, I have used C* 2.1.0-rc1, 2.1.0-rc2, 2.1.0-rc3 and I currently use 2.1.0-rc5. Since 2.1.0-rc2, it appears that the secondary indexes are not always working. Just after the INSERT of a row, the index seems to be there. But after a while (I do not know when or why), SELECT statements based

dropping secondary indexes

2014-07-30 Thread Parag Patel
Hi, I've noticed that our datamodel has many unnecessary secondary indexes. Are there a recommended procedure to drop a secondary index on a very large table? Is there any sort of repair/cleanup that should be done after calling the DROP command? Thanks, Parag

Re: Tombstones on secondary indexes

2014-05-28 Thread Robert Coli
On Thu, May 15, 2014 at 1:43 AM, Joel Samuelsson wrote: > https://issues.apache.org/jira/browse/CASSANDRA-4314 seems to say that > tombstones on secondary indexes are not removed by a compaction. Do I need > to do it manually? > The ticket you have pasted says : "It'

Tombstones on secondary indexes

2014-05-16 Thread Joel Samuelsson
e not removed. https://issues.apache.org/jira/browse/CASSANDRA-4314 seems to say that tombstones on secondary indexes are not removed by a compaction. Do I need to do it manually? Best regards, Joel Samuelsson

Re: Cannot get secondary indexes on fields in compound primary key to work (Cassandra 2.0.0)

2013-09-19 Thread Petter von Dolwitz (Hem)
Sep 6, 2013 at 6:18 AM, Petter von Dolwitz (Hem) < >> petter.von.dolw...@gmail.com> wrote: >> >>> I am struggling with getting secondary indexes to work. I have created >>> secondary indexes on some fields that are part of the compound primary key >>> but on

Re: Cannot get secondary indexes on fields in compound primary key to work (Cassandra 2.0.0)

2013-09-08 Thread Petter von Dolwitz (Hem)
://www.datastax.com/dev/blog/cql3-for-cassandra-experts 2013/9/6 Robert Coli > On Fri, Sep 6, 2013 at 6:18 AM, Petter von Dolwitz (Hem) < > petter.von.dolw...@gmail.com> wrote: > >> I am struggling with getting secondary indexes to work. I have created >> secondary indexes on some

Re: Cannot get secondary indexes on fields in compound primary key to work (Cassandra 2.0.0)

2013-09-06 Thread Robert Coli
On Fri, Sep 6, 2013 at 6:18 AM, Petter von Dolwitz (Hem) < petter.von.dolw...@gmail.com> wrote: > I am struggling with getting secondary indexes to work. I have created > secondary indexes on some fields that are part of the compound primary key > but only one of the indexes see

Cannot get secondary indexes on fields in compound primary key to work (Cassandra 2.0.0)

2013-09-06 Thread Petter von Dolwitz (Hem)
I am struggling with getting secondary indexes to work. I have created secondary indexes on some fields that are part of the compound primary key but only one of the indexes seems to work (the one set on the field 'e' on the table definition below). Using any other secondary index

Scrub on secondary indexes

2013-09-01 Thread Boris Yen
ot; seems to prevent the log message from being logged into system.log again. However, when the cf name is for secondary indexes, there seems no way to make this error go away unless I manually remove the data files from cassandra. I tried to patch the cassandra to make running scrub on second

Re: Secondary Indexes On Partitioned Time Series Data Question

2013-08-02 Thread Gareth Collins
OK, thanks for the information. Gareth On Thu, Aug 1, 2013 at 3:53 PM, Robert Coli wrote: > On Thu, Aug 1, 2013 at 12:49 PM, Gareth Collins > wrote: >> >> Would this be correct? Just making sure I understand how to best use >> secondary indexes in Cassandra with ti

Re: Secondary Indexes On Partitioned Time Series Data Question

2013-08-01 Thread Shahab Yunus
Thanks a lot. Regards, Shahab On Thu, Aug 1, 2013 at 8:32 PM, Robert Coli wrote: > On Thu, Aug 1, 2013 at 2:34 PM, Shahab Yunus wrote: > >> Can you shed some more light (or point towards some other resource) that >> why you think built-in Secondary Indexes should not

Re: Secondary Indexes On Partitioned Time Series Data Question

2013-08-01 Thread Robert Coli
On Thu, Aug 1, 2013 at 2:34 PM, Shahab Yunus wrote: > Can you shed some more light (or point towards some other resource) that > why you think built-in Secondary Indexes should not be used easily or > without much consideration? Thanks. > 1) Secondary indexes are more or less mo

Re: Secondary Indexes On Partitioned Time Series Data Question

2013-08-01 Thread Shahab Yunus
Hi Robert, Can you shed some more light (or point towards some other resource) that why you think built-in Secondary Indexes should not be used easily or without much consideration? Thanks. Regards, Shahab On Thu, Aug 1, 2013 at 3:53 PM, Robert Coli wrote: > On Thu, Aug 1, 2013 at 12:49

Re: Secondary Indexes On Partitioned Time Series Data Question

2013-08-01 Thread Robert Coli
On Thu, Aug 1, 2013 at 12:49 PM, Gareth Collins wrote: > Would this be correct? Just making sure I understand how to best use > secondary indexes in Cassandra with time series data. > In general unless you ABSOLUTELY NEED the one unique feature of built-in Secondary Indexes (atomic

Secondary Indexes On Partitioned Time Series Data Question

2013-08-01 Thread Gareth Collins
condary_key1/secondary_key2 as Cassandra secondary indexes for these queries I assume that secondary_key1/secondary_key_2 would really need to be composites combined into one column (in SQL I would create multi-column indexes)? i.e.: secondary_key_1 - pk_part1 + partition_key + real_secondary_key_1 sec

Re: Slow retrieval using secondary indexes

2013-05-06 Thread aaron morton
ultant New Zealand @aaronmorton http://www.thelastpickle.com On 4/05/2013, at 12:11 AM, Francisco Nogueira Calmon Sobral wrote: > Thanks! > > The creation of the new CF worked pretty well and fast! Unfortunately, I was > unable to trace the request made using secondary indexe

Re: Slow retrieval using secondary indexes

2013-05-03 Thread Francisco Nogueira Calmon Sobral
Thanks! The creation of the new CF worked pretty well and fast! Unfortunately, I was unable to trace the request made using secondary indexes: cqlsh:Sessions> select * from "Items" where key = '687474703a2f2f6573706f7'; key

Re: Slow retrieval using secondary indexes

2013-04-28 Thread aaron morton
ry key > and gets using secondary indexes: > > > [default@Sessions] get Users where mahoutUserid = 30127944399716352; > --- > RowKey: STQ0TTNII2LS211YYJI4GEV80M1SE8 > => (column=mahoutUserid, value=30127944399716352, timestamp=1366820944696000) > > 1 Row

Slow retrieval using secondary indexes

2013-04-26 Thread Francisco Nogueira Calmon Sobral
Hi all! We are using Cassandra 1.2.1 with a 8 node cluster running at Amazon. We started with 6 nodes and added the 2 later. When performing some reads in Cassandra, we observed a high difference between gets using the primary key and gets using secondary indexes: [default@Sessions] get

Re: Secondary Indexes

2013-03-22 Thread Mayank
Ok. I always know the row key before I start the Cassandra read operation. A full system could have 300-500k columns so secondary indexes don't seem a good idea here. I think the best option will be to query a range of columns for the given row key. Thanks a bunch guys. On Mar 20, 2013,

Re: Secondary Indexes

2013-03-21 Thread aaron morton
> When I query for user_id = "user1" and order_attr1 = 1991 I want to get the > order_num. Is this possible without super columns? If you only have a few hundred columns you can read them all back and filter client side. Secondary indexes are used when you do not know the row

Re: Secondary Indexes

2013-03-20 Thread Mayank
gt; safely assume that its related indexes will also expire? > Yes > > > > > http://www.datastax.com/docs/1.1/ddl/indexes > > Maintaining Secondary Indexes > Emailed to ask for clarification. > > Cheers > > > - > Aaron Morton > Freela

Re: Secondary Indexes

2013-03-19 Thread aaron morton
It is compacted when it needs it, not when the parent CF is compacted. > About expirations: if the column in the user defined column family expires > can I > safely assume that its related indexes will also expire? Yes > > http://www.datastax.com/docs/1.1/ddl/indexes >

Re: Secondary Indexes

2013-03-19 Thread Mayank
Thanks guys. I am working with Andy on this project. Further questions on the secondary indexes: Assuming we have 1000 columns in 1 row of the column family and about 900 of them have NamedColumn1=1 and of those 900 only 10 of them also have NamedColumn2=1. If I query for columns which have

Re: Secondary Indexes

2013-03-19 Thread aaron morton
ng/search, and Cassandra provides the bulk store. > > From: Andy Stec [mailto:andys...@gmail.com] > Sent: Saturday, March 16, 2013 12:10 AM > To: user@cassandra.apache.org > Subject: Secondary Indexes > > We need to provide search capability based on a field that is a bitm

RE: Secondary Indexes

2013-03-17 Thread moshe.kranc
...@gmail.com] Sent: Saturday, March 16, 2013 12:10 AM To: user@cassandra.apache.org Subject: Secondary Indexes We need to provide search capability based on a field that is a bitmap combination of 18 possible values. We want to use secondary indexes to improve performance. One possible solution is

Secondary Indexes

2013-03-15 Thread Andy Stec
We need to provide search capability based on a field that is a bitmap combination of 18 possible values. We want to use secondary indexes to improve performance. One possible solution is to create a named column for each value and have a secondary index for each of the 18 columns. Questions we

Re: Limitations on secondary indexes

2013-03-07 Thread Sylvain Lebresne
that it is a replica for. > So there is a limitation currently indeed, but it is of 2 billion indexed > rows per-node. > > > On Thu, Mar 7, 2013 at 8:05 PM, Edward Sargisson wrote: > >> Hi, >> Please correct me if this statement is wrong. >> >> Secondary indexe

Re: Limitations on secondary indexes

2013-03-07 Thread Sylvain Lebresne
Not exactly. Each replica only indexes the rows that it is a replica for. So there is a limitation currently indeed, but it is of 2 billion indexed rows per-node. On Thu, Mar 7, 2013 at 8:05 PM, Edward Sargisson wrote: > Hi, > Please correct me if this statement is wrong. > > Secon

Limitations on secondary indexes

2013-03-07 Thread Edward Sargisson
Hi, Please correct me if this statement is wrong. Secondary indexes are limited to indexing 2 billion rows - because they turn a row into a column and C* has a limit of 2 billion columns. Cheers, Edward

Re: neither 'nodetool repair' nor 'hinted hanoff/read repair' work for secondary indexes

2013-02-05 Thread Alexei Bakanov
Made a d-test for easier reproduction and created https://issues.apache.org/jira/browse/CASSANDRA-5223 On 1 February 2013 15:14, Alexei Bakanov wrote: > Hi again, > > Once started playing with CCM it's hard to stop, such a great tool. > My issue with secondary indexes is f

neither 'nodetool repair' nor 'hinted hanoff/read repair' work for secondary indexes

2013-02-01 Thread Alexei Bakanov
Hi again, Once started playing with CCM it's hard to stop, such a great tool. My issue with secondary indexes is following: neither explicit 'nodetool repair' nor implicit 'hinted handoffs/read repairs' resolve inconsistencies in data I get from secondary indexes. I obser

Re: Why Secondary indexes is so slowly by my test?

2012-12-13 Thread Chengying Fang
redundancy and maintenance. The most , what items I can query are decided at design time, that is, No Design No Query, even all data is there. Thanks to all. -- Original -- From: "Edward Capriolo"; Date: Fri, Dec 14, 2012 01:31 AM To: "user"; Sub

Re: Why Secondary indexes is so slowly by my test?

2012-12-13 Thread Tyler Hobbs
If anyone's interested in a little more background on the read-before-write fix that Ed mentioned, see: https://issues.apache.org/jira/browse/CASSANDRA-2897 On Thu, Dec 13, 2012 at 11:31 AM, Edward Capriolo wrote: > Here is a good start. > > http://www.anuff.com/2011/02/indexing-in-cassandra.htm

Re: Why Secondary indexes is so slowly by my test?

2012-12-13 Thread Edward Capriolo
Here is a good start. http://www.anuff.com/2011/02/indexing-in-cassandra.html On Thu, Dec 13, 2012 at 11:35 AM, Alain RODRIGUEZ wrote: > Hi Edward, can you share the link to this blog ? > > Alain > > 2012/12/13 Edward Capriolo > >> Ed ENuff s > > >

Re: Why Secondary indexes is so slowly by my test?

2012-12-13 Thread Alain RODRIGUEZ
Hi Edward, can you share the link to this blog ? Alain 2012/12/13 Edward Capriolo > Ed ENuff s

Re: Why Secondary indexes is so slowly by my test?

2012-12-13 Thread Edward Capriolo
Until the secondary indexes do not read before write is in a release and stabilized you should follow Ed ENuff s blog and do your indexing yourself with composites. On Thursday, December 13, 2012, aaron morton wrote: > The IndexClause for the get_indexed_slices takes a start key. You can p

Re: Why Secondary indexes is so slowly by my test?

2012-12-12 Thread aaron morton
1. > Also we can not always set row range fulfill the query conditions when doing > query. Maybe I should redesign the CF model to fix it. > > -- Original -- > From: "Hiller, Dean"; > Date: Wed, Dec 12, 2012 10:51 PM > To: "

Re: Why Secondary indexes is so slowly by my test?

2012-12-12 Thread Chengying Fang
redesign the CF model to fix it. -- Original -- From: "Hiller, Dean"; Date: Wed, Dec 12, 2012 10:51 PM To: "user@cassandra.apache.org"; Subject: Re: Why Secondary indexes is so slowly by my test? You could always try PlayOrm's quer

Re: Why Secondary indexes is so slowly by my test?

2012-12-12 Thread Hiller, Dean
day, December 11, 2012 8:22 PM To: user mailto:user@cassandra.apache.org>> Subject: Re: Why Secondary indexes is so slowly by my test? Thanks to Low. We use CompositeColumn to substitue it in single not-equality and definite equalitys query. And we will give up cassandra because of the we

Re: Why Secondary indexes is so slowly by my test?

2012-12-11 Thread Chengying Fang
1, 2012 07:44 PM To: "user"; Subject: Re: Why Secondary indexes is so slowly by my test? Hi, Secondary index lookups are more complicated than normal queries so will be slower. Items have to first be queried in the index, then retrieved from their actual location. Also, insert

Re: Why Secondary indexes is so slowly by my test?

2012-12-11 Thread Richard Low
you need to retrieve large amounts of data with your query, you would be better off changing your data model to not use secondary indexes. Richard. On 7 December 2012 03:08, Chengying Fang wrote: > Hi guys, > > I found Secondary indexes too slowly in my product(amazon large instanc

Why Secondary indexes is so slowly by my test?

2012-12-06 Thread Chengying Fang
Hi guys, I found Secondary indexes too slowly in my product(amazon large instance) with cassandra, then I did test again as describe here. But the result is the same as product. What's wrong with cassandra or me? Now my test: newly installed ubuntu-12.04 LTS , apache-cassandra-1.1.6, de

Re: How to query secondary indexes

2012-11-28 Thread Blake Eggleston
You're going to have a problem doing this in a single query because you're asking cassandra to select a non-contiguous set of rows. Also, to my knowledge, you can only use non equal operators on clustering keys. The best solution I could come up with would be to define you table like so: CREATE TA

How to query secondary indexes

2012-11-28 Thread Oren Karmi
Hi, According to the documentation on Indexes ( http://www.datastax.com/docs/1.1/ddl/indexes ), in order to use WHERE on a column which is not part of my key, I must define a secondary index on it. However, I can only use equality comparison on it but I wish to use other comparisons methods like g

Re: Concurrency and secondary indexes

2012-11-25 Thread aaron morton
...@thelastpickle.com] > Sent: November-22-12 8:04 PM > To: user@cassandra.apache.org > Subject: Re: Concurrency and secondary indexes > > What version are you on ? > > > but we are finding a secondary index is performing slow > Not sure what you mean here.

RE: Concurrency and secondary indexes

2012-11-23 Thread Simon Guindon
@cassandra.apache.org Subject: Re: Concurrency and secondary indexes What version are you on ? but we are finding a secondary index is performing slow Not sure what you mean here. Are secondary indexes concurrent or single threaded? Rebuilding a secondary index (via node tool) is a single threaded operation

Re: Concurrency and secondary indexes

2012-11-22 Thread aaron morton
What version are you on ? > but we are finding a secondary index is performing slow Not sure what you mean here. > Are secondary indexes concurrent or single threaded? Rebuilding a secondary index (via node tool) is a single threaded operation, but *all* indexes specified on the c

Concurrency and secondary indexes

2012-11-22 Thread Simon Guindon
We are importing data from one column family into a second column family via "nodetool refresh" but we are finding a secondary index is performing slow and the machine CPU is pretty much idle. We are trying to bulk load data as fast as possible. Are secondary indexes concurrent

Re: Query over secondary indexes

2012-10-09 Thread Hiller, Dean
>> >Date: Tuesday, October 9, 2012 7:39 AM >To: Nrel mailto:dean.hil...@nrel.gov>>, >"user@cassandra.apache.org<mailto:user@cassandra.apache.org>" >mailto:user@cassandra.apache.org>> >Subject: Re: Query over secondary indexes > >Thanks . This is w

Re: Query over secondary indexes

2012-10-09 Thread Hiller, Dean
a.v...@gmail.com>> Date: Tuesday, October 9, 2012 7:39 AM To: Nrel mailto:dean.hil...@nrel.gov>>, "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: Re: Query over secondary indexes Thanks . This is what i have

Re: Query over secondary indexes

2012-10-09 Thread Vivek Mishra
t; To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" < > user@cassandra.apache.org<mailto:user@cassandra.apache.org>> > Subject: Re: Query over secondary indexes > > I did wait for atleast 5 minutes before terminating it. Also sometimes it > re

Re: Query over secondary indexes

2012-10-08 Thread Vivek Mishra
@impetus.co.in> wrote: >> >>> Try making *user_name* a primary key in combination with some other >>> unique column and see if results are improving. >>> >>> -Rishabh >>> >>> *From:* Vivek Mishra [mailto:mishra.v...@gmail.com] >>>

Re: Query over secondary indexes

2012-10-08 Thread Vivek Mishra
ng *user_name* a primary key in combination with some other >> unique column and see if results are improving. >> >> -Rishabh >> >> *From:* Vivek Mishra [mailto:mishra.v...@gmail.com] >> *Sent:* Friday, October 05, 2012 2:35 PM >> *To:* user@cassandra.apache.org

Re: Query over secondary indexes

2012-10-08 Thread aaron morton
2:45 PM, Rishabh Agrawal > wrote: > Try making user_name a primary key in combination with some other unique > column and see if results are improving. > > -Rishabh > > From: Vivek Mishra [mailto:mishra.v...@gmail.com] > Sent: Friday, October 05, 2012 2:35 PM >

Re: Query over secondary indexes

2012-10-05 Thread Vivek Mishra
> > -Rishabh > > *From:* Vivek Mishra [mailto:mishra.v...@gmail.com] > *Sent:* Friday, October 05, 2012 2:35 PM > *To:* user@cassandra.apache.org > *Subject:* Query over secondary indexes > > > > I have a column family "User" which is having a indexed colum

RE: Query over secondary indexes

2012-10-05 Thread Rishabh Agrawal
Try making user_name a primary key in combination with some other unique column and see if results are improving. -Rishabh From: Vivek Mishra [mailto:mishra.v...@gmail.com] Sent: Friday, October 05, 2012 2:35 PM To: user@cassandra.apache.org Subject: Query over secondary indexes I have a column

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-30 Thread Hiller, Dean
pache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Thursday, August 30, 2012 2:14 PM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: Re: Why Cassandra secondary indexes are so slow

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-30 Thread Edward Kibardin
Thanks Guys for the answers... The main issue here seems not the secondary index, but speed of searching for random keys in column family. I've done the experiment and queried the same 5000 rows not using index but providing a list of keys to Pycassa... the speed was the same. Although, using Sup

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-30 Thread Tyler Hobbs
pycassa already breaks up the query into smaller chunks, but you should try playing with the buffer_size kwarg for get_indexed_slices, perhaps lowering it to ~300, as Aaron suggests: http://pycassa.github.com/pycassa/api/pycassa/columnfamily.html#pycassa.columnfamily.ColumnFamily.get_indexed_slices

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-29 Thread aaron morton
> *from 12 to 20 seconds (!!!) to find 5000 rows*. More is not always better. Cassandra must materialise the full 5000 rows and send them all over the wire to be materialised on the other side. Try asking for a few hundred at a time and see how it goes. Cheers - Aaron Morto

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-28 Thread Robin Verlangen
@Edward: I think you should consider a queue for exporting the new rows. Just store the rowkey in a queue (you might want to consider looking at http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Distributed-work-queues-td5226248.html ) and process that row every couple of minutes. Th

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-28 Thread Robin Verlangen
"What this means is that eventually you will have 1 row in the secondary index table with 350K columns" Is this really true? I would have expected that Cassandra used internal index sharding/bucketing? With kind regards, Robin Verlangen *Software engineer* * * W http://www.robinverlangen.nl E ro

Re: Why Cassandra secondary indexes are so slow on just 350k rows?

2012-08-28 Thread Dave Brosius
If i understand you correctly, you are only ever querying for the rows where is_exported = false, and turning them into trues. What this means is that eventually you will have 1 row in the secondary index table with 350K columns that you will never look at. It seems to me you that perhaps you

Re: How to manually build and maintain secondary indexes

2012-07-26 Thread Rajat Mathur
http://www.anuff.com/2011/02/indexing-in-cassandra.html On Thu, Jul 26, 2012 at 11:43 PM, Brian O'Neill wrote: > Alon, > > We came to the same conclusion regarding secondary indexes, and instead of > using them we implemented our own wide-row indexing capability and > open

Re: How to manually build and maintain secondary indexes

2012-07-26 Thread Brian O'Neill
Alon, We came to the same conclusion regarding secondary indexes, and instead of using them we implemented our own wide-row indexing capability and open-sourced it. Its available here: https://github.com/hmsonline/cassandra-indexing We still have challenges rebuilding indexes, etc. It

How to manually build and maintain secondary indexes

2012-07-26 Thread Alon Pilberg
Hello, My company is working on transition of our relational data model to Cassandra. Naturally, one of the basic demands is to have secondary indexes to answer queries quickly according to the application's needs. After looking at Cassandra's native support for secondary indexes, we d

Re: Secondary Indexes, Quorum and Cluster Availability

2012-06-07 Thread Jim Ancona
Secondary Index FAQ at > http://wiki.apache.org/cassandra/SecondaryIndexes > > Q: How does choice of Consistency Level affect cluster availability when > using secondary indexes? > A: Because secondary indexes are distributed, you must have CL level nodes > available for *all* token

Re: Secondary Indexes, Quorum and Cluster Availability

2012-06-07 Thread aaron morton
d idea for the documentation to reflect the tradeoffs more > clearly. > > Here's a proposed addition to the Secondary Index FAQ at > http://wiki.apache.org/cassandra/SecondaryIndexes > > Q: How does choice of Consistency Level affect cluster availability when > using

Re: Secondary Indexes, Quorum and Cluster Availability

2012-06-06 Thread Jim Ancona
evel affect cluster availability when using secondary indexes? A: Because secondary indexes are distributed, you must have CL level nodes available for *all* token ranges in the cluster in order to complete a query. For example, with RF = 3, when two out of three consecutive nodes in the ring are

Re: Secondary Indexes, Quorum and Cluster Availability

2012-06-05 Thread Jim Ancona
we observed. When I reasoned my way through what I knew about how secondary indexes work, I came to the same conclusion about all token ranges having to be available. My surprise at the behavior was because I *hadn't* reasoned my way through it until we had the issue. Somehow I doubt I&#

Re: Secondary Indexes, Quorum and Cluster Availability

2012-06-04 Thread aaron morton
IIRC index slices work a little differently with consistency, they need to have CL level nodes available for all token ranges. If you drop it to CL ONE the read is local only for a particular token range. The problem when doing index reads is the nodes that contain the results can no longer be

Secondary Indexes, Quorum and Cluster Availability

2012-06-01 Thread Jim Ancona
Hi, We have an application with two code paths, one of which uses a secondary index query and the other, which doesn't. While testing node down scenarios in our cluster we got a result which surprised (and concerned) me, and I wanted to find out if the behavior we observed is expected. Background

Re: How can we use composite indexes and secondary indexes together

2012-06-01 Thread Vivek Mishra
to make this > kind of queries fast? > > First I thought, it would be natural to make composite index on > (Firstname,Lastname,Age) and add separate secondary index on remaining > fields (Country and ChildCount). But I can't insert rows into table after > creating secondary indexe

Re: How can we use composite indexes and secondary indexes together

2012-05-31 Thread aaron morton
ast? > > First I thought, it would be natural to make composite index on > (Firstname,Lastname,Age) and add separate secondary index on remaining fields > (Country and ChildCount). But I can't insert rows into table after creating > secondary indexes. And also, I can't

How can we use composite indexes and secondary indexes together

2012-05-31 Thread Nury Redjepow
can't insert rows into table after creating secondary indexes. And also, I can't query the table. I'm using cassandra 1.1.0, and cqlsh with --cql3 option. Any other suggestions to solve our problem (complex queries with mandatory and additional options) are welcome.The main point

Re: CQL 3.0 composite keys and secondary indexes

2012-05-07 Thread Sylvain Lebresne
On Tue, May 8, 2012 at 12:08 AM, Roland Mechler wrote: > It seems as though secondary indexes are not supported in tables (column > families) that have composite keys. Is that true? It is. > If so, are there plans to suport that combination in the future? There is: https://issues.a

CQL 3.0 composite keys and secondary indexes

2012-05-07 Thread Roland Mechler
It seems as though secondary indexes are not supported in tables (column families) that have composite keys. Is that true? If so, are there plans to suport that combination in the future? -Roland

Re: Compression on secondary indexes

2012-03-31 Thread aaron morton
I've not checked the code but (reading https://issues.apache.org/jira/browse/CASSANDRA-3877) I would guess it is not possible to set compression on secondary indexes pre 1.1. Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 30/03

Compression on secondary indexes

2012-03-30 Thread Paolo Bernardi
Greetings, how may I set the compression algorithm on secondary indexes? I saw that from Cassandra 1.1.0 it will be inherited from the CF, but in the previous releases I must set it explicity; I just don't know how to do that. Paolo -- @bernarpa http://paolobernardi.wordpress.com

Re: Secondary indexes don't go away after metadata change

2012-03-07 Thread aaron morton
rsion of the > schema on a new node? Is there a reason to apply the migrations? > > - Mike > > From: aaron morton [mailto:aa...@thelastpickle.com] > Sent: Tuesday, March 06, 2012 4:14 AM > To: user@cassandra.apache.org > Subject: Re: Secondary indexes don't go away afte

RE: Secondary indexes don't go away after metadata change

2012-03-06 Thread Frisch, Michael
on of the schema on a new node? Is there a reason to apply the migrations? - Mike From: aaron morton [mailto:aa...@thelastpickle.com] Sent: Tuesday, March 06, 2012 4:14 AM To: user@cassandra.apache.org Subject: Re: Secondary indexes don't go away after metadata change When the new node co

Re: Secondary indexes don't go away after metadata change

2012-03-06 Thread aaron morton
elastpickle.com] > Sent: Monday, March 05, 2012 3:58 AM > To: user@cassandra.apache.org > Subject: Re: Secondary indexes don't go away after metadata change > > The secondary index CF's are marked as no longer required / marked as > compacted. under 1.x they would then be delet

  1   2   3   >