Re: Error when using CQL driver : No indexed columns present in by-columns clause with "equals" operator

[Sylvain Lebresne]

birth_year is secondary indexed. When querying using 2nd indexes, Cassandra
(it's not CQL specific) requires that you use an '=' for at least one of
the indexed column in the where clause (in your example you only have one
such column so it should be an '='). This is a limitation of 2ndary indexes
(that will hopefully be lifted someday -
https://issues.apache.org/jira/browse/CASSANDRA-4476).

In other words, what you can do is:
 select * from users where birth_year = 1965
but your query is not supported (for 2nd indexes that is).

--
Sylvain


On Thu, Jan 31, 2013 at 7:22 AM, Dinusha Dilrukshi
wrote:

> Hi All,
>
> I have created a column family as follows. (With secondary indexes.)
>
> create column family users with comparator=UTF8Type and
> key_validation_class = 'UTF8Type' and default_validation_class = 'UTF8Type'
>  and column_metadata=[{column_name: full_name, validation_class: UTF8Type},
>  {column_name: birth_year, validation_class: LongType, index_type: KEYS},
>  {column_name: state, validation_class:  UTF8Type, index_type: KEYS}];
>
> And I am using CQL driver-1.1.1 with Cassandra server-1.1.1. Once I try to
> execute the following query, it gives  an exception saying 'No indexed
> columns present in by-columns clause with "equals" operator'.
>
> CQL :
> select * from users where birth_year<1965
>
> Caused by: java.sql.SQLSyntaxErrorException: No indexed columns present in
> by-columns clause with "equals" operator
> at
> org.apache.cassandra.cql.jdbc.CassandraPreparedStatement.doExecute(CassandraPreparedStatement.java:155)
> at
> org.apache.cassandra.cql.jdbc.CassandraPreparedStatement.executeQuery(CassandraPreparedStatement.java:199)
>
> Appreciate any help to resolve this..
>
> Regards,
> Dinusha.
>

Re: too many warnings of Heap is full

[Alain RODRIGUEZ]

@Bryan

"Other space is used but can be more easily controlled by tuning for
- memtable
- compaction
- key cache
- *row cache*"

Isn't row cache stored off-heap since a while ?

2013/1/31 Bryan Talbot 

> On Wed, Jan 30, 2013 at 2:44 PM, Guillermo Barbero <
> guillermo.barb...@spotbros.com> wrote:
>
>>  WARN [MemoryMeter:1] 2013-01-30 21:37:48,079 Memtable.java (line 202)
>> setting live ratio to maximum of 64.0 instead of 751.6512549537648
>>
>>
> This looks interesting.  Doesn't this mean that the ratio of space used
> for that CF for memory to serialized form is 751:1 but was forced to a
> lower "sane" value?
>
> -Bryan
>
>

Re: Understanding Virtual Nodes on Cassandra 1.2

[Manu Zhang]

On Thu 31 Jan 2013 03:43:32 AM CST, Zhong Li wrote:

Are there tickets/documents explain how data be replicated on Virtual
Nodes? If there are multiple tokens on one physical host, may a chance
two or more tokens chosen by replication strategy located on same
host? If move/remove/add a token
manually, does Cassandra Engine validate the case?

Thanks.


On Jan 30, 2013, at 12:46 PM, Zhong Li wrote:


You add a physical node and that in turn adds num_token tokens to
the ring.


No, I am talking about Virtual Nodes with order preserving
partitioner. For an existing host with multiple tokens setting list
on cassandra.inital_token. After initial bootstrapping, the host will
not aware changes of cassandra.inital_token. If I want add a new
token( virtual node), I have to rebuild the host with new token list.

My question is if there is way to add a virtual nodes without rebuild it?

Thanks,

On Jan 30, 2013, at 10:21 AM, Manu Zhang wrote:


On Wed 30 Jan 2013 02:29:27 AM CST, Zhong Li wrote:

One more question, can I add a virtual node manually without reboot
and rebuild a host data?

I checked nodetool command, there is no option to add a node.

Thanks.

Zhong


On Jan 29, 2013, at 11:09 AM, Zhong Li wrote:


I was misunderstood this
http://www.datastax.com/dev/blog/virtual-nodes-in-cassandra-1-2 ,
especially
"If you want to get started with vnodes on a fresh cluster, however,
that is fairly straightforward. Just don’t set the
|initial_token| parameter in your|conf/cassandra.yaml| and instead
enable the |num_tokens| parameter. A good default value for this
is 256"

Also I couldn't find document about set multiple tokens
for cassandra.inital_token

Anyway, I just tested, it does work to set  comma separated list of
tokens.

Thanks,

Zhong


On Jan 29, 2013, at 3:06 AM, aaron morton wrote:


After I searched some document on Datastax website and some old
ticket, seems that it works for random partitioner only, and leaves
order preserved partitioner out of the luck.

Links ?


 or allow add Virtual Nodes manually?

If not looked into it but there is a cassandra.inital_token startup
param that takes a comma separated list of tokens for the node.

There also appears to be support for the ordered partitions to
generate random tokens.

But you would still have the problem of having to balance your row
keys around the token space.

Cheers
-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com 


On 29/01/2013, at 10:31 AM, Zhong Li mailto:z...@voxeo.com>
> wrote:


Hi All,

Virtual Nodes is great feature. After I searched some document on
Datastax website and some old ticket, seems that it works for
random partitioner only, and leaves order preserved partitioner out
of the luck. I may misunderstand, please correct me. if it doesn't
love order preserved partitioner, would be possible to add support
multiple initial_token(s) for  order preserved partitioner  or
allow add Virtual Nodes manually?

Thanks,

Zhong








You add a physical node and that in turn adds num_token tokens to
the ring.






no, those tokens will be skipped

Re: too many warnings of Heap is full

[Guillermo Barbero]

Hi, replies to your answers:

The CFs that seems to flush more often (users and messages) do not
have multiget queries...

There are 4 secondary indexes for users and 3 for messages. I've seen
something odd here: the index of the CFs is also stored in a column
AND is also indexed. There isn't any query by that secondary index. I
should probably erase the index, shouldn't I?

I post some lines of the log before 21:35:

Node 150:
 INFO [HintedHandoff:1] 2013-01-30 21:07:43,795
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:07:43,796
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:17:43,796
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:17:43,796
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:27:43,796
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:27:43,797
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153

Node 151:
 INFO [HintedHandoff:1] 2013-01-30 21:04:45,821
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:04:45,822
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 INFO [ScheduledTasks:1] 2013-01-30 21:14:30,112 GCInspector.java
(line 122) GC for ParNew: 213 ms for 1 collections, 1920192480 used;
max is 3886022656
 INFO [HintedHandoff:1] 2013-01-30 21:14:45,821
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:14:45,822
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:24:45,822
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:24:45,822
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 INFO [ScheduledTasks:1] 2013-01-30 21:28:25,294 GCInspector.java
(line 122) GC for ParNew: 236 ms for 1 collections, 2264010544 used;
max is 3886022656
 INFO [HintedHandoff:1] 2013-01-30 21:34:45,822
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:34:45,822
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153

Node 152:
 INFO [HintedHandoff:1] 2013-01-30 21:02:03,706
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:02:03,707
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 WARN [MemoryMeter:1] 2013-01-30 21:09:57,220 Memtable.java (line 197)
setting live ratio to minimum of 1.0 instead of 0.15873949606408785
 INFO [MemoryMeter:1] 2013-01-30 21:09:57,221 Memtable.java (line 213)
CFS(Keyspace='SB_NNCD', ColumnFamily='CF_profilePics') liveRatio is
1.0 (just-counted was 1.0).  calculation took 3ms for 301 columns
 INFO [MemoryMeter:1] 2013-01-30 21:09:57,223 Memtable.java (line 213)
CFS(Keyspace='SB_NNCD',
ColumnFamily='CF_profilePics.CF_profilePics_pPicMD5_idx') liveRatio is
18.952013237727524 (just-counted was 18.952013237727524).  calculation
took 2ms for 74 columns
 INFO [MemoryMeter:1] 2013-01-30 21:09:57,242 Memtable.java (line 213)
CFS(Keyspace='SB_NNCD',
ColumnFamily='CF_profilePics.CF_profilePics_userSBCode_idx') liveRatio
is 12.820966340897906 (just-counted was 12.209597352454495).
calculation took 19ms for 74 columns
 INFO [HintedHandoff:1] 2013-01-30 21:12:03,707
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:12:03,707
HintedHandOffManager.java (line 392) Finished hinted handoff of 0 rows
to endpoint /10.0.0.153
 INFO [MemoryMeter:1] 2013-01-30 21:12:47,775 Memtable.java (line 213)
CFS(Keyspace='SB_NNCD', ColumnFamily='CF_NewsFeedTimeline') liveRatio
is 3.605170679607766 (just-counted was 3.4883667978539283).
calculation took 110ms for 1462 columns
 INFO [HintedHandoff:1] 2013-01-30 21:22:03,707
HintedHandOffManager.java (line 296) Started hinted handoff for token:
85070591730234615865843651857942052864 with IP: /10.0.0.153
 INFO [HintedHandoff:1] 2013-01-30 21:22:03,708
Hint

Re: JDBC, Select * Cql2 vs Cql3 problem ?

[Andy Cobley]

Hi Edward,

I've used Hector for years now myself and know how good it is.  I think I've 
tracked this to a problem in the JDBC driver, I'll post a message and raise a 
ticket over there.  (there is still a possibility that Cassandra 1.2 is sending 
an incorrect CQLresult set)

Andy



On 30 Jan 2013, at 14:26, Edward Capriolo  wrote:

> Darn auto correct cql2 , is only good for compact tables. Make sure you are 
> setting you cql version. Or frankly just switch to Hector / thrift and use 
> things that are know to work for years now.
> 
> On Wednesday, January 30, 2013, Edward Capriolo  wrote:
> > You really can't mix cql2 and cql3. Cql2 does not understand cql3s sparse 
> > tables. Technically it ,barfs all over the place. Cql2 is only good for 
> > contact tables.
> >
> > On Wednesday, January 30, 2013, Andy Cobley 
> >  wrote:
> >> Well this is getting stranger,   for me  with this simple table 
> >> definition, 
> >> select key,gender from users
> >> is also failing with a null pointer exception
> >> Andy
> >> On 29 Jan 2013, at 13:50, Andy Cobley  
> >> wrote:


The University of Dundee is a Scottish Registered Charity, No. SC015096.

JDBC : CreateresultSet fails with null column in CqlResultSet

[Andy Cobley]

As you may be aware I've been trying to track down a problem using JDBC 1.1.2 
with Cassandra 1.2.0  I was getting a null pointer exception in the result set. 
 I've done some digging into the JDBC driver  and found the following.

In CassandraResultSet.java the new result set is Instantiated in 

CassandraResultSet(Statement statement, CqlResult resultSet, String keyspace)

I decided to trace the result set with the following code:

rowsIterator = resultSet.getRowsIterator();
System.out.println("---");
while(rowsIterator.hasNext()){
   CqlRow row = rowsIterator.next();
   curRowKey = row.getKey();
   System.out.println("Row Key "+curRowKey);
   List cols = row.getColumns();
   Iterator iterator;
   iterator = cols.iterator(); 
   while (iterator.hasNext()){
   Column col=(Column)iterator.next();
   String Name= new String(col.getName());
   String Value = new String(col.getValue());
   System.out.println("Col "+Name+ " : "+Value);
}
}

This produced the following output:

---
Row Key [B@617e53c9
Col key : jsmith
Col  : 
Col password : ch@ngem3a
Row Key [B@2caee320
Col key : jbrown
Col  : 
Col gender : male
---

As you can see there is a black column at position 2 in each of the rows.  As 
this resultset has come from the Cassandra thrift client ( I believe) the 
problem amy lay there.  There is no blank column defined by my SQL create 
statements I believe. 

If I'm correct here, should I raise a ticket with JDBC or Cassandra ? (for now 
I've patched my local JDBC driver so it doesn't create a TypedColumn if the 
result set produces a null column)

Andy


The University of Dundee is a Scottish Registered Charity, No. SC015096.

Re: Error when using CQL driver : No indexed columns present in by-columns clause with "equals" operator

[Dinusha Dilrukshi]

Thanks Sylvain..

Regards,
Dinusha.



On Thu, Jan 31, 2013 at 2:47 PM, Sylvain Lebresne wrote:

> birth_year is secondary indexed. When querying using 2nd indexes,
> Cassandra (it's not CQL specific) requires that you use an '=' for at least
> one of the indexed column in the where clause (in your example you only
> have one such column so it should be an '='). This is a limitation of
> 2ndary indexes (that will hopefully be lifted someday -
> https://issues.apache.org/jira/browse/CASSANDRA-4476).
>
> In other words, what you can do is:
>  select * from users where birth_year = 1965
> but your query is not supported (for 2nd indexes that is).
>
> --
> Sylvain
>
>
> On Thu, Jan 31, 2013 at 7:22 AM, Dinusha Dilrukshi  > wrote:
>
>> Hi All,
>>
>> I have created a column family as follows. (With secondary indexes.)
>>
>> create column family users with comparator=UTF8Type and
>> key_validation_class = 'UTF8Type' and default_validation_class = 'UTF8Type'
>>  and column_metadata=[{column_name: full_name, validation_class:
>> UTF8Type},
>>  {column_name: birth_year, validation_class: LongType, index_type: KEYS},
>>  {column_name: state, validation_class:  UTF8Type, index_type: KEYS}];
>>
>> And I am using CQL driver-1.1.1 with Cassandra server-1.1.1. Once I try
>> to execute the following query, it gives  an exception saying 'No indexed
>> columns present in by-columns clause with "equals" operator'.
>>
>> CQL :
>> select * from users where birth_year<1965
>>
>> Caused by: java.sql.SQLSyntaxErrorException: No indexed columns present
>> in by-columns clause with "equals" operator
>> at
>> org.apache.cassandra.cql.jdbc.CassandraPreparedStatement.doExecute(CassandraPreparedStatement.java:155)
>> at
>> org.apache.cassandra.cql.jdbc.CassandraPreparedStatement.executeQuery(CassandraPreparedStatement.java:199)
>>
>> Appreciate any help to resolve this..
>>
>> Regards,
>> Dinusha.
>>
>
>

Re: CQL binary protocol

[Gabriel Ciuloaica]

Hi,

You may take a look to java-driver 
 project. It has an 
implementation for connection pool.


Cheers,
Gabi

On 1/31/13 6:48 PM, Vivek Mishra wrote:

Hi,
Any connection pool API available for cassandra transport 
Client(org.apache.cassandra.transport.Client)?


-Vivek

General question regarding bootstrap and nodetool repair

[Wei Zhu]

Hi,
After messing around with my Cassandra cluster recently, I think I need some 
basic understanding on how things work behind scene regarding data streaming.
Let's say we have three node cluster with RF = 3.  If node 3 for some reason 
dies and I want to replace it with a new node with the same (maybe minus one) 
range. During the bootstrap, how the data is streamed?
From what I observed, Node 3 has replicates for its primary range on node 4, 5. 
So it streams the data from them and starts to compact them. Also, node 3 holds 
replicates for primary range of node 2, so it streams data from node 2 and node 
4. Similarly, it holds replicates for node 1. So data streamed from node 1 and 
node 2. So during the bootstaping, it basically gets the data from all the 
replicates (2 copies each), so it will require double the disk space in order 
to hold the data? Over the time, those SStables will be compacted and redundant 
will be removed? Is it true?

if we issue nodetool repair -pr on node 3, apart from streaming data from node 
4, 5 to 3. We also see data stream between node 4, 5 since they hold the 
replicates. But I don't see log regarding "merkle tree calculation" on node 
4,5. Just wondering how they know what data to stream in order to repair node 
4, 5?

Thanks.
-Wei

initial_token

[Stephen.M.Thompson]

Hi folks, I'm trying to get a multimode setup working, which seems like it 
should be really simple from the documentation.

ERROR 11:41:20,773 Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: For input string: 
"85070591730234615865843651857942052864"
at 
org.apache.cassandra.dht.Murmur3Partitioner$1.validate(Murmur3Partitioner.java:180)
at 
org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:433)
at 
org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:121)
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:178)
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:397)
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:440)
For input string: "85070591730234615865843651857942052864"
Fatal configuration error; unable to start server.  See log for stacktrace.

>From my cassandra.yaml ...

initial_token: 85070591730234615865843651857942052864

>From the wiki this certainly looks correct:
http://www.datastax.com/docs/1.1/initialize/cluster_init

I've tried this with a couple of values but seem to always get the same result 
... am I missing something?

Thanks,
Steve

Re: initial_token

[Edward Capriolo]

Now by default a new partitioner is chosen Murmer3. The range of
tokens used to be something like 0 - 2^127. Now the range of its
tokens is -2^64 <->  2^64 . You can switch back to random partitioner
and follow the old instructions or try to find a new doc with the new
instructions.

On Thu, Jan 31, 2013 at 2:47 PM,   wrote:
> Hi folks, I’m trying to get a multimode setup working, which seems like it
> should be really simple from the documentation.
>
>
>
> ERROR 11:41:20,773 Fatal configuration error
>
> org.apache.cassandra.exceptions.ConfigurationException: For input string:
> "85070591730234615865843651857942052864"
>
> at
> org.apache.cassandra.dht.Murmur3Partitioner$1.validate(Murmur3Partitioner.java:180)
>
> at
> org.apache.cassandra.config.DatabaseDescriptor.loadYaml(DatabaseDescriptor.java:433)
>
> at
> org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:121)
>
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:178)
>
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:397)
>
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:440)
>
> For input string: "85070591730234615865843651857942052864"
>
> Fatal configuration error; unable to start server.  See log for stacktrace.
>
>
>
> From my cassandra.yaml …
>
>
>
> initial_token: 85070591730234615865843651857942052864
>
>
>
> From the wiki this certainly looks correct:
>
> http://www.datastax.com/docs/1.1/initialize/cluster_init
>
>
>
> I’ve tried this with a couple of values but seem to always get the same
> result … am I missing something?
>
>
>
> Thanks,
>
> Steve

Re: General question regarding bootstrap and nodetool repair

[Wei Zhu]

I decided to dig in to the source code, looks like in the case of nodetool 
repair, if the current node sees the difference between the remote nodes based 
on the merkle tree calculation, it will start a streamrepair session to ask the 
remote nodes to stream data between  each other. 

But I am still not sure how about the my first question regarding the 
bootstrap, anyone?

Thanks.
-Wei


 From: Wei Zhu 
To: Cassandr usergroup  
Sent: Thursday, January 31, 2013 10:50 AM
Subject: General question regarding bootstrap and nodetool repair
 

Hi,
After messing around with my Cassandra cluster recently, I think I need some 
basic understanding on how things work behind scene regarding data streaming.
Let's say we have three node cluster with RF = 3.  If node 3 for some reason 
dies and I want to replace it with a new node with the same (maybe minus one) 
range. During the bootstrap, how the data is streamed?
From what I observed, Node 3 has replicates for its primary range on node 4, 5. 
So it streams the data from them and starts to compact them. Also, node 3 holds 
replicates for primary range of node 2, so it streams data from node 2 and node 
4. Similarly, it holds replicates for node 1. So data streamed from node 1 and 
node 2. So during the bootstaping, it basically gets the data from all the 
replicates (2 copies each), so it will require double the disk space in order 
to hold the data? Over the time, those SStables will be compacted and redundant 
will be removed? Is it true?

if we issue nodetool repair -pr on node 3, apart from streaming data from node 
4, 5 to 3. We also see data stream between node 4, 5 since they hold the 
replicates. But I don't see log regarding "merkle tree calculation" on node 
4,5. Just wondering how they know what data to stream in order to repair node 
4, 5?

Thanks.
-Wei

Re: General question regarding bootstrap and nodetool repair

[Rob Coli]

On Thu, Jan 31, 2013 at 12:19 PM, Wei Zhu  wrote:
> But I am still not sure how about the my first question regarding the
> bootstrap, anyone?

As I understand it, bootstrap occurs from a single replica. Which
replica is chosen is based on some internal estimation of which is
closest/least loaded/etc. But only from a single replica, so in RF=3,
in order to be consistent with both you still have to run a repair.

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: initial_token

[Rob Coli]

On Thu, Jan 31, 2013 at 12:17 PM, Edward Capriolo  wrote:
> Now by default a new partitioner is chosen Murmer3.

"Now" = as of 1.2, to be unambiguous.

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: initial_token

[Edward Capriolo]

This is the bad side of changing default. There are going to be a few
groups unfortunates.

The first group, who only can not setup their cluster, and eventually
figure out their tokens. (this thread)
The second group, who assume their tokens were correct and run around
with an unbalanced cluster thinking the performance sucks. (the
threads for the next few months)
The third group, who will google "how to balance my ring" and find a
page with random partitioner instructions. (the occasional thread for
the next N years)
The fourth group, because as of now map reduce is highly confused by this.

On Thu, Jan 31, 2013 at 4:52 PM, Rob Coli  wrote:
> On Thu, Jan 31, 2013 at 12:17 PM, Edward Capriolo  
> wrote:
>> Now by default a new partitioner is chosen Murmer3.
>
> "Now" = as of 1.2, to be unambiguous.
>
> =Rob
>
> --
> =Robert Coli
> AIM>ALK - rc...@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb

Re: General question regarding bootstrap and nodetool repair

[Wei Zhu]

Thanks Rob. I think you are right on it.

Here is what I found:

https://github.com/apache/cassandra/blob/cassandra-1.1.0/src/java/org/apache/cassandra/dht/RangeStreamer.java#L140


It sorts the end point by proximity and in 

https://github.com/apache/cassandra/blob/cassandra-1.1.0/src/java/org/apache/cassandra/dht/RangeStreamer.java#L171


It fetches the data from the only one source.

That answers my question. So we will have to run repair after the bootstrap to 
make sure the consistency. 

Thanks.
-Wei




 From: Rob Coli 
To: user@cassandra.apache.org 
Sent: Thursday, January 31, 2013 1:50 PM
Subject: Re: General question regarding bootstrap and nodetool repair
 
On Thu, Jan 31, 2013 at 12:19 PM, Wei Zhu  wrote:
> But I am still not sure how about the my first question regarding the
> bootstrap, anyone?

As I understand it, bootstrap occurs from a single replica. Which
replica is chosen is based on some internal estimation of which is
closest/least loaded/etc. But only from a single replica, so in RF=3,
in order to be consistent with both you still have to run a repair.

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: too many warnings of Heap is full

[Nate McCall]

>
> There are 4 secondary indexes for users and 3 for messages. I've seen
> something odd here: the index of the CFs is also stored in a column
> AND is also indexed. There isn't any query by that secondary index. I
> should probably erase the index, shouldn't I?

What is the cardinality like on these indexes? Can you provide the
schema creation for these two column families?

Re: General question regarding bootstrap and nodetool repair

[Wei Zhu]

One more question though, 
I tried to replace a node with a new node with the same IP, Here is what we 
did: 

* drain the node 
* check nodetool on other nodes, and this node is marked down (the token for 
this node is 100) 
* clear the data, commit log, saved cache on the down node. 
* change initial_token from 100 to 99 in the yaml file 
* start the node 
* check nodetool, the down node of 100 disappeared by itself (!!) and new node 
with token 99 showed up 
* checked log, see the message saying bootstrap completed. But only a couple of 
MB streamed. 
* nodetool movetoken 98 
* nodetool, see the node with token 98 comes up. 
* check log, see the message saying bootstrap completed. But still only a 
couple of MB streamed. 

The only reason I can think of is that the new node has the same IP as the 
"dead" node we tried to replace? After reading the bootstrap code, it shouldn't 
be the case. Is it a bug? Or anyone tried to replace a dead node with the same 
IP? 

Thanks. 
-Wei 


- Original Message -

From: "Wei Zhu"  
To: user@cassandra.apache.org 
Sent: Thursday, January 31, 2013 3:14:59 PM 
Subject: Re: General question regarding bootstrap and nodetool repair 



Thanks Rob. I think you are right on it. 


Here is what I found: 


https://github.com/apache/cassandra/blob/cassandra-1.1.0/src/java/org/apache/cassandra/dht/RangeStreamer.java#L140
 



It sorts the end point by proximity and in 


https://github.com/apache/cassandra/blob/cassandra-1.1.0/src/java/org/apache/cassandra/dht/RangeStreamer.java#L171
 



It fetches the data from the only one source. 


That answers my question. So we will have to run repair after the bootstrap to 
make sure the consistency. 


Thanks. 
-Wei 







From: Rob Coli  
To: user@cassandra.apache.org 
Sent: Thursday, January 31, 2013 1:50 PM 
Subject: Re: General question regarding bootstrap and nodetool repair 

On Thu, Jan 31, 2013 at 12:19 PM, Wei Zhu < wz1...@yahoo.com > wrote: 


But I am still not sure how about the my first question regarding the 
bootstrap, anyone? 

As I understand it, bootstrap occurs from a single replica. Which 


replica is chosen is based on some internal estimation of which is 
closest/least loaded/etc. But only from a single replica, so in RF=3, 
in order to be consistent with both you still have to run a repair. 

=Rob 

-- 
=Robert Coli 
AIM>ALK - rc...@palominodb.com 
YAHOO - rcoli.palominob 
SKYPE - rcoli_palominodb 

Re: Multi-range gets/deletes in Thrift/CQL

[Manu Zhang]

On Fri 25 Jan 2013 07:42:03 PM CST, Ravikumar Govindarajan wrote:

I came across two issue fixes in 1.2 release

a) Cassandra-3885 - Specify Multiple Ranges in SliceQueryFilter
b) Cassandra-3708 - Specify range during tombstone ops

We have composite columns in our app and wish to utilize the above 2 fixes

Ex: Delete all columns with only first-part of Composite column key,
containing 2 parts
  Get all columns based on first-part of Composite column,
containing 2 parts. {key --> [firstpart1-*, firstpart2-*,
firstpart3-*]}

Are these available in CQL/Thrift of 1.2 release?

--
Ravi




yes, put the first part into the primary key when you create table in 
CQL3 and when doing the query you'd better provide the row key as well

rangeQuery to traverse keys backward?

[Yuhan Zhang]

Hi all,

I'm tryinng to use get_range to traverse the rows by page by providing a
:start_key and an :finish_key.

This works fine when I traverse forward with :start_key=>last_key,
:finish_key=>""
However, when I tried to traversed backward with :start_key="",
:finish_key=>first_key, this always gave me the first few rows in the
column family.
(my goal is to get  the rows adjacent to my "first_key")

looks like it always takes priority of :start_key over the :finish_key.

as for column range,  there is an option to reverse the order. but there is
an option for  traversing rows.
so I'm wondering whether cassandra is capable of doing this task with the
current api

I tried both twitter cassandra client and hector client, but couldn't find
a way to perform it.
have someone been able to do this?


Thank you

Yuhan

-- 
The information contained in this e-mail is for the exclusive use of the 
intended recipient(s) and may be confidential, proprietary, and/or legally 
privileged. Inadvertent disclosure of this message does not constitute a 
waiver of any privilege.  If you receive this message in error, please do 
not directly or indirectly print, copy, retransmit, disseminate, or 
otherwise use the information. In addition, please delete this e-mail and 
all copies and notify the sender.

Re: General question regarding bootstrap and nodetool repair

[Rob Coli]

On Thu, Jan 31, 2013 at 3:31 PM, Wei Zhu  wrote:
> The only reason I can think of is that the new node has the same IP as the
> "dead" node we tried to replace? After reading the bootstrap code, it
> shouldn't be the case. Is it a bug? Or anyone tried to replace a dead node
> with the same IP?

You can use replace_token property to accomplish this. I would expect
cassandra to get confused by having two nodes with the same ip.

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Re: Pycassa vs YCSB results.

[Tyler Hobbs]

Can you provide the python script that you're using?

(I'm moving this thread to the pycassa mailing list (
pycassa-disc...@googlegroups.com), which is a better place for this
discussion.)


On Thu, Jan 31, 2013 at 6:25 PM, Pradeep Kumar Mantha
wrote:

> Hi,
>
> I am trying to benchmark cassandra on a 12 Data Node cluster using 16
> clients ( each client uses 32 threads) using custom pycassa client and YCSB.
>
> I found the maximum number of operations/seconds achieved using pycassa
> client is nearly 70k+ reads/second.
> Whereas with YCSB it is ~ 120k reads/second.
>
> Any thoughts, why I see this huge difference in performance?
>
>
> Here is the description of setup.
>
> Pycassa client (a simple python script).
> 1. Each pycassa client starts 4 threads - where each thread queries 76896
> queries.
> 2. a shell script is used to submit 4threads/each core using taskset unix
> command on a 8 core single node. ( 8 * 4 * 76896 queries)
> 3. Another shell script is used to scale the single node shell script to
> 16 nodes  ( total queries now - 16 * 8 * 4 * 76896 queries )
>
> I tried to keep YCSB configuration as much as similar to my custom pycassa
> benchmarking setup.
>
> YCSB -
>
> Launched 16 YCSB clients on 16 nodes where each client uses 32 threads for
> execution and need to query ( 32 * 76896 keys ), i.e 100% reads
>
> The dataset is different in each case, but has
>
> 1. same number of total records.
> 2. same number of fields.
> 3. field length is almost same.
>
> Could you please let me know, why I see this huge performance difference
> and is there any way I can improve the operations/second using pycassa
> client.
>
> thanks
> pradeep
>
>



-- 
Tyler Hobbs
DataStax 

Re: Pycassa vs YCSB results.

[Pradeep Kumar Mantha]

Thanks.. Please find the script as attachment.

Just re-iterating.
Its just a simple python script which submit 4 threads.
This script has been scheduled on 8 cores using taskset unix command , thus
running 32 threads/node.
and then scaling to 16 nodes

thanks
pradeep

On Thu, Jan 31, 2013 at 4:38 PM, Tyler Hobbs  wrote:

> Can you provide the python script that you're using?
>
> (I'm moving this thread to the pycassa mailing list (
> pycassa-disc...@googlegroups.com), which is a better place for this
> discussion.)
>
>
> On Thu, Jan 31, 2013 at 6:25 PM, Pradeep Kumar Mantha <
> pradeep...@gmail.com> wrote:
>
>> Hi,
>>
>> I am trying to benchmark cassandra on a 12 Data Node cluster using 16
>> clients ( each client uses 32 threads) using custom pycassa client and YCSB.
>>
>> I found the maximum number of operations/seconds achieved using pycassa
>> client is nearly 70k+ reads/second.
>> Whereas with YCSB it is ~ 120k reads/second.
>>
>> Any thoughts, why I see this huge difference in performance?
>>
>>
>> Here is the description of setup.
>>
>> Pycassa client (a simple python script).
>> 1. Each pycassa client starts 4 threads - where each thread queries 76896
>> queries.
>> 2. a shell script is used to submit 4threads/each core using taskset unix
>> command on a 8 core single node. ( 8 * 4 * 76896 queries)
>> 3. Another shell script is used to scale the single node shell script to
>> 16 nodes  ( total queries now - 16 * 8 * 4 * 76896 queries )
>>
>> I tried to keep YCSB configuration as much as similar to my custom
>> pycassa benchmarking setup.
>>
>> YCSB -
>>
>> Launched 16 YCSB clients on 16 nodes where each client uses 32 threads
>> for execution and need to query ( 32 * 76896 keys ), i.e 100% reads
>>
>> The dataset is different in each case, but has
>>
>> 1. same number of total records.
>> 2. same number of fields.
>> 3. field length is almost same.
>>
>> Could you please let me know, why I see this huge performance difference
>> and is there any way I can improve the operations/second using pycassa
>> client.
>>
>> thanks
>> pradeep
>>
>>
>
>
>
> --
> Tyler Hobbs
> DataStax 
>


pycassa_client.py
Description: Binary data

Re: Start token sorts after end token

[aaron morton]

Out of interest do you have the ticket?

A

-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 31/01/2013, at 3:19 AM, Edward Capriolo  wrote:

> Fix is simply to switch to random partitioner.
> 
> On Wednesday, January 30, 2013, Edward Capriolo  wrote:
> > This was unexpected fallout fro the change to murmur partitioner. A jira is 
> > open but if you need map red murmers is currently out of the question.
> >
> > On Wednesday, January 30, 2013, Tejas Patil  
> > wrote:
> >> While reading data from Cassandra in map-reduce, I am getting 
> >> "InvalidRequestException(why:Start token sorts after end token)"
> >> Below is the code snippet that I used and the entire stack trace.
> >> (I am using Cassandra 1.2.0 and hadoop 0.20.2)
> >> Can you point out the issue here ?
> >> Code snippet:
> >>SlicePredicate predicate = new SlicePredicate();
> >> SliceRange sliceRange = new SliceRange();
> >> sliceRange.start = ByteBuffer.wrap(("1".getBytes()));
> >> sliceRange.finish = ByteBuffer.wrap(("100".getBytes()));
> >> sliceRange.reversed = false;
> >> //predicate.slice_range = sliceRange;
> >> List colNames = new ArrayList();
> >> colNames.add(ByteBuffer.wrap("url".getBytes()));
> >> colNames.add(ByteBuffer.wrap("Parent".getBytes()));
> >> predicate.column_names = colNames;
> >> ConfigHelper.setInputSlicePredicate(job.getConfiguration(), predicate);
> >> Full stack trace:
> >> java.lang.RuntimeException: InvalidRequestException(why:Start token sorts 
> >> after end token)
> >> at 
> >> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.maybeInit(ColumnFamilyRecordReader.java:384)
> >> at 
> >> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:390)
> >> at 
> >> org.apache.cassandra.hadoop.ColumnFamilyRecordReader$StaticRowIterator.computeNext(ColumnFamilyRecordReader.java:313)
> >> at 
> >> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> >> at 
> >> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> >> at 
> >> org.apache.cassandra.hadoop.ColumnFamilyRecordReader.nextKeyValue(ColumnFamilyRecordReader.java:184)
> >> at 
> >> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
> >> at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> >> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:
> >>

Re: Node selection when both partition key and secondary index field constrained?

[aaron morton]

> So basically it's merging the results 2 separate queries:   Indexed scan 
> (token-range) intersect foo.flag_index=true  
NO.
It is doing one query, one the secondary index. When it reads the row keys in 
that index is discards any outside of the token range, 
That query is sent to nodes which have a token range that intersect with the 
token range you have supplied. 

So if your query token range is included in one Node Token Range, the query 
will be sent to CL nodes that replicate that token range. 

Cheers
 
-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 31/01/2013, at 3:49 AM, Peter Lin  wrote:

> I'd also point out, Hector has better support for CQL3 features than
> Astyanax. I contributed some stuff to hector back in December, but I
> don't have time to apply those changes to astyanax.
> 
> I have other contributions in mind for hector, which I hope to work on
> later this year.
> 
> On Wed, Jan 30, 2013 at 9:45 AM, Edward Capriolo  
> wrote:
>> Hector has this feature because Hector is awesome sauce, but aystynsnax is
>> new,sexy, and bogged about by netflix.
>> 
>> So the new cassandra trend to force everyone to use less functional new
>> stuff is at work here making you wish for something that already exists
>> elsewhere.
>> 
>> 
>> On Wednesday, January 30, 2013, Hiller, Dean  wrote:
>>> I recall someone doing some work in Astyanax and I don't know if it made
>>> it back in where astyanax would retry at a lower CL level when 2 nodes were
>>> down so things could continue to work which was a VERY VERY cool feature.
>>> You may want to look into that….I know at some point, I plan to.
>>> 
>>> Later,
>>> Dean
>>> 
>>> From: Edward Capriolo
>>> mailto:edlinuxg...@gmail.com>>
>>> Reply-To: "user@cassandra.apache.org"
>>> mailto:user@cassandra.apache.org>>
>>> Date: Wednesday, January 30, 2013 7:31 AM
>>> To: "user@cassandra.apache.org"
>>> mailto:user@cassandra.apache.org>>
>>> Subject: Re: Node selection when both partition key and secondary index
>>> field constrained?
>>> 
>>> Any query is going to fail quorum + rf3 + 2 nodes down.
>>> 
>>> One thing about 2x indexes (both user defined and built in) is that
>>> finding an answer using them requires more nodes to be up then just a single
>>> get or slice.
>>> 
>>> On Monday, January 28, 2013, Mike Sample
>>> mailto:mike.sam...@gmail.com>> wrote:
 Thanks Aaron.   So basically it's merging the results 2 separate queries:
 Indexed scan (token-range) intersect foo.flag_index=true where the
 latter query hits the entire cluster as per the secondary index FAQ entry.
 Thus the overall query would fail if LOCAL_QUORUM was requested, RF=3 and 2
 nodes in a given replication group were down. Darn.  Is there any way of
 efficiently getting around this (ie scope the query to just the nodes in 
 the
 token range)?
 
 
 
 
 On Mon, Jan 28, 2013 at 11:44 AM, aaron morton
 mailto:aa...@thelastpickle.com>> wrote:
> 
> It uses the index...
> 
> cqlsh:dev> tracing on;
> Now tracing requests.
> cqlsh:dev>
> cqlsh:dev>
> cqlsh:dev> SELECT id, flag from foo WHERE TOKEN(id) > '-9939393' AND
> TOKEN(id) <= '0' AND flag=true;
> 
> Tracing session: 128cab90-6982-11e2-8cd1-51eaa232562e
> 
> activity   | timestamp|
> source| source_elapsed
> 
> +--+---+
> execute_cql3_query | 08:36:55,244 |
> 127.0.0.1 |  0
>  Parsing statement | 08:36:55,244 |
> 127.0.0.1 |600
> Peparing statement | 08:36:55,245 |
> 127.0.0.1 |   1408
>  Determining replicas to query | 08:36:55,246 |
> 127.0.0.1 |   1924
> Executing indexed scan for (max(-9939393), max(0)] | 08:36:55,247 |
> 127.0.0.1 |   2956
> Executing single-partition query on foo.flag_index | 08:36:55,247 |
> 127.0.0.1 |   3192
>   Acquiring sstable references | 08:36:55,247 |
> 127.0.0.1 |   3220
>  Merging memtable contents | 08:36:55,247 |
> 127.0.0.1 |   3265
>   Scanned 0 rows and matched 0 | 08:36:55,247 |
> 127.0.0.1 |   3396
>   Request complete | 08:36:55,247 |
> 127.0.0.1 |   3644
> 
> 
> It reads from the secondary index and discards keys that are outside of
> the token range.
> 
> Cheers
> 
> 
> -
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
> 
> @aaronmorton
> http:

Re: Nodetool can not get to 7199 after migrating to 1.2.1

[aaron morton]

Check the JMX port listed in /etc/cassandra/cassandra-env.sh 

If it's not default you can pass it to nodetool using the --jmx-port option. 

Cheers

-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 31/01/2013, at 3:52 AM, Shahryar Sedghi  wrote:

> I migrated my test environment from 1.2.0 to 1.2.1 (DataStax Community) and 
> nodetool  can not communicate to 7199, even if it is listening. in one node I 
> get   
> Failed to connect to 'cassandra4:7199': Connection refused
> 
> in another node I get timeout.
> 
> Did I do anything wrong, when upgrading?
> 
> Thanks In advance
> 
> Shahryar
> -- 
> "Life is what happens while you are making other plans." ~ John Lennon

Re: Suggestion: Move some threads to the client-dev mailing list

[aaron morton]

+1

Also where can I learn more about pyhtondra ?

Cheers
-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 31/01/2013, at 8:09 AM, Rob Coli  wrote:

> On Wed, Jan 30, 2013 at 7:21 AM, Edward Capriolo  
> wrote:
>> My suggestion: At minimum we should re-route these questions to client-dev
>> or simply say, "If it is not part of core Cassandra, you are looking in the
>> wrong place for support"
> 
> +1, I find myself scanning past all those questions in order to find
> questions I am able to answer based solely on my operational knowledge
> of the Cassandra daemon.
> 
> =Rob
> 
> -- 
> =Robert Coli
> AIM>ALK - rc...@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb