Re: Proper Use of PreparedStatements in DataStax driver

[Sylvain Lebresne]

> Am I posting this to the wrong place?

Actually, kind of. The proper place for that type of questions is probably
the
java driver mailing (
https://groups.google.com/a/lists.datastax.com/forum/#!forum/java-driver-user
).
But anyway, answers follows.

> 1. My question is should I only instantiate the PreparedStatement once and
> reuse *the same instance* every time I want to execute an INSERT?  Is that
> necessary to get the performance benefits of using PreparedStatements?

Yes, you should. You should only call Session.prepare once.


> If the answer is #1, then that means PreparedStatements are effectively
> singletons and need to be multi-thread-safe and the app is responsible for
> dealing with lifecycle issues presuming that the original Session dies or
> whatever

PreparedStatement is immutable and thus thread-safe. However BoundStatement
are
not. You should have one PreparedStatement, but you everytime you bind it
for
execution, you'll created a new BoundStatement.

As for the lifecycle, there is nothing really to deal with. A Session in the
driver is *not* just one server connection. It's a thread-safe object that
encapsulate pools of connections to the nodes in the cluster and that
abstract
from the application the issues of handling node failures. A Session never
dies, and as a consequence a PreparedStatement is always valid (unless you
manually shutdown the Session that is).

--
Sylvain

COPY command times out

[Petter von Dolwitz (Hem)]

Hi,

I'm trying to import CSV data using the COPY ... FROM command. After
importing 10% of my 2.5 GB csv file the operation aborts with the message:

"Request did not complete within rpc_timeout.
Aborting import at record #504631 (line 504632). Previously-inserted values
still present."

There are no exceptions in the log. I'm using Cassandra 2.0.1 on ubuntu
using a two machine setup with 4 cores, 15 GB RAM each.

The table design incorporates many secondary indexes (which someone
discouraged me to use).

Can anybody tells me what is going on?

Thanks,
Petter

Bulk Loader in cassandra : String as row keys in cassandra

[ashish sanadhya]

I have done with bulk loader with key_validation_class=LexicalUUIDType for
new row with the help of this [code][1] but i have changed my
**key_validation_class=AsciiType** in order to make **string as row keys**

  create column family Users1
  with key_validation_class=AsciiType
   and comparator=AsciiType
  AND column_metadata = [
  {column_name: timestamp1, validation_class: AsciiType}
  {column_name: symbol, validation_class: AsciiType}
  {column_name: Bid_Price, validation_class:AsciiType}
  {column_name: Ask_Price, validation_class:AsciiType}
  ];


i have tried all possible changes to code in order to make row keys as
string type but getting an error or even without **usersWriter.newRow** not
able to write into sstable


  while ((line = reader.readLine()) != null)
{
 if (entry.parse(line, lineNumber))
{
//usersWriter.newRow(uuid);
usersWriter.newRow(String.valueOf(lineNumber));
usersWriter.addColumn(bytes("symbol"), bytes(entry.symbol),
timestamp);
usersWriter.addColumn(bytes("timestamp1"),
bytes(entry.timestamp1), timestamp);
usersWriter.addColumn(bytes("Bid_Price"),
bytes(entry.Bid_Price), timestamp);
usersWriter.addColumn(bytes("Ask_Price"),
bytes(entry.Ask_Price), timestamp);
}
lineNumber++;
}

  getting an error as expected it is only taking **ByteBuffer**

 usersWriter.newRow(String.valueOf(lineNumber));
   ^
 required: ByteBuffer
 found: String
 reason: actual argument String cannot be converted to ByteBuffer by
method invocation  conversion

Any help to make string as row keys in sstable for the above column family
definition.thanks.






  [1]:
http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java

Re: COPY command times out

[Vivek Mishra]

If not getting any exception.

Reason for "Request did not complete within rpc_timeout."  => socket time
out

As per http://www.datastax.com/docs/1.1/references/cql/COPY( COPY from CSV
section)

COPY FROM is intended for importing small datasets (a few million rows or
less) into Cassandra. For importing larger datasets, use Cassandra Bulk
Loader  or
the sstable2json /
json2sstable
utility.


-Vivek




On Fri, Oct 11, 2013 at 3:02 PM, Petter von Dolwitz (Hem) <
petter.von.dolw...@gmail.com> wrote:

> Hi,
>
> I'm trying to import CSV data using the COPY ... FROM command. After
> importing 10% of my 2.5 GB csv file the operation aborts with the message:
>
> "Request did not complete within rpc_timeout.
> Aborting import at record #504631 (line 504632). Previously-inserted
> values still present."
>
> There are no exceptions in the log. I'm using Cassandra 2.0.1 on ubuntu
> using a two machine setup with 4 cores, 15 GB RAM each.
>
> The table design incorporates many secondary indexes (which someone
> discouraged me to use).
>
> Can anybody tells me what is going on?
>
> Thanks,
> Petter
>
>
>

Re: Bulk Loader in cassandra : String as row keys in cassandra

[Vivek Mishra]

but i have changed my **key_validation_class=AsciiType** in order to make
**string as row keys**

why not key_validation_class=UTF8Type ?

-Vivek


On Fri, Oct 11, 2013 at 3:55 PM, ashish sanadhya wrote:

> I have done with bulk loader with key_validation_class=LexicalUUIDType for
> new row with the help of this [code][1] but i have changed my
> **key_validation_class=AsciiType** in order to make **string as row keys**
>
>   create column family Users1
>   with key_validation_class=AsciiType
>and comparator=AsciiType
>   AND column_metadata = [
>   {column_name: timestamp1, validation_class: AsciiType}
>   {column_name: symbol, validation_class: AsciiType}
>   {column_name: Bid_Price, validation_class:AsciiType}
>   {column_name: Ask_Price, validation_class:AsciiType}
>   ];
>
>
> i have tried all possible changes to code in order to make row keys as
> string type but getting an error or even without **usersWriter.newRow** not
> able to write into sstable
>
>
>   while ((line = reader.readLine()) != null)
> {
>  if (entry.parse(line, lineNumber))
> {
> //usersWriter.newRow(uuid);
> usersWriter.newRow(String.valueOf(lineNumber));
> usersWriter.addColumn(bytes("symbol"),
> bytes(entry.symbol), timestamp);
> usersWriter.addColumn(bytes("timestamp1"),
> bytes(entry.timestamp1), timestamp);
> usersWriter.addColumn(bytes("Bid_Price"),
> bytes(entry.Bid_Price), timestamp);
> usersWriter.addColumn(bytes("Ask_Price"),
> bytes(entry.Ask_Price), timestamp);
> }
> lineNumber++;
> }
>
>   getting an error as expected it is only taking **ByteBuffer**
>
>  usersWriter.newRow(String.valueOf(lineNumber));
>^
>  required: ByteBuffer
>  found: String
>  reason: actual argument String cannot be converted to ByteBuffer by
> method invocation  conversion
>
> Any help to make string as row keys in sstable for the above column family
> definition.thanks.
>
>
>
>
>
>
>   [1]:
> http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java
>

Re: Bulk Loader in cassandra : String as row keys in cassandra

[Vivek Mishra]

Also, please use ByteBufferUtils for byte conversions.


On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra  wrote:

> but i have changed my **key_validation_class=AsciiType** in order to make
> **string as row keys**
>
> why not key_validation_class=UTF8Type ?
>
> -Vivek
>
>
> On Fri, Oct 11, 2013 at 3:55 PM, ashish sanadhya 
> wrote:
>
>> I have done with bulk loader with key_validation_class=LexicalUUIDType
>> for new row with the help of this [code][1] but i have changed my
>> **key_validation_class=AsciiType** in order to make **string as row keys**
>>
>>   create column family Users1
>>   with key_validation_class=AsciiType
>>and comparator=AsciiType
>>   AND column_metadata = [
>>   {column_name: timestamp1, validation_class: AsciiType}
>>   {column_name: symbol, validation_class: AsciiType}
>>   {column_name: Bid_Price, validation_class:AsciiType}
>>   {column_name: Ask_Price, validation_class:AsciiType}
>>   ];
>>
>>
>> i have tried all possible changes to code in order to make row keys as
>> string type but getting an error or even without **usersWriter.newRow** not
>> able to write into sstable
>>
>>
>>   while ((line = reader.readLine()) != null)
>> {
>>  if (entry.parse(line, lineNumber))
>> {
>> //usersWriter.newRow(uuid);
>> usersWriter.newRow(String.valueOf(lineNumber));
>> usersWriter.addColumn(bytes("symbol"),
>> bytes(entry.symbol), timestamp);
>> usersWriter.addColumn(bytes("timestamp1"),
>> bytes(entry.timestamp1), timestamp);
>> usersWriter.addColumn(bytes("Bid_Price"),
>> bytes(entry.Bid_Price), timestamp);
>> usersWriter.addColumn(bytes("Ask_Price"),
>> bytes(entry.Ask_Price), timestamp);
>> }
>> lineNumber++;
>> }
>>
>>   getting an error as expected it is only taking **ByteBuffer**
>>
>>  usersWriter.newRow(String.valueOf(lineNumber));
>>^
>>  required: ByteBuffer
>>  found: String
>>  reason: actual argument String cannot be converted to ByteBuffer by
>> method invocation  conversion
>>
>> Any help to make string as row keys in sstable for the above column
>> family definition.thanks.
>>
>>
>>
>>
>>
>>
>>   [1]:
>> http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java
>>
>
>

Re: Bulk Loader in cassandra : String as row keys in cassandra

[ashish sanadhya]

Hi vivek key_validation_class=UTF8Type will do ,but i certainly want
*string as row keys, *so will it work ?? *
*


On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra  wrote:

> Also, please use ByteBufferUtils for byte conversions.
>
>
> On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:
>
>> but i have changed my **key_validation_class=AsciiType** in order to
>> make **string as row keys**
>>
>> why not key_validation_class=UTF8Type ?
>>
>> -Vivek
>>
>>
>> On Fri, Oct 11, 2013 at 3:55 PM, ashish sanadhya 
>> wrote:
>>
>>> I have done with bulk loader with key_validation_class=LexicalUUIDType
>>> for new row with the help of this [code][1] but i have changed my
>>> **key_validation_class=AsciiType** in order to make **string as row keys**
>>>
>>>   create column family Users1
>>>   with key_validation_class=AsciiType
>>>and comparator=AsciiType
>>>   AND column_metadata = [
>>>   {column_name: timestamp1, validation_class: AsciiType}
>>>   {column_name: symbol, validation_class: AsciiType}
>>>   {column_name: Bid_Price, validation_class:AsciiType}
>>>   {column_name: Ask_Price, validation_class:AsciiType}
>>>   ];
>>>
>>>
>>> i have tried all possible changes to code in order to make row keys as
>>> string type but getting an error or even without **usersWriter.newRow** not
>>> able to write into sstable
>>>
>>>
>>>   while ((line = reader.readLine()) != null)
>>> {
>>>  if (entry.parse(line, lineNumber))
>>> {
>>> //usersWriter.newRow(uuid);
>>> usersWriter.newRow(String.valueOf(lineNumber));
>>> usersWriter.addColumn(bytes("symbol"),
>>> bytes(entry.symbol), timestamp);
>>> usersWriter.addColumn(bytes("timestamp1"),
>>> bytes(entry.timestamp1), timestamp);
>>> usersWriter.addColumn(bytes("Bid_Price"),
>>> bytes(entry.Bid_Price), timestamp);
>>> usersWriter.addColumn(bytes("Ask_Price"),
>>> bytes(entry.Ask_Price), timestamp);
>>> }
>>> lineNumber++;
>>> }
>>>
>>>   getting an error as expected it is only taking **ByteBuffer**
>>>
>>>  usersWriter.newRow(String.valueOf(lineNumber));
>>>^
>>>  required: ByteBuffer
>>>  found: String
>>>  reason: actual argument String cannot be converted to ByteBuffer by
>>> method invocation  conversion
>>>
>>> Any help to make string as row keys in sstable for the above column
>>> family definition.thanks.
>>>
>>>
>>>
>>>
>>>
>>>
>>>   [1]:
>>> http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java
>>>
>>
>>
>

Re: Bulk Loader in cassandra : String as row keys in cassandra

[Vivek Mishra]

I am not able to get your meaning for "*string as row keys" ? *
*
*
Row key values will be of type "key_validation_class"  only
*
*

On Fri, Oct 11, 2013 at 4:25 PM, ashish sanadhya wrote:

> Hi vivek key_validation_class=UTF8Type will do ,but i certainly want
> *string as row keys, *so will it work ?? *
> *
>
>
> On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:
>
>> Also, please use ByteBufferUtils for byte conversions.
>>
>>
>> On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:
>>
>>> but i have changed my **key_validation_class=AsciiType** in order to
>>> make **string as row keys**
>>>
>>> why not key_validation_class=UTF8Type ?
>>>
>>> -Vivek
>>>
>>>
>>> On Fri, Oct 11, 2013 at 3:55 PM, ashish sanadhya >> > wrote:
>>>
 I have done with bulk loader with key_validation_class=LexicalUUIDType
 for new row with the help of this [code][1] but i have changed my
 **key_validation_class=AsciiType** in order to make **string as row keys**

   create column family Users1
   with key_validation_class=AsciiType
and comparator=AsciiType
   AND column_metadata = [
   {column_name: timestamp1, validation_class: AsciiType}
   {column_name: symbol, validation_class: AsciiType}
   {column_name: Bid_Price, validation_class:AsciiType}
   {column_name: Ask_Price, validation_class:AsciiType}
   ];


 i have tried all possible changes to code in order to make row keys as
 string type but getting an error or even without **usersWriter.newRow** not
 able to write into sstable


   while ((line = reader.readLine()) != null)
 {
  if (entry.parse(line, lineNumber))
 {
 //usersWriter.newRow(uuid);
 usersWriter.newRow(String.valueOf(lineNumber));
 usersWriter.addColumn(bytes("symbol"),
 bytes(entry.symbol), timestamp);
 usersWriter.addColumn(bytes("timestamp1"),
 bytes(entry.timestamp1), timestamp);
 usersWriter.addColumn(bytes("Bid_Price"),
 bytes(entry.Bid_Price), timestamp);
 usersWriter.addColumn(bytes("Ask_Price"),
 bytes(entry.Ask_Price), timestamp);
 }
 lineNumber++;
 }

   getting an error as expected it is only taking **ByteBuffer**

  usersWriter.newRow(String.valueOf(lineNumber));
^
  required: ByteBuffer
  found: String
  reason: actual argument String cannot be converted to ByteBuffer
 by method invocation  conversion

 Any help to make string as row keys in sstable for the above column
 family definition.thanks.






   [1]:
 http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java

>>>
>>>
>>
>

Re: Bulk Loader in cassandra : String as row keys in cassandra

[ashish sanadhya]

Here i mean that key_validation_class=AsciiType or
key_validation_class=UTF8Type
but I am unable to create an sstable for this column family

create column family Users1
  with key_validation_class=UTF8Type
   and comparator=AsciiType
  AND column_metadata = [
  {column_name: timestamp1, validation_class: AsciiType}
  {column_name: symbol, validation_class: AsciiType}
  {column_name: Bid_Price, validation_class:AsciiType}
  {column_name: Ask_Price, validation_class:AsciiType}
  ];

how do i get from this usersWriter.newRow(String.
valueOf(lineNumber));  ?
thanks.



On Fri, Oct 11, 2013 at 4:30 PM, Vivek Mishra  wrote:

> I am not able to get your meaning for "*string as row keys" ? *
> *
> *
> Row key values will be of type "key_validation_class"  only
> *
> *
>
> On Fri, Oct 11, 2013 at 4:25 PM, ashish sanadhya 
> wrote:
>
>> Hi vivek key_validation_class=UTF8Type will do ,but i certainly want
>> *string as row keys, *so will it work ?? *
>> *
>>
>>
>> On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:
>>
>>> Also, please use ByteBufferUtils for byte conversions.
>>>
>>>
>>> On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:
>>>
 but i have changed my **key_validation_class=AsciiType** in order to
 make **string as row keys**

 why not key_validation_class=UTF8Type ?

 -Vivek


 On Fri, Oct 11, 2013 at 3:55 PM, ashish sanadhya <
 sanadhyaa...@gmail.com> wrote:

> I have done with bulk loader with key_validation_class=LexicalUUIDType
> for new row with the help of this [code][1] but i have changed my
> **key_validation_class=AsciiType** in order to make **string as row keys**
>
>   create column family Users1
>   with key_validation_class=AsciiType
>and comparator=AsciiType
>   AND column_metadata = [
>   {column_name: timestamp1, validation_class: AsciiType}
>   {column_name: symbol, validation_class: AsciiType}
>   {column_name: Bid_Price, validation_class:AsciiType}
>   {column_name: Ask_Price, validation_class:AsciiType}
>   ];
>
>
> i have tried all possible changes to code in order to make row keys as
> string type but getting an error or even without **usersWriter.newRow** 
> not
> able to write into sstable
>
>
>   while ((line = reader.readLine()) != null)
> {
>  if (entry.parse(line, lineNumber))
> {
> //usersWriter.newRow(uuid);
> usersWriter.newRow(String.valueOf(lineNumber));
> usersWriter.addColumn(bytes("symbol"),
> bytes(entry.symbol), timestamp);
> usersWriter.addColumn(bytes("timestamp1"),
> bytes(entry.timestamp1), timestamp);
> usersWriter.addColumn(bytes("Bid_Price"),
> bytes(entry.Bid_Price), timestamp);
> usersWriter.addColumn(bytes("Ask_Price"),
> bytes(entry.Ask_Price), timestamp);
> }
> lineNumber++;
> }
>
>   getting an error as expected it is only taking **ByteBuffer**
>
>  usersWriter.newRow(String.valueOf(lineNumber));
>^
>  required: ByteBuffer
>  found: String
>  reason: actual argument String cannot be converted to ByteBuffer
> by method invocation  conversion
>
> Any help to make string as row keys in sstable for the above column
> family definition.thanks.
>
>
>
>
>
>
>   [1]:
> http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java
>


>>>
>>
>

Re: Bulk Loader in cassandra : String as row keys in cassandra

[Vivek Mishra]

Change key_validation_class to UTF8Type and

usersWriter.newRow(ByteBufferUtil.bytes(String.valueOf(lineNumber)));



On Fri, Oct 11, 2013 at 4:42 PM, ashish sanadhya wrote:

> Here i mean that key_validation_class=AsciiType or 
> key_validation_class=UTF8Type
> but I am unable to create an sstable for this column family
>
> create column family Users1
>   with key_validation_class=UTF8Type
>
>and comparator=AsciiType
>   AND column_metadata = [
>   {column_name: timestamp1, validation_class: AsciiType}
>   {column_name: symbol, validation_class: AsciiType}
>   {column_name: Bid_Price, validation_class:AsciiType}
>   {column_name: Ask_Price, validation_class:AsciiType}
>   ];
>
> how do i get from this usersWriter.newRow(String.
> valueOf(lineNumber));  ?
> thanks.
>
>
>
> On Fri, Oct 11, 2013 at 4:30 PM, Vivek Mishra wrote:
>
>> I am not able to get your meaning for "*string as row keys" ? *
>> *
>> *
>> Row key values will be of type "key_validation_class"  only
>> *
>> *
>>
>> On Fri, Oct 11, 2013 at 4:25 PM, ashish sanadhya 
>> wrote:
>>
>>> Hi vivek key_validation_class=UTF8Type will do ,but i certainly want
>>> *string as row keys, *so will it work ?? *
>>> *
>>>
>>>
>>> On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:
>>>
 Also, please use ByteBufferUtils for byte conversions.


 On Fri, Oct 11, 2013 at 4:17 PM, Vivek Mishra wrote:

> but i have changed my **key_validation_class=AsciiType** in order to
> make **string as row keys**
>
> why not key_validation_class=UTF8Type ?
>
> -Vivek
>
>
> On Fri, Oct 11, 2013 at 3:55 PM, ashish sanadhya <
> sanadhyaa...@gmail.com> wrote:
>
>> I have done with bulk loader with
>> key_validation_class=LexicalUUIDType for new row with the help of this
>> [code][1] but i have changed my **key_validation_class=AsciiType** in 
>> order
>> to make **string as row keys**
>>
>>   create column family Users1
>>   with key_validation_class=AsciiType
>>and comparator=AsciiType
>>   AND column_metadata = [
>>   {column_name: timestamp1, validation_class: AsciiType}
>>   {column_name: symbol, validation_class: AsciiType}
>>   {column_name: Bid_Price, validation_class:AsciiType}
>>   {column_name: Ask_Price, validation_class:AsciiType}
>>   ];
>>
>>
>> i have tried all possible changes to code in order to make row keys
>> as string type but getting an error or even without 
>> **usersWriter.newRow**
>> not able to write into sstable
>>
>>
>>   while ((line = reader.readLine()) != null)
>> {
>>  if (entry.parse(line, lineNumber))
>> {
>> //usersWriter.newRow(uuid);
>> usersWriter.newRow(String.valueOf(lineNumber));
>> usersWriter.addColumn(bytes("symbol"),
>> bytes(entry.symbol), timestamp);
>> usersWriter.addColumn(bytes("timestamp1"),
>> bytes(entry.timestamp1), timestamp);
>> usersWriter.addColumn(bytes("Bid_Price"),
>> bytes(entry.Bid_Price), timestamp);
>> usersWriter.addColumn(bytes("Ask_Price"),
>> bytes(entry.Ask_Price), timestamp);
>> }
>> lineNumber++;
>> }
>>
>>   getting an error as expected it is only taking **ByteBuffer**
>>
>>  usersWriter.newRow(String.valueOf(lineNumber));
>>^
>>  required: ByteBuffer
>>  found: String
>>  reason: actual argument String cannot be converted to ByteBuffer
>> by method invocation  conversion
>>
>> Any help to make string as row keys in sstable for the above column
>> family definition.thanks.
>>
>>
>>
>>
>>
>>
>>   [1]:
>> http://www.datastax.com/wp-content/uploads/2011/08/DataImportExample.java
>>
>
>

>>>
>>
>

Re: COPY command times out

[Petter von Dolwitz (Hem)]

Hi,

thanks for you reply. My CSV contained roughly 5 million rows. It aborted
after half a million row. It seems to me that some throttling mechanism is
missing in the implementation of the COPY command?

I'll try the other way of getting data in.

/Petter


2013/10/11 Vivek Mishra 

> If not getting any exception.
>
> Reason for "Request did not complete within rpc_timeout."  => socket time
> out
>
> As per http://www.datastax.com/docs/1.1/references/cql/COPY( COPY from
> CSV section)
>
>  COPY FROM is intended for importing small datasets (a few million rows or
> less) into Cassandra. For importing larger datasets, use Cassandra Bulk
> Loader  or
> the sstable2json / 
> json2sstable
> utility.
>
>
> -Vivek
>
>
>
>
> On Fri, Oct 11, 2013 at 3:02 PM, Petter von Dolwitz (Hem) <
> petter.von.dolw...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm trying to import CSV data using the COPY ... FROM command. After
>> importing 10% of my 2.5 GB csv file the operation aborts with the message:
>>
>> "Request did not complete within rpc_timeout.
>> Aborting import at record #504631 (line 504632). Previously-inserted
>> values still present."
>>
>> There are no exceptions in the log. I'm using Cassandra 2.0.1 on ubuntu
>> using a two machine setup with 4 cores, 15 GB RAM each.
>>
>> The table design incorporates many secondary indexes (which someone
>> discouraged me to use).
>>
>> Can anybody tells me what is going on?
>>
>> Thanks,
>> Petter
>>
>>
>>
>

Heap requirement for Off-heap space

[Artur Kronenberg]

Hi,

I was playing around with cassandra off-heap options. I configured 3 GB 
off-heap for my row cache and 2 GB Heap space for cassandra. After 
running a bunch of load tests against it I saw the cache warm up. Doing 
a jmap histogram I noticed a lot of offHeapkey objects. At that point my 
row cache was only 500 MB big. So I got worried on how this will affect 
Heap space when I manage to fill the entire off-heap.


Is there a formular that maps heap space to off-heap? Or more 
spicifically how much heap space does it require to manage 3 GB off-heap 
space?


thanks!

artur

Denial of Service Issue

[Thorsten.Sinn]

I found the issue below concerning inactive client connections (see Cassandra 
Security). We 
are using Cassandra 1.2.4 and the Cassandra JDBC driver as client. Is this 
still an existing issue?
Quoted from site above:
Denial of Service problem:
Cassandra uses a Thread- Per-Client model in its network code. Since setting up 
a connection requires the Cassandra server to start a new thread on each 
connection (in addition to the TCP overhead incurred by the network), the 
Cassandra project recommends utilizing some sort of connection pooling. An 
attacker can prevent the Cassandra server from accepting new client connections 
by causing the Cassandra server to allocate all its resources to fake 
connection attempts. The only pieces of information required by an attacker are 
the IP addresses of the cluster members, and this information can be obtained 
by passively sniffing the network. The current implementation doesn't timeout 
inactive connections, so any connection that is opened without actually passing 
data consumes a thread and a file-descriptor that are never released.

Re: Cassandra Agent

[Edward Capriolo]

Stick sandra on the end. Restsandra.

On Friday, October 11, 2013, Ran Tavory  wrote:
> Seems like the greeks are all used out, how about moving the the japanese
mythology? it's a brand new pool of names...
> http://en.wikipedia.org/wiki/Japanese_mythology
>
>
> On Fri, Oct 11, 2013 at 8:29 AM, Blair Zajac  wrote:
>>
>> On 10/10/2013 10:28 PM, Blair Zajac wrote:
>>>
>>> On 10/10/2013 08:53 PM, Sean McCully wrote:

 On Thursday, October 10, 2013 08:30:42 PM Blair Jacuzzi wrote:
>
> On 10/10/2013 07:54 PM, Sean McCully wrote:
>>
>> Hello Cassandra Users,
>>
>> I've recently created a Cassandra Agent as part of Netflix's Cloud
>> Prize
>> competition, the submission which I've named Hector is largely based
on
>> Netflix's Priam. I would be very interested in getting feedback, from
>> anyone willing to give Hector (https://github.com/seanmccully/hector)
a
>> try. I am very interested in seeing if this is something the
Cassandra
>> Community is interested in using with their Cassandra installs.
>
> For one, there's a name conflict with the well known Hector Cassandra
> client project:
>
> http://hector-client.github.io/hector/build/html/index.html

 Any suggestions on a new name?
>>>
>>> Helenus, the twin brother of the prophetess Cassandra???
>>>
>>> http://en.wikipedia.org/wiki/Helenus
>>
>> Oops, should have Googled myself before suggesting this, they are NodeJS
Bindings for Cassandra:
>>
>> https://github.com/simplereach/helenus
>>
>> Well, I'll leave it to you to find a free name ;)
>>
>> Blair
>>
>
>
>
> --
> /Ran
> http://tavory.com

Re: Cassandra Agent

[David Schairer]

http://en.wikipedia.org/wiki/List_of_children_of_Priam

You've got plenty of children of Priam to go around.  Doesn't anyone read the 
Iliad any more?  :)

--DRS

On Oct 11, 2013, at 6:55 AM, Edward Capriolo  wrote:

> Stick sandra on the end. Restsandra.
> 
> On Friday, October 11, 2013, Ran Tavory  wrote:
> > Seems like the greeks are all used out, how about moving the the japanese 
> > mythology? it's a brand new pool of names...
> > http://en.wikipedia.org/wiki/Japanese_mythology
> >
> >
> > On Fri, Oct 11, 2013 at 8:29 AM, Blair Zajac  wrote:
> >>
> >> On 10/10/2013 10:28 PM, Blair Zajac wrote:
> >>>
> >>> On 10/10/2013 08:53 PM, Sean McCully wrote:
> 
>  On Thursday, October 10, 2013 08:30:42 PM Blair Jacuzzi wrote:
> >
> > On 10/10/2013 07:54 PM, Sean McCully wrote:
> >>
> >> Hello Cassandra Users,
> >>
> >> I've recently created a Cassandra Agent as part of Netflix's Cloud
> >> Prize
> >> competition, the submission which I've named Hector is largely based on
> >> Netflix's Priam. I would be very interested in getting feedback, from
> >> anyone willing to give Hector (https://github.com/seanmccully/hector) a
> >> try. I am very interested in seeing if this is something the Cassandra
> >> Community is interested in using with their Cassandra installs.
> >
> > For one, there's a name conflict with the well known Hector Cassandra
> > client project:
> >
> > http://hector-client.github.io/hector/build/html/index.html
> 
>  Any suggestions on a new name?
> >>>
> >>> Helenus, the twin brother of the prophetess Cassandra???
> >>>
> >>> http://en.wikipedia.org/wiki/Helenus
> >>
> >> Oops, should have Googled myself before suggesting this, they are NodeJS 
> >> Bindings for Cassandra:
> >>
> >> https://github.com/simplereach/helenus
> >>
> >> Well, I'll leave it to you to find a free name ;)
> >>
> >> Blair
> >>
> >
> >
> >
> > --
> > /Ran
> > http://tavory.com

Re: Cassandra Agent

[Ran Tavory]

Now that's a fun problem to solve!

> On 11 באוק 2013, at 17:17, David Schairer  wrote:
> 
> http://en.wikipedia.org/wiki/List_of_children_of_Priam
> 
> You've got plenty of children of Priam to go around.  Doesn't anyone read the 
> Iliad any more?  :)
> 
> --DRS
> 
>> On Oct 11, 2013, at 6:55 AM, Edward Capriolo  wrote:
>> 
>> Stick sandra on the end. Restsandra.
>> 
>>> On Friday, October 11, 2013, Ran Tavory  wrote:
>>> Seems like the greeks are all used out, how about moving the the japanese 
>>> mythology? it's a brand new pool of names...
>>> http://en.wikipedia.org/wiki/Japanese_mythology
>>> 
>>> 
 On Fri, Oct 11, 2013 at 8:29 AM, Blair Zajac  wrote:
 
> On 10/10/2013 10:28 PM, Blair Zajac wrote:
> 
>> On 10/10/2013 08:53 PM, Sean McCully wrote:
>> 
>>> On Thursday, October 10, 2013 08:30:42 PM Blair Jacuzzi wrote:
>>> 
 On 10/10/2013 07:54 PM, Sean McCully wrote:
 
 Hello Cassandra Users,
 
 I've recently created a Cassandra Agent as part of Netflix's Cloud
 Prize
 competition, the submission which I've named Hector is largely based on
 Netflix's Priam. I would be very interested in getting feedback, from
 anyone willing to give Hector (https://github.com/seanmccully/hector) a
 try. I am very interested in seeing if this is something the Cassandra
 Community is interested in using with their Cassandra installs.
>>> 
>>> For one, there's a name conflict with the well known Hector Cassandra
>>> client project:
>>> 
>>> http://hector-client.github.io/hector/build/html/index.html
>> 
>> Any suggestions on a new name?
> 
> Helenus, the twin brother of the prophetess Cassandra???
> 
> http://en.wikipedia.org/wiki/Helenus
 
 Oops, should have Googled myself before suggesting this, they are NodeJS 
 Bindings for Cassandra:
 
 https://github.com/simplereach/helenus
 
 Well, I'll leave it to you to find a free name ;)
 
 Blair
>>> 
>>> 
>>> 
>>> --
>>> /Ran
>>> http://tavory.com
> 

Re: Denial of Service Issue

[Richard Low]

On 11 October 2013 14:03,  wrote:

>  I found the issue below concerning inactive client connections (see 
> *Cassandra
> Security*).
> We are using Cassandra 1.2.4 and the Cassandra JDBC driver as client. Is
> this still an existing issue?
> Quoted from site above:
> Denial of Service problem:
> Cassandra uses a Thread- Per-Client model in its network code. Since
> setting up a connection requires the Cassandra server to start a new thread
> on each connection (in addition to the TCP overhead incurred by the
> network), the Cassandra project recommends utilizing some sort of
> connection pooling. An attacker can prevent the Cassandra server from
> accepting new client connections by causing the Cassandra server to
> allocate all its resources to fake connection attempts. The only pieces of
> information required by an attacker are the IP addresses of the cluster
> members, and this information can be obtained by passively sniffing the
> network. The current implementation doesn’t timeout inactive connections,
> so any connection that is opened without actually passing data consumes a
> thread and a file-descriptor that are never released.
>

 This is still an issue, but you must not expose Cassandra to untrusted
users.  Just like you wouldn't let untrusted users have network access to
your Oracle, MySQL, etc. servers.

Richard.

Re: Denial of Service Issue

[Edward Capriolo]

While you normally would not allow access to a MySQL server, it is done in
many instances like shared hosting.
Also mysql does support a max fail connection attempts feature that will
blacklist an IP for a time.


On Fri, Oct 11, 2013 at 3:37 PM, Richard Low  wrote:

> On 11 October 2013 14:03,  wrote:
>
>>  I found the issue below concerning inactive client connections (see 
>> *Cassandra
>> Security*).
>> We are using Cassandra 1.2.4 and the Cassandra JDBC driver as client. Is
>> this still an existing issue?
>> Quoted from site above:
>> Denial of Service problem:
>> Cassandra uses a Thread- Per-Client model in its network code. Since
>> setting up a connection requires the Cassandra server to start a new thread
>> on each connection (in addition to the TCP overhead incurred by the
>> network), the Cassandra project recommends utilizing some sort of
>> connection pooling. An attacker can prevent the Cassandra server from
>> accepting new client connections by causing the Cassandra server to
>> allocate all its resources to fake connection attempts. The only pieces of
>> information required by an attacker are the IP addresses of the cluster
>> members, and this information can be obtained by passively sniffing the
>> network. The current implementation doesn’t timeout inactive connections,
>> so any connection that is opened without actually passing data consumes a
>> thread and a file-descriptor that are never released.
>>
>
>  This is still an issue, but you must not expose Cassandra to untrusted
> users.  Just like you wouldn't let untrusted users have network access to
> your Oracle, MySQL, etc. servers.
>
> Richard.
>
>>
>

Re: Heap requirement for Off-heap space

[Robert Coli]

On Fri, Oct 11, 2013 at 5:47 AM, Artur Kronenberg <
artur.kronenb...@openmarket.com> wrote:

> I was playing around with cassandra off-heap options. I configured 3 GB
> off-heap for my row cache and 2 GB Heap space for cassandra.
>

The row cache is only really useful in very specific cases involving very
small, hot, and uniform in size datasets. Many more people have shot
themselves in the foot by using it than have had great success.

The off-heap row cache also invalidates on UPDATE, which is significantly
different from what many people expect.


> Is there a formula that maps heap space to off-heap?


FWIW, I'd also like to know the answer to this question.

=Rob

vnode + multi dc migration

[Chris Burroughs]

I know there is a good deal of interest [1] on feasible methods for 
enabling vnodes on clusters that did not start with them.


We recently completed a migration from a production cluster not using 
vnodes and in a single DC to one using vnodes in two DCs.  We used the 
"just spin up a new DC and rebuild" strategy instead of shuffle and it 
worked.  The checklist was long but it really wasn't more complicated 
than that.  Thanks to several people in #cassandra for suggesting the 
technique and reviewing procedures.


One oddity we noticed is that when nodes in a new DC join 
(auto_bootstrap:false) CL.ONE performance tanked [2].  The spike is when 
the nodes came online, and the drop is when reads were switched to 
CL.LOCAL_QUORUM  This only happened when the new DC was cross-continent 
(not a logical DC in the same colo).


[1] 
http://mail-archives.apache.org/mod_mbox/cassandra-user/201308.mbox/%3CCAEDUwd12vhRJbPZpVJ6QzTOx3pwU=11hhgkkipghhgvosbj...@mail.gmail.com%3E


[2] http://i.imgur.com/ZW5Ob8V.png

Re: Cassandra Agent

[Sean McCully]

On Friday, October 11, 2013 09:55:36 AM Edward Capriolo wrote:
> Stick sandra on the end. Restsandra.
> 
Polites, isn't a rest API wrapper for Cassandra really, it does more 
configuration and management of a Cassandra Node. Though it does have a REST 
API



> On Friday, October 11, 2013, Ran Tavory  wrote:
> > Seems like the greeks are all used out, how about moving the the japanese
> 
> mythology? it's a brand new pool of names...
> 
> > http://en.wikipedia.org/wiki/Japanese_mythology
> > 
> > On Fri, Oct 11, 2013 at 8:29 AM, Blair Zajac  wrote:
> >> On 10/10/2013 10:28 PM, Blair Zajac wrote:
> >>> On 10/10/2013 08:53 PM, Sean McCully wrote:
>  On Thursday, October 10, 2013 08:30:42 PM Blair Jacuzzi wrote:
> > On 10/10/2013 07:54 PM, Sean McCully wrote:
> >> Hello Cassandra Users,
> >> 
> >> I've recently created a Cassandra Agent as part of Netflix's Cloud
> >> Prize
> >> competition, the submission which I've named Hector is largely based
> 
> on
> 
> >> Netflix's Priam. I would be very interested in getting feedback, from
> >> anyone willing to give Hector (https://github.com/seanmccully/hector)
> 
> a
> 
> >> try. I am very interested in seeing if this is something the
> 
> Cassandra
> 
> >> Community is interested in using with their Cassandra installs.
> > 
> > For one, there's a name conflict with the well known Hector Cassandra
> > client project:
> > 
> > http://hector-client.github.io/hector/build/html/index.html
>  
>  Any suggestions on a new name?
> >>> 
> >>> Helenus, the twin brother of the prophetess Cassandra???
> >>> 
> >>> http://en.wikipedia.org/wiki/Helenus
> >> 
> >> Oops, should have Googled myself before suggesting this, they are NodeJS
> 
> Bindings for Cassandra:
> >> https://github.com/simplereach/helenus
> >> 
> >> Well, I'll leave it to you to find a free name ;)
> >> 
> >> Blair
> > 
> > --
> > /Ran
> > http://tavory.com
-- 
Sean McCully

Re: Cassandra Agent

[Sean McCully]

On Friday, October 11, 2013 07:17:53 AM David Schairer wrote:
> http://en.wikipedia.org/wiki/List_of_children_of_Priam
> 
> You've got plenty of children of Priam to go around.  Doesn't anyone read
> the Iliad any more?  :)
> 
> --DRS
And so we have Polites,
https://github.com/seanmccully/polites



> 
> On Oct 11, 2013, at 6:55 AM, Edward Capriolo  wrote:
> > Stick sandra on the end. Restsandra.
> > 
> > On Friday, October 11, 2013, Ran Tavory  wrote:
> > > Seems like the greeks are all used out, how about moving the the
> > > japanese mythology? it's a brand new pool of names...
> > > http://en.wikipedia.org/wiki/Japanese_mythology
> > > 
> > > On Fri, Oct 11, 2013 at 8:29 AM, Blair Zajac  wrote:
> > >> On 10/10/2013 10:28 PM, Blair Zajac wrote:
> > >>> On 10/10/2013 08:53 PM, Sean McCully wrote:
> >  On Thursday, October 10, 2013 08:30:42 PM Blair Jacuzzi wrote:
> > > On 10/10/2013 07:54 PM, Sean McCully wrote:
> > >> Hello Cassandra Users,
> > >> 
> > >> I've recently created a Cassandra Agent as part of Netflix's Cloud
> > >> Prize
> > >> competition, the submission which I've named Hector is largely
> > >> based on
> > >> Netflix's Priam. I would be very interested in getting feedback,
> > >> from
> > >> anyone willing to give Hector
> > >> (https://github.com/seanmccully/hector) a
> > >> try. I am very interested in seeing if this is something the
> > >> Cassandra
> > >> Community is interested in using with their Cassandra installs.
> > > 
> > > For one, there's a name conflict with the well known Hector
> > > Cassandra
> > > client project:
> > > 
> > > http://hector-client.github.io/hector/build/html/index.html
> >  
> >  Any suggestions on a new name?
> > >>> 
> > >>> Helenus, the twin brother of the prophetess Cassandra???
> > >>> 
> > >>> http://en.wikipedia.org/wiki/Helenus
> > >> 
> > >> Oops, should have Googled myself before suggesting this, they are
> > >> NodeJS Bindings for Cassandra:
> > >> 
> > >> https://github.com/simplereach/helenus
> > >> 
> > >> Well, I'll leave it to you to find a free name ;)
> > >> 
> > >> Blair
> > > 
> > > --
> > > /Ran
> > > http://tavory.com
-- 
Sean McCully