Re: write performance thrift interfaces

[Martin Probst (RobHost Support)]

Hi Roger,

we've only adjusted the names for the keyspaces and the columnfamilies. This is 
the second perl benchmark code, which switches the node after 100 datasets:

#!/usr/bin/perl

use strict;
use warnings;

use Data::Dumper qw( Dumper );

use Net::Cassandra;

my $host1 = "localhost";
my $host2 = "10.11.12.1";
my $keyspace = "TestCassandra";
my $column = "Customer";
$| = 1;

my $cassandra1 = Net::Cassandra->new( hostname => $host1 );
my $client1    = $cassandra1->client;
my $cassandra2 = Net::Cassandra->new( hostname => $host2 );
my $client2    = $cassandra2->client;

my @stuff;
my $i = 0;
my $c_level = Net::Cassandra::Backend::ConsistencyLevel::ZERO;
while( my $line = <STDIN> )
{

    @stuff = split( /\|/, $line );
    my $ts = time();
    my $client = ( ( $i % 100 ) == 0 ) ? $client2 : $client1;
    
    my $c1 = new Net::Cassandra::Backend::Column( { name => 'NAME', value => 
$stuff[1], timestamp => $ts } );
    my $c2 = new Net::Cassandra::Backend::Column( { name => 'ADDRESS', value => 
$stuff[2], timestamp => $ts } );
    my $c3 = new Net::Cassandra::Backend::Column( { name => 'MAIL', value => 
$stuff[3], timestamp => $ts } );
    my $c4 = new Net::Cassandra::Backend::Column( { name => 'CONTACT', value => 
$stuff[4], timestamp => $ts } );
    my $c5 = new Net::Cassandra::Backend::Column( { name => 'TEL', value => 
$stuff[5], timestamp => $ts } );
    my $c6 = new Net::Cassandra::Backend::Column( { name => 'WEB', value => 
$stuff[6], timestamp => $ts } );

    my $cs1 = new  Net::Cassandra::Backend::ColumnOrSuperColumn( { column => 
$c1 } );
    my $cs2 = new  Net::Cassandra::Backend::ColumnOrSuperColumn( { column => 
$c2 } );
    my $cs3 = new  Net::Cassandra::Backend::ColumnOrSuperColumn( { column => 
$c3 } );
    my $cs4 = new  Net::Cassandra::Backend::ColumnOrSuperColumn( { column => 
$c4 } );
    my $cs5 = new  Net::Cassandra::Backend::ColumnOrSuperColumn( { column => 
$c5 } );
    my $cs6 = new  Net::Cassandra::Backend::ColumnOrSuperColumn( { column => 
$c6 } );
    
    my %mutation = ( $column => [$cs1, $cs2, $cs3, $cs4, $cs5, $cs6]);

    $client->batch_insert(
        $keyspace,
        $stuff[0],
        \%mutation,
        $c_level,
    );

$i++;
print "insert element $i\r";

}

exit(0);

Thanks in advance,
Martin

Am 18.03.2010 um 19:08 schrieb Roger Schildmeijer:

> Yes, 30 writes / s sounds a little bit poor. 
> 
> Maybe you could show your benchmark code? And what adjustments had to be done 
> to the CF?
> 
> // Roger
> 
> 
> On 18 mar 2010, at 19.03em, Martin Probst (RobHost Support) wrote:
> 
>> Hi,
>> 
>> we've tested the write performance on a single and dual node cluster and the 
>> results are strangely poor. We've got about 30 inserts per second which 
>> seems a little bit slow?! The strange about is, that the node's we've used 
>> (single-cpu, 3gb ram, single disk) got a load of 0.02-0.05 while the inserts 
>> run. There is no disk/network bottlenecks, no swapping. The insert-script 
>> reads some values from stdin and inserts this values into cassandra. So it 
>> seems thats the used thrift interface are the bootleneck (we've tested php, 
>> perl, java, results a nearly the same). We've used version 0.5.1 and the 
>> default config with adjusted ColumnFamilies.
>> 
>> Are there some hints or suggestions?
>> 
>> Cheers,
>> Martin
>