I am just wondering, why the stress test tools (python, java) need more
threads ?
Is the bottleneck of a single thread in the client, or in the server?
Thanks.
Sean
2011/3/22 Ryan King
> On Mon, Mar 21, 2011 at 4:02 AM, pob wrote:
> > Hi,
> > I'm inserting data from client node with stress.py
Hi,
Is there documentation available anywhere that describes how one can
use org.apache.cassandra.security.streaming.* ? After the EC2 posts
yesterday, one question I was asked was about the security of data
being shifted between nodes. Is it done in clear text, or
encrypted..? I haven't seen
A client thread need to wait for response, during the server can
handle multiple requests simultaneously.
2011/3/22 Sheng Chen :
> I am just wondering, why the stress test tools (python, java) need more
> threads ?
> Is the bottleneck of a single thread in the client, or in the server?
> Thanks.
>
aaron morton wrote:
>
>
> Also a node is be responsible for storing it's token range and acting as a
> replica for other token ranges. So reducing the token range may not have a
> dramatic affect on the storage requirements.
>
Aaron,
is there a way to configure wimpy nodes such that the repl
On Mar 22, 2011, at 5:09 AM, aaron morton wrote:
> 1) You should use nodes with the same capacity (CPU, RAM, HDD), cassandra
> assumes they are all equal.
Care to elaborate? While equal node will certainly make life easier I would
have thought that dynamic snitch would take care of performan
Not that I know of.
Aaron
On 22/03/2011, at 10:45 PM, buddhasystem wrote:
>
> aaron morton wrote:
>>
>>
>> Also a node is be responsible for storing it's token range and acting as a
>> replica for other token ranges. So reducing the token range may not have a
>> dramatic affect on the storage
My assumption is from not seeing anything in the code to explicitly support
nodes of different specs (also think I saw it somewhere ages ago). AFAIK the
dynamic snitch is there to detect nodes with a temporarily reduced throughput
and try to reduce the read load on them.
I may be wrong on this
Should be: not that I know of without making code changes.
Aaron
On 22/03/2011, at 11:05 PM, Aaron Morton wrote:
> Not that I know of.
> Aaron
>
> On 22/03/2011, at 10:45 PM, buddhasystem wrote:
>
>>
>> aaron morton wrote:
>>>
>>>
>>> Also a node is be responsible for storing it's token ra
According to the Wiki Page on compaction: once compaction is finished, the old
SSTable files may be deleted*
* http://wiki.apache.org/cassandra/MemtableSSTable
I thought the old SSTables would be deleted automatically, but this wiki page
got me thinking otherwise.
Question is, if it is true
Hi,
> contrib/py_stress is the easiest way to shake out any issues with your
> install and get a benchmark.
> There is also https://github.com/brianfrankcooper/YCSB but I would go with
> py_stress until it stops been useful.
Very good, thank's.. !
> Note: These are abstract benchmarks to be use
It seems some settings like memtable_throughput_in_mb are Keyspace-specific
(at least with 0.7.4).
How can these settings best be changed on a running cluster?
PS - preferable by a sysadmin using nodetool or cassandra-cli
Thanks!
Jon
Thanks Milind for sharing!
As Sasha already asked, ec2 sends data across regions over the
internet without any encryption. So you may consider to tunnel the traffic
thru ssh.
I don't know how to do that with cassandra. Any?
Regards, mike
On Tue, Mar 22, 2011 at 5:29 AM, Milind Parikh wrote:
>
Milind,
Thank you for attaching the patch here, but it would be really nice if you
could create a jira account so you could participate in the discussion on the
ticket and put the patch on there - that is the way people license their
contributions with the apache 2 license. You just need to cr
Force a GC to remove the unused sstables. Use something like jconsole or cmd
line "jmap -histo:live ". You would run the jmap command as the
cassandra user or root. The jmap will give you a bunch of output on live
objects in the heap if you choose to look at it.
Sridhar
On Tue, Mar 22, 2011 at 8
Hi Everyone,
Can the Ec2Snitch be enabled by adjusting the parameter in the
cassandra.yaml and restarting the node?
More, I suppose the question I'm after is, can the snitch method be
adjusted adhoc (with node restart) or once it's changed from
SimpleSnitch to Ec2Snitch that's it? What influence
>From the next paragraph of the same wiki page:
SSTables that are obsoleted by a compaction are deleted asynchronously
when the JVM performs a GC. You can force a GC from jconsole if
necessary, but Cassandra will force one itself if it detects that it
is low on space. A compaction marker is also a
doooh. thanks!
On Mar 22, 2011, at 3:27 PM, Jonathan Ellis wrote:
> From the next paragraph of the same wiki page:
>
> SSTables that are obsoleted by a compaction are deleted asynchronously
> when the JVM performs a GC. You can force a GC from jconsole if
> necessary, but Cassandra will force on
I see now that this is too new:
https://issues.apache.org/jira/browse/CASSANDRA-1567 and that it's
scheduled for the 0.8 release.
Is it right to assume the following from the accepted patch:
1. keystore and truststore passwords are kept in clear text in the
cassandra.yaml ?
2. It's all or nothi
Hi -
On our recently live cassandra cluster of 5 nodes, we've noticed that the
latency readings, especially Reads have gone up drastically.
TotalReadLatencyMicros 5413483
TotalWriteLatencyMicros 1811824
I understand these are in microseconds, but what meaning do they have for
Just as what it named, it's the total microseconds spent on read operations
so far.
Not average.
Regards,
Chen
Senior Developer, EvidentSoftware(Leaders in Monitoring of NoSQL & JAVA )
http://www.evidentsoftware.com
On Tue, Mar 22, 2011 at 11:11 AM, Jonathan Colby
wrote:
> Hi -
>
> On our re
On Mon, Mar 21, 2011 at 8:33 PM, A J wrote:
> I am trying to estimate the time it will take to rebuild a node. After
> loading reasonable data,
> ...
> For some reason, the repair command runs forever. I just have 3G of
> data per node but still the repair is running for more than an hour !
What
0.7.4
On Tue, Mar 22, 2011 at 11:49 AM, Robert Coli wrote:
> On Mon, Mar 21, 2011 at 8:33 PM, A J wrote:
>> I am trying to estimate the time it will take to rebuild a node. After
>> loading reasonable data,
>> ...
>> For some reason, the repair command runs forever. I just have 3G of
>> data per
On Tue, Mar 22, 2011 at 7:19 AM, Sasha Dolgy wrote:
> More, I suppose the question I'm after is, can the snitch method be
> adjusted adhoc (with node restart) or once it's changed from
> SimpleSnitch to Ec2Snitch that's it?
You can change Snitches on a cluster with data on it, as long as you
are
> I may be wrong on this, so anyone else feel free to jump in. Here are some
> issues to consider...
>
> - keyspace memory requirements are global, all nodes must have enough memory
> to support the CFs.
> - During node moves, additions or deletions the token range may increase,
> nodes with les
Thanks for the good response.
my thought was as aws becomes more and more expensive (no option to swap out
small cheap disks for larger cheap disks...) i'll need to switch to
dedicated hardware and the topology will change. didnt want to back myself
into a corner early on when the amount of data
On Tue, Mar 22, 2011 at 8:53 AM, A J wrote:
> 0.7.4
>
> On Tue, Mar 22, 2011 at 11:49 AM, Robert Coli wrote:
>> On Mon, Mar 21, 2011 at 8:33 PM, A J wrote:
>>> I am trying to estimate the time it will take to rebuild a node. After
>>> loading reasonable data,
http://issues.apache.org/jira/brows
Never mind - I had thought it was more generalizable but since it's just going
against the public IP between regions, that's not going to be something that
makes it into trunk. I had just wanted to see if there was a way that it could
be done, but it sounds like since amazon doesn't provide dec
Thanks. The problem is intermittent meaning we have separate CA cluster
environments: In some cases there is no problem running a snapshot followed
by a later clear snapshot (or for that matter physical delete of the
snapshot file).
When I stop Cassandra the snapshot file can be deleted. As soon as
Milind,
Among the limitation you might want to add that 'nodetool repair' does
not work with this patch.
I tried several times and the repair hangs.
When I run it directly on the trunk of 0.7.4 (without the patch) it
completes successfully within reasonable time.
Thanks.
On Tue, Mar 22, 2011 at 1
there are some other knock on issues too. the SSL work that has been
done would also have to be changed ...
-sd
On Tue, Mar 22, 2011 at 6:58 PM, A J wrote:
> Milind,
> Among the limitation you might want to add that 'nodetool repair' does
> not work with this patch.
> I tried several times and
On Tue, Mar 22, 2011 at 12:23 PM, Peter Schuller
wrote:
>> I may be wrong on this, so anyone else feel free to jump in. Here are some
>> issues to consider...
>>
>> - keyspace memory requirements are global, all nodes must have enough memory
>> to support the CFs.
>> - During node moves, additio
> Wait! maybe this is a quadruple-whammy since we have to account for
> the data being replicated to other nodes. At replication factor 3 only
> 1/3rd of the data on the node actually belongs in that TokenRange, So
> it is not as simple as having small nodes with smaller ranges, you
> also have to
Actually I had modified the source code (to put a patch for cassandra
to work across EC2 regions). That patch seems to be causing issue with
'nodetool repair'
When I run without the patch (and within an ec2 region), the repair
completes within reasonable time.
On Tue, Mar 22, 2011 at 12:40 PM, Rob
I know it has zero utility, but I think it has a tremendous coolness and
propaganda value -- has anyone tried to run cassandra on a recent generation
cell phone/tablet? Or a cluster of these ;)
--
View this message in context:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cas
What is the process of changing the IP address for a node in a cluster?
Casey
i'd +1 if it were on a PS3 ...
On Tue, Mar 22, 2011 at 7:53 PM, buddhasystem wrote:
> I know it has zero utility, but I think it has a tremendous coolness and
> propaganda value -- has anyone tried to run cassandra on a recent generation
> cell phone/tablet? Or a cluster of these ;)
Hello
I have 3 node cluster. But i think that something wrong happens on one of
this nodes. On two nodes cache uses all system memory and disc usage is
small, but on one node cash doesn't use all memory and top utility shows
about 2GB of free memory, and disk usage on that node is more intensive.
I have been going through the mailing list and compiling suggestions to
address the swapping due to mmap issue.
1) Use JNA (done but)
Are these steps also required:
- Start Cassandra with CAP_IPC_LOCK (or as "root"). (not done)
grep Unevictable /proc/meminfo
- set /proc/sys/vm/swappiness to 0
2011/3/22 Adi
> I have been going through the mailing list and compiling suggestions to
> address the swapping due to mmap issue.
>
> 1) Use JNA (done but)
> Are these steps also required:
> - Start Cassandra with CAP_IPC_LOCK (or as "root"). (not done)
>
And what is CAP_IPC_LOCK?
On Tue, Mar 22, 2011 at 3:44 PM, ruslan usifov wrote:
>
>
> 2011/3/22 Adi
>
>> I have been going through the mailing list and compiling suggestions to
>> address the swapping due to mmap issue.
>>
>> 1) Use JNA (done but)
>> Are these steps also required:
>> - Start Cassandra with CAP_IPC_LOCK (o
The easiest way to get memlock to work after putting the jna jar in your
classpath is just run this before:
ulimit -a unlimited
in your init script or before starting cassandra. The default for max locked
memory is 32KB on older kernels, and 64KB on newer ones.
-Chris
On Mar 22, 2011, at 12:5
On Tue, Mar 22, 2011 at 2:54 PM, Sasha Dolgy wrote:
> i'd +1 if it were on a PS3 ...
>
> On Tue, Mar 22, 2011 at 7:53 PM, buddhasystem wrote:
>> I know it has zero utility, but I think it has a tremendous coolness and
>> propaganda value -- has anyone tried to run cassandra on a recent generation
I probably could have saved myself some time by saying (as Peter and Edward
pointed out) "if you use nodes with different capabilities you will need treat
all nodes as having the lowest spec and that could be a waste." :)
Aaron
On 23 Mar 2011, at 07:26, Peter Schuller wrote:
>> Wait! maybe th
Sounds interesting, please let the community know your findings.
Aaron
On 23 Mar 2011, at 01:31, FernandoVM wrote:
> Hi,
>
>> contrib/py_stress is the easiest way to shake out any issues with your
>> install and get a benchmark.
>> There is also https://github.com/brianfrankcooper/YCSB but I w
They are memtable settings are Column Family specific and can be changed via
the cassandra-cli, see the help in cassandra-cli "help update column family;"
Aaron
On 23 Mar 2011, at 01:53, Jonathan Colby wrote:
> It seems some settings like memtable_throughput_in_mb are Keyspace-specific
> (at
AFAIK upgrade from 0.6.2 to 0.6.12 should be a straight forward rolling
restart. Do check the changes.txt file first and if you have a test env test it
there. (The large gap in versions makes me a little nervous).
If you feel it's reproducible (even sometimes) can you create a jira ticket ?
ht
On Sun, Mar 20, 2011 at 4:42 PM, aaron morton wrote:
> When compacting it will use the path with the greatest free space. When
> compaction completes successfully the files will lose their temporary status
> and that will be their new home.
>
> On 18 Mar 2011, at 14:10, John Lewis wrote:
>
>> |
Jonathan,
for all of us just tinker with test clusters, building confidence in the
product, it would be nice to be able to do same with nodetool, without
jconsole, just my 0.5 penny. Thanks.
Jonathan Ellis-3 wrote:
>
> From the next paragraph of the same wiki page:
>
> SSTables that are obsol
Hi,
I've seen some strange occurrence of a deleted node reappearing all of
a sudden in the ring, which leads to my question: where is the ring
structure maintained (memory with local copies?) and what prompts it
to change? I appreciate any thoughts on the events below.
I'm running 0.7.4 on 4 EC2
One machine cluster, low load, 0.7.2
INFO 18:22:31,155 reading saved cache
/data1/cassandra_data/saved_caches/system-Schema-KeyCache
WARN 18:22:31,155 error reading saved cache
/data1/cassandra_data/saved_caches/system-Schema-KeyCache
java.io.EOFException
at
java.io.ObjectInputStream$Pee
Hi Sasha
Thank you so much for your advice.
I changed JMX_PORT from 10036 to 8080 in cassandra-env.sh.
Now nodetool ring is working as following.
# nodetool --host **.**.254.54 ring
Address Status State LoadOwnsToken
I'm writing a row with about 45k columns. Most of them are quite small, and
there are a few of 2 MB and one of 5 MB. The write procedure times out.
Total data load is 9 MB.
What would be the cause?
--
View this message in context:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com
This is a known (harmless) bug in 0.7.2. You should upgrade so you
don't run into bugs that have been already fixed.
On Tue, Mar 22, 2011 at 6:24 PM, buddhasystem wrote:
> One machine cluster, low load, 0.7.2
>
> INFO 18:22:31,155 reading saved cache
> /data1/cassandra_data/saved_caches/system-
"I'm writing a row with about 45k columns."
On Tue, Mar 22, 2011 at 7:39 PM, buddhasystem wrote:
> I'm writing a row with about 45k columns. Most of them are quite small, and
> there are a few of 2 MB and one of 5 MB. The write procedure times out.
> Total data load is 9 MB.
>
> What would be the
Hi there,
Installed a new 4 node 0.7.4 cluster on ec2. Brought up the first
node without issue with Ec2Snitch configured in the cassandra.yaml.
Brought up a second node, with the first node defined as the seed. No
visible issues. 3 & 4 however are giving me problems as shown in the
output belo
Jonathan, wide rows have been discussed. I thought that the limit on number
of columns is way bigger than 45k. What can one expect in reality?
--
View this message in context:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/0-7-2-choking-on-a-5-MB-column-tp6198387p6198548.html
Se
On Tue, Mar 22, 2011 at 8:59 PM, buddhasystem wrote:
> Jonathan, wide rows have been discussed. I thought that the limit on number
> of columns is way bigger than 45k. What can one expect in reality?
>
> --
> View this message in context:
> http://cassandra-user-incubator-apache-org.3065146.n2.na
How do you define your Keyspace?
As you may know, in Cassandra, replication (factor) is defined as the
attribute of Keyspace.
And what do you mean:
> However replication never happened.
> I can't get data I set at other node.
What did you do on cassandra, and what did you get in response?
maki
I see. I'm doing something even more drastic then, because I'm only inserting
one row in this case, and just use cf.insert(), without batch mutator. It
didn't occur to me that was a bad idea.
So I take it, this method will fail. Hmm.
--
View this message in context:
http://cassandra-user-incuba
@aj
are you sure that all ports are accessible from all node?
@sasha
I think that being able to have the semantics of address aNAT address can
emable security from different perspective. Describing an overlay nw will
take long hete. But that may solve your security concerns over the internet.
/*
60 matches
Mail list logo
