unsubscribe
--
http://about.me/BrianTarbox
--
http://about.me/BrianTarbox
My group is seeing the same thing and also can not figure out why its
happening.
On Mon, Mar 23, 2015 at 8:36 AM, Anishek Agarwal wrote:
> Forgot to mention I am using Cassandra 2.0.13
>
> On Mon, Mar 23, 2015 at 5:59 PM, Anishek Agarwal
> wrote:
>
>> Hello,
>>
>> I am using a single node serv
C* seems to have more than its share of "version x doesn't work, use
version y " type issues
On Thu, Jan 8, 2015 at 2:23 PM, Robert Coli wrote:
> On Thu, Jan 8, 2015 at 11:14 AM, Roni Balthazar
> wrote:
>
>> We are using C* 2.1.2 with 2 DCs. 30 nodes DC1 and 10 nodes DC2.
>>
>
> https://eng
We're running into a problem where things are fine if our client runs
single threaded but gets TransportException if we use multiple threads.
The datastax driver gets an NIO checkBounds error.
Here is a link to a stack overflow question we found that describes the
problem we're seeing. This quest
te-then-read outside of tests, you
> should be aware that it's a Bad Idea™, but your email reads like you
> already know that =)
>
> On Thu Nov 06 2014 at 7:16:25 AM Brian Tarbox
> wrote:
>
>> We're doing development on a single node cluster (and yes of course w
set the consistency level to ALL with no effect.
I've read CASSANDRA-876 which seems spot-on but it was closed as
won't-fix...and I don't see what the solution is.
Thanks in advance for any help.
Brian Tarbox
--
http://about.me/BrianTarbox
The last guidance I heard from DataStax was to use m2.2xlarge's on AWS and
put data on the ephemeral drivehave they changed this guidance?
Brian
On Tue, Aug 19, 2014 at 9:41 AM, Oleg Dulin wrote:
> Distinguished Colleagues:
>
> Our current Cassandra cluster on AWS looks like this:
>
> 3 no
We're considering a C* setup with very large columns and I have a question
about the details of read.
I understand that a read request gets handled by the coordinator which
sends read requests to of the nodes holding replicas of the data,
and once nodes have replied with consistent data it is re
anks,
On Tue, Jul 1, 2014 at 2:57 PM, Robert Coli wrote:
> On Tue, Jul 1, 2014 at 11:54 AM, Brian Tarbox
> wrote:
>
>> Given that an upgrade is (for various internal reasons) not an option at
>> this point...is there anything I can do to get repair working again? I
at 11:09 AM, Brian Tarbox
> wrote:
>
>> We're running 1.2.13.
>>
>
> 1.2.17 contains a few streaming fixes which might help.
>
>
>> Any chance that doing a rolling-restart would help?
>>
>
> Probably not.
>
>
>> Would running wit
oolExecutor.getTask(ThreadPoolExecutor.java:1068)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
On Tue, Jul 1
We're running 1.2.13.
Any chance that doing a rolling-restart would help?
Would running without the "-pr" improve the odds?
Thanks.
On Tue, Jul 1, 2014 at 1:40 PM, Robert Coli wrote:
> On Tue, Jul 1, 2014 at 9:24 AM, Brian Tarbox
> wrote:
>
>> I have a six no
I have a six node cluster in AWS (repl:3) and recently noticed that repair
was hanging. I've run with the "-pr" switch.
I see this output in the nodetool command line (and also in that node's
system.log):
Starting repair command #9, repairing 256 ranges for keyspace dev_a
but then no other outp
I don't think I have the space to run a major compaction right now (I'm
above 50% disk space used already) and compaction can take extra space I
think?
On Wed, Jun 18, 2014 at 3:24 PM, Robert Coli wrote:
> On Wed, Jun 18, 2014 at 12:05 PM, Brian Tarbox
> wrote:
>
>>
ocess on a rolling basis if I understand you.
Thanks,
Brian
On Wed, Jun 18, 2014 at 2:37 PM, Robert Coli wrote:
> On Wed, Jun 18, 2014 at 10:56 AM, Brian Tarbox
> wrote:
>
>> I have a column family that only stores the last 5 days worth of some
>> data...and yet I have
have
been deleted but were not.
The files are tens or hundreds of gigs so deleting would be good, unless
its really bad!
Thanks,
Brian Tarbox
sk on a separate volume.
Thanks.
On Wed, Jun 18, 2014 at 12:03 PM, Jeremy Jongsma
wrote:
> One option is to add new nodes, and do a node repair/cleanup on
> everything. That will at least reduce your per-node data size.
>
>
> On Wed, Jun 18, 2014 at 11:01 AM, Brian Tarbox
>
tions require
double the space of one's data.
Since I can't change the size of attached storage for my instance type my
question is can I somehow get these maintenance operations to use other
volumes?
Failing that, what are my options? Thanks.
Brian Tarbox
I have a composite key consisting of: (integer, bytes) and I have rows like:
(1,abc), (1,def), (2,abc), (2,def) and I want to find all rows with the
integer part = 2.
I need to create a startBeyondName using CompositeType.Builder class and am
wondering if specifying (2, Bytes.Empty) will sort corr
the other nodes
went right back into the "dropping messages" state.
Help please.
Brian
On Mon, Mar 24, 2014 at 10:01 AM, Brian Tarbox wrote:
> I'm getting "messages dropped" messages in my cluster even when (like
> right now) there are no clients running against t
I'm getting "messages dropped" messages in my cluster even when (like right
now) there are no clients running against the cluster.
1) who could be generating the traffic if there are no clients?
2) is there a way to list active clients...on the off chance that there is
a client I don't know about?
u, Mar 20, 2014 at 3:07 PM, Robert Coli wrote:
>
>> On Thu, Mar 20, 2014 at 1:03 PM, Brian Tarbox
>> wrote:
>>
>>> Does this still apply since we're using 1.2.13? (should have said that
>>> in the original message)>
>>>
>>
>> I check
Does this still apply since we're using 1.2.13? (should have said that in
the original message)>
Thank you.
On Thu, Mar 20, 2014 at 3:57 PM, Robert Coli wrote:
> On Thu, Mar 20, 2014 at 12:31 PM, Brian Tarbox
> wrote:
>
>> I've seen this problem with
as a
means of picking servers is almost always liable to death spirals when a
server can have a failure.
Is there any way to configure away from this in C*?
Thanks,
Brian Tarbox
I have a six node cluster (running m2-2xlarge instances in AWS) with RF=3
and I'm seeing two of the six nodes reporting lots of dropped messages.
The six machines are identical (created from same AWS AMI) so this local
behavior has me puzzled.
BTW this is mostly happening when I'm reading via sec
gt; otherwise the servers still try to read the data from all other data
>>> centers.
>>>
>>> I can understand the latency, but I cant understand how it would save
>>> money? The amount of data transferred from the AWS server to the client
>>> should be
in its own zone would help?
Thank you,
Brian Tarbox
A vaguely related question...my OpsCenter now has two separate tabs for the
same cluster...one tab shows all six nodes and has their agents...the other
tab has the same six nodes but no agents.
I see no way to get rid of the spurious tab.
On Thu, Jan 23, 2014 at 12:47 PM, Ken Hancock wrote:
> M
ated. Not sure if that issue was fixed in 2.0.4, I'm
> avoiding altering tables completely for now.
>
> __
> Sent from iPhone
>
> On 22 Jan 2014, at 7:50 am, Brian Tarbox wrote:
>
> We're trying to use CompositeTypes and Secondary indexe
they shouldn't have an interaction.
Its also puzzling to us that ExtendedFilter asserts in this case...if it
find no columns I would have expected an empty return but not a failure
(that our client code saw as a Timeout exception).
Any clues would be appreciated.
Thanks,
Brian Tarbox
servers just in case but figured I'd ask the experts
first in case I'm doing something foolish.
Thanks,
Brian Tarbox
features we should really consider?
would be most appreciated.
Thank you,
Brian Tarbox
The advice I heard at the New York C* conference...which we follow is to
use the m2.2xlarge and give it about 8 GB. The m2.4xlarge seems overkill
(or at least over price).
Brian
On Fri, Aug 23, 2013 at 6:12 PM, David Laube wrote:
> Hi All,
>
> We are evaluating our JVM heap size configuration
We run a cluster in EC2 and it's working very well for us. The standard
seems to be M2.2XLarge instances with data living on the ephemeral drives
(which means its local and fast) and backups either to EBS, S3 or just
relying on cluster size and replication (we avoid that last idea).
Brian
On Su
Odd that this discussion happens now as I'm also getting this error. I get
a burst of error messages and then the system continues...with no apparent
ill effect.
I can't tell what the system was doing at the timehere is the stack.
BTW Opscenter says I only have 4 or 5 SSTables in each of my 6
Perhaps I should already know this but why is running a major compaction
considered so bad? We're running 1.1.6.
Thanks.
On Thu, Jul 11, 2013 at 7:51 AM, Takenori Sato wrote:
> Hi,
>
> I think it is a common headache for users running a large Cassandra
> cluster in production.
>
>
> Running a
Have to disagree with the "does no harm" comment just a tiny bit. I had a
similar situation recently and coincidentally needed to do a CF truncate.
The system rejected the request saying that not all nodes were up.
Nodetool ring said everyone was up but nodetool gossipinfo said there were
vestig
>From this list and the NYC* conference it seems that the consensus
configuration of C* on EC2 is to put the data on an ephemeral drive and
then periodically back it the drive to S3...relying on C*'s inherent fault
tolerance to deal with any data loss.
Fine, and we're doing this...but we find that
ms odd for only about 300 gig total data).
Anyway, any suggestions for monitoring / speeding up cleanup would be
appreciated.
Brian Tarbox
/budget on it
and so now can't attend SFSummit25. NYC* was great but I feel that I got
misled...or did I misunderstand somehow???
Brian Tarbox
On Fri, Apr 12, 2013 at 11:50 AM, Jonathan Ellis wrote:
> Hi all,
>
> Last year's Summit saw fantastic talks [1] and over 800
I think we all go through this learning curve. Here is the answer I gave
last time this question was asked:
The output of this command seems to make no sense unless I think of it as 5
completely separate histograms that just happen to be displayed together.
Using this example output should I rea
Last week I attended DataStax's NYC* conference and one of the give-aways
was a wooden USB stick. Finally getting around to loading it I find it
empty.
Anyone else have this problem? Are the conference presentations available
somewhere else?
Brian Tarbox
> On 12/02/2013, at 10:04 AM, Andrey Ilinykh wrote:
>
> You have to use private IPs, but if an instance dies you have to bootstrap
> it with replace token flag. If you use EC2 I'd recommend Netflix's Priam
> tool. It manages all that stuff, plus you have S3 backup.
>
addresses can change out
from under you?
Thanks.
Brian Tarbox
At what level will the NY talks be? I had been planning on attending
Datastax's big summer conference and I might not be able to get approval
for bothso I'd like to hear more about this one.
On Wed, Jan 30, 2013 at 12:40 PM, Jonathan Ellis wrote:
> ApacheCon North America (Portland, Feb 26
I've heard that on Amazon EC2 I should be using ephemeral drives...but I
want/need to be using encrypted volumes.
On my local machine I use cryptsetup to encrypt a device and then mount it
and so on...but on Amazon I get the error:
"Cannot open device /dev/xvdb for read-only access".
Reading fur
stumble
> across that page? Just saying that the explanation of the very powerful
> nodetool commands should be more front and center.
>
> Brian
>
>
> On Tue, Jan 22, 2013 at 10:03 AM, Edward Capriolo
> wrote:
>
> This was described in good detail here:
>
> http://t
ail here:
>
> http://thelastpickle.com/2011/04/28/Forces-of-Write-and-Read/
>
> On Tue, Jan 22, 2013 at 9:41 AM, Brian Tarbox wrote:
>
>> Thank you! Since this is a very non-standard way to display data it
>> might be worth a better explanation in the various online docu
Thank you! Since this is a very non-standard way to display data it might
be worth a better explanation in the various online documentation sets.
Thank you again.
Brian
On Tue, Jan 22, 2013 at 9:19 AM, Mina Naguib wrote:
>
>
> On 2013-01-22, at 8:59 AM, Brian Tarbox wrote:
The output of this command seems to make no sense unless I think of it as 5
completely separate histograms that just happen to be displayed together.
Using this example output should I read it as: my reads all took either 1
or 2 sstable. And separately, I had write latencies of 3,7,19. And
separ
ow?
Any suggestions how to proceed are appreciated.
Thanks.
Brian Tarbox
the reads to fetch metric data.
>
> Normally the number of reads required to view metrics are small enough
> that they only make a minor difference in your overall read latency
> average, but when you have no other reads occurring, they're the only reads
> that are included in the average.
&
it
stays up even after all the reads are done. At last glance I've had next
to zero reads for 10 minutes but still have a read request latency thats
basically unchanged from when there were actual reads.
How am I to interpret this?
Thanks.
Brian Tarbox
doing this I've turned off all of the
programs I know of.
But, DataStax still saying there are clients running against us...how can I
find them?
Thanks.
Brian Tarbox
paction. This is especially true if your reads are fairly random and
> don’t focus on a single, hot dataset."
>
> From: Brian Tarbox
> Reply-To: "user@cassandra.apache.org"
> Date: Monday, January 7, 2013 12:56 PM
> To: "user@cassandra.apache.org"
> Su
on
> http://www.thelastpickle.com
>
> On 8/01/2013, at 9:13 AM, Michael Kjellman
> wrote:
>
> Size tiered or leveled compaction?
>
> From: Brian Tarbox
> Reply-To: "user@cassandra.apache.org"
> Date: Monday, January 7, 2013 12:03 PM
> To: "user@cass
I have not specified leveled compaction so I guess I'm defaulting to size
tiered? My data (in the column family causing the trouble) insert once,
ready many, update-never.
Brian
On Mon, Jan 7, 2013 at 3:13 PM, Michael Kjellman wrote:
> Size tiered or leveled compaction?
>
> From
I have a column family where I'm doing 500 inserts/sec for 12 hours or so
at time. At some point my performance falls off a cliff due to time spent
doing compactions.
I'm seeing row after row of logs saying that after 1 or 2 hours of
compactiing it reduced to 100% of 99% of the original.
I'm try
hat can I check next?
Thanks!
Brian Tarbox
Anyone still use Pelops?
On Sun, Dec 30, 2012 at 12:19 PM, Shahryar Sedghi wrote:
> I use JDBC with Cassandra 1.1 with CQL 3. I tried both Hector and Thrift
> and JDBC is much easier to code, I never tried Astyanax. Application
> servers have built-in connection pooling support for JDBC, but d
error is
>
> [default@cf] get
> token['fbc1e9f7cc2c0c2fa186138ed28e5f691613409c0bcff648c651ab1f79f9600b'];
> => (column=client_id, value=8ec4c29de726ad4db3f89a44cb07909c04f90932d,
> timestamp=1355836425784329, ttl=648000)
> A long is exactly 8 bytes: 10
>
>
>
>
> -Bryan
>
>
>
> really hard and have no issues whatsoever.
>>
>>
> A related question is "which which version of java 7 did you try"? The
> first releases of java 7 were apparently famous for having many issues but
> it seems the more recent updates are much more stable.
>
> --
We've reverted all machines back to Java 6 after running into numerous Java
7 issues...some running Cassandra, some running Zookeeper, others just
general problems. I don't recall any other major language release being
such a mess.
On Fri, Dec 14, 2012 at 5:07 PM, Bill de hÓra wrote:
> "At lea
Can you supply your java parameters?
On Mon, Nov 12, 2012 at 7:29 AM, Everton Lima wrote:
> Hi people,
>
> I was using cassandra on distributed project. I am using java 6 and
> cassandra 1.1.6. My problem is in Memory manager (I think). My system was
> throwing heap limit exception.
> The problem
ions would be appreciated, thanks.
Brian Tarbox
I got some new ubuntu servers to add to my cluster and found that the file
system is "fuseblk" which really means NTFS.
All else being equal would I expect to get any performance boost if I
converted the file system to EXT4? Edward Capriolo's "Cookbook" book seems
to su
I had a two node cluster that I expanded to four nodes. I ran the token
generation script and moved all the nodes so that when I run "nodetool
ring" each node reports 25% Effective-Ownership.
However, my load numbers map out to 39%, 30%, 15%, 17%.
How can that be?
Thanks.
I have a two node cluster hosting a 45 gig dataset. I periodically have to
read a high fraction (20% or so) of my 'rows', grabbing a few thousand at a
time and then processing them.
This used to result in about 300-500 reads a second which seemed quite
good. Recently that number has plummeted to
I can't imagine why this would be a problem but I wonder if anyone has
experience with running a mix of 32 and 64 bit nodes in a cluster.
(I'm not going to do this in production, just trying to make use of the
gear I have for my local system).
Thanks.
70 matches
Mail list logo
