On Wed, Nov 5, 2014 at 11:00 PM, Jimmy Lin wrote:
> Sorry I have late follow up question
>
> In the Cassandra.yaml file the concurrent_read section has the following
> comment:
>
> What does it mean by " the operations to enqueue low enough in the stack
> that the OS and drives can reorder t
On Mon, Nov 3, 2014 at 7:44 AM, Sebastian Martinka <
sebastian.marti...@mercateo.com> wrote:
> System and Keyspace Information:
>
> 4 Nodes
>
>
> CREATE KEYSPACE restore_test WITH replication = { 'class':
> 'SimpleStrategy',
>
> 'replication_factor': '3'};
>
>
>
>
>
> I assumed, that a flush
On Tue, Oct 28, 2014 at 9:02 AM, Adria Arcarons <
adria.arcar...@greenpowermonitor.com> wrote:
> Hi,
>
> Hi
>
>
> We have about 50.000 CFs of varying size
>
>
>
>
> The writing test consists of a continuous flow of inserts. The inserts are
> done inside BATCH statements in groups of 1.000 t
With a 4.5 TB table and just 4 nodes, repair will likely take forever for
any version.
-Bryan
On Fri, Sep 26, 2014 at 10:40 AM, Jonathan Haddad wrote:
> Are you using Cassandra 2.0 & vnodes? If so, repair takes forever.
> This problem is addressed in 2.1.
>
> On Fri, Sep 26, 2014 at 9:52 AM,
On Mon, May 19, 2014 at 6:39 AM, mahesh rajamani
wrote:
> Sorry I just realized the table name in 2 schema are slightly different,
> but still i am not sure why i should not use same index name across
> different schema. Below is the instruction to reproduce.
>
>
> Created 2 keyspace using cassand
he guidelines don’t mention setting noatime and
> nodiratime flags in the fstab for data volumes, but I wonder if that’s a
> common practice.
>
> James
>
>
>
>
> --
>
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> Skype: *burtonator*
> blog: http
I think there are several issues in your schema and queries.
First, the schema can't efficiently return the single newest post for every
author. It can efficiently return the newest N posts for a particular
author.
On Fri, May 16, 2014 at 11:53 PM, 後藤 泰陽 wrote:
>
> But I consider LIMIT to be a
How should nodetool command be run as the user "nobody"?
The nodetool command fails with an exception if it cannot create a
.cassandra directory in the current user's home directory.
I'd like to schedule some nodetool commands to run with least privilege as
cron jobs. I'd like to run them as the
Running
#> cat /proc/$(cat /var/run/cassandra.pid)/limits
as root or your cassandra user will tell you what limits it's actually
running with.
On Sun, May 4, 2014 at 10:12 PM, Yatong Zhang wrote:
> I am running 'repair' when the error occurred. And just a few days before
> I changed the com
I think the options for using CQL from PHP pretty much don't exist. Those
that do are very old, haven't been updated in months, and don't support
newer CQL features. Also I don't think any of them use the binary protocol
but use thrift instead.
>From what I can tell, you'll be stuck using old CQL
bloom_filter_fp_chance = 0.7 is probably way too large to be effective and
you'll probably have issues compacting deleted rows and get poor read
performance with a value that high. I'd guess that anything larger than
0.1 might as well be 1.0.
-Bryan
On Fri, Jun 21, 2013 at 5:58 AM, srmore wro
Manual compaction for LCS doesn't really do much. It certainly doesn't
compact all those little files into bigger files. What makes you think
that compactions are not occurring?
-Bryan
On Tue, Jun 18, 2013 at 3:59 PM, Franc Carter wrote:
> On Sat, Jun 15, 2013 at 11:49 AM, Franc Carter
> wr
For generic questions like this, google is your friend:
http://lmgtfy.com/?q=cassandra+conflict+resolution
-Bryan
On Thu, Jun 6, 2013 at 11:23 AM, Emalayan Vairavanathan <
svemala...@yahoo.com> wrote:
> Hi All,
>
> Can someone tell me about the conflict resolution mechanisms provided by
> Cassa
sstables. This means you WILL see
obsolete
# data at CL.ONE!
# ignore: ignore fatal errors and let requests fail, as in pre-1.2 Cassandra
disk_failure_policy: stop
On Wed, Jun 5, 2013 at 2:59 PM, Bryan Talbot wrote:
> If you're using cassandra 1.2 then you have a choice spec
If you're using cassandra 1.2 then you have a choice specified in the yaml
# policy for data disk failures:
# stop: shut down gossip and Thrift, leaving the node effectively dead, but
# can still be inspected via JMX.
# best_effort: stop using the failed disk and respond to requests based o
One or more of these might be effective depending on your particular usage
- remove data (rows especially)
- add nodes
- add ram (has limitations)
- reduce bloom filter space used by increasing fp chance
- reduce row and key cache sizes
- increase index sample ratio
- reduce compaction concurrency
I think what you're asking for (efficient removal of TTL'd write-once data)
is already in the works but not until 2.0 it seems.
https://issues.apache.org/jira/browse/CASSANDRA-5228
-Bryan
On Tue, May 28, 2013 at 1:26 PM, Hiller, Dean wrote:
> Oh and yes, astyanax uses client side response la
On Mon, May 20, 2013 at 10:01 AM, Pinak Pani <
nishant.has.a.quest...@gmail.com> wrote:
> Assume NetworkTopologyStrategy. So, I wanted to know whether a data-center
> will contain all the keys?
>
> This is the case:
>
> CREATE KEYSPACE appKS
> WITH placement_strategy = 'NetworkTopologyStrategy'
Option #3 since it depends on the placement strategy and not the
partitioner.
-Bryan
On Mon, May 20, 2013 at 6:24 AM, Pinak Pani <
nishant.has.a.quest...@gmail.com> wrote:
> I just wanted to verify the fact that if I happen to setup a multi
> data-center Cassandra setup, will each data center
I think you're conflating "may" with "must". That article says that
updates "may" still be applied to some replicas when there is a failure and
I believe that still is the case. However, if the coordinator knows that
the CL can't be met before even attempting the write, I don't think it will
atte
512 sectors for read-ahead. Are your new fancy SSD drives using large
sectors? If your read-ahead is really reading 512 x 4KB per random IO,
then that 2 MB per read seems like a lot of extra overhead.
-Bryan
On Thu, May 16, 2013 at 12:35 PM, Keith Wright wrote:
> We actually have it set to
ithout adding hardware or removing data.
>
> -Bryan
>
>
> On Fri, May 10, 2013 at 7:44 PM, Edward Capriolo wrote:
>
>> If you use your off heap memory linux has an OOM killer, that will kill a
>> random tasks.
>>
>>
>> On Fri, May 10, 2013 at 11:3
data.
-Bryan
On Fri, May 10, 2013 at 7:44 PM, Edward Capriolo wrote:
> If you use your off heap memory linux has an OOM killer, that will kill a
> random tasks.
>
>
> On Fri, May 10, 2013 at 11:34 AM, Bryan Talbot wrote:
>
>> If off-heap memory (for indes samples, bloom filte
If off-heap memory (for indes samples, bloom filters, row caches, key
caches, etc) is exhausted, will cassandra experience a memory allocation
error and quit? If so, are there plans to make the off-heap usage more
dynamic to allow less used pages to be replaced with "hot" data and the
paged-out /
On Sat, May 4, 2013 at 9:22 PM, Aiman Parvaiz wrote:
>
> When starting this cluster we set
> > JVM_OPTS="$JVM_OPTS -Xss1000k"
>
>
>
Why did you increase the stack-size to 5.5 times greater than recommended?
Since each threads now uses 1000KB minimum just for the stack, a large
number of threads
It's true that a 16GB heap is generally not a good idea; however, it's not
clear from the data provided what problem you're trying to solve.
What is it that you don't like about the default settings?
-Bryan
On Fri, May 3, 2013 at 4:27 AM, Oleg Dulin wrote:
> Here is my question. It can't pos
I believe that "nodetool rebuild" is used to add a new datacenter, not just
a new host to an existing cluster. Is that what you ran to add the node?
-Bryan
On Fri, Apr 26, 2013 at 1:27 PM, John Watson wrote:
> Small relief we're not the only ones that had this issue.
>
> We're going to try r
On Thu, Apr 4, 2013 at 1:27 AM, wrote:
>
> After some time (1 hour / 2 hour) cassandra shut services on one or two
> nodes with follwoing errors;
>
Wonder what the workload and schema is like ...
We can see from below that you've tweaked and disabled many of the memory
"safety valve" and other
In the worst case, that is possible, but compaction strategies try to
minimize the number of SSTables that a row appears in so a row being in ALL
SStables is not likely for most cases.
-Bryan
On Wed, Mar 27, 2013 at 12:17 PM, Kanwar Sangha wrote:
> Hi – I have a query on Read with Cassandra.
Those older files won't be included in a compaction until there are
min_compaction_threshold (4) files of that size. When you get another SS
table -Data.db file that is about 12-18GB then you'll have 4 and they will
be compacted together into one new file. At that time, if there are any
rows with
On Thu, Feb 28, 2013 at 5:08 PM, Víctor Hugo Oliveira Molinar <
vhmoli...@gmail.com> wrote:
> Ok guys let me try to ask it in a different way:
>
> Will repair totally ensure a data synchronism among nodes?
If there are no writes happening on the cluster then yes. Otherwise, the
answer is "it de
I see from your read and write count that your nreldata CF has nearly equal
number of reads as writes. I would expect that disabling your bloom filter
is going to hurt your read performance quite a bit.
Also, beware that disabling your bloom filter may also cause tombstoned
rows to never be delet
rofile than
> previous cassandra systems I have setup. We don't need really any caching
> as disk access is typically fine on reads.
>
> Thanks,
> Dean
>
> From: Bryan Talbot mailto:btal...@aeriagames.com>>
> Reply-To: "user@cassandra.apache.org<m
This calculation is incorrect btw. 10,000 GB transferred at 1.25 GB / sec
would complete in about 8,000 seconds which is just 2.2 hours and not 5.5
days. The error is in the conversion (1hr/60secs) which is off by 2 orders
of magnitude since (1hr/3600secs) is the correct conversion.
-Bryan
On
With a RF and CL of one, there is no replication so there can be no issue
with distributed deletes. Writes (and reads) can only go to the one host
that has the data and will be refused if that node is down. I'd guess that
your app isn't deleting records when you think that it is, or that the
dele
Aren't bloom filters kept off heap in 1.2?
https://issues.apache.org/jira/browse/CASSANDRA-4865
Disabling bloom filters also disables tombstone removal as well, so don't
disable them if you delete anything.
https://issues.apache.org/jira/browse/CASSANDRA-5182
I believe that the index samples (by
Generally data isn't written to whatever node the client connects to. In
your case, a row is written to one of the nodes based on the hash of the
row key. If that one replica node is down, it won't matter which
coordinator node you attempt a write with CL.ONE: the write will fail.
If you want th
Wow, that's pretty ambitions expecting an upgrade which skips 4 major
versions (0.7, 0.8, 1.0, 1.1) to work.
I think you're going to have to follow the upgrade path for each of those
intermediate steps and not upgrade in one big jump.
-Bryan
On Thu, Feb 7, 2013 at 3:41 AM, Sergey Leschenko wro
On Wed, Jan 30, 2013 at 2:44 PM, Guillermo Barbero <
guillermo.barb...@spotbros.com> wrote:
> WARN [MemoryMeter:1] 2013-01-30 21:37:48,079 Memtable.java (line 202)
> setting live ratio to maximum of 64.0 instead of 751.6512549537648
>
>
This looks interesting. Doesn't this mean that the ratio of
My guess is that those one or two nodes with the gc pressure also have more
rows in your big CF. More rows could be due to imbalanced distribution if
your'e not using a random partitioner or from those nodes not yet removing
deleted rows which other nodes may have done.
JVM heap space is used for
have stabilized the load as desired
at the expense of additional jvm memory.
-Bryan
On Thu, Jan 17, 2013 at 6:50 PM, Bryan Talbot wrote:
> Bleh, I rushed out the email before some meetings and I messed something
> up. Working on reproducing now with better notes this time.
>
> -B
ere you are setting gc_grace to 0 (although I could just be blind, it
> happens).
>
>
> On Thu, Jan 17, 2013 at 5:01 PM, Bryan Talbot wrote:
>
>> I'm able to reproduce this behavior on my laptop using 1.1.5, 1.1.7,
>> 1.1.8, a trivial schema, and a simple script that
array("a" => 1), null, 120);
}
// Close our connections
$pool->close();
$sys->close();
?>
-Bryan
On Thu, Jan 17, 2013 at 10:11 AM, Bryan Talbot wrote:
> We are using LCS and the particular row I've referenced has been involved
> in several compactions after
he column
> which already exist on disk. So when the compaction ran your ExpiringColumn
> was turned into a DeletedColumn and placed on disk.
>
> ** **
>
> I would expect the next round of compaction to remove these columns.
>
> ** **
>
> There is a new feature in
t; 2. after that CF gets compacted
>
> I guess your expired columns are propagated to high tier CF, which gets
> compacted rarely.
> So, you have to wait when high tier CF gets compacted.
>
> Andrey
>
>
>
> On Wed, Jan 16, 2013 at 11:39 AM, Bryan Talbot wrote:
>
>&g
On cassandra 1.1.5 with a write heavy workload, we're having problems
getting rows to be compacted away (removed) even though all columns have
expired TTL. We've tried size tiered and now leveled and are seeing the
same symptom: the data stays around essentially forever.
Currently we write all co
Brian, did any of your issues with java 7 result in corrupting data in
cassandra?
We just ran into an issue after upgrading a test cluster from Cassandra
1.1.5 and Oracle JDK 1.6.0_29-b11 to Cassandra 1.1.7 and 7u10.
What we saw is values in columns with validation
Class=org.apache.cassandra.db.m
ebin.com/MrSTHH9F
> Node 154: http://pastebin.com/7p0Usvwd
>
> @Bryan
>
> "clients always connect to that server"
>
> I didn't join it in the screenshot from AWS console, but AWS report an
> (almost) equal network within the nodes (same for output and cpu). The c
Or maybe the clients always connect to that server which can satisfy all
reads. They have 3 nodes with RF=3.
-Bryan
On Wed, Dec 19, 2012 at 12:15 PM, aaron morton wrote:
> Is there a sustained difference or did it settle back ?
> Could this have been compaction or repair or upgrade tables work
With 1.1.5, the TS is displayed with the local timezone and seems correct.
cqlsh:bat> create table test (id uuid primary key, ts timestamp );
cqlsh:bat> insert into test (id,ts) values (
'89d09c88-40ac-11e2-a1e2-6067201fae78', '2012-12-07T10:00:00-');
cqlsh:bat> select * from test;
id
Well, asking for 500MB of data at once for a server with such modest
specs is asking for troubles. Here are my suggestions.
Disable the 1 GB row cache
Consider allocating that memory for the java heap "Xms2500m Xmx2500m"
Don't fetch all the columns at once -- page through them a slice at a time
I
; Thank you very much for this information. So in other words, the settings
>> such as row_cache_size_in_mb in YAML alone are not enough, and I must also
>> specify the caching attribute on a per column family basis?
>>
>> -- Y.
>>
>>
>> On Tue, Nov 27, 201
On Tue, Nov 27, 2012 at 8:16 PM, Yiming Sun wrote:
> Hello,
>
> but it is not clear to me where this setting belongs to, because even in the
> v1.1.6 conf/cassandra.yaml, there is no such property, and apparently
> adding this property to the yaml causes a fatal configuration error upon
> server
The https://github.com/sebgiroux/Cassandra-Cluster-Admin app does some
of what you're asking. It allows basic browsing and some admin
functionality. If you want to run actual CQL queries though, you
currently need to use another app for that (like cqlsh).
-Bryan
On Thu, Nov 15, 2012 at 11:30 P
On Tue, Nov 6, 2012 at 8:27 AM, horschi wrote:
>
>
>> it is a big itch for my use case. Repair ends up streaming tens of
>> gigabytes of data which has expired TTL and has been compacted away on some
>> nodes but not yet on others. The wasted work is not nice plus it drives up
>> the memory usa
As the OP of this thread, it is a big itch for my use case. Repair ends up
streaming tens of gigabytes of data which has expired TTL and has been
compacted away on some nodes but not yet on others. The wasted work is not
nice plus it drives up the memory usage (for bloom filters, indexes, etc)
of
Do a rolling upgrade of the ring to 1.0.12 first and then upgrade to 1.1.x.
After each rolling upgrade, you should probably do the recommend "nodetool
upgradesstables", etc. The datastax documentation about upgrading might be
helpful for you: http://www.datastax.com/docs/1.1/install/upgrading
-B
Note that 1.0.7 came out before 1.1 and I know there were
some compatibility issues that were fixed in later 1.0.x releases which
could affect your upgrade. I think it would be best to first upgrade to
the latest 1.0.x release, and then upgrade to 1.1.x from there.
-Bryan
On Thu, Nov 1, 2012 a
It seems like CASSANDRA-3442 might be an effective fix for this issue
assuming that I'm reading it correctly. It sounds like the intent is to
automatically compact SSTables when a certain percent of the columns are
gcable by being deleted or with expired tombstones. Is my understanding
correct?
I've been experiencing a behavior that is undesirable and it seems like a
bug that causes a high amount of wasted work.
I have a CF where all columns have a TTL, are generally all inserted in a
very short period of time (less than a second) and are never over-written
or explicitly deleted. Eventu
On Thu, Oct 25, 2012 at 4:15 AM, aaron morton wrote:
> This sounds very much like "my heap is so consumed by (mostly) bloom
>> filters that I am in steady state GC thrash."
>>
>
> Yes, I think that was at least part of the issue.
>
>
> The rough numbers I've used to estimate working set are:
>
>
On Wed, Oct 24, 2012 at 2:38 PM, Rob Coli wrote:
> On Mon, Oct 22, 2012 at 8:38 AM, Bryan Talbot
> wrote:
> > The nodes with the most data used the most memory. All nodes are
> affected
> > eventually not just one. The GC was on-going even when the nodes were
> not
>
On Mon, Oct 22, 2012 at 6:05 PM, aaron morton wrote:
> The GC was on-going even when the nodes were not compacting or running a
> heavy application load -- even when the main app was paused constant the GC
> continued.
>
> If you restart a node is the onset of GC activity correlated to some event?
tingOccupancyFraction=75"
> JVM_OPTS="$JVM_OPTS -XX:+UseCMSInitiatingOccupancyOnly"
> JVM_OPTS="$JVM_OPTS -XX:+UseCompressedOops"
>
> You are too far behind the reference JVM's. Parallel GC is the preferred
> and highest performing form in the curren
inline with the static data load.
>
> Now days GC issues are typically due to more dynamic forces, like
> compaction, repair and throughput.
>
> Hope that helps.
>
> -----
> Aaron Morton
> Freelance Developer
> @aaronmorton
> http://www.thelastpickle.com
>
erver actually is. That's not such a good thing.
-Bryan
On Thu, Oct 18, 2012 at 11:06 AM, Bryan Talbot wrote:
> In a 4 node cluster running Cassandra 1.1.5 with sun jvm 1.6.0_29-b11
> (64-bit), the nodes are often getting "stuck" in state where CMS
> collections of
I believe that reading with CL.ONE will still cause read repair to be run
(in the background) 'read_repair_chance' of the time.
-Bryan
On Thu, Oct 18, 2012 at 1:52 PM, Andrey Ilinykh wrote:
> On Thu, Oct 18, 2012 at 1:34 PM, Michael Kjellman
> wrote:
> > Not sure I understand your question (i
In a 4 node cluster running Cassandra 1.1.5 with sun jvm 1.6.0_29-b11
(64-bit), the nodes are often getting "stuck" in state where CMS
collections of the old space are constantly running.
The JVM configuration is using the standard settings in cassandra-env --
relevant settings are included below.
er then lifetime
> history.
>
>
>
> On Fri, Oct 5, 2012 at 7:27 PM, Bryan Talbot
> wrote:
> > I've recently added compaction rate (in bytes / second) to my monitors
> for
> > cassandra and am seeing some odd values. I wasn't expecting the values
> for
I've recently added compaction rate (in bytes / second) to my monitors for
cassandra and am seeing some odd values. I wasn't expecting the values for
TotalBytesCompacted to sometimes decrease from one reading to the next. It
seems that the value should be monotonically increasing while a server i
ully wise to use this. Is 1.1.4 the one to use?
> >
> > Cheers,
> > Alex
>
--
Bryan Talbot
Architect / Platform team lead, Aeria Games and Entertainment
Silicon Valley | Berlin | Tokyo | Sao Paulo
anges.
-Bryan
On Wed, Sep 12, 2012 at 2:42 PM, Bryan Talbot wrote:
> I'm testing upgrading a multi-node cluster from 1.0.9 to 1.1.5 and ran
> into the error message described here:
> https://issues.apache.org/jira/browse/CASSANDRA-4195
>
> What I can't tell is if this is a
I'm testing upgrading a multi-node cluster from 1.0.9 to 1.1.5 and ran into
the error message described here:
https://issues.apache.org/jira/browse/CASSANDRA-4195
What I can't tell is if this is a serious issue or if it can be safely
ignored.
If it is a serious issue, shouldn't the migration guid
73 matches
Mail list logo
