Re: Cassandra read throughput with little/no caching.

[James Masson]

On 31/12/12 18:45, Tyler Hobbs wrote:

On Mon, Dec 31, 2012 at 11:24 AM, James Masson mailto:james.mas...@opigram.com>> wrote:


Well, it turns out the Read-Request Latency graph in Ops-Center is
highly misleading.

Using jconsole, the read-latency for the column family in question
is actually normally around 800 microseconds, punctuated by
occasional big spikes that drive up the averages.

Towards the end of the batch process, the Opscenter reported average
latency is up above 4000 microsecs, and forced compactions no longer
help drive the latency down again.

I'm going to stop relying on OpsCenter for data for performance
analysis metrics, it just doesn't have the resolution.


James, it's worth pointing out that Read Request Latency in OpsCenter is
measuring at the coordinator level, so it includes the time spent
sending requests to replicas and waiting for a response.  There's
another latency metric that is per-column family named Local Read
Latency; it sounds like this is the equivalent number that you were
looking at in jconsole.  This metric basically just includes the time to
read local caches/memtables/sstables.

We are looking to rename one or both of the metrics for clarity; any
input here would be helpful. For example, we're considering "Coordinated
Read Request Latency" or "Client Read Request Latency" in place of just
"Read Request Latency".

--
Tyler Hobbs
DataStax 



Hi Tyler,

thanks for clarifying this. So you're saying the difference between the 
global Read Request latency in opscenter, and the column family specific 
one is in the effort coordinating a validated read across multiple 
replicas? Is this not part of what Hector does for itself?


Essentially, I'm looking to see whether I can use this to derive where 
any extra latency from a client request comes from.


As for names, I'd suggest "cluster coordinated read request latency", 
bit of a mouthful, I know.


Is there anywhere I can find concrete definitions of what the stats in 
OpsCenter, and raw Cassandra via JMX mean? The docs I've found seem 
quite ambiguous.


I still think that the data resolution that OpsCenter gives makes it 
more suitable for trending/alerting rather than chasing down tricky 
performance issues. This sort of investigation work is what I do for a 
living, I typically use intervals of 10 seconds or lower, and don't 
average my data. Although, storing your data inside the database your 
measuring does restrict your options a little :-)


regards

James M

Inter-Cluster Communication

[Everton Lima]

Hi people,

I would to know if it is possible to create 2 clusters, in the first
constain just meta-data and in the second just the real data. How the
system will comunicate with this both cluster and one cluster communicate
with other? Could any one help me?

Thanks.

-- 
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Re: Question on TTLs and Tombstones

[Michal Michalski]

Yup, I know it was pretty long mail and it was Christmas time, so I 
thought it might be left without a reply for some time, but as some time 
has passed, I'll try to remind you about my question with additional help:


TL;DR version:

WHEN does Cassandra remove expired (because of TTL) data? Which 
operations cause Cassandra to check for TTL and create Tombstones for 
them if needed? It happens during compaction, for sure. How about scrub, 
repair? Others?


Regards,
Michał

W dniu 28.12.2012 09:08, Michal Michalski pisze:

Hi,

I have a question regarding TTLs and Tombstones with a pretty long
scenario + solution question. My first, general question is - when
Cassandra checks for the TTL (if it expired) and creates the Tombstone
if needed? I know it happens during compaction, but is this the only
situation? How about checking it on reads? How about the
"nodetool-based" actions? Scrub? Repair?

The reason of my question is such scenario - I add the same amount of
rows to CF every month. All of them have TTL of 6 months - when I add
data from July, data from January should expire. I do NOT modify these
data any later. However, because of SizeTiered compaction and large
SSTables my old data do not expire in terms of disk usage - the're in
the biggest/oldest SSTable which is not going to be compacted any soon.
I want to get rid of the data I don't need. So my solution is to perform
a user defined compaction on the single file that contains the oldest
data (I make an assumption that in my use case it's the biggest / oldest
SSTable). It works (at least the first compaction - see below), but I
want to make sure that I'm right and I understand why it happens ;-)

Heres how I understand how it works (it's December, my oldest data are
from November, so I want to have nothing older than June):

I have a large SSTable which was compacted in August for the last time
and it's the oldest SSTable, much larger than the rest, so I can assume
that it contains:
(a) some Tombstones for the January data (when it was compacted for the
last time January was the month to be expired so the Tombstones were
created) which haven't been removed so far
(b) some data from February - May which are NOT marked for deletion so
far because when compaction has occured for the last time they were
"fresh" enough to stay
(c) some newer data (June+)
So I compact it. Tombstones (a) are removed. Expired data (b) are marked
for deletion by creating Tombstones for them. The rest of data is
untouched. This reduces the file size by ~10-20%. This is what I checked
and it worked.
Then I wait 10 days (gc_grace) and compact it once again. It should
remove all the Tombstones created during previous compaction, so file
size should be reduced significantly (let's say it should be like 20% of
the initial size or so). This is what I wait for.
Am I right?

How about repair? As compaction is a "per-node" task, I guess I should
run repair between these two compactions to make sure that Tombstones
have been transfered to other replicas?

Or maybe - returning to my first question - Cassandra checks TTLs much
more often (like with every single read?) so they're "spread" among many
SSTables and they won't get compacted efficiently during compacting the
oldest SSTable only? Or maybe jobs like scrub check TTLs and create
Tombstones too? Or repair?

I know that I could check some of these things with new nodetool
features (like checking % of Tombstones in SSTable), but I run 1.1.1 and
it's unavailable here. I know that 1.2 (or 1.1.7?) handles Tombstones in
a better way, but - still - it's not my case unless I upgrade.

Kind regards,
Michał

Re: Question on TTLs and Tombstones

[Sylvain Lebresne]

> WHEN does Cassandra remove expired (because of TTL) data?

When a compaction reads an expired column, it removes it and replaces it by
a tombstone (i.e. a deleted marker). So the first compaction after the
expiration is what actually removes the data, but it won't reclaim all the
disk space yet due to the tombstone. Said tombstone then follow the usual
rules for tombstones. I.e. it will be fully removed by compaction once
gc_grace seconds after the tombstone creation has elapsed. I note that
during reads, if a column is expired it is simply ignored by the read, but
nothing more is done.

> Which operations cause Cassandra to check for TTL and create Tombstones
for
> them if needed?

Only compaction does (for the 'create tombstones' part at least I mean).
 For the rest of the system, an expired column is always handled exactly as
if it was a tombstone (so reads ignore them, scrub don't care specially
about them and repair don't do anything special either). I note that for
repair this could be a source of inconsistency between nodes; see more
details on https://issues.apache.org/jira/browse/CASSANDRA-4905.

--
Sylvain

[RELEASE] Apache Cassandra 1.2 released

[Sylvain Lebresne]

The Cassandra team wishes you a very happy new year 2013, and is very
pleased
to announce the release of Apache Cassandra version 1.2.0. Cassandra 1.2.0
is a
new major release for the Apache Cassandra distributed database. This
version
adds numerous improvements[1,2] including (but not restricted to):
- Virtual nodes[4]
- The final version of CQL3 (featuring many improvements)
- Atomic batches[5]
- Request tracing[6]
- Numerous performance improvements[7]
- A new binary protocol for CQL3[8]
- Improved configuration options[9]
- And much more...

Please make sure to carefully read the release notes[2] before upgrading.

Both source and binary distributions of Cassandra 1.2.0 can be downloaded
at:

 http://cassandra.apache.org/download/

Or you can use the debian package available from the project APT
repository[3]
(you will need to use the 12x series).

The Cassandra Team

[1]: http://goo.gl/JmKp3 (CHANGES.txt)
[2]: http://goo.gl/47bFz (NEWS.txt)
[3]: http://wiki.apache.org/cassandra/DebianPackaging
[4]: http://www.datastax.com/dev/blog/virtual-nodes-in-cassandra-1-2
[5]: http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2
[6]: http://www.datastax.com/dev/blog/tracing-in-cassandra-1-2
[7]:
http://www.datastax.com/dev/blog/performance-improvements-in-cassandra-1-2
[8]: http://www.datastax.com/dev/blog/binary-protocol
[9]: http://www.datastax.com/dev/blog/configuration-changes-in-cassandra-1-2

Re: Question on TTLs and Tombstones

[Michal Michalski]

Thanks for your answer. Moreover, the issue you mentioned in the end was 
the answer to the question I was going to ask next ;-)


Regards,
Michał

W dniu 02.01.2013 15:42, Sylvain Lebresne pisze:

WHEN does Cassandra remove expired (because of TTL) data?


When a compaction reads an expired column, it removes it and replaces it by
a tombstone (i.e. a deleted marker). So the first compaction after the
expiration is what actually removes the data, but it won't reclaim all the
disk space yet due to the tombstone. Said tombstone then follow the usual
rules for tombstones. I.e. it will be fully removed by compaction once
gc_grace seconds after the tombstone creation has elapsed. I note that
during reads, if a column is expired it is simply ignored by the read, but
nothing more is done.


Which operations cause Cassandra to check for TTL and create Tombstones

for

them if needed?


Only compaction does (for the 'create tombstones' part at least I mean).
  For the rest of the system, an expired column is always handled exactly as
if it was a tombstone (so reads ignore them, scrub don't care specially
about them and repair don't do anything special either). I note that for
repair this could be a source of inconsistency between nodes; see more
details on https://issues.apache.org/jira/browse/CASSANDRA-4905.

--
Sylvain

Re: Cassadra API for Java

[Hiller, Dean]

You can check the cassandra list of java clients

http://wiki.apache.org/cassandra/ClientOptions

From: Brian Tarbox mailto:tar...@cabotresearch.com>>
Reply-To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Date: Sunday, December 30, 2012 11:03 AM
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Subject: Re: Cassadra API for Java

Anyone still use Pelops?


On Sun, Dec 30, 2012 at 12:19 PM, Shahryar Sedghi 
mailto:shsed...@gmail.com>> wrote:
I use JDBC with Cassandra 1.1 with CQL 3. I  tried both Hector and Thrift and 
JDBC is  much easier to code, I never tried Astyanax. Application servers have 
built-in connection pooling support for JDBC, but do not provide fail over to 
other machines, you need to do it at the application level.

Another Caveat: With Both Hector and Thrift without CQL you can retrieve all or 
portion of the row keys, CQL (at least on 1.1) does not give you distinct row 
keys. If you have a use case like this, either you need a Hybrid API solution 
or stick with another API,

Regards

Shahryar


On Fri, Dec 28, 2012 at 10:34 PM, Michael Kjellman 
mailto:mkjell...@barracuda.com>> wrote:
This was asked as recently as one month + 1 day btw:

http://grokbase.com/t/cassandra/user/12bve4d8e8/java-high-level-client if you 
weren't subscribed to the group to see the messages to see a longer discussion.

From: Baskar Sikkayan mailto:techba...@gmail.com>>
Reply-To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Date: Friday, December 28, 2012 7:24 PM
To: "user@cassandra.apache.org" 
mailto:user@cassandra.apache.org>>
Subject: Cassadra API for Java

Hi,
  I am new to Apache Cassandra.
Could you please suggest me good java API( Hector, thrift or .) for 
Cassandra?

Thanks,
Baskar.S
+91 97394 76008



--
Join Barracuda Networks in the fight against hunger.
To learn how you can help in your community, please visit: 
http://on.fb.me/UAdL4f
  ­­



--
"Life is what happens while you are making other plans." ~ John Lennon

Re: Cassandra read throughput with little/no caching.

[Tyler Hobbs]

On Wed, Jan 2, 2013 at 5:28 AM, James Masson wrote:

>
> thanks for clarifying this. So you're saying the difference between the
> global Read Request latency in opscenter, and the column family specific
> one is in the effort coordinating a validated read across multiple replicas?


Yes.


> Is this not part of what Hector does for itself?
>

No.  Here's the basic order of events:

1) Hector sends a request to some node in the cluster, which will act as
the coordinator.
2) The coordinator then sends the actual read requests out to each of the
(RF) replicas.
3a) The coordinator waits for responses from the replicas; how many it
waits for depends on the consistency level.
3b) The replicas perform actual cache/memtable/sstable reads and respond to
the coordinator when complete
4) Once the required number of replicas have responded, the coordinator
replies to the client (Hector).

The Read Request Latency metric is measuring the time taken in steps 2
through 4.  The CF Local Read Latency metric is only capturing the time
taken in step 3b.


> Essentially, I'm looking to see whether I can use this to derive where any
> extra latency from a client request comes from.
>

Yes, using the two numbers in conjunction can be very informative.  Also,
you might be interested in the new query tracing feature in 1.2, which
shows very detailed steps and their latencies.


>
> As for names, I'd suggest "cluster coordinated read request latency", bit
> of a mouthful, I know.
>

Awesome, thanks for your input.


>
> Is there anywhere I can find concrete definitions of what the stats in
> OpsCenter, and raw Cassandra via JMX mean? The docs I've found seem quite
> ambiguous.
>

This has pretty good writeups of each:
http://www.datastax.com/docs/opscenter/online_help/performance/index#opscenter-performance-metrics


>
> I still think that the data resolution that OpsCenter gives makes it more
> suitable for trending/alerting rather than chasing down tricky performance
> issues. This sort of investigation work is what I do for a living, I
> typically use intervals of 10 seconds or lower, and don't average my data.
> Although, storing your data inside the database your measuring does
> restrict your options a little :-)


True, there's a limit to what you can detect with 60 second resolution.
We've considered being able to report metrics at a finer resolution without
durably storing them anywhere, which would be useful for when you're
actively watching the cluster.

Thanks!

-- 
Tyler Hobbs
DataStax 

Re: Cassandra read throughput with little/no caching.

[James Masson]

On 02/01/13 16:18, Tyler Hobbs wrote:

On Wed, Jan 2, 2013 at 5:28 AM, James Masson mailto:james.mas...@opigram.com>> wrote:

>

1) Hector sends a request to some node in the cluster, which will act as
the coordinator.
2) The coordinator then sends the actual read requests out to each of
the (RF) replicas.
3a) The coordinator waits for responses from the replicas; how many it
waits for depends on the consistency level.
3b) The replicas perform actual cache/memtable/sstable reads and respond
to the coordinator when complete
4) Once the required number of replicas have responded, the coordinator
replies to the client (Hector).

The Read Request Latency metric is measuring the time taken in steps 2
through 4.  The CF Local Read Latency metric is only capturing the time
taken in step 3b.




Great, that's exactly the level of detail I'm looking for.




Is there anywhere I can find concrete definitions of what the stats
in OpsCenter, and raw Cassandra via JMX mean? The docs I've found
seem quite ambiguous.


This has pretty good writeups of each:
http://www.datastax.com/docs/opscenter/online_help/performance/index#opscenter-performance-metrics


Your description above was much better :-) I'm more interested in docs 
for the raw metrics provided in JMX.





I still think that the data resolution that OpsCenter gives makes it
more suitable for trending/alerting rather than chasing down tricky
performance issues. This sort of investigation work is what I do for
a living, I typically use intervals of 10 seconds or lower, and
don't average my data. Although, storing your data inside the
database your measuring does restrict your options a little :-)


True, there's a limit to what you can detect with 60 second resolution.
We've considered being able to report metrics at a finer resolution
without durably storing them anywhere, which would be useful for when
you're actively watching the cluster.


That would be a great feature, but it's quite difficult taking 
high-resolution data capture without disturbing the system you're trying 
to measure.


Perhaps worth taking the data-capture points off-list?

James M

Re: Inter-Cluster Communication

[Rob Coli]

On Wed, Jan 2, 2013 at 4:33 AM, Everton Lima  wrote:
> I would to know if it is possible to create 2 clusters, in the first
> constain just meta-data and in the second just the real data. How the system
> will comunicate with this both cluster and one cluster communicate with
> other? Could any one help me?

Cassandra does not have a mechanism for clusters to talk to each
other. Your application could talk to both clusters, but they would be
independent of each other.

=Rob

-- 
=Robert Coli
AIM>ALK - rc...@palominodb.com
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

Force data to a specific node

[Everton Lima]

It is possible to force a data to stay in a specific node?

-- 
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Re: Inter-Cluster Communication

[Everton Lima]

Ok.
Thanks.

2013/1/2 Rob Coli 

> On Wed, Jan 2, 2013 at 4:33 AM, Everton Lima 
> wrote:
> > I would to know if it is possible to create 2 clusters, in the first
> > constain just meta-data and in the second just the real data. How the
> system
> > will comunicate with this both cluster and one cluster communicate with
> > other? Could any one help me?
>
> Cassandra does not have a mechanism for clusters to talk to each
> other. Your application could talk to both clusters, but they would be
> independent of each other.
>
> =Rob
>
> --
> =Robert Coli
> AIM>ALK - rc...@palominodb.com
> YAHOO - rcoli.palominob
> SKYPE - rcoli_palominodb
>



-- 
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Re: Force data to a specific node

[Edward Sargisson]

Why would you want to?


From: Everton Lima 
To: Cassandra-User 
Sent: Wed Jan 02 18:03:49 2013
Subject: Force data to a specific node

It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Re: Force data to a specific node

[Everton Lima]

We need to do this to minimize the network I/O. We have our own load data
balance algorithm. We have some data that is best to process in a local
machine.
Is it possible? How?

2013/1/2 Edward Sargisson 

> Why would you want to?
>
> --
>  *From*: Everton Lima 
> *To*: Cassandra-User 
> *Sent*: Wed Jan 02 18:03:49 2013
> *Subject*: Force data to a specific node
>
> It is possible to force a data to stay in a specific node?
>
> --
> Everton Lima Aleixo
> Bacharel em Ciência da Computação pela UFG
> Mestrando em Ciência da Computação pela UFG
> Programador no LUPA
>
>


-- 
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

How to show unread messages counts?

[Drew Kutcharian]

Happy New Year Everyone!

What's the best way to model "unread messages count" in Cassandra? I have a 
UserMessage CF where the row key is the user id and the column name is the 
message id (timeuuid) and I store the message and the status (READ/UNREAD) in 
the value column. I would like to be able to show the number of unread messages 
to the user. Currently I'm thinking of having a separate Counter CF just to 
keep the number of unread messages in there. Is this a good approach? Is there 
a better one?

Thanks,

Drew

Re: Force data to a specific node

[Aaron Turner]

You'd have to use the ordered partitioner or something like that and
choose your row key according to the node you want it placed.

But that's in general a really bad idea because you end up with
unbalanced nodes and hot spots.

That said, are your nodes on a LAN?  I have my 9+3 node cluster (two
datacenters) on 100Mbps ports (which everyone says not to do) and it's
working just fine.  Even node rebuilds haven't been that bad so far.
If you're trying to avoid WAN replication, then use a dedicated
cluster.

On Wed, Jan 2, 2013 at 10:20 AM, Everton Lima  wrote:
> We need to do this to minimize the network I/O. We have our own load data
> balance algorithm. We have some data that is best to process in a local
> machine.
> Is it possible? How?
>
>
> 2013/1/2 Edward Sargisson 
>>
>> Why would you want to?
>>
>> 
>> From: Everton Lima 
>> To: Cassandra-User 
>> Sent: Wed Jan 02 18:03:49 2013
>> Subject: Force data to a specific node
>>
>> It is possible to force a data to stay in a specific node?
>>
>> --
>> Everton Lima Aleixo
>> Bacharel em Ciência da Computação pela UFG
>> Mestrando em Ciência da Computação pela UFG
>> Programador no LUPA
>>
>
>
>
> --
> Everton Lima Aleixo
> Bacharel em Ciência da Computação pela UFG
> Mestrando em Ciência da Computação pela UFG
> Programador no LUPA
>



-- 
Aaron Turner
http://synfin.net/ Twitter: @synfinatic
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
-- Benjamin Franklin
"carpe diem quam minimum credula postero"

Re: Force data to a specific node

[Edward Capriolo]

There is a crazy, very bad, don't do it way to do this. You can set RF=1
and hack the "LocalPartitioner" (because the local partitioner has been
made not to do this)

Then the node you connect to and write is the node the data will get stored
on.

Its like memcache "do it yourself" style sharding.

Did I say not suggested.

If not "not suggested"


On Wed, Jan 2, 2013 at 2:54 PM, Aaron Turner  wrote:

> You'd have to use the ordered partitioner or something like that and
> choose your row key according to the node you want it placed.
>
> But that's in general a really bad idea because you end up with
> unbalanced nodes and hot spots.
>
> That said, are your nodes on a LAN?  I have my 9+3 node cluster (two
> datacenters) on 100Mbps ports (which everyone says not to do) and it's
> working just fine.  Even node rebuilds haven't been that bad so far.
> If you're trying to avoid WAN replication, then use a dedicated
> cluster.
>
> On Wed, Jan 2, 2013 at 10:20 AM, Everton Lima 
> wrote:
> > We need to do this to minimize the network I/O. We have our own load data
> > balance algorithm. We have some data that is best to process in a local
> > machine.
> > Is it possible? How?
> >
> >
> > 2013/1/2 Edward Sargisson 
> >>
> >> Why would you want to?
> >>
> >> 
> >> From: Everton Lima 
> >> To: Cassandra-User 
> >> Sent: Wed Jan 02 18:03:49 2013
> >> Subject: Force data to a specific node
> >>
> >> It is possible to force a data to stay in a specific node?
> >>
> >> --
> >> Everton Lima Aleixo
> >> Bacharel em Ciência da Computação pela UFG
> >> Mestrando em Ciência da Computação pela UFG
> >> Programador no LUPA
> >>
> >
> >
> >
> > --
> > Everton Lima Aleixo
> > Bacharel em Ciência da Computação pela UFG
> > Mestrando em Ciência da Computação pela UFG
> > Programador no LUPA
> >
>
>
>
> --
> Aaron Turner
> http://synfin.net/ Twitter: @synfinatic
> http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix &
> Windows
> Those who would give up essential Liberty, to purchase a little temporary
> Safety, deserve neither Liberty nor Safety.
> -- Benjamin Franklin
> "carpe diem quam minimum credula postero"
>

Re: How to show unread messages counts?

[aaron morton]

> Currently I'm thinking of having a separate Counter CF just to keep the 
> number of unread messages in there. Is this a good approach?
Yup. 
Add a UserMetrics CF with columns for the counts you want to keep. 

Cheers

-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 3/01/2013, at 8:37 AM, Drew Kutcharian  wrote:

> Happy New Year Everyone!
> 
> What's the best way to model "unread messages count" in Cassandra? I have a 
> UserMessage CF where the row key is the user id and the column name is the 
> message id (timeuuid) and I store the message and the status (READ/UNREAD) in 
> the value column. I would like to be able to show the number of unread 
> messages to the user. Currently I'm thinking of having a separate Counter CF 
> just to keep the number of unread messages in there. Is this a good approach? 
> Is there a better one?
> 
> Thanks,
> 
> Drew

Re: How to show unread messages counts?

[Drew Kutcharian]

Thanks Aaron.

On Jan 2, 2013, at 2:55 PM, aaron morton  wrote:

>> Currently I'm thinking of having a separate Counter CF just to keep the 
>> number of unread messages in there. Is this a good approach?
> Yup. 
> Add a UserMetrics CF with columns for the counts you want to keep. 
> 
> Cheers
> 
> -
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
> 
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 3/01/2013, at 8:37 AM, Drew Kutcharian  wrote:
> 
>> Happy New Year Everyone!
>> 
>> What's the best way to model "unread messages count" in Cassandra? I have a 
>> UserMessage CF where the row key is the user id and the column name is the 
>> message id (timeuuid) and I store the message and the status (READ/UNREAD) 
>> in the value column. I would like to be able to show the number of unread 
>> messages to the user. Currently I'm thinking of having a separate Counter CF 
>> just to keep the number of unread messages in there. Is this a good 
>> approach? Is there a better one?
>> 
>> Thanks,
>> 
>> Drew
> 

Re: Column Family migration/tombstones

[aaron morton]

> 1) As one can imagine, the index and bloom filter for this column family is 
> large.  Am I correct to assume that bloom filter and index space will not be 
> reduced until after gc_grace_period?
Yes. 

> 2) If I would manually run repair across a cluster, is there a process I can 
> use to safely remove these tombstones before gc_grace period to free this 
> memory sooner?
There is nothing to specifically purge tombstones. 

You can temporarily reduce the gc_grace_seconds and then trigger compaction. 
Either by reducing the min_compaction_threshold to 2 and doing a flush. Or by 
kicking of a user defined compaction using the JMX interface. 

> 3) Any words of warning when undergoing this?
Make sure you have a good breakfast. 
(It's more general advice than Cassandra specific.)


Cheers

-
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 30/12/2012, at 8:51 AM, Mike  wrote:

> Hello,
> 
> We are undergoing a change to our internal datamodel that will result in the 
> eventual deletion of over a hundred million rows from a Cassandra column 
> family.  From what I understand, this will result in the generation of 
> tombstones, which will be cleaned up during compaction, after gc_grace_period 
> time (default: 10 days).
> 
> A couple of questions:
> 
> 1) As one can imagine, the index and bloom filter for this column family is 
> large.  Am I correct to assume that bloom filter and index space will not be 
> reduced until after gc_grace_period?
> 
> 2) If I would manually run repair across a cluster, is there a process I can 
> use to safely remove these tombstones before gc_grace period to free this 
> memory sooner?
> 
> 3) Any words of warning when undergoing this?
> 
> We are running Cassandra 1.1.2 on a 6 node cluster and a Replication Factor 
> of 3.  We use LOCAL_QUORM consistency for all operations.
> 
> Thanks!
> -Mike