Looking for pointers about replication internal working

branch; Apache Cassandra has always avoided it. Is there any paper or blog or other pointers to understand what Apache Cassandra did to avoid Zookeeper? Thanks! Han

Re: understand bootstrapping

request to node A during this time? Is it possible to happen? Thanks Han On Wed, Jan 27, 2021 at 5:48 PM Yifan Cai wrote: > Your thoughts regarding Gossip are correct. There could be a time that > nodes in the cluster hold different views of the ring locally. > > In the case of b

Re: understand bootstrapping

e point of view, even when it is successful, at the end of `StorageService::joinTokenRing`, it is possible that some existing nodes have not updated their token ring yet, is this correct? Thanks Han > > Hopefully, the answers help. > > - Yifan > > On Sun, Jan 24, 2021 at 1:00 PM Han

understand bootstrapping

seed node and non-seed node? I'm particularly trying to understand the fault-tolerant part of updating Token Ring state on every node, but couldn't find relevant info by searching. Any info or pointers are appreciated. Thanks! Han

Cassandra data retention policy

Hi guys, We have a processing system that just uses the data for the past six months in Cassandra. Any suggestions on the best way to manage the old data in order to save disk space? We want to keep it as backup but it will not be used unless we need to do recovery. Thanks in advance! -John

which replica has your data?

Hi all, I have a data item whose row key is 7573657238353137303937323637363334393636363230 and I have a five node Cassandra cluster with replication factor set to 3. Each replica's token is listed below TOK: 0 TOK: 34028236692093846346337460743176821145 TOK: 6805647338418769269267492148635364

Re: How to extract information from commit log?

sions of partitions. > Jonathan > > Jonathan Lacefield > Solutions Architect, DataStax > (404) 822 3487 > [1] > [2] > > On Tue, Mar 18, 2014 at 2:25 PM, Han,Meng wrote: > >> Hi Cassandra hackers! >> >> I have a question regarding extracting use

How to extract information from commit log?

Hi Cassandra hackers! I have a question regarding extracting useful information from commit log. Since its a binary log, how should I extract information such as timestamp, values from it? Does anyone know any binary log reader that I can use directly to read commit log? If there is no such

Re: Cassandra Data Archiving

On Fri, Jun 1, 2012 at 12:28 PM, Harshvardhan Ojha < harshvardhan.o...@makemytrip.com> wrote: > Problem statement: > > We are keeping daily generated data(user generated content) in > Cassandra, but our application is using only 15 days old data. So how can > we archive data older than 15 da

Re: will compaction delete empty rows after all columns expired?

On Thu, May 31, 2012 at 9:31 AM, Curt Allred wrote: > No, these were not wide rows. They are rows that formerly had one or 2 > columns. The columns are deleted but the empty rows dont go away, even > after gc_grace_secs. > The empty row goes away only during a compaction after the gc_grace_secs

Re: Repair Process Taking too long

On Sat, Apr 14, 2012 at 1:57 PM, Igor wrote: > Hi! > > What is the difference between 'repair' and '-pr repair'? Simple repair > touch all token ranges (for all nodes) and -pr touch only range for which > given node responsible? > > -pr only touches the primary range of the node. If you executes

Re: sstable2json and resurrected rows

Did you hit the bug here? https://issues.apache.org/jira/browse/CASSANDRA-4054 best regards, 坚果云 , 最简捷易用的云存储 无限空间, 文件同步, 备份和分享! 2012/3/30 Jonas Borgström > Let me rephrase my question: > > Is it true that deleted rows will still be present in the sstable after a > m

Re: Peregrine: A new map reduce framework for iterative/pipelined jobs.

On Tue, Dec 27, 2011 at 2:31 PM, Kevin Burton wrote: > > I'm pleased to announce Peregrine 0.5.0 - a new map reduce framework > optimized > for iterative and pipelined map reduce jobs. > > http://peregrine_mapreduce.bitbucket.org/ > > This originally started off with some internal work at Spinn3r

Re: split large sstable

best regards, 韩竹(Zhu Han) 坚果铺子 <https://jianguopuzi.com>, 最简单易用的云存储 同步文件, 分享照片, 文档备份! On Mon, Nov 21, 2011 at 11:07 PM, Dan Hendry wrote: > Pretty sure your argument about indirect blocks making large files > inefficient only pertains to ext2/3 and not ext4. It seems ext4 re

Re: Upgrading to 1.0

I'd like to know whether it is possible to upgrade from 0.6.13 to 1.0.x directly, either. Is there anything we should notice that "nodetool scrub" might not fix? On Wed, Nov 2, 2011 at 7:46 PM, Jake Maizel wrote: > Hello, > > We run a medium sized cluster of 12 nodes on 0.6.13 and would like t

Re: Moving to a new cluster

On Thu, Sep 22, 2011 at 11:04 PM, Yan Chunlu wrote: > > hi Aaron: > > could you explain more about the issue about repair make space usage going > crazy? > I guess repair defers the compaction progress as it brings a lot of load. For update-heavy workload, the space usage goes higher and higher

Re: Planet Cassandra is now live

On Sat, Aug 13, 2011 at 4:35 AM, Konstantin Naryshkin wrote: > Would you consider adding an RSS feed to the site for the benefit of those > who like to use feed readers to keep track of unread posts and what not? > Here it is: http://planetcassandra.org/aggregator/rss > > - Original Message

Re: migrating from 0.6 to 0.8, "java.io.IOError: ... cannot extend file to required size"

On Wed, Aug 10, 2011 at 5:24 PM, aaron morton wrote: > I remember seeing this once before upgrading a system from 0.6 to 0.7 on a > Ubuntu EC2 (non data stax build) with EBS disks. I did the same thing and > just assumed it was an EBS or 0.6 bug. From memory after the upgrade that > node had some

Re: Survey: Cassandra/JVM Resident Set Size increase

Chris, I've deployed the patch to the cluster for two days. Everything is quite good since then. Thank you! best regards, 韩竹(Zhu Han) On Sat, Jul 30, 2011 at 3:52 AM, Chris Burroughs wrote: > Thanks to everyone who responded (I think I learned a few new tricks > from seeing wha

Re: Cassandra 0.6.8 snapshot problem?

On Thu, Jul 28, 2011 at 10:47 PM, Jian Fang wrote: > Hi, > > We have an old production Cassandra 0.6.8 instance without replica, i.e., > the replication factor is 1. Recently, we noticed that > the snapshot data we took from this instance are inconsistent with the > running instance data. For exam

Re: Cassandra Storage Sizing

Very helpful. Thank you! best regards, Zhu Han On Thu, Jul 21, 2011 at 12:24 PM, Todd Burruss wrote: > I put together a blog post on Cassandra Storage Sizing so I don’t need to > keep figuring it out again and again. Hope everyone finds it useful, and > give feedback if you fi

Re: Commit log is not emptied after "nodetool drain"

2011/7/15 Zhu Han > > 2011/7/15 Jonathan Ellis > >> If you have non-empty segments post-drain that is a bug. Is it >> reproducible? >> > > I think it is always reproducible on 0.6.x branch. Here is a simple > experiment: > Should I raise an issue

Re: Commit log is not emptied after "nodetool drain"

mmitLog-1310702265959.log > -rw-r--r-- 1 root root 125 2011-07-15 11:58 CommitLog-1310702291383.log > > 2011/7/14 Zhu Han : > > Jonathan, > > > > But all the old non-empty log segments are kept on the disk. And > cassandra > > takes some time to apply the opera

Re: Commit log is not emptied after "nodetool drain"

Jonathan, But all the old non-empty log segments are kept on the disk. And cassandra takes some time to apply the operations from these closed log segments after restart of the process. Is it expected? best regards, 韩竹(Zhu Han) 坚果铺子 <https://jianguopuzi.com>, 最简单易用的云存储 同步文件, 分享照片

Commit log is not emptied after "nodetool drain"

0.8.x branch. https://issues.apache.org/jira/browse/CASSANDRA-2874 best regards, 韩竹(Zhu Han) 坚果铺子 <https://jianguopuzi.com>, 最简单易用的云存储 同步文件, 分享照片, 文档备份!

Re: Survey: Cassandra/JVM Resident Set Size increase

On Wed, Jul 13, 2011 at 9:45 PM, Konstantin Naryshkin wrote: > Do you mean that it is using all of the available heap? That is the > expected behavior of most long running Java applications. The JVM will not > GC until it needs memory (or you explicitly ask it to) and will only free up > a bit of

Re: copy data from multi-node cluster to single node

On Tue, Jul 5, 2011 at 8:58 AM, aaron morton wrote: > How do you change the name of a cluster? The FAQ instructions do not seem > to work for me - are they still valid for 0.7.5? > Is the backup / restore mechanism going to work, or is there a > better/simpler to copy data from multi-node to sing

Re: compaction behaviour

best regards, Zhu Han On Sun, Apr 3, 2011 at 9:21 AM, Anurag Gujral wrote: > Hi All, >I have loaded data into cassandra using batch processing the > response times for reads are in the range of 0.8 ms but I am using SSDs. so > I expect the read times to be even fas

Re: reduced cached mem; resident set size growth

On Thu, Mar 17, 2011 at 10:27 AM, Zhu Han wrote: > > > On Thu, Feb 3, 2011 at 1:49 AM, Ryan King wrote: > >> On Wed, Feb 2, 2011 at 6:22 AM, Chris Burroughs >> wrote: >> > On 01/28/2011 09:19 PM, Chris Burroughs wrote: >> >> Thanks Oleg and Zhu. I

Re: reduced cached mem; resident set size growth

On Thu, Feb 3, 2011 at 1:49 AM, Ryan King wrote: > On Wed, Feb 2, 2011 at 6:22 AM, Chris Burroughs > wrote: > > On 01/28/2011 09:19 PM, Chris Burroughs wrote: > >> Thanks Oleg and Zhu. I swear that wasn't a new hotspot version when I > >> checked, but that's obviously not the case. I'll update

Re: FW: Very slow batch insert using version 0.7.2

On Fri, Mar 11, 2011 at 10:40 AM, Erik Forkalsrud wrote: > > I see the same behavior with smaller batch sizes. It appears to happen > when starting Cassandra with the defaults on relatively large systems. > Attached is a script I created to reproduce the problem. (usage: mutate.sh > /path/to/ap

Re: reduced cached mem; resident set size growth

Chris, Somebody else and I have the same problem as you, and reported it here: http://www.apacheserver.net/Very-high-memory-utilization-not-caused-by-mmap-on-sstables-at1082970.htm [NB: It is not solved although the titles said so. Some response from me in the thread is not accurate.] IMHO, you

Re: reduced cached mem; resident set size growth

On Fri, Jan 28, 2011 at 4:15 PM, Oleg Anastasyev wrote: > > > > http://img24.imageshack.us/img24/1754/cassandrarss.png > > > This looks like cassandra leaking memory inside java heap. > I remember, there was some leaking issues with java versions <1.6.u21, > correct > me if I wrong. Try to upgrad

Re: Cassandra on iSCSI?

On Fri, Jan 21, 2011 at 3:00 PM, Mick Semb Wever wrote: > > It should work fine; the main reason to go with local storage is the > > huge cost advantage. > > [OT] They're quoting roughly the same price for both (claiming that the > extra cost goes into having for each node a separate disk cabinet

Re: Tombstone lifespan after multiple deletions

On Wed, Jan 19, 2011 at 8:41 PM, Germán Kondolf wrote: > On Wed, Jan 19, 2011 at 12:59 AM, Zhu Han wrote: > > > > > > On Wed, Jan 19, 2011 at 11:35 AM, Germán Kondolf < > german.kond...@gmail.com> > > wrote: > >> > >> Yes, that's

Re: Tombstone lifespan after multiple deletions

On Wed, Jan 19, 2011 at 11:35 AM, Germán Kondolf wrote: > Yes, that's what I meant, but correct me if I'm wrong, when a deletion > comes after another deletion for the same row or column will the gc-before > count against the last one, isn't it? > > IIRC, after compaction. even if the row key is n

Re: Tombstone lifespan after multiple deletions

I'm not clear here. Are you worried about the later inserted tombstone prevents the whole row from being reclaimed and the storage space can not be freed? To my knowledge, after major compaction, only the row key and tombstone are kept. Is it a big deal? best regards, hanzhu On Tue, Jan 18,

Re: Tombstone lifespan after multiple deletions

If the tombstone is older than the row or column inserted later, is the tombstone skipped entirely after compaction? best regards, hanzhu On Wed, Jan 19, 2011 at 11:16 AM, Jonathan Ellis wrote: > If you mean that multiple tombstones for the same row or column should > be merged into a single o

Re: Which Java on Fedora? Sun's or GNU's?

Eric, Do you use the default GC settings? Can you show me the openJDK version by "java -version"? Thank you! If everything is the same, I suspect I need to upgrade the kernel. best regards, hanzhu On Wed, Dec 29, 2010 at 11:44 PM, Eric Evans wrote: > On Wed, 2010-12-29 at 11:11 +0100, Peter

Re: Which Java on Fedora? Sun's or GNU's?

I met with native memory leak with OpenJDK. Still try to figure it out... best regards, hanzhu On Wed, Dec 29, 2010 at 6:11 PM, Peter Schuller wrote: > > Which is best? > > Which is preferred? > > If by "GNU" you mean the gcj stuff, then absolutely no. :) > > If you mean OpenJDK it's less obvi

Re: complexity

Yep. I forgot about the binary search part. Thank you! regards, hanzhu On Fri, Dec 24, 2010 at 9:35 PM, Jonathan Ellis wrote: > On Fri, Dec 24, 2010 at 4:42 AM, Zhu Han wrote: > > When the row is stored on disk as SSTable, the complexity of getting a > row > > is constant,

Re: complexity

On Fri, Dec 24, 2010 at 6:42 PM, Zhu Han wrote: > When the row is stored on disk as SSTable, the complexity of getting a row > is constant, as it always know where to get the row by in-memory indices. > BTW: not the whole indices are kept in memory, just part of them are. This is cont

Re: complexity

When the row is stored on disk as SSTable, the complexity of getting a row is constant, as it always know where to get the row by in-memory indices. When the row is stored in memory as memtable, it is stored as skip list[1]. The complexity is O(logN). N is the total number of rows in the skip li

Re: [SOLVED] Very high memory utilization (not caused by mmap on sstables)

Can anybody recommend a stable enough JDK environment for 0.6.x branch on ubuntu server? Thank you! best regards, hanzhu On Sun, Dec 19, 2010 at 10:29 AM, Zhu Han wrote: > The problem seems still like the C-heap of JVM, which leaks 70MB every day. > Here is the summary: > &

Re: [SOLVED] Very high memory utilization (not caused by mmap on sstables)

Here is a typo, sorry... best regards, hanzhu On Sun, Dec 19, 2010 at 10:29 AM, Zhu Han wrote: > The problem seems still like the C-heap of JVM, which leaks 70MB every day. > Here is the summary: > > on 12/19: 010c3000 178548K rw---[ anon ] > on 12/18: 010c

Re: [SOLVED] Very high memory utilization (not caused by mmap on sstables)

ble:0 kB Bounce: 0 kB CommitLimit: 0 kB Committed_AS:0 kB VmallocTotal:0 kB VmallocUsed: 0 kB VmallocChunk:0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 Hugepagesize: 2048 kB > thanks, > Sri > > On Fr

Re: [SOLVED] Very high memory utilization (not caused by mmap on sstables)

? best regards, hanzhu On Thu, Dec 16, 2010 at 9:28 PM, Zhu Han wrote: > I've tried it. But it does not work for me this afternoon. > > Thank you! > > best regards, > hanzhu > > > > On Thu, Dec 16, 2010 at 8:59 PM, Matthew Conway wrote: > >> Thanks fo

Re: Memory leak with Sun Java 1.6 ?

This bug is present in both Sun JDK and open JDK because they share the same hotspot VM. Sun JDK got the fix earlier than open JDK. best regards, hanzhu On Thu, Dec 16, 2010 at 6:43 PM, Jedd Rashbrooke < jedd.rashbro...@imagini.net> wrote: > Hi Peter, > > I've read through the "Very high memo

Re: [SOLVED] Very high memory utilization (not caused by mmap on sstables)

/bugdatabase/view_bug.do?bug_id=6824570 [2] http://blog.fuseyism.com/index.php/2010/09/10/icedtea6-19-released/ best regards, hanzhu On Thu, Dec 16, 2010 at 3:10 PM, Zhu Han wrote: > The test node is behind a firewall. So I took some time to find a way to > get JMX diagnostic informatio

Re: Very high memory utilization (not caused by mmap on sstables)

estart the instance. best regards, hanzhu On Thu, Dec 16, 2010 at 1:00 PM, Zhu Han wrote: > After investigating it deeper, I suspect it's native memory leak of JVM. > The large anonymous map on lower address space should be the native heap of > JVM, but not java object heap. H

Re: Very high memory utilization (not caused by mmap on sstables)

2010 at 10:50 AM, Zhu Han wrote: > Hi, > > I have a test node with apache-cassandra-0.6.8 on ubuntu 10.4. The > hardware environment is an OpenVZ container. JVM settings is > # java -Xmx128m -version > java version "1.6.0_18" > OpenJDK Runtime Environment (IcedTea6

Very high memory utilization (not caused by mmap on sstables)

Hi, I have a test node with apache-cassandra-0.6.8 on ubuntu 10.4. The hardware environment is an OpenVZ container. JVM settings is # java -Xmx128m -version java version "1.6.0_18" OpenJDK Runtime Environment (IcedTea6 1.8.2) (6b18-1.8.2-4ubuntu2) OpenJDK 64-Bit Server VM (build 16.0-b13, mixed m