Thanks Jonathan. I added an extra directory big enough and at least the bootstrap worked.
However, it brings me to questions that i cant seem to find an answer to. In my case, i had a 2 node cluster to start with, replication factor of 2, basically all data residing on both machines. Both nodes had about 150 gigs of data, and about 30 gigs free space. Suddenly, one of the machines had a disk failure, so i decided to remove the node, using nodetool removetoken. Brought up another node to replace the down node, and tried to bootstrap it. Thats when I ran into the space issue. Resolved it by adding the extra disk, about 100 gigs. While bootstrapping I noticed anticompaction happenning for each columnfamily, and saw the createion of a 92 gig file for the biggest columnfamily i have, which was then streamed across to the other node. This file was then deleted. I followed this up with a nodetool cleanup, as was mentioned on the wiki to delete any unused references (probably none in my case, but just wanted to do it!). This triggered an anti-compaction again, took 2 more hours and created a 92 gig file similar to the last step. However, nothing happenned after this. This big file just sits there though, and no other files were deleted. So, now i have 240 gigs of disk used up on the node where i triggered the cleanup as no files were deleted. When I do a nodetool ring, it says 93 gigs load, as opposed to 150 gigs previously. Questions; 1. what is the purpose of this anticompacted file created during cleanup? 2. compaction is also supposed to create a big file after reconciling the sstable files.. i see that anti-compaction is also creating a big file. How are these files going to be different? 3. Other than during streaming (bootstrapping), when is anti-compaction triggered? 4. Now i have 240 gigs of disk used for probably around 92 gigs of useful data. When will the other files get deleted? Do i need to run compaction separately to take care of this. Thanks for your patience, Gurpreet On Wed, Sep 8, 2010 at 9:02 AM, Jonathan Ellis <jbel...@gmail.com> wrote: > if you have a data directory defined with enough room, Cassandra will > use that one. > > On Wed, Sep 8, 2010 at 6:25 AM, Gurpreet Singh <gurpreet.si...@gmail.com> > wrote: > > Hi, > > version: cassandra 0.6.5 > > I am trying to bootstrap a new node from an existing seed node. > > The new node seems to be stuck with the bootstrapping message, and did > not > > show any activity. > > Only after i checked the logs of the seed node, i realise there has been > an > > error: > > Caused by: java.lang.UnsupportedOperationException: disk full > > at > > > org.apache.cassandra.db.CompactionManager.doAntiCompaction(CompactionManager.java:345) > > at > > > org.apache.cassandra.db.CompactionManager.access$500(CompactionManager.java:49) > > at > > > org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:143) > > at > > > org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:140) > > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > > The data size on the seed node is about 140 gigs, and it has abt 20 gig > free > > space left. > > How do i get rid of this error and get the bootstrapping going ? > > Can it anti-compact to some other place that has disk space available or > any > > other ways to get this working? > > Thanks > > Gurpreet > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of Riptano, the source for professional Cassandra support > http://riptano.com >
