The only true drain is 1) turn on ip tables to stop all incoming traffic 2) flush 3) wait 4) delete files 5) upgrade 6) restart
On Thu, Jan 3, 2013 at 2:59 PM, Michael Kjellman <mkjell...@barracuda.com>wrote: > That's why I didn’t create a ticket as I knew there was one. But, I > thought this had been fixed in 1.1.7 ?? > > From: Edward Capriolo <edlinuxg...@gmail.com> > Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> > Date: Thursday, January 3, 2013 11:57 AM > To: "user@cassandra.apache.org" <user@cassandra.apache.org> > Subject: Re: Error after 1.2.0 upgrade > > There is a bug on this, drain has been in a weird state for a long time. > In 1.0 it did not work labeled as a known limitation. > > https://issues.apache.org/jira/browse/CASSANDRA-4446 > > > > On Thu, Jan 3, 2013 at 2:49 PM, Michael Kjellman > <mkjell...@barracuda.com>wrote: > >> Another thing: for those that use counters this might be a problem. >> >> I always do a nodetool drain before upgrading a node (as is good practice >> btw). However, in every case on every one of my nodes, the commit log was >> replayed on each node and mutations were created. Could lead to double >> counting of counters… >> >> No bug for that yet >> >> Best, >> Micahel >> >> From: Michael Kjellman <mkjell...@barracuda.com> >> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> >> Date: Thursday, January 3, 2013 11:42 AM >> To: "user@cassandra.apache.org" <user@cassandra.apache.org> >> Subject: Re: Error after 1.2.0 upgrade >> >> Tracking Issues: >> >> https://issues.apache.org/jira/browse/CASSANDRA-5101 >> https://issues.apache.org/jira/browse/CASSANDRA-5104 which was created >> because of https://issues.apache.org/jira/browse/CASSANDRA-5103 >> https://issues.apache.org/jira/browse/CASSANDRA-5102 >> >> Also friendly reminder to all that cql2 created indexes will not work >> with cql3. You need to drop them and recreate in cql3, otherwise you'll see >> rpc_timeout issues. >> >> I'll update with more issues as I see them. The fun bugs never happen in >> your dev environment do they :) >> >> From: aaron morton <aa...@thelastpickle.com> >> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> >> Date: Thursday, January 3, 2013 11:38 AM >> To: "user@cassandra.apache.org" <user@cassandra.apache.org> >> Subject: Re: Error after 1.2.0 upgrade >> >> Michael, >> Could you share some of your problems ? May be of help for others. >> >> Cheers >> >> ----------------- >> Aaron Morton >> Freelance Cassandra Developer >> New Zealand >> >> @aaronmorton >> http://www.thelastpickle.com >> >> On 4/01/2013, at 5:45 AM, Michael Kjellman <mkjell...@barracuda.com> >> wrote: >> >> I'm having huge upgrade issues from 1.1.7 -> 1.2.0 atm but in a 12 node >> cluster which I am slowly massaging into a good state I haven't seen this >> in 15+ hours of operation… >> >> This looks related to JNA? >> >> From: Alain RODRIGUEZ <arodr...@gmail.com> >> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> >> Date: Thursday, January 3, 2013 8:42 AM >> To: "user@cassandra.apache.org" <user@cassandra.apache.org> >> Subject: Error after 1.2.0 upgrade >> >> In a dev env, C* 1.1.7 -> 1.2.0, 1 node. >> >> I run Cassandra in a 8GB memory environment. >> >> The upgrade went well, but I sometimes have the following error: >> >> INFO 17:31:04,143 Node /192.168.100.201 state jump to normal >> INFO 17:31:04,149 Enqueuing flush of Memtable-local@1654799672(32/32 >> serialized/live bytes, 2 ops) >> INFO 17:31:04,149 Writing Memtable-local@1654799672(32/32 >> serialized/live bytes, 2 ops) >> INFO 17:31:04,371 Completed flushing >> /home/stockage/cassandra/data/system/local/system-local-ia-12-Data.db (91 >> bytes) for commitlog position ReplayPosition(segmentId=1357230649515, >> position=49584) >> INFO 17:31:04,376 Startup completed! Now serving reads. >> INFO 17:31:04,798 Compacted to >> [/var/lib/cassandra/data/system/local/system-local-ia-13-Data.db,]. 950 to >> 471 (~49% of original) bytes for 1 keys at 0,000507MB/s. Time: 886ms. >> INFO 17:31:04,889 mx4j successfuly loaded >> HttpAdaptor version 3.0.2 started on port 8081 >> INFO 17:31:04,967 Not starting native transport as requested. Use JMX >> (StorageService->startNativeTransport()) to start it >> INFO 17:31:04,980 Binding thrift service to /0.0.0.0:9160 >> INFO 17:31:05,007 Using TFramedTransport with a max frame size of >> 15728640 bytes. >> INFO 17:31:09,964 Using synchronous/threadpool thrift server on 0.0.0.0 >> : 9160 >> INFO 17:31:09,965 Listening for thrift clients... >> *** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with >> message transform method call failed at >> ../../../src/share/instrument/JPLISAgent.c line: 806 >> ERROR 17:33:56,002 Exception in thread Thread[Thrift:1702,5,main] >> java.lang.StackOverflowError >> at java.net.SocketInputStream.socketRead0(Native Method) >> at java.net.SocketInputStream.read(Unknown Source) >> at java.io.BufferedInputStream.fill(Unknown Source) >> at java.io.BufferedInputStream.read1(Unknown Source) >> at java.io.BufferedInputStream.read(Unknown Source) >> at >> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) >> at >> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) >> at >> org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129) >> at >> org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101) >> at >> org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:378) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:297) >> at >> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:204) >> at >> org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:22) >> at >> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:199) >> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown >> Source) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown >> Source) >> at java.lang.Thread.run(Unknown Source) >> >> ---------------------------------- >> Join Barracuda Networks in the fight against hunger. >> To learn how you can help in your community, please visit: >> http://on.fb.me/UAdL4f >> >> >> >> >> ---------------------------------- >> Join Barracuda Networks in the fight against hunger. >> To learn how you can help in your community, please visit: >> http://on.fb.me/UAdL4f >> >> >> ---------------------------------- >> Join Barracuda Networks in the fight against hunger. >> To learn how you can help in your community, please visit: >> http://on.fb.me/UAdL4f >> >> > > > ---------------------------------- > Join Barracuda Networks in the fight against hunger. > To learn how you can help in your community, please visit: > http://on.fb.me/UAdL4f > >
