Simple solution is to stop the nodes and delete the contents of the commitlog, data and saved_caches directories. Normally these are in /var/lib/cassandra
This will give you a clean starting point. The cli statement is using secondary indexes http://www.datastax.com/dev/blog/whats-new-cassandra-07-secondary-indexes phpcassa has support for it, http://thobbs.github.com/phpcassa/api/index.html I've managed to avoid php so far so cannot help there. Aaron On 25 Mar 2011, at 12:18, ko...@vivinavi.com wrote: > Hi Aaron > > Thank you so much for your reply and advice. > I did describe cluster. > Then I found multiple version as following. > > [default@unknown] describe cluster; > Cluster Information: > Snitch: org.apache.cassandra.locator.SimpleSnitch > Partitioner: org.apache.cassandra.dht.RandomPartitioner > Schema versions: > d24cdd50-5567-11e0-a96a-cc92cc024db2: [**.**.254.53] > 2db1c0ff-51b1-11e0-8dc8-e700f669bcfc: [**.**.254.54] > > So maybe I have to drop cluster. > I have to drop both clusters? or only one side? > > And then I don't know how to drop the entire cluster. > I checked help of Cassandra-cli or Cassandra wiki etc. > I can find only drop keyspace,CF but drop cluster. > Please let me know how to drop cluster. > > And I have another question. > I found a command "get <cf> where <column> = <value> [and <column> > <value> > and ...] [limit int];" > on Cassandra-Cli help. > Is this command available in PHP program? > > Thank you for your reply in advance. > > Mamoru Mac Kondo > > > (2011/03/23 18:45), aaron morton wrote: >> >> If you are getting the "Cluster schema does not agree" then you have a sick >> cluster and replication will not be working properly. >> >> Open the cassandra-cli client and type "describe cluster;" you should see a >> single schema version such as... >> >> [default@unknown] describe cluster; >> Cluster Information: >> Snitch: org.apache.cassandra.locator.SimpleSnitch >> Partitioner: org.apache.cassandra.dht.RandomPartitioner >> Schema versions: >> e69ffb30-5525-11e0-0000-242d50cf1fbd: [127.0.0.1, 127.0.0.2] >> >> But I think you will see multiple versions. >> >> The easiest thing to do is drop the entire cluster and start again. >> >> Hope that helps. >> Aaron >> >> On 24 Mar 2011, at 11:45, ko...@vivinavi.com wrote: >> >>> Hi Aaron >>> >>> Thank you so much for your reply and advice. >>> I watched your presentation. it"s helpful for me. >>> >>> Anyway I did as following with 2 node servers.(53;1st node,54;2nd node) >>> I started the following write/read.php program by through Thrift at 53 >>> server. >>> <?php >>> include_once... >>> $servers[0]['host'] = '**.**.254.53'; >>> $servers[0]['port'] = '9160'; >>> $KeyspaceName = 'Twissandra'; >>> $ColumnFamilyName = 'user'; >>> $Key = 'LA'; >>> $conn = new Connection($KeyspaceName, $servers); >>> $users = new ColumnFamily($conn, $ColumnFamilyName); >>> $users->insert($Key, array('Movie' => 'Hollywood', 'Seaside' => >>> 'SantaMonica' , 'Museum' => 'Getty' , 'Japanese' => 'Little Tokyo' )); >>> $ret=$users->get($Key); >>> print_r($ret); >>> ?> >>> I could get $ret at 53 server. >>> Then I started the following read.php program at 54 server. >>> <?php >>> include_once... >>> $servers[0]['host'] = '**.**.254.54'; >>> $servers[0]['port'] = '9160'; >>> $KeyspaceName = 'Twissandra'; >>> $ColumnFamilyName = 'user'; >>> $Key = 'LA'; >>> $conn = new Connection($KeyspaceName, $servers); >>> $users = new ColumnFamily($conn, $ColumnFamilyName); >>> $ret=$users->get($Key); >>> print_r($ret); >>> ?> >>> I could get $ret at 54 server. >>> So I confirmed these data are replicated from 53 to 54. >>> However I did same thing from 54 to 53 with only changed data(key='hahaha'). >>> (These are same keyspace,CF) >>> I could see data on 54 server.(no problem) >>> But on 53 server I had an following error message. >>> Fatal error: Uncaught exception 'cassandra_TimedOutException' in >>> /usr/share/php5/Thrift/thrift/packages/cassandra/Cassandra.php:253 >>> Stack trace: >>> #0 /usr/share/php5/Thrift/thrift/packages/cassandra/Cassandra.php(253): >>> thrift_protocol_read_binary(Object(TBinaryProtocolAccelerated), >>> 'cassandra_Cassa...', false) >>> #1 /usr/share/php5/Thrift/thrift/packages/cassandra/Cassandra.php(226): >>> CassandraClient->recv_get_slice() >>> #2 /usr/share/php5/Thrift/columnfamily.php(254): >>> CassandraClient->get_slice('hahaha', Object(cassandra_ColumnParent), >>> Object(cassandra_SlicePredicate), 1) >>> #3 /var/www/thrift/PhpcassaRead.php(44): ColumnFamily->get('hahaha') >>> #4 {main} >>> thrown in /usr/share/php5/Thrift/thrift/packages/cassandra/Cassandra.php >>> on line 253 >>> I checked on Cassandra-cli at 53 server. >>> [default@Twissandra] get user['hahaha']; >>> null >>> >>> So I think replication doesn't work from 54 to 53. >>> I know when I add node and then set data at 1st node, this data will be >>> provided to all other nodes. >>> But I'm not sure when I set data at 2nd node, this data should be >>> replicated from 2nd to 1st node. >>> I though this will be replicated because of ring. >>> >>> And when I tried to create keyspace or CF several times on 54(2nd server), >>> I had a following error message. >>> "Cluster schema does not yet agree" >>> i don't know how to schema agree. >>> >>> Am I wrong? >>> Thank you for your advice in advance. >>> >>> Mamoru Mac Kondo >>> >>> (2011/03/23 11:32), Aaron Morton wrote: >>>> >>>> What process are you using to confirm the data was replicated to another >>>> server ? And what makes you say the data is not replicated ? I think your >>>> understanding of the replication may be a little off, you rarely read from >>>> one node. Have a look at >>>> http://thelastpickle.com/2011/02/07/Introduction-to-Cassandra/ >>>> >>>> What happens when you try to create a keyspace or CF and it fails? >>>> >>>> It's easier to help if you provide the commands you use and any error >>>> messages. Also remember to check the logs on the servers for errors. >>>> >>>> Aaron >>>> >>>> On 24/03/2011, at 5:58 AM, "ko...@vivinavi.com" <ko...@vivinavi.com> wrote: >>>> >>>>> Hi maki-san >>>>> >>>>> I am so sorry this was my mistake. >>>>> I expected when I set data at one node, the data should be copied to the >>>>> other node >>>>> at the same keyspace and same column family by replication. >>>>> This replication was working. >>>>> I just made a mistake to get data by wrong Key.(First character of key >>>>> was lower-case character >>>>> but I thought first character was capitalized. I tried to get by key=User >>>>> but key=user) >>>>> I am so sorry for this confusion. >>>>> So now I confirmed the data I set at **.**.254.53 was replicated to >>>>> **.**.254.54. >>>>> (At first I started 53 with autobootstrap=false then started 54 with >>>>> autobootstrap=true and then >>>>> finally restarted 53 with autobootstrap=true. So 53 server is like a >>>>> master) >>>>> >>>>> However when I set data at 54 server, this data never been replicated. >>>>> So this is Not bidirectional replication ? >>>>> Cassandra at multi-node is ring (This means no master, no slave?). >>>>> (Off course seeds in cassandra.yaml of both server are the same) >>>>> So I expected I can replicate bidirectinally. >>>>> Am I wrong? >>>>> And when I run at multi-node, I can't create keyspace, column family on >>>>> Cassandra-cli at other 54 server >>>>> (like a slave server). >>>>> Is this normal? >>>>> How can I create same keyspace and column family at multi-node? >>>>> or I have to restart at single-node, then can I create same keyspace and >>>>> column family? >>>>> >>>>> Thank you for your help in advance. >>>>> >>>>> Mamoru Mac Kondo >>>>> >>>>> (2011/03/22 18:43), Maki Watanabe wrote: >>>>>> How do you define your Keyspace? >>>>>> As you may know, in Cassandra, replication (factor) is defined as the >>>>>> attribute of Keyspace. >>>>>> And what do you mean: >>>>>>> However replication never happened. >>>>>>> I can't get data I set at other node. >>>>>> What did you do on cassandra, and what did you get in response? >>>>>> >>>>>> maki >>>>>> >>>>>> >>>>>> 2011/3/23 ko...@vivinavi.com<ko...@vivinavi.com>: >>>>>>> Hi Sasha >>>>>>> Thank you so much for your advice. >>>>>>> I changed JMX_PORT from 10036 to 8080 in cassandra-env.sh. >>>>>>> Now nodetool ring is working as following. >>>>>>> >>>>>>> # nodetool --host **.**.254.54 ring >>>>>>> Address Status State Load Owns Token >>>>>>> >>>>>>> 31247585259092561925693111230676487333 >>>>>>> **.**.254.53 Up Normal 51.3 KB 84.50% >>>>>>> 4871825541058236750403047111542070004 >>>>>>> **.**.254.54 Up Normal 66.71 KB 15.50% >>>>>>> 31247585259092561925693111230676487333 >>>>>>> >>>>>>> Then it seems I could set data to other node by Cassandra-cli --host >>>>>>> other >>>>>>> node IP --port 9160.(Currently only 2 nodes) >>>>>>> However replication never happened. >>>>>>> I can't get data I set at other node. >>>>>>> I don't know what's wrong. >>>>>>> (I thought replication starts when cassandra -p restart) >>>>>>> Please advice me how to do to start replication. >>>>>>> Thank you for your advice in advance. >>>>>>> >>>>>>> >>>>>>> (2011/03/18 23:38), Sasha Dolgy wrote: >>>>>>> >>>>>>> You need to specify the -jmxport with nodetool >>>>>>> >>>>>>> On Mar 19, 2011 2:48 AM, "ko...@vivinavi.com"<ko...@vivinavi.com> >>>>>>> wrote: >>>>>>>> Hi everyone >>>>>>>> >>>>>>>> I am still new to Cassandra, Thrift. >>>>>>>> But anyway Cassandra 0.7.4, Thrift 0.5.0 are working on java 1.6.0.18 >>>>>>>> of >>>>>>>> Debian 5.0.7.at single node. >>>>>>>> Then I had to try and check multi node on 2 servers. >>>>>>>> (JVM_PORT=10036 on /etc/cassandra-env.sh) >>>>>>>> I modified /etc/cassandra/cassandra.yaml as following. >>>>>>>> auto_bootstrap:false ->true >>>>>>>> seeds: -127.0.0.1 -> add Global IP addres of 2 servers(incl.own >>>>>>>> server) >>>>>>>> listen_address:localhost -> Own Global IP address(or own host name on >>>>>>>> /etc/hosts) >>>>>>>> rpc_address:localhost ->0.0.0.0 >>>>>>>> I run master server and then slave server. >>>>>>>> netstat -nl is as following. on both servers. >>>>>>>> Proto Recv-Q Send-Q Local Address Foreign Address State >>>>>>>> tcp 0 0 0.0.0.0:9160 0.0.0.0:* LISTEN >>>>>>>> tcp 0 0 0.0.0.0:10036 0.0.0.0:* LISTEN >>>>>>>> tcp 0 0 **.**.**.**:7000 0.0.0.0:* LISTEN >>>>>>>> >>>>>>>> However it seems Cassandra doesn't work. >>>>>>>> Because I can't get any data from Cluster (always null, data is >>>>>>>> broken?) >>>>>>>> So I checked the nodetool (nodetool --host IP ring). >>>>>>>> The nodetool had errors as following. >>>>>>>> Error connection to remote JMX agent! >>>>>>>> java.io.IOException: Failed to retrieve RMIServer stub: >>>>>>>> javax.naming.ServiceUnavailableException [Root exception is >>>>>>>> java.rmi.ConnectException: Connection refused to host: **.**.**.**; >>>>>>>> nested exception is: >>>>>>>> java.net.ConnectException: Connection refused] >>>>>>>> at >>>>>>>> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:342) >>>>>>>> at >>>>>>>> >>>>>>>> javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267) >>>>>>>> at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:137) >>>>>>>> at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:107) >>>>>>>> at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:511) >>>>>>>> Caused by: javax.naming.ServiceUnavailableException [Root exception is >>>>>>>> java.rmi.ConnectException: Connection refused to host: **.**.**.**; >>>>>>>> nested exception is: >>>>>>>> java.net.ConnectException: Connection refused] >>>>>>>> at >>>>>>>> com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118) >>>>>>>> at >>>>>>>> >>>>>>>> com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203) >>>>>>>> at javax.naming.InitialContext.lookup(InitialContext.java:409) >>>>>>>> at >>>>>>>> >>>>>>>> javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1902) >>>>>>>> at >>>>>>>> >>>>>>>> javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1871) >>>>>>>> at >>>>>>>> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:276) >>>>>>>> ... 4 more >>>>>>>> Caused by: java.rmi.ConnectException: Connection refused to host: >>>>>>>> **.**.**.**; nested exception is: >>>>>>>> java.net.ConnectException: Connection refused >>>>>>>> at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) >>>>>>>> at >>>>>>>> sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) >>>>>>>> at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) >>>>>>>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340) >>>>>>>> at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source) >>>>>>>> at >>>>>>>> com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114) >>>>>>>> ... 9 more >>>>>>>> Caused by: java.net.ConnectException: Connection refused >>>>>>>> at java.net.PlainSocketImpl.socketConnect(Native Method) >>>>>>>> at >>>>>>>> >>>>>>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:310) >>>>>>>> at >>>>>>>> >>>>>>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:176) >>>>>>>> at >>>>>>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:163) >>>>>>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384) >>>>>>>> at java.net.Socket.connect(Socket.java:546) >>>>>>>> at java.net.Socket.connect(Socket.java:495) >>>>>>>> at java.net.Socket.<init>(Socket.java:392) >>>>>>>> at java.net.Socket.<init>(Socket.java:206) >>>>>>>> at >>>>>>>> >>>>>>>> sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40) >>>>>>>> at >>>>>>>> >>>>>>>> sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146) >>>>>>>> at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613) >>>>>>>> ... 14 more >>>>>>>> >>>>>>>> I don't know how to do. >>>>>>>> Please advise what am I wrong. >>>>>>>> And please let me know how to delete or re-build cluster. >>>>>>>> Because I can't even set data or make new column family currently on >>>>>>>> Cassandra-Cli. >>>>>>>> Thank you for your help in advance. >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>> ** >>>>> >>> >> > >
