> > Should I use nodetool repair utility > That wouldn't help, this an anti-entropy mechanism (see https://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsRepair.html#toolsRepair__description_unique_27 ).
It is something really important too often left aside. Yet, your issue here is not about consistency. The client can't find any node in charge of the read / written tokens in the ring. This depends on the topology, the replication factor and your network mainly. I think there is something wrong in your setup. I would try this: - Make sure connection / port are ok - Try increasing the RF / Strategy in the stress tool - Try with an other consistency level (not LOCAL_*, as mentioned here : http://stackoverflow.com/questions/32055251/not-enough-replica-available-for-query-at-consistency-local-one-1-required-but ) Good luck, ----------------- Alain The Last Pickle http://www.thelastpickle.com 2016-01-22 23:02 GMT+01:00 Bhuvan Rawal <bhu1ra...@gmail.com>: > Getting same exception again. Should I use nodetool repair utility? > > On Sat, Jan 23, 2016 at 3:10 AM, Sebastian Estevez < > sebastian.este...@datastax.com> wrote: > >> https://github.com/brianmhess/cassandra-loader >> >> All the best, >> >> >> [image: datastax_logo.png] <http://www.datastax.com/> >> >> Sebastián Estévez >> >> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >> >> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image: >> facebook.png] <https://www.facebook.com/datastax> [image: twitter.png] >> <https://twitter.com/datastax> [image: g+.png] >> <https://plus.google.com/+Datastax/about> >> <http://feeds.feedburner.com/datastax> >> <http://goog_410786983> >> >> >> <http://www.datastax.com/gartner-magic-quadrant-odbms> >> >> DataStax is the fastest, most scalable distributed database technology, >> delivering Apache Cassandra to the world’s most innovative enterprises. >> Datastax is built to be agile, always-on, and predictably scalable to any >> size. With more than 500 customers in 45 countries, DataStax is the >> database technology and transactional backbone of choice for the worlds >> most innovative companies such as Netflix, Adobe, Intuit, and eBay. >> >> On Fri, Jan 22, 2016 at 4:37 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >> wrote: >> >>> Yes im specifying -node parameter to stress, otherwise it throws network >>> connection failed. >>> >>> Can you point me to a sample java application to test pushing data from >>> external server? Let's see if that works >>> >>> On Sat, Jan 23, 2016 at 2:55 AM, Sebastian Estevez < >>> sebastian.este...@datastax.com> wrote: >>> >>>> when i opened my cassandra-rackdc.properties i saw that DC names were >>>>> DC1 & DC2, rack name was RAC1 . Please note that this is the default >>>>> configuration, I have not modified any file. >>>> >>>> >>>> cassandra-rackdc.properties is only respected based on your snitch >>>> <https://docs.datastax.com/en/cassandra/2.1/cassandra/architecture/architectureSnitchesAbout_c.html> >>>> . >>>> >>>> $ cqlsh >>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': >>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: >>>>> Connection refused")}) >>>>> whereas >>>>> $ cqlsh <ip> >>>>> works fine >>>>> is that the reason why the cassandra-stress is not able to communicate >>>>> with other replicas? >>>> >>>> >>>> Are you providing the -node parameter to stress >>>> <http://docs.datastax.com/en/cassandra/2.1/cassandra/tools/toolsCStress_t.html> >>>> ? >>>> >>>> >>>> >>>> All the best, >>>> >>>> >>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>> >>>> Sebastián Estévez >>>> >>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >>>> >>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image: >>>> facebook.png] <https://www.facebook.com/datastax> [image: twitter.png] >>>> <https://twitter.com/datastax> [image: g+.png] >>>> <https://plus.google.com/+Datastax/about> >>>> <http://feeds.feedburner.com/datastax> >>>> <http://goog_410786983> >>>> >>>> >>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>> >>>> DataStax is the fastest, most scalable distributed database >>>> technology, delivering Apache Cassandra to the world’s most innovative >>>> enterprises. Datastax is built to be agile, always-on, and predictably >>>> scalable to any size. With more than 500 customers in 45 countries, >>>> DataStax >>>> is the database technology and transactional backbone of choice for the >>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay. >>>> >>>> On Fri, Jan 22, 2016 at 4:07 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >>>> wrote: >>>> >>>>> I had a look at the jira below: >>>>> https://issues.apache.org/jira/browse/CASSANDRA-7905 >>>>> >>>>> when i opened my cassandra-rackdc.properties i saw that DC names were >>>>> DC1 & DC2, rack name was RAC1 . Please note that this is the default >>>>> configuration, I have not modified any file. >>>>> >>>>> There is another point of concern here which might be relevant to >>>>> previous one as well, im not able to login to cqlsh directly, i.e. I have >>>>> to specify ip as well even when im logged in to that machine. >>>>> >>>>> $ cqlsh >>>>> Connection error: ('Unable to connect to any servers', {'127.0.0.1': >>>>> error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: >>>>> Connection refused")}) >>>>> >>>>> whereas >>>>> $ cqlsh <ip> >>>>> works fine >>>>> >>>>> is that the reason why the cassandra-stress is not able to communicate >>>>> with other replicas? >>>>> >>>>> On Sat, Jan 23, 2016 at 1:37 AM, Sebastian Estevez < >>>>> sebastian.este...@datastax.com> wrote: >>>>> >>>>>> Sorry I missed that. >>>>>> >>>>>> Both your nodetool status and keyspace replication settings say >>>>>> Cassandra and Analytics for the DC names. I'm not sure where you're >>>>>> seeing >>>>>> DC1, DC2, etc. and why you suspect that is the problem. >>>>>> >>>>>> All the best, >>>>>> >>>>>> >>>>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>>>> >>>>>> Sebastián Estévez >>>>>> >>>>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >>>>>> >>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> [image: >>>>>> facebook.png] <https://www.facebook.com/datastax> [image: >>>>>> twitter.png] <https://twitter.com/datastax> [image: g+.png] >>>>>> <https://plus.google.com/+Datastax/about> >>>>>> <http://feeds.feedburner.com/datastax> >>>>>> <http://goog_410786983> >>>>>> >>>>>> >>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>>>> >>>>>> DataStax is the fastest, most scalable distributed database >>>>>> technology, delivering Apache Cassandra to the world’s most innovative >>>>>> enterprises. Datastax is built to be agile, always-on, and predictably >>>>>> scalable to any size. With more than 500 customers in 45 countries, >>>>>> DataStax >>>>>> is the database technology and transactional backbone of choice for the >>>>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and >>>>>> eBay. >>>>>> >>>>>> On Fri, Jan 22, 2016 at 1:45 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Hi Sebastian, >>>>>>> >>>>>>> I had attached nodetool status output in previous mail, pasting it >>>>>>> again : >>>>>>> >>>>>>> $ nodetool status Datacenter: Analytics ===================== >>>>>>> Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address >>>>>>> Load Tokens Owns Host ID Rack UN 10.41.55.17 428.5 KB 256 ? >>>>>>> 39d6d585-e641-4046-9d0b-797356597b5e rack1 UN 10.41.55.19 404.44 KB 256 >>>>>>> ? >>>>>>> 69edf930-efd9-4d74-a798-f3d4ac02e516 rack1 UN 10.41.55.18 423.21 KB 256 >>>>>>> ? >>>>>>> b74bab13-09b2-4760-bce9-c8ef05e50f6d rack1 UN 10.41.55.20 683.23 KB 256 >>>>>>> ? >>>>>>> fb5c4fed-6e1e-4ea8-838d-358106906830 rack1 Datacenter: Cassandra >>>>>>> ===================== Status=Up/Down |/ >>>>>>> State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns >>>>>>> Host ID Rack UN 10.41.55.15 209.4 KB 256 ? >>>>>>> ffc3b9a0-5d5c-4a3d-a99e-49d255731278 rack1 UN 10.41.55.21 227.44 KB 256 >>>>>>> ? >>>>>>> c68deba4-b9a2-43fc-bb13-6af74c88c210 rack1 UN 10.41.55.23 222.71 KB 256 >>>>>>> ? >>>>>>> 8229aa87-af00-48fa-ad6b-3066d3dc0e58 rack1 UN 10.41.55.22 218.72 KB 256 >>>>>>> ? >>>>>>> c7ba84fd-7992-41de-8c88-11574a72db99 rack1 >>>>>>> >>>>>>> Regards, >>>>>>> Bhuvan Rawal >>>>>>> >>>>>>> On Sat, Jan 23, 2016 at 12:11 AM, Sebastian Estevez < >>>>>>> sebastian.este...@datastax.com> wrote: >>>>>>> >>>>>>>> The output of `nodetool status` would help us diagnose. >>>>>>>> >>>>>>>> All the best, >>>>>>>> >>>>>>>> >>>>>>>> [image: datastax_logo.png] <http://www.datastax.com/> >>>>>>>> >>>>>>>> Sebastián Estévez >>>>>>>> >>>>>>>> Solutions Architect | 954 905 8615 | sebastian.este...@datastax.com >>>>>>>> >>>>>>>> [image: linkedin.png] <https://www.linkedin.com/company/datastax> >>>>>>>> [image: >>>>>>>> facebook.png] <https://www.facebook.com/datastax> [image: >>>>>>>> twitter.png] <https://twitter.com/datastax> [image: g+.png] >>>>>>>> <https://plus.google.com/+Datastax/about> >>>>>>>> <http://feeds.feedburner.com/datastax> >>>>>>>> <http://goog_410786983> >>>>>>>> >>>>>>>> >>>>>>>> <http://www.datastax.com/gartner-magic-quadrant-odbms> >>>>>>>> >>>>>>>> DataStax is the fastest, most scalable distributed database >>>>>>>> technology, delivering Apache Cassandra to the world’s most innovative >>>>>>>> enterprises. Datastax is built to be agile, always-on, and predictably >>>>>>>> scalable to any size. With more than 500 customers in 45 countries, >>>>>>>> DataStax >>>>>>>> is the database technology and transactional backbone of choice for the >>>>>>>> worlds most innovative companies such as Netflix, Adobe, Intuit, and >>>>>>>> eBay. >>>>>>>> >>>>>>>> On Fri, Jan 22, 2016 at 1:39 PM, Bhuvan Rawal <bhu1ra...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks for the response Alain, >>>>>>>>> >>>>>>>>> cqlsh> create keyspace mykeyspace WITH replication = >>>>>>>>> {'class':'NetworkTopologyStrategy', 'Analytics':2, 'Cassandra':3} >>>>>>>>> cqlsh> use mykeyspace; >>>>>>>>> cqlsh:mykeyspace>create table mytable (id int primary key, name >>>>>>>>> text, address text, phone text); >>>>>>>>> cqlsh:mykeyspace> insert into mytable (id, name, address, phone) >>>>>>>>> values (1, 'Kiyu','Texas', '555-1212'); # and other similar statement >>>>>>>>> I then issued the below command from every node and found >>>>>>>>> consistent results. >>>>>>>>> cqlsh:mykeyspace> select * from mytable; >>>>>>>>> >>>>>>>>> // Then i repeated the above steps for NetworkTopologyStrategy and >>>>>>>>> found same results >>>>>>>>> >>>>>>>>> I ran basic cassandra stress >>>>>>>>> seed1 - seed of datacenter 1 >>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node >>>>>>>>> any_random_ip >>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node seed1 >>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node seed1,seed2 >>>>>>>>> $ cassandra-stress write n=50000 -rate threads=4 -node >>>>>>>>> all_8_ip_comma_seperated >>>>>>>>> $ cassandra-stress write n=1000000 cl=one -mode native cql3 >>>>>>>>> -schema keyspace="keyspace1" -pop seq=1..1000000 -node ip1,ip2,ip3,ip4 >>>>>>>>> >>>>>>>>> All of them threw the exception >>>>>>>>> *com.datastax.driver.core.exceptions.UnavailableException: Not >>>>>>>>> enough replica available for query at consistency LOCAL_ONE (1 >>>>>>>>> required but >>>>>>>>> only 0 alive)* >>>>>>>>> >>>>>>>>> >>>>>>>>> I have a feeling that the issue is with datacenter name for some >>>>>>>>> reason, because in some config files I found DC name to be like >>>>>>>>> DC1/DC2/DC3 >>>>>>>>> in some it is like Cassandra/Analytics (The ones I had specified while >>>>>>>>> installation). Im unsure which yaml/property file to look for correct >>>>>>>>> inconsistency. >>>>>>>>> >>>>>>>>> (C*heers :) - im so tempted to copy that) >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Bhuvan >>>>>>>>> >>>>>>>>> On Fri, Jan 22, 2016 at 8:47 PM, Alain RODRIGUEZ < >>>>>>>>> arodr...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> The the exact command you ran (stress-tool with options) could be >>>>>>>>>> useful to help you on that. >>>>>>>>>> >>>>>>>>>> However, Im able to create keyspace, tables and insert data using >>>>>>>>>>> cqlsh and it is replicating fine to all the nodes. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Having the schema might be useful too. >>>>>>>>>> >>>>>>>>>> Did you ran the cqlsh and the stress-tool from the same server ? >>>>>>>>>> If not, you might want to check the port you use (9042/9160/...) are >>>>>>>>>> open. >>>>>>>>>> Also, cqlsh uses local_one by default too. If both commands were >>>>>>>>>> run against the same DC, from the same machine they should behave >>>>>>>>>> the same >>>>>>>>>> way. Are they ? >>>>>>>>>> >>>>>>>>>> C*heers, >>>>>>>>>> >>>>>>>>>> ----------------- >>>>>>>>>> Alain >>>>>>>>>> >>>>>>>>>> The Last Pickle >>>>>>>>>> http://www.thelastpickle.com >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2016-01-22 9:57 GMT+01:00 Bhuvan Rawal <bhu1ra...@gmail.com>: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> i have created a POC cluster with 2 DC , each having 4 nodes >>>>>>>>>>> with DSE 4.8.1 installed. >>>>>>>>>>> >>>>>>>>>>> On issuing cassandra stress im getting an error and data is not >>>>>>>>>>> being inserted: >>>>>>>>>>> *com.datastax.driver.core.exceptions.UnavailableException: Not >>>>>>>>>>> enough replica available for query at consistency LOCAL_ONE (1 >>>>>>>>>>> required but >>>>>>>>>>> only 0 alive)* >>>>>>>>>>> >>>>>>>>>>> However, Im able to create keyspace, tables and insert data >>>>>>>>>>> using cqlsh and it is replicating fine to all the nodes. >>>>>>>>>>> >>>>>>>>>>> Details of the cluster can be found below (all the nodes seem to >>>>>>>>>>> be alive and kicking): >>>>>>>>>>> >>>>>>>>>>> $ nodetool status Datacenter: Analytics ===================== >>>>>>>>>>> Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address >>>>>>>>>>> Load Tokens Owns Host ID Rack UN 10.41.55.17 428.5 KB 256 ? >>>>>>>>>>> 39d6d585-e641-4046-9d0b-797356597b5e rack1 UN 10.41.55.19 404.44 KB >>>>>>>>>>> 256 ? >>>>>>>>>>> 69edf930-efd9-4d74-a798-f3d4ac02e516 rack1 UN 10.41.55.18 423.21 KB >>>>>>>>>>> 256 ? >>>>>>>>>>> b74bab13-09b2-4760-bce9-c8ef05e50f6d rack1 UN 10.41.55.20 683.23 KB >>>>>>>>>>> 256 ? >>>>>>>>>>> fb5c4fed-6e1e-4ea8-838d-358106906830 rack1 Datacenter: >>>>>>>>>>> Cassandra ===================== Status=Up/Down |/ >>>>>>>>>>> State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns >>>>>>>>>>> Host ID Rack UN 10.41.55.15 209.4 KB 256 ? >>>>>>>>>>> ffc3b9a0-5d5c-4a3d-a99e-49d255731278 rack1 UN 10.41.55.21 227.44 KB >>>>>>>>>>> 256 ? >>>>>>>>>>> c68deba4-b9a2-43fc-bb13-6af74c88c210 rack1 UN 10.41.55.23 222.71 KB >>>>>>>>>>> 256 ? >>>>>>>>>>> 8229aa87-af00-48fa-ad6b-3066d3dc0e58 rack1 UN 10.41.55.22 218.72 KB >>>>>>>>>>> 256 ? >>>>>>>>>>> c7ba84fd-7992-41de-8c88-11574a72db99 rack1 >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Bhuvan Rawal >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >
