Random NoHostAvailableException using DataStax Java driver
Hi All :) I have an application which use DataStax Java driver v2.0.2 to connect to Cassandra Cluster (6 nodes, v 2.0.11). Application is deployed in three copies on 3 different servers. From time to time on random application server I get this exception: 2014-11-04 10:37:15,301 - ERROR: Servlet.service() for servlet [Unique Identifier Service] in context with path [/uis] threw exception [com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: [cassandra-11/10.0.0.11:9042, /10.0.0.10:9042, /10.0.0.12:9042, /10.0.0.7:9042, /10.0.0.9:9042, /10.0.0.8:9042] - use getErrors() for details)] with root cause com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: [cassandra-11/10.0.0.11:9042, /10.0.0.10:9042, /10.0.0.12:9042, /10.0.0.7:9042, /10.0.0.9:9042, /10.0.0.8:9042] - use getErrors() for details) at com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:103) at com.datastax.driver.core.SessionManager.execute(SessionManager.java:418) at com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:454) at com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:104) at com.datastax.driver.core.SessionManager.execute(SessionManager.java:92) at eu.europeana.cloud.service.uis.database.dao.CassandraDataProviderDAO.getProviders(CassandraDataProviderDAO.java:92) at eu.europeana.cloud.service.uis.CassandraDataProviderService.getProviders(CassandraDataProviderService.java:34) at eu.europeana.cloud.service.uis.rest.DataProvidersResource.getProviders(DataProvidersResource.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:353) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:343) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:318) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:983) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:359) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:372) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:335) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:218) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) at
Re: Random NoHostAvailableException using DataStax Java driver
Hi, Let's move the discussion to the Java driver mailing list: https://groups.google.com/a/lists.datastax.com/forum/#!forum/java-driver-user I'm posting a reply to this message there. -- Olivier Michallat Driver & tools engineer, DataStax On Tue, Nov 4, 2014 at 12:45 PM, Ola Nowak wrote: > Hi All :) > I have an application which use DataStax Java driver v2.0.2 to connect > to Cassandra Cluster (6 nodes, v 2.0.11). > Application is deployed in three copies on 3 different servers. From > time to time on random application server I get this exception: > > 2014-11-04 10:37:15,301 - ERROR: Servlet.service() for servlet [Unique > Identifier Service] in context with path [/uis] threw exception > [com.datastax.driver.core.exceptions.NoHostAvailableException: All > host(s) tried for query failed (tried: [cassandra-11/10.0.0.11:9042, > /10.0.0.10:9042, /10.0.0.12:9042, /10.0.0.7:9042, /10.0.0.9:9042, > /10.0.0.8:9042] - use getErrors() for details)] with root cause > com.datastax.driver.core.exceptions.NoHostAvailableException: All > host(s) tried for query failed (tried: [cassandra-11/10.0.0.11:9042, > /10.0.0.10:9042, /10.0.0.12:9042, /10.0.0.7:9042, /10.0.0.9:9042, > /10.0.0.8:9042] - use getErrors() for details) > at > com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:103) > at > com.datastax.driver.core.SessionManager.execute(SessionManager.java:418) > at > com.datastax.driver.core.SessionManager.executeQuery(SessionManager.java:454) > at > com.datastax.driver.core.SessionManager.executeAsync(SessionManager.java:104) > at > com.datastax.driver.core.SessionManager.execute(SessionManager.java:92) > at > eu.europeana.cloud.service.uis.database.dao.CassandraDataProviderDAO.getProviders(CassandraDataProviderDAO.java:92) > at > eu.europeana.cloud.service.uis.CassandraDataProviderService.getProviders(CassandraDataProviderService.java:34) > at > eu.europeana.cloud.service.uis.rest.DataProvidersResource.getProviders(DataProvidersResource.java:54) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171) > at > org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:195) > at > org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:353) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:343) > at > org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) > at > org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) > at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) > at org.glassfish.jersey.internal.Errors.process(Errors.java:315) > at org.glassfish.jersey.internal.Errors.process(Errors.java:297) > at org.glassfish.jersey.internal.Errors.process(Errors.java:267) > at > org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:318) > at > org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) > at > org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:983) > at > org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:359) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:372) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:335) > at > org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:218) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Appl
Re: Unsubscribe
You should have received an email when you signed up which gives you instructions on how to unsubscribe. Otherwise, send an email to user-h...@cassandra.apache.org On Mon, Nov 3, 2014 at 10:30 PM, Malay Nilabh wrote: > Hi > > > > It was great to be part of this group. Thanks for helping out. Please > unsubscribe me now. > > > > *Regards,* > > *Malay Nilabh* > > BIDW BU/ Big Data CoE > > L&T Infotech Ltd, Hinjewadi,Pune > > [image: Description: image001]: +91-20-66571746 > > [image: Description: Description: Description: Description: > cid:image002.png@01CF1EAD.959B9290]+91-73-879-00727 > > Email: malay.nil...@lntinfotech.com > > *|| Save Paper - Save Trees || * > > > > -- > The contents of this e-mail and any attachment(s) may contain confidential > or privileged information for the intended recipient(s). Unintended > recipients are prohibited from taking action on the basis of information in > this e-mail and using or disseminating the information, and must notify the > sender and delete it from their system. L&T Infotech will not accept > responsibility or liability for the accuracy or completeness of, or the > presence of any virus or disabling code in this e-mail" >
random thought: rewrite Cassandra using Scala ?
I got introduced to Scala recently and found its features on Futures and Function composition very neat. Cassandra utilizes Futures and essentially passing around functions (using anonymous inner subclasses) a lot, if expressed in Scala , it could be much simpler and smoother. just a random thought: would there be any value to rewrite the code in Scala? thanks Yang
Re: random thought: rewrite Cassandra using Scala ?
I guess to answer that, you would need to first ask if it would be worth redoing the 4+ years of development that has gone into Cassandra. There have been many discussions about porting it to C/C++ and always is the same response; it's just not worth the rewrite for the incremental gain. Also, since Scala is in the JVM, the gain would really only be syntactical sugar. This is of course my own opinion. Sent from my iPhone > On Nov 4, 2014, at 7:19 PM, Yang wrote: > > I got introduced to Scala recently and found its features on Futures and > Function composition very neat. > > Cassandra utilizes Futures and essentially passing around functions (using > anonymous inner subclasses) a lot, if expressed in Scala , it could be much > simpler and smoother. just a random thought: would there be any value to > rewrite the code in Scala? > > > thanks > Yang
Cassandra heap pre-1.1
Is there a good formula to calculate heap utilization in Cassandra pre-1.1, specifically 1.0.10. We are seeing gc pressure on our nodes. And I am trying to estimate what could be causing this? Using node tool info my steady state heap is at about 10GB. XMX is 12G. I have 4.5 GB of bloom filters which I can derive looking at cfstats I have negligible row caching. I have key caching enabled on my cfs. I couldn't find an easy way to estimate how much this is using, but I tried to invalidate the key cache and I got 1.3 GB back. That still only adds up to 5.8 GB. I know there is index sampling going on as well. I have around 800 million rows. Is there a way to estimate how much space this would add up to? What else? -Raj
