I tried on the latest trunk (through CLI connecting to Hive Server) and there is no disconnection after 10 mins for a long query.
@Ayush, is this Java client using JDBC connection? If so the client may have set a timeout for JDBC queries. I'm suspecting the disconnection is from the Java client you are using. On Feb 24, 2011, at 11:43 PM, Carl Steinbach wrote: Hi Viral, Hive 0.5.0 and 0.6.0 use the same version of libthrift, so the problem is more likely related to some difference in the way 0.5.0 and 0.6.0 configure/initialize Thrift, or to some other issue related to the way the Thrift connection is managed on the client or server side (though it looks like the connection is getting dropped on the server side). Carl On Thu, Feb 24, 2011 at 11:31 PM, Viral Bajaria <viral.baja...@gmail.com<mailto:viral.baja...@gmail.com>> wrote: Carl, Do you think this issue was not there before 0.6 ? We run our thrift servers for hours and have never faced this issue. I don't think I have restarted any of my thrift servers for days. My hive wrapper does have logic to handle timeouts, it reconnects whenever it sees that the thrift connection has died. I think I will add some logging in my wrapper code to see if my setup does see regular timeouts but I never see any issues due to the retry logic. -Viral On Thu, Feb 24, 2011 at 11:24 PM, Carl Steinbach <c...@cloudera.com<mailto:c...@cloudera.com>> wrote: I filed a JIRA ticket to track the task of making the Thrift socket timeout configurable: https://issues.apache.org/jira/browse/HIVE-2006 On Thu, Feb 24, 2011 at 11:21 PM, Carl Steinbach <c...@cloudera.com<mailto:c...@cloudera.com>> wrote: Hi Ayush, I suspect you're running into Thrift's default socket timeout setting. I recommend checking out a copy of the Hive source code, and modifying the Thrift setup code in HiveServer.java to explicitly set the socket timeout on the TServerSocket, e.g. in HiveServer.main() change TServerTransport serverTransport = new TServerSocket(port); to TServerTransport serverTransport = new TServerSocket(port, timeout); The Thrift javadoc doesn't specify whether timeout is in seconds or milliseconds, so you'll probably have to play around with this value. Hope this helps. Carl On Thu, Feb 24, 2011 at 10:52 PM, Ayush Gupta <ay...@glugbot.com<mailto:ay...@glugbot.com>> wrote: On Fri, Feb 25, 2011 at 12:17 PM, Viral Bajaria <viral.baja...@gmail.com<mailto:viral.baja...@gmail.com>> wrote: What do the logs of the thrift server say ?? If it does not give any relevant information, I would enable DEBUG level logging on the console. the hiveserver is pretty quiet, the connection appears to be terminated silently. I'll up the logging to DEBUG, thanks for that suggestion. Also a point to remember is the single-threaded nature of the hive thrift server (atleast upto v0.5) yeah, there is only this one client connected in this scenario. But looking at the logs is what will be the first thing that I would do. The query (map/reduce job) will continue to run even if you shutdown the server since a shutdown does not kill the job submitted to the JobTracker. sure On Thu, Feb 24, 2011 at 9:36 PM, Ayush Gupta <ay...@glugbot.com<mailto:ay...@glugbot.com>> wrote: Probing this further reveals that the connection is reset by the server in exactly 10 minutes every time. I'm running Hive 0.6. I do not see anything relevant at http://wiki.apache.org/hadoop/Hive/AdminManual/Configuration but is there some configuration property which controls this? -ayush On Fri, Feb 25, 2011 at 8:23 AM, Ayush Gupta <ay...@glugbot.com<mailto:ay...@glugbot.com>> wrote: Hi! I'm having some trouble running queries from a java client against a remote Thrift Hive server. Its all setup and quicker queries do run through fine. But queries which run longer than about 10 minutes disconnect the client with a "TTransportException: Connection reset" exception.. The query continues to run on the Hive server but since the client is disconnected the results are "lost". The complete stack trace is below. Does this sound familiar to anyone? org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:314) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:262) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:192) at org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:72) at org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:57) at com.wordnik.analytics.data.ReportsRunner$.refreshReport(ReportsRunner.scala:105) at com.wordnik.analytics.data.ReportsRunner$.refreshDailyReport(ReportsRunner.scala:34) at com.wordnik.analytics.data.ReportsRunner.refreshDailyReport(ReportsRunner.scala) at com.wordnik.analytics.util.Temp.main(Temp.java:11) Caused by: java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:168) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:125) ... 10 more -ayush
