And setting httpclient.max-connections=100 doesn't seem to be picked up. --Wouter de Bie Developer Business Intelligence, Spotify wou...@spotify.com (mailto:wou...@spotify.com) +46 72 018 0777
On Friday, July 22, 2011 at 4:54 PM, Wouter de Bie wrote: > Hi, > > When I use 0.6.1, hive gets stuck at the following point after about 6 > "select * from table limit 10": > > 2011-07-22 14:50:44,153 DEBUG httpclient.HttpClient > (HttpClient.java:executeMethod(322)) - enter > HttpClient.executeMethod(HttpMethod) > 2011-07-22 14:50:44,153 DEBUG httpclient.HttpClient > (HttpClient.java:executeMethod(373)) - enter > HttpClient.executeMethod(HostConfiguration,HttpMethod,HttpState) > 2011-07-22 14:50:44,153 DEBUG httpclient.MultiThreadedHttpConnectionManager > (MultiThreadedHttpConnectionManager.java:getConnectionWithTimeout(383)) - > enter HttpConnectionManager.getConnectionWithTimeout(HostConfiguration, long) > 2011-07-22 14:50:44,154 DEBUG httpclient.MultiThreadedHttpConnectionManager > (MultiThreadedHttpConnectionManager.java:getConnectionWithTimeout(390)) - > HttpConnectionManager.getConnection: config = > HostConfiguration[host=https://MYBUCKET.s3.amazonaws.com], timeout = 0 > 2011-07-22 14:50:44,154 DEBUG httpclient.MultiThreadedHttpConnectionManager > (MultiThreadedHttpConnectionManager.java:getHostPool(775)) - enter > HttpConnectionManager.ConnectionPool.getHostPool(HostConfiguration) > 2011-07-22 14:50:44,155 DEBUG httpclient.MultiThreadedHttpConnectionManager > (MultiThreadedHttpConnectionManager.java:doGetConnection(494)) - Unable to > get a connection, waiting..., > hostConfig=HostConfiguration[host=https://MYBUCKET.s3.amazonaws.com] > > > --Wouter de Bie > Developer Business Intelligence, Spotify > wou...@spotify.com (mailto:wou...@spotify.com) > +46 72 018 0777 > > On Friday, July 22, 2011 at 11:30 AM, Florin Diaconeasa wrote: > > > Hi, > > > > First of all i think jets3t is called by hadoop and not by hive. Hive, from > > my knowledge, doesn't know about the storage system and i see that the > > jets3t jar is actually inside the lib directory of hadoop. > > > > Apparently there is a way to configure jets3t: > > http://www.jets3t.org/toolkit/configuration.html > > > > I tried putting a jets3t.properties file with > > httpclient.max-connections=100 in the conf dir of hadoop, but not sure it > > took into consideration. > > > > Also, another question: How did you reach the conclusion that the pool got > > depleted? > > > > On 21 July 2011 23:45, Wouter de Bie <wou...@spotify.com > > (mailto:wou...@spotify.com)> wrote: > > > hive 0.7.0+27.1-2~maverick-cdh3 and hadoop 0.20.2+923.21-1 > > > > > > --Wouter de Bie > > > Developer Business Intelligence, Spotify > > > wou...@spotify.com (mailto:wou...@spotify.com) > > > +46 72 018 0777 > > > > > > On Thursday, July 21, 2011 at 9:05 PM, Florin Diaconeasa wrote: > > > > > > > What hive version are you using? > > > > > > > > On Jul 21, 2011, at 1:10 PM, Wouter de Bie wrote: > > > > > Hi guys, > > > > > > > > > > I've just trying to upgrade to jets3t 0.7.4 from 0.6.1, because the > > > > > connection pool gets depleted after 20 requests. Now, I'm getting the > > > > > following stack trace when trying to access s3. Does anyone have a > > > > > clue how to solve this? > > > > > > > > > > FAILED: Hive Internal Error: > > > > > java.lang.RuntimeException(org.apache.hadoop.fs.s3.S3Exception: > > > > > org.jets3t.service.S3ServiceException: Request Error. HEAD '/tmp' on > > > > > Host MYBUCKET.s3.amazonaws.com (http://MYBUCKET.s3.amazonaws.com)' @ > > > > > 'Thu, 21 Jul 2011 09:10:20 GMT' -- ResponseCode: 404, ResponseStatus: > > > > > Not Found, RequestId: 37E8E1DD4E8FC3D8, HostId: > > > > > abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop) > > > > > java.lang.RuntimeException: org.apache.hadoop.fs.s3.S3Exception: > > > > > org.jets3t.service.S3ServiceException: Request Error. HEAD '/tmp' on > > > > > Host 'MYBUCKET.s3.amazonaws.com (http://MYBUCKET.s3.amazonaws.com)' @ > > > > > 'Thu, 21 Jul 2011 09:10:20 GMT' -- ResponseCode: 404, ResponseStatus: > > > > > Not Found, RequestId: 37E8E1DD4E8FC3D8, HostId: > > > > > abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop > > > > > at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:151) > > > > > at org.apache.hadoop.hive.ql.Context.getMRScratchDir(Context.java:190) > > > > > at org.apache.hadoop.hive.ql.Context.getMRTmpFileURI(Context.java:247) > > > > > at > > > > > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:900) > > > > > at > > > > > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:6594) > > > > > at > > > > > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238) > > > > > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:340) > > > > > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:736) > > > > > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164) > > > > > at > > > > > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241) > > > > > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:456) > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > > > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > > > > at java.lang.reflect.Method.invoke(Method.java:597) > > > > > at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > > > > > Caused by: org.apache.hadoop.fs.s3.S3Exception: > > > > > org.jets3t.service.S3ServiceException: Request Error. HEAD '/tmp' on > > > > > Host 'MYBUCKET.s3.amazonaws.com (http://MYBUCKET.s3.amazonaws.com)' @ > > > > > 'Thu, 21 Jul 2011 09:10:20 GMT' -- ResponseCode: 404, ResponseStatus: > > > > > Not Found, RequestId: 37E8E1DD4E8FC3D8, HostId: > > > > > abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop > > > > > at > > > > > org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.handleServiceException(Jets3tNativeFileSystemStore.java:229) > > > > > at > > > > > org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:111) > > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > > > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > > > > at java.lang.reflect.Method.invoke(Method.java:597) > > > > > at > > > > > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) > > > > > at > > > > > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) > > > > > at org.apache.hadoop.fs.s3native.$Proxy4.retrieveMetadata(Unknown > > > > > Source) > > > > > at > > > > > org.apache.hadoop.fs.s3native.NativeS3FileSystem.getFileStatus(NativeS3FileSystem.java:394) > > > > > at > > > > > org.apache.hadoop.fs.s3native.NativeS3FileSystem.mkdir(NativeS3FileSystem.java:507) > > > > > at > > > > > org.apache.hadoop.fs.s3native.NativeS3FileSystem.mkdirs(NativeS3FileSystem.java:500) > > > > > at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1226) > > > > > at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:147) > > > > > ... 15 more > > > > > Caused by: org.jets3t.service.S3ServiceException: Request Error. HEAD > > > > > '/tmp' on Host 'MYBUCKET.s3.amazonaws.com > > > > > (http://MYBUCKET.s3.amazonaws.com)' @ 'Thu, 21 Jul 2011 09:10:20 GMT' > > > > > -- ResponseCode: 404, ResponseStatus: Not Found, RequestId: > > > > > 37E8E1DD4E8FC3D8, HostId: > > > > > abkdf12EFCJBY9FyBwZrC9SEcSibtW7moeZHnSBdUoyKwIO8btedy0Q8AjdNQqop > > > > > at > > > > > org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:520) > > > > > at > > > > > org.jets3t.service.impl.rest.httpclient.RestS3Service.performRestHead(RestS3Service.java:868) > > > > > at > > > > > org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectImpl(RestS3Service.java:2016) > > > > > at > > > > > org.jets3t.service.impl.rest.httpclient.RestS3Service.getObjectDetailsImpl(RestS3Service.java:1944) > > > > > at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:3059) > > > > > at org.jets3t.service.S3Service.getObjectDetails(S3Service.java:1940) > > > > > at > > > > > org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.retrieveMetadata(Jets3tNativeFileSystemStore.java:103) > > > > > ... 27 more > > > > > Caused by: org.jets3t.service.impl.rest.HttpException > > > > > at > > > > > org.jets3t.service.impl.rest.httpclient.RestS3Service.performRequest(RestS3Service.java:518) > > > > > ... 33 more > > > > > > > > > > > > > > > --Wouter de Bie > > > > > Developer Business Intelligence, Spotify > > > > > wou...@spotify.com (mailto:wou...@spotify.com) > > > > > +46 72 018 0777 > > > > > > > > > > > > > > > -- > > > > > > Florin >
