Re: Query on Hadoop-1849 with respect to latest working trunk.

2011-02-04 Thread Aaron Kimball
Mohit, Thanks for tracking this down -- it sounds like a bug. Please file a bug report at https://issues.apache.org/jira/browse/HADOOP - Aaron On Thu, Feb 3, 2011 at 8:53 PM, Mohit wrote: > Hello Authors, > > > > I suspect there is a problem in there, > > > > I configured a property ipc.server

Re: [VOTE] Release Hadoop 0.21.0 (candidate 0)

2010-07-08 Thread Aaron Kimball
I downloaded the "combined" tarball of 0.21.0-rc0 and set it up as a pseudo-distributed Hadoop cluster. Everything seems to work; basic smoke tests pass. Did not run internal unit tests. I tested Sqoop 1.0.0 against this release. All sqoop unit tests pass. Sqoop can operate on the command-line as

[jira] Resolved: (HADOOP-6708) New file format for very large records

2010-05-14 Thread Aaron Kimball (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Kimball resolved HADOOP-6708. --- Resolution: Won't Fix After thinking more about this, I don't think this issue i

[jira] Created: (HADOOP-6754) DefaultCodec.createOutputStream() leaks memory

2010-05-06 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Attachments: CompressionBug.java DefaultCodec.createOutputStream() creates a new Compressor instance in each OutputStream. Even if the OutputStream is closed, this leaks memory. -- This message is automatically generated by JIRA. - You can reply to this email

[jira] Created: (HADOOP-6708) New file format for very large records

2010-04-15 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Assignee: Aaron Kimball A file format that handles multi-gigabyte records efficiently, with lazy disk access -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org

[jira] Created: (HADOOP-6673) nightly builds have incorrect VersionInfo

2010-04-01 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Snapshots of Hadoop trunk downloaded from Ivy have VersionInfo.getVersion() returning "Unknown" -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

Re: Static state in Configuration and elsewhere

2010-02-11 Thread Aaron Kimball
There are an enormous number of examples of the following line in user-side code: Configuration conf = new Configuration(); ... This is going to need to still work transparently after any refactoring. The new Configuration in this case needs to be populated with values from the appropriate defaul

[jira] Created: (HADOOP-6541) An Interactive Hadoop FS shell

2010-02-04 Thread Aaron Kimball (JIRA)
An Interactive Hadoop FS shell -- Key: HADOOP-6541 URL: https://issues.apache.org/jira/browse/HADOOP-6541 Project: Hadoop Common Issue Type: New Feature Reporter: Aaron Kimball Assignee

[jira] Created: (HADOOP-6540) Contrib unit tests have invalid XML for core-site, etc.

2010-02-04 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Blocker The *-site.xml files in src/contrib/test are not valid XML. the declaration must appear above the license header. -- This message is automatically generated by JIRA. - You can reply to this email to add a

Re: build and use hadoop-git

2010-01-24 Thread Aaron Kimball
See http://wiki.apache.org/hadoop/HowToContribute for more step-by-step instructions. - Aaron On Fri, Jan 22, 2010 at 7:36 PM, Kay Kay wrote: > Start with hadoop-common to start building . > > hadoop-hdfs / hadoop-mapred pull the dependencies from apache snapshot > repository that contains the n

[jira] Created: (HADOOP-6497) Introduce wrapper around FSDataInputStream providing Avro SeekableInput interface

2010-01-15 Thread Aaron Kimball (JIRA)
: Hadoop Common Issue Type: New Feature Components: fs Reporter: Aaron Kimball Assignee: Aaron Kimball Reading data from avro files requires using Avro's SeekableInput interface; we need to be able to wrap FSDataInputStream in this interface. --

[jira] Created: (HADOOP-6492) Make avro serialization APIs public

2010-01-13 Thread Aaron Kimball (JIRA)
Make avro serialization APIs public --- Key: HADOOP-6492 URL: https://issues.apache.org/jira/browse/HADOOP-6492 Project: Hadoop Common Issue Type: Improvement Reporter: Aaron Kimball

[jira] Created: (HADOOP-6443) Serialization classes accept invalid metadata

2009-12-15 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Assignee: Aaron Kimball The {{SerializationBase.accept()}} methods of several serialization implementations use incorrect metadata when determining whether they are the correct serializer for the user's metadata. -- This message is automati

[jira] Resolved: (HADOOP-6438) Add configuration getters/setters to serialization classes

2009-12-15 Thread Aaron Kimball (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-6438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Kimball resolved HADOOP-6438. --- Resolution: Invalid After discussion here and on MAPREDUCE-1126, the conclusion is that

[jira] Created: (HADOOP-6438) Add configuration getters/setters to serialization classes

2009-12-11 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Assignee: Aaron Kimball Needed for MAPREDUCE-1126, getter and setter methods to inject specific metadata into configurations to (de)serialize various data types. -- This message is automatically generated by JIRA. - You can reply to this email to add

[jira] Created: (HADOOP-6420) String-to-String Maps should be embeddable in Configuration

2009-12-08 Thread Aaron Kimball (JIRA)
Components: conf Reporter: Aaron Kimball Assignee: Aaron Kimball Per MAPREDUCE-1126, we need to be able to take a map of (key, value) pairs and embed that inside a Configuration object. -- This message is automatically generated by JIRA. - You can reply to this

Re: conf.Configuration: java.io.IOException: config(config)

2009-11-27 Thread Aaron Kimball
When running Hadoop with DEBUG logging on, this IOException was actually responsible for well-over 90% of the lines of text in my logs, making them unreadable. We actually removed this on trunk: https://issues.apache.org/jira/browse/HADOOP-6312 - Aaron On Thu, Nov 19, 2009 at 5:29 AM, Steve Lou

Re: Hadoop coding style guideline

2009-11-27 Thread Aaron Kimball
Thanks for getting that list of examples together. That's a pretty good mix! I went through these too without looking at Todd's comments first to avoid prejudice. Here's my results.. 1) ugly dangling ')' 6-7) would prefer 4 spaces before 'throws' 11-12) ok. 16-17) ok. I don't think we should manda

[jira] Created: (HADOOP-6370) Contrib project ivy dependencies are not included in binary target

2009-11-12 Thread Aaron Kimball (JIRA)
: Bug Components: build Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Critical Attachments: HADOOP-6370.patch Only Hadoop's own library dependencies are promoted to ${build.dir}/lib; any libraries required by contribs ar

Re: How to build and deploy Hadoop 0.21 ?

2009-11-08 Thread Aaron Kimball
On Thu, Nov 5, 2009 at 2:34 AM, Andrei Dragomir wrote: > Hello everyone. > We ran into a bunch of issues with building and deploying hadoop 0.21. > It would be great to get some answers about how things should work, so > we can try to fix them. > > 1. When checking out the repositories, each of t

[jira] Created: (HADOOP-6324) FileSystem API should allow progress callbacks in rename and delete

2009-10-20 Thread Aaron Kimball (JIRA)
Type: Improvement Components: fs Reporter: Aaron Kimball Some operations (e.g., rename and delete) can take very long running times on some filesystem implementations (e.g., S3). The API should provide the ability to include progress callbacks during these operations

[jira] Created: (HADOOP-6315) GzipCodec should not represent BuiltInZlibInflater as decompressorType

2009-10-15 Thread Aaron Kimball (JIRA)
Issue Type: Bug Components: io Reporter: Aaron Kimball Assignee: Aaron Kimball It is possible to pollute CodecPool in such a way that Hadoop cannot read gzip-compressed data. -- This message is automatically generated by JIRA. - You can reply to this email to add

[jira] Created: (HADOOP-6312) Configuration sends too much data to log4j

2009-10-13 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Assignee: Aaron Kimball Attachments: HADOOP-6312.patch Configuration objects send a DEBUG-level log message every time they're instantiated, which include a full stack trace. This is more appropriate for TRACE-level logging, as it renders

Re: Using Hadoop with executables and binary data

2009-08-20 Thread Aaron Kimball
Look into "typed bytes": http://dumbotics.com/2009/02/24/hadoop-1722-and-typed-bytes/ On Thu, Aug 20, 2009 at 10:29 AM, Jaliya Ekanayake wrote: > Hi Stefan, > > > > I am sorry, for the late reply. Somehow the response email has slipped my > eyes. > > Could you explain a bit on how to use Hadoop s

[jira] Created: (HADOOP-6152) Hadoop scripts do not correctly put jars on the classpath

2009-07-15 Thread Aaron Kimball (JIRA)
Reporter: Aaron Kimball Assignee: Aaron Kimball Priority: Blocker Attachments: HADOOP-6152.patch The various Hadoop scripts (bin/hadoop, bin/hdfs, bin/mapred) do not properly identify the jars needed to run Hadoop. They try to include hadoop-*-hdfs.jar, etc, rather

[jira] Reopened: (HADOOP-5482) org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle

2009-07-06 Thread Aaron Kimball (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-5482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Kimball reopened HADOOP-5482: --- This issue was marked "Resolved" but it was never applied to the 20 branch or trunk

Re: Apache git mirrors, post project split

2009-06-29 Thread Aaron Kimball
Chris, No operations in git ever require connectivity to an upstream remote, except for the obvious ones of "pull more down from remote" and "push local refs up to remote." All history and associated metadata is fully replicated to each clone. - Aaron On Mon, Jun 29, 2009 at 12:01 PM, Chris Doug