ORC with Map Column Type using Hive 0.11.0-RC1

2013-05-03 Thread Andrew Psaltis
Hello, I am trying to evaluate Hive 0.11.0-RC1, in particular I am very interested in the ORC storage mechanism. We have a need to have one column be a Map in a table and from what I have read this is supported with the ORC format, however when trying to do a select on a table with a Map column

Re: ORC with Map Column Type using Hive 0.11.0-RC1

2013-05-03 Thread Andrew Psaltis
che.org>" mailto:user@hive.apache.org>> Date: Friday, May 3, 2013 2:20 PM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: ORC with Map Column Type using Hive 0.11.0-RC1 On Fri, May 3, 2013 at 10:20 AM, Andrew Psal

OrcFile writing failing with multiple threads

2013-05-24 Thread Andrew Psaltis
All, I have a test application that is attempting to add rows to an OrcFile from multiple threads, however, every time I do I get exceptions with stack traces like the following: java.lang.IndexOutOfBoundsException: Index 4 is outside of 0..5 at org.apache.hadoop.hive.ql.io.orc.DynamicIntArray.g

Re: OrcFile writing failing with multiple threads

2013-05-24 Thread Andrew Psaltis
he Java collections API don't lock themselves. You should synchronize on the writer before adding a row. I'm open to making the writers synchronized. -- Owen On Fri, May 24, 2013 at 11:39 AM, Andrew Psaltis mailto:andrew.psal...@webtrends.com>> wrote: All, I have a test application th

Re: OrcFile writing failing with multiple threads

2013-06-11 Thread Andrew Psaltis
Fri, May 24, 2013 at 1:28 PM, Andrew Psaltis mailto:andrew.psal...@webtrends.com>> wrote: Here is a snippet from the file header comment the WriterImpl for ORC: /** * This class is synchronized so that multi-threaded access is ok. In * particular, because the MemoryManager is sh