Re: External Table with unclosed orc files.

> "ga...@apache.org <mailto:ga...@apache.org>" <mailto:ga...@apache.org>> > Date: Wednesday, April 15, 2015 at 4:07 AM > To: "user@hive.apache.org <mailto:user@hive.apache.org>" > mailto:user@hive.apache.org>> > Subject: Re: External

Re: External Table with unclosed orc files.

o:user@hive.apache.org>>, "ga...@apache.org<mailto:ga...@apache.org>" mailto:ga...@apache.org>> Date: Wednesday, April 15, 2015 at 4:07 AM To: "user@hive.apache.org<mailto:user@hive.apache.org>" mailto:user@hive.apache.org>> Subject: Re: External

Re: External Table with unclosed orc files.

Grant Overby (groverby) wrote: > Thanks for the link to the hive streaming bolt. We rolled our own bolt > many moons ago to utilize hive streaming. We’ve tried it against 0.13 and > 0.14 . Acid tables have been a real pain for us. We don’t believe they are > production ready. At least in our use

Re: External Table with unclosed orc files.

IIRC the HW Trucking Demo creates a temporary table from csv files of the new data then issues a select … insert into an orc table. For the love of google, I can’t find this demo atm, and I’m out of time. If I recall correctly, this strikes me as suboptimal compared to writing orc files directly

Re: External Table with unclosed orc files.

Submitting patches or test cases is tricky business for a Cisco employee. I’ll put in the legal admin effort to get approval to do this. :/ The majority of the issues I mentioned /should/ find their way to apache via hortonworks. Additional responses are inline. On 4/14/15, 5:28 PM, "Gop

Re: External Table with unclosed orc files.

loyees >accept >any responsibility. > > >-----Original Message----- >From: Grant Overby (groverby) [mailto:grove...@cisco.com] >Sent: 14 April 2015 22:02 >To: Gopal Vijayaraghavan; user@hive.apache.org >Subject: Re: External Table with unclosed orc files. > >Thanks fo

Re: External Table with unclosed orc files.

unsubscribe On Tue, Apr 14, 2015 at 4:28 PM, Gopal Vijayaraghavan wrote: > > >0.14 . Acid tables have been a real pain for us. We don¹t believe they are > >production ready. At least in our use cases, Tez crashes for assorted > >reasons or only assigns 1 mapper to the partition. Having delta fil

Re: External Table with unclosed orc files.

>0.14 . Acid tables have been a real pain for us. We don¹t believe they are >production ready. At least in our use cases, Tez crashes for assorted >reasons or only assigns 1 mapper to the partition. Having delta files and >no base files borks mapper assignments. Some of the chicken-egg problems f

RE: External Table with unclosed orc files.

ility. -Original Message- From: Grant Overby (groverby) [mailto:grove...@cisco.com] Sent: 14 April 2015 22:02 To: Gopal Vijayaraghavan; user@hive.apache.org Subject: Re: External Table with unclosed orc files. Thanks for the link to the hive streaming bolt. We rolled our own bolt many moons

Re: External Table with unclosed orc files.

Thanks for the link to the hive streaming bolt. We rolled our own bolt many moons ago to utilize hive streaming. We’ve tried it against 0.13 and 0.14 . Acid tables have been a real pain for us. We don’t believe they are production ready. At least in our use cases, Tez crashes for assorted reasons o

Re: External Table with unclosed orc files.

> What will Hive do if querying an external table containing orc files >that are still being written to? Doing that directly won¹t work at all. Because ORC files are only readable after the Footer is written out, which won¹t be for any open files. > I won¹t be able to test these scenarios till t

Re: External Table with unclosed orc files.

It will fail. Orc writes info in the footers that are required to properly read the file. If close hasn't been called, then that footer hasn't been written yet. Alan. Grant Overby (groverby) April 14, 2015 at 20:46 What will Hive do if querying an external table

RE: External Table with unclosed orc files.

che.org Subject: External Table with unclosed orc files. What will Hive do if querying an external table containing orc files that are still being written to? If the process writing the orc files exits without calling .close()? Sorry for taking the cheap way out and asking instead of

External Table with unclosed orc files.

What will Hive do if querying an external table containing orc files that are still being written to? If the process writing the orc files exits without calling .close()? Sorry for taking the cheap way out and asking instead of testing. I couldn’t find anything on this via google. I won’t be a