Re: ORC: duplicate record - rowid meaning ?

2019-12-01 Thread Peter Vary
Thanks David, Hope that Hive 3 streaming will help you soon to avoid these kind of headaches :) Peter > On Dec 1, 2019, at 17:57, David Morin wrote: > > Hi Peter, > > At the moment I have a pipeline based on Flink to write Orc Files. These Orc > Files can be read from Hive thanks to external

Re: ORC: duplicate record - rowid meaning ?

2019-12-01 Thread David Morin
Hi Peter, At the moment I have a pipeline based on Flink to write Orc Files. These Orc Files can be read from Hive thanks to external tables and, then, a merge statement (triggered by oozie) push these data into tables managed by Hive (transactional tables => ORC). Hive version is 2.1 because t

Re: hive error: "Too many bytes before delimiter: 2147483648"

2019-12-01 Thread Shawn Weeks
That looks like you’ve encountered a file with no delimiter as that’s near the max size for an array or string. Also I don’t think you can terminate fields with a line feed as that’s the hard coded row delimiter. Thanks Shawn From: xuanhuang <18351886...@163.com> Reply-To: "user@hive.apache.org