Thanks Peter, Any workaround on HDP 2.6.x with Hive 2 ? Otherwise, the only way is to reduce time it takes for this "merge" queries in order to cancel locks and related transactions. Am I right ?
Le mar. 2 juin 2020 à 11:52, Peter Vary <pv...@cloudera.com> a écrit : > Hi David, > > I think this jira describes your situation: > https://issues.apache.org/jira/browse/HIVE-16360 > > "The reason is that compactor won't compact anything above the level of an > open transaction. So if there is a very long running txn, it may filter > out all deltas in a partition. This looks very confusing when looking at > the logs." > > Thanks, > Peter > > On Jun 2, 2020, at 11:44, David Morin <morin.david....@gmail.com> wrote: > > I don't get it. > The transaction id in the error message "No delta files or original files > found to compact in hdfs://... with min Open txnid:6704143" is related to a > current transaction (transaction + lock) for another user, another database > and table. But still it blocks the compaction for the current database/table > > On 2020/06/01 20:13:08, David Morin <morin.david....@gmail.com> wrote: > > Hi, > > I have a compaction issue on my cluster. When I force a compaction (major) > on one table I get this error in Metastore logs: > > 2020-06-01 19:49:35,512 ERROR [XXXX-78]: compactor.CompactorMR > (CompactorMR.java:run(264)) - No delta files or original files found to > compact in hdfs://...hive/warehouse/my.db/test for compactionId=57944651 > with min Open txnid:6704143. Compaction cannot compact above this txnid > > Any idea about the root cause and how to fix it ? > thx > > Current env: HDP 2.6.5 > Hive table in transactional mode (ORC) > > >
