tooptoop4 opened a new issue #1801:
URL: https://github.com/apache/hudi/issues/1801
COW data was written with hoodie 0.4.6, now when doing writes with 0.5.3 get
this warning (job still completes)
```
2020-07-06 08:39:05,801 [main] INFO
org.apache.hudi.common.table.HoodieTableConfig - Loading table properties from
s3a://redact/.hoodie/hoodie.properties
2020-07-06 08:39:05,830 [main] INFO
org.apache.hudi.common.table.HoodieTableMetaClient - Finished Loading Table of
type COPY_ON_WRITE(version=1) from s3a://redact/
2020-07-06 08:39:05,830 [main] INFO
org.apache.hudi.common.table.HoodieTableMetaClient - Loading Active commit
timeline for s3a://redact/
2020-07-06 08:39:05,907 [main] INFO
org.apache.hudi.common.table.timeline.HoodieActiveTimeline - Loaded instants
[[==>20200520013324__commit__INFLIGHT], [==>20200520013643__commit__INFLIGHT],
[==>20200528042521__commit__INFLIGHT], [==>20200528042827__commit__INFLIGHT],
[==>20200528043104__commit__INFLIGHT], [==>20200528043932__commit__INFLIGHT],
[==>20200528073521__commit__INFLIGHT], [==>20200603095037__commit__INFLIGHT],
[==>20200624045351__clean__INFLIGHT], [==>20200701042622__commit__INFLIGHT],
[==>20200701044730__clean__INFLIGHT], [20200706081301__clean__COMPLETED],
[20200706081301__commit__COMPLETED], [20200706083709__clean__COMPLETED],
[20200706083709__commit__COMPLETED], [20200706083850__commit__COMPLETED]]
2020-07-06 08:39:05,910 [main] INFO
org.apache.hudi.client.HoodieCleanClient - There were previously unfinished
cleaner operations. Finishing Instant=[==>20200624045351__clean__INFLIGHT]
2020-07-06 08:39:05,968 [main] WARN
org.apache.hudi.client.HoodieCleanClient - Failed to perform previous clean
operation, instant: [==>20200624045351__clean__INFLIGHT]
org.apache.avro.AvroTypeException: Found
com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting
org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
at
org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at
org.apache.avro.io.ResolvingDecoder.readFieldOrder(ResolvingDecoder.java:130)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:215)
at
org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)
at
org.apache.hudi.common.util.AvroUtils.deserializeAvroMetadata(AvroUtils.java:148)
at
org.apache.hudi.common.util.CleanerUtils.getCleanerPlan(CleanerUtils.java:87)
at
org.apache.hudi.client.HoodieCleanClient.runClean(HoodieCleanClient.java:145)
at
org.apache.hudi.client.HoodieCleanClient.lambda$clean$32(HoodieCleanClient.java:89)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at
org.apache.hudi.client.HoodieCleanClient.clean(HoodieCleanClient.java:86)
at
org.apache.hudi.client.HoodieWriteClient.clean(HoodieWriteClient.java:836)
at
org.apache.hudi.client.HoodieWriteClient.postCommit(HoodieWriteClient.java:513)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:157)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:101)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:92)
2020-07-06 08:39:05,971 [main] INFO
org.apache.hudi.client.HoodieCleanClient - There were previously unfinished
cleaner operations. Finishing Instant=[==>20200701044730__clean__INFLIGHT]
2020-07-06 08:39:06,015 [main] WARN
org.apache.hudi.client.HoodieCleanClient - Failed to perform previous clean
operation, instant: [==>20200701044730__clean__INFLIGHT]
org.apache.avro.AvroTypeException: Found
com.uber.hoodie.avro.model.HoodieCleanMetadata, expecting
org.apache.hudi.avro.model.HoodieCleanerPlan, missing required field policy
at
org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at
org.apache.avro.io.ResolvingDecoder.readFieldOrder(ResolvingDecoder.java:130)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:215)
at
org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)
at
org.apache.hudi.common.util.AvroUtils.deserializeAvroMetadata(AvroUtils.java:148)
at
org.apache.hudi.common.util.CleanerUtils.getCleanerPlan(CleanerUtils.java:87)
at
org.apache.hudi.client.HoodieCleanClient.runClean(HoodieCleanClient.java:145)
at
org.apache.hudi.client.HoodieCleanClient.lambda$clean$32(HoodieCleanClient.java:89)
at
java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:647)
at
org.apache.hudi.client.HoodieCleanClient.clean(HoodieCleanClient.java:86)
at
org.apache.hudi.client.HoodieWriteClient.clean(HoodieWriteClient.java:836)
at
org.apache.hudi.client.HoodieWriteClient.postCommit(HoodieWriteClient.java:513)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:157)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:101)
at
org.apache.hudi.client.AbstractHoodieWriteClient.commit(AbstractHoodieWriteClient.java:92)
2020-07-06 08:39:06,022 [main] INFO
org.apache.hudi.common.table.HoodieTableMetaClient - Loading
HoodieTableMetaClient from
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
