Posted my answer on SO, reposting here for visibility: Note that the HBaseCoderProviderRegistrar[1] already registers the HBaseMutationCoder for the Mutation type automatically already.
Using the maven-shade plugin without handling service files contained in META-INF/ inside your output jar is a common pitfall for users. You want to add the ServicesResourceTransformer to your maven-shade transformers list. See Apache documentation[2] and ServiceLoader javadoc[3] for more details. There are many other questions like this on StackOverflow. [1]: https://github.com/apache/beam/blob/95f1a5aeba85cd4a6e0eb1ce6eee141e3667d86d/sdks/java/io/hbase/src/main/java/org/apache/beam/sdk/io/hbase/HBaseCoderProviderRegistrar.java#L30 [2]: https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#ServicesResourceTransformer [3]: https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html On Thu, Nov 8, 2018 at 10:13 AM K Fred <[email protected]> wrote: > Hi， > > I am running into an issue using HBaseIO with FlinkRunner. > The exception is below: > Exception in thread "main" java.lang.IllegalStateException: Unable to > return a default Coder for > ParDo(HBaseProfile)/ParMultiDo(HBaseProfile).output [PCollection]. Correct > one of the following root causes: > No Coder has been manually specified; you may do so using .setCoder(). > Inferring a Coder from the CoderRegistry failed: Unable to provide a > Coder for org.apache.hadoop.hbase.client.Mutation. > Building a Coder using a registered CoderProvider failed. > See suppressed exceptions for detailed failures. > Using the default output Coder from the producing PTransform failed: > PTransform.getOutputCoder called. > at > org.apache.beam.repackaged.beam_sdks_java_core.com.google.common.base.Preconditions.checkState(Preconditions.java:444) > at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:278) > at > org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:115) > at > org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:190) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:536) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488) > at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:370) > at ac.cn.iie.process.ProfileProcess.process(ProfileProcess.java:91) > at ac.cn.iie.Bootstrap.main(Bootstrap.java:25) > > I have no idea about this question. Just guess that may be related to > Coder register. So i regist Coder provider manually. The detail is : > https://stackoverflow.com/questions/53200867/unable-to-provide-a-coder-for-org-apache-hadoop-hbase-client-mutation-using-hbas/53200894#53200894 > > Does anyone have better solution? I think it shouldn't register by user. > It camn register automatically by pipeline. > -- > > Fred >
