I used "mvn compile exec:java -P direct-runner -D exec.mainClass=org.apache.beam.examples.WordCount -D exec.args="--inputFile=C%3A/users/ges/workspace/word-count-beam/pom.xml --output=counts"" however got a "NOT FOUND" error. It didn't create any temp files yet.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. null: InvocationTargetException: java.lang.IllegalArgumentException: Error matching the pattern or glob C%3A/users/ges/workspace/word-count-beam/pom.xml: status NOT_FOUND -> [Help 1] I do believe without specifying the URI the compile works on the most parts - mvn compile exec:java -P direct-runner -D exec.mainClass=org.apache.beam.examples.WordCount -D exec.args="--inputFile=pom.xml --output=counts" Failed with [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project word-count-beam: An exception occured while executing the Java class. null: InvocationTargetException: java.lang.IllegalStateException: Unable to find registrar for c -> [Help 1] Is there a way to know which "c" beams tried to read but failed to find registrar? The reason I ask is because from what I can see, the compile and execution definitely already passed the input stage. The content of the file "pom.xml" had already been stored and counted in the temp files generated. Is it possible that Beams was reading some other "c"s somewhere else and reported this error? Thanks, Silas Ge 2017-05-23 22:41 GMT+08:00 Lukasz Cwik <lc...@google.com>: > The paths are URI-like so "C:/path/to/file.txt" resolves to a URI with a > scheme of "C", an authority of "path" and a path of "/to/file.txt" which is > likely to never work since no filesystem handler will be bound to the > scheme "C". > > Try "file:///C%3A/Users/ges/Workspace/word-count-beampom.xml" > (% is the way to escape reserved characters in URIs and 3A is the hex for > ":") > > > On Tue, May 23, 2017 at 2:43 AM, Shuangyin Ge <gosoy...@gmail.com> wrote: > >> When I simply used "C:/Users/ges/Workspace/word-count-beam/pom.xml" as >> input file, I am pretty sure that it has gone farther than using URI >> "file:///", because it has already generated the temp pipeline files and >> wordcount. Just the compile failed with "Unable to find registrar for c". >> To my gut this is a different issue than not able to locate the input file. >> >> Thanks, >> Silas Ge >> >> 2017-05-23 17:31 GMT+08:00 Shuangyin Ge <gosoy...@gmail.com>: >> >>> Hello Luke, >>> >>> I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml" or >>> "C:/Users/ges/Workspace/word-count-beampom.xml" as input file as seen >>> in https://issues.apache.org/jira/browse/BEAM-2298, however both failed. >>> >>> "file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed >>> with Illegal char <:> at index 4 >>> "C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable to >>> find registrar for c" >>> >>> Any ideas? >>> >>> Thanks, >>> Silas Ge >>> >>> 2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>: >>> >>>> This is a known problem with how the local file system is being >>>> interacted with as is evident by the similar test failures when running the >>>> Apache Beam unit tests on Windows: >>>> https://issues.apache.org/jira/browse/BEAM-2299 >>>> >>>> I would suggest trying to use "file:///c:/my/path/to/file.txt" >>>> >>>> There is a general proposal to change how we work with resource paths: >>>> https://issues.apache.org/jira/browse/BEAM-2283 >>>> >>>> >>>> >>>> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <gosoy...@gmail.com> >>>> wrote: >>>> >>>>> Hello team, >>>>> >>>>> Not sure if any can share some lights. I am getting "Unable to find >>>>> registrar for c" error while running >>>>> >>>>> $ mvn compile exec:java >>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount \ >>>>> -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner >>>>> >>>>> following the Java Quickstart for Beam https://beam.apache.org/g >>>>> et-started/quickstart-java/ >>>>> >>>>> Complete output is shown below: >>>>> >>>>> [INFO] Scanning for projects... >>>>> [WARNING] >>>>> [WARNING] Some problems were encountered while building the effective >>>>> model for org.example:word-count-beam:jar:0.1 >>>>> [WARNING] 'build.plugins.plugin.version' for >>>>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80, >>>>> column 15 >>>>> [WARNING] >>>>> [WARNING] It is highly recommended to fix these problems because they >>>>> threaten the stability of your build. >>>>> [WARNING] >>>>> [WARNING] For this reason, future Maven versions might no longer >>>>> support building such malformed projects. >>>>> [WARNING] >>>>> [INFO] >>>>> [INFO] ------------------------------------------------------------ >>>>> ------------ >>>>> [INFO] Building word-count-beam 0.1 >>>>> [INFO] ------------------------------------------------------------ >>>>> ------------ >>>>> [INFO] >>>>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ >>>>> word-count-beam --- >>>>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered >>>>> resources, i.e. build is platform dependent! >>>>> [INFO] skip non existing resourceDirectory >>>>> C:\Users\ges\Workspace\word-count-beam\src\main\resources >>>>> [INFO] >>>>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ >>>>> word-count-beam --- >>>>> [INFO] Changes detected - recompiling the module! >>>>> [WARNING] File encoding has not been set, using platform encoding >>>>> Cp1252, i.e. build is platform dependent! >>>>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co >>>>> unt-beam\target\classes >>>>> [INFO] >>>>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ >>>>> word-count-beam --- >>>>> [WARNING] >>>>> java.lang.reflect.InvocationTargetException >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>>>> ssorImpl.java:62) >>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>>>> thodAccessorImpl.java:43) >>>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293) >>>>> at java.lang.Thread.run(Thread.java:748) >>>>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException: >>>>> java.lang.IllegalStateException: Unable to find registrar for c >>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe >>>>> sult.waitUntilFinish(DirectRunner.java:322) >>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe >>>>> sult.waitUntilFinish(DirectRunner.java:292) >>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner >>>>> .java:200) >>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner >>>>> .java:63) >>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295) >>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281) >>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:184) >>>>> ... 6 more >>>>> Caused by: java.lang.IllegalStateException: Unable to find registrar >>>>> for c >>>>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil >>>>> eSystems.java:447) >>>>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111) >>>>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem >>>>> s.java:174) >>>>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321) >>>>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase >>>>> dSink.java:905) >>>>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces >>>>> sElement(WriteFiles.java:376) >>>>> [INFO] ------------------------------------------------------------ >>>>> ------------ >>>>> [INFO] BUILD FAILURE >>>>> [INFO] ------------------------------------------------------------ >>>>> ------------ >>>>> [INFO] Total time: 10.279 s >>>>> [INFO] Finished at: 2017-05-22T08:56:16+08:00 >>>>> [INFO] Final Memory: 32M/450M >>>>> [INFO] ------------------------------------------------------------ >>>>> ------------ >>>>> [ERROR] Failed to execute goal >>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java >>>>> (default-cli) on project word-count-beam: An exception occured while >>>>> executing the Java class. null: InvocationTargetException: >>>>> java.lang.IllegalStateException: Unable to find registrar for c -> >>>>> [Help 1] >>>>> [ERROR] >>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with >>>>> the -e switch. >>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging. >>>>> [ERROR] >>>>> [ERROR] For more information about the errors and possible solutions, >>>>> please read the following articles: >>>>> [ERROR] [Help 1] http://cwiki.apache.org/con >>>>> fluence/display/MAVEN/MojoExecutionException >>>>> >>>>> Pom.xml is as below >>>>> >>>>> <?xml version="1.0" encoding="UTF-8"?> >>>>> <!-- >>>>> Licensed to the Apache Software Foundation (ASF) under one or more >>>>> contributor license agreements. See the NOTICE file distributed with >>>>> this work for additional information regarding copyright ownership. >>>>> The ASF licenses this file to You under the Apache License, Version 2.0 >>>>> (the "License"); you may not use this file except in compliance with >>>>> the License. You may obtain a copy of the License at >>>>> >>>>> http://www.apache.org/licenses/LICENSE-2.0 >>>>> >>>>> Unless required by applicable law or agreed to in writing, software >>>>> distributed under the License is distributed on an "AS IS" BASIS, >>>>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or >>>>> implied. >>>>> See the License for the specific language governing permissions and >>>>> limitations under the License. >>>>> --> >>>>> <project xmlns="http://maven.apache.org/POM/4.0.0"; >>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 >>>>> http://maven.apache.org/xsd/maven-4.0.0.xsd";> >>>>> <modelVersion>4.0.0</modelVersion> >>>>> >>>>> <groupId>org.example</groupId> >>>>> <artifactId>word-count-beam</artifactId> >>>>> <version>0.1</version> >>>>> >>>>> <packaging>jar</packaging> >>>>> >>>>> <properties> >>>>> <beam.version>2.0.0</beam.version> >>>>> <surefire-plugin.version>2.20</surefire-plugin.version> >>>>> </properties> >>>>> >>>>> <repositories> >>>>> <repository> >>>>> <id>apache.snapshots</id> >>>>> <name>Apache Development Snapshot Repository</name> >>>>> <url>https://repository.apache.org/content/repositories/snapshots/</ >>>>> url> >>>>> <releases> >>>>> <enabled>false</enabled> >>>>> </releases> >>>>> <snapshots> >>>>> <enabled>true</enabled> >>>>> </snapshots> >>>>> </repository> >>>>> </repositories> >>>>> >>>>> <build> >>>>> <plugins> >>>>> <plugin> >>>>> <groupId>org.apache.maven.plugins</groupId> >>>>> <artifactId>maven-compiler-plugin</artifactId> >>>>> <version>3.5.1</version> >>>>> <configuration> >>>>> <source>1.7</source> >>>>> <target>1.7</target> >>>>> </configuration> >>>>> </plugin> >>>>> >>>>> <plugin> >>>>> <groupId>org.apache.maven.plugins</groupId> >>>>> <artifactId>maven-surefire-plugin</artifactId> >>>>> <version>2.20</version> >>>>> <configuration> >>>>> <parallel>all</parallel> >>>>> <threadCount>4</threadCount> >>>>> <redirectTestOutputToFile>true</redirectTestOutputToFile> >>>>> </configuration> >>>>> <dependencies> >>>>> <dependency> >>>>> <groupId>org.apache.maven.surefire</groupId> >>>>> <artifactId>surefire-junit47</artifactId> >>>>> <version>2.20</version> >>>>> </dependency> >>>>> </dependencies> >>>>> </plugin> >>>>> >>>>> <!-- Ensure that the Maven jar plugin runs before the Maven >>>>> shade plugin by listing the plugin higher within the file. --> >>>>> <plugin> >>>>> <groupId>org.apache.maven.plugins</groupId> >>>>> <artifactId>maven-jar-plugin</artifactId> >>>>> </plugin> >>>>> >>>>> <!-- >>>>> Configures `mvn package` to produce a bundled jar ("fat jar") for >>>>> runners >>>>> that require this for job submission to a cluster. >>>>> --> >>>>> <plugin> >>>>> <groupId>org.apache.maven.plugins</groupId> >>>>> <artifactId>maven-shade-plugin</artifactId> >>>>> <version>3.0.0</version> >>>>> <executions> >>>>> <execution> >>>>> <phase>package</phase> >>>>> <goals> >>>>> <goal>shade</goal> >>>>> </goals> >>>>> <configuration> >>>>> <filters> >>>>> <filter> >>>>> <artifact>*:*</artifact> >>>>> <excludes> >>>>> <exclude>META-INF/LICENSE</exclude> >>>>> <exclude>META-INF/*.SF</exclude> >>>>> <exclude>META-INF/*.DSA</exclude> >>>>> <exclude>META-INF/*.RSA</exclude> >>>>> </excludes> >>>>> </filter> >>>>> </filters> >>>>> <transformers> >>>>> <transformer implementation="org.apache.mav >>>>> en.plugins.shade.resource.ServicesResourceTransformer"/> >>>>> </transformers> >>>>> </configuration> >>>>> </execution> >>>>> </executions> >>>>> </plugin> >>>>> </plugins> >>>>> >>>>> <pluginManagement> >>>>> <plugins> >>>>> <plugin> >>>>> <groupId>org.codehaus.mojo</groupId> >>>>> <artifactId>exec-maven-plugin</artifactId> >>>>> <version>1.4.0</version> >>>>> <configuration> >>>>> <cleanupDaemonThreads>false</cleanupDaemonThreads> >>>>> </configuration> >>>>> </plugin> >>>>> </plugins> >>>>> </pluginManagement> >>>>> </build> >>>>> >>>>> <profiles> >>>>> <profile> >>>>> <id>direct-runner</id> >>>>> <activation> >>>>> <activeByDefault>true</activeByDefault> >>>>> </activation> >>>>> <!-- Makes the DirectRunner available when running a pipeline. --> >>>>> <dependencies> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-runners-direct-java</artifactId> >>>>> <version>2.0.0</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> </dependencies> >>>>> </profile> >>>>> >>>>> <profile> >>>>> <id>apex-runner</id> >>>>> <!-- Makes the ApexRunner available when running a pipeline. --> >>>>> <dependencies> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-runners-apex</artifactId> >>>>> <version>${beam.version}</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> <!-- >>>>> Apex depends on httpclient version 4.3.5, project has a transitive >>>>> dependency to httpclient 4.0.1 from >>>>> google-http-client. Apex dependency version being specified explicitly >>>>> so that it gets picked up. This >>>>> can be removed when the project no longer has a dependency on a >>>>> different httpclient version. >>>>> --> >>>>> <dependency> >>>>> <groupId>org.apache.httpcomponents</groupId> >>>>> <artifactId>httpclient</artifactId> >>>>> <version>4.3.5</version> >>>>> <scope>runtime</scope> >>>>> <exclusions> >>>>> <exclusion> >>>>> <groupId>commons-codec</groupId> >>>>> <artifactId>commons-codec</artifactId> >>>>> </exclusion> >>>>> </exclusions> >>>>> </dependency> >>>>> </dependencies> >>>>> </profile> >>>>> >>>>> <profile> >>>>> <id>dataflow-runner</id> >>>>> <!-- Makes the DataflowRunner available when running a pipeline. --> >>>>> <dependencies> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId> >>>>> <version>${beam.version}</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> </dependencies> >>>>> </profile> >>>>> >>>>> <profile> >>>>> <id>flink-runner</id> >>>>> <!-- Makes the FlinkRunner available when running a pipeline. --> >>>>> <dependencies> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-runners-flink_2.10</artifactId> >>>>> <version>${beam.version}</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> </dependencies> >>>>> </profile> >>>>> >>>>> <profile> >>>>> <id>spark-runner</id> >>>>> <!-- Makes the SparkRunner available when running a pipeline. >>>>> Additionally, >>>>> overrides some Spark dependencies to Beam-compatible versions. --> >>>>> <dependencies> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-runners-spark</artifactId> >>>>> <version>${beam.version}</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId> >>>>> <version>${beam.version}</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> <dependency> >>>>> <groupId>org.apache.spark</groupId> >>>>> <artifactId>spark-streaming_2.10</artifactId> >>>>> <version>1.6.2</version> >>>>> <scope>runtime</scope> >>>>> <exclusions> >>>>> <exclusion> >>>>> <groupId>org.slf4j</groupId> >>>>> <artifactId>jul-to-slf4j</artifactId> >>>>> </exclusion> >>>>> </exclusions> >>>>> </dependency> >>>>> <dependency> >>>>> <groupId>com.fasterxml.jackson.module</groupId> >>>>> <artifactId>jackson-module-scala_2.10</artifactId> >>>>> <version>2.8.8</version> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> </dependencies> >>>>> </profile> >>>>> </profiles> >>>>> >>>>> <dependencies> >>>>> <!-- Adds a dependency on the Beam SDK. --> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-sdks-java-core</artifactId> >>>>> <version>${beam.version}</version> >>>>> </dependency> >>>>> >>>>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module. --> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId> >>>>> <version>${beam.version}</version> >>>>> </dependency> >>>>> >>>>> <!-- Dependencies below this line are specific dependencies needed by >>>>> the examples code. --> >>>>> <dependency> >>>>> <groupId>com.google.api-client</groupId> >>>>> <artifactId>google-api-client</artifactId> >>>>> <version>1.22.0</version> >>>>> <exclusions> >>>>> <!-- Exclude an old version of guava that is being pulled >>>>> in by a transitive dependency of google-api-client --> >>>>> <exclusion> >>>>> <groupId>com.google.guava</groupId> >>>>> <artifactId>guava-jdk5</artifactId> >>>>> </exclusion> >>>>> </exclusions> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>com.google.apis</groupId> >>>>> <artifactId>google-api-services-bigquery</artifactId> >>>>> <version>v2-rev295-1.22.0</version> >>>>> <exclusions> >>>>> <!-- Exclude an old version of guava that is being pulled >>>>> in by a transitive dependency of google-api-client --> >>>>> <exclusion> >>>>> <groupId>com.google.guava</groupId> >>>>> <artifactId>guava-jdk5</artifactId> >>>>> </exclusion> >>>>> </exclusions> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>com.google.http-client</groupId> >>>>> <artifactId>google-http-client</artifactId> >>>>> <version>1.22.0</version> >>>>> <exclusions> >>>>> <!-- Exclude an old version of guava that is being pulled >>>>> in by a transitive dependency of google-api-client --> >>>>> <exclusion> >>>>> <groupId>com.google.guava</groupId> >>>>> <artifactId>guava-jdk5</artifactId> >>>>> </exclusion> >>>>> </exclusions> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>com.google.apis</groupId> >>>>> <artifactId>google-api-services-pubsub</artifactId> >>>>> <version>v1-rev10-1.22.0</version> >>>>> <exclusions> >>>>> <!-- Exclude an old version of guava that is being pulled >>>>> in by a transitive dependency of google-api-client --> >>>>> <exclusion> >>>>> <groupId>com.google.guava</groupId> >>>>> <artifactId>guava-jdk5</artifactId> >>>>> </exclusion> >>>>> </exclusions> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>joda-time</groupId> >>>>> <artifactId>joda-time</artifactId> >>>>> <version>2.4</version> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>com.google.guava</groupId> >>>>> <artifactId>guava</artifactId> >>>>> <version>20.0</version> >>>>> </dependency> >>>>> >>>>> <!-- Add slf4j API frontend binding with JUL backend --> >>>>> <dependency> >>>>> <groupId>org.slf4j</groupId> >>>>> <artifactId>slf4j-api</artifactId> >>>>> <version>1.7.14</version> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>org.slf4j</groupId> >>>>> <artifactId>slf4j-jdk14</artifactId> >>>>> <version>1.7.14</version> >>>>> <!-- When loaded at runtime this will wire up slf4j to the JUL backend >>>>> --> >>>>> <scope>runtime</scope> >>>>> </dependency> >>>>> >>>>> <!-- Hamcrest and JUnit are required dependencies of PAssert, >>>>> which is used in the main code of DebuggingWordCount example. --> >>>>> <dependency> >>>>> <groupId>org.hamcrest</groupId> >>>>> <artifactId>hamcrest-all</artifactId> >>>>> <version>1.3</version> >>>>> </dependency> >>>>> >>>>> <dependency> >>>>> <groupId>junit</groupId> >>>>> <artifactId>junit</artifactId> >>>>> <version>4.12</version> >>>>> </dependency> >>>>> >>>>> <!-- The DirectRunner is needed for unit tests. --> >>>>> <dependency> >>>>> <groupId>org.apache.beam</groupId> >>>>> <artifactId>beam-runners-direct-java</artifactId> >>>>> <version>${beam.version}</version> >>>>> <scope>test</scope> >>>>> </dependency> >>>>> </dependencies> >>>>> </project> >>>>> >>>>> >>>>> >>>>> Thanks in advance. >>>>> >>>> >>>> >>> >> >
