Ok, so the bug in the examples is an absence of resources. Having classes in the classpath is not sufficient Modules.java is using ServiceLoader, which is setting private static final String PREFIX = "META-INF/services/" So all the modules have to be listed in the resource files
> On May 27, 2020, at 11:29 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org> wrote: > > > > On Thu, May 28, 2020, 12:19 AM Boris Lublinsky <boris.lublin...@lightbend.com > <mailto:boris.lublin...@lightbend.com>> wrote: > I think I figured this out. > The project seems to be missing > > resources > <https://github.com/ververica/flink-statefun-workshop/tree/master/statefun-workshop-functions/src/main/resources>/META-INF > > <https://github.com/ververica/flink-statefun-workshop/tree/master/statefun-workshop-functions/src/main/resources/META-INF>/services > directory, which should contain services > > Yes, the functions / ingresses / regresses etc. are not discoverable if the > service file isnt present in the classpath. > > For the examples, if you are running it straight from the repo, should all > have that service file defined and therefore readily runnable. > > If you are creating your own application project, you'll have to add that > yourself. > > > Another question: > I see org.apache.flink.statefun.flink.io.datastream.SourceSinkModule > > Class, which I think allows to use existing data streams as ingress/egress. > > Are there any examples of its usage > > On the Harness class, there is a withFlinkSourceFunction method in which you > can directly add a Flink source function as the ingress. > > If you want to use that directly in a normal application (not just execution > in IDE with the Harness), you can define your ingesses/egresses by binding > SourceFunctionSpec / SinkFunctionSpec. > Please see how they are being used in the Harness class for examples. > > Gordon > > > >> On May 27, 2020, at 11:10 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org >> <mailto:tzuli...@apache.org>> wrote: >> >> Hi, >> >> The example is working fine on my side (also using IntelliJ). >> This could most likely be a problem with your project setup in the IDE, >> where the classpath isn't setup correctly. >> >> What do you see when you right click on the statefun-flink-harness-example >> directory (in the IDE) --> Open Module Settings, and then under the >> "Sources" / "Dependencies" tab? >> Usually this should all be automatically setup correctly when importing the >> project. >> >> Gordon >> >> On Wed, May 27, 2020 at 11:46 PM Boris Lublinsky >> <boris.lublin...@lightbend.com <mailto:boris.lublin...@lightbend.com>> wrote: >> The project >> https://github.com/apache/flink-statefun/tree/release-2.0/statefun-examples/statefun-flink-harness-example >> >> <https://github.com/apache/flink-statefun/tree/release-2.0/statefun-examples/statefun-flink-harness-example> >> Does not work in Intellij. >> >> The problem is that when running in Intellij, method public static Modules >> loadFromClassPath() { >> Does not pick up classes, which are local in Intellij >> >> Any work arounds? >> >> >> >> >>> On May 22, 2020, at 12:03 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org >>> <mailto:tzuli...@apache.org>> wrote: >>> >>> Hi, >>> >>> Sorry, I need to correct my comment on using the Kafka ingress / egress >>> with the Harness. >>> >>> That is actually doable, by adding an extra dependency to >>> `statefun-flink-distribution` in your Harness program. >>> That pulls in all the other required dependencies required by the Kafka >>> ingress / egress, such as the source / sink providers and Flink Kafka >>> connectors. >>> >>> Cheers, >>> Gordon >>> >>> On Fri, May 22, 2020 at 12:04 PM Tzu-Li (Gordon) Tai <tzuli...@apache.org >>> <mailto:tzuli...@apache.org>> wrote: >>> Are you getting an exception from running the Harness? >>> The Harness should already have the required configurations, such as the >>> parent first classloading config. >>> >>> Otherwise, if you would like to add your own configuration, use the >>> `withConfiguration` method on the `Harness` class. >>> >>> On Fri, May 22, 2020 at 7:19 AM Boris Lublinsky >>> <boris.lublin...@lightbend.com <mailto:boris.lublin...@lightbend.com>> >>> wrote: >>> Also, where do I put flint-conf.yaml in Idea to add additional required >>> config parameter: >>> classloader.parent-first-patterns.additional: >>> org.apache.flink.statefun;org.apache.kafka;com.google.protobuf >>> >>> >>>> On May 21, 2020, at 12:22 PM, Boris Lublinsky >>>> <boris.lublin...@lightbend.com <mailto:boris.lublin...@lightbend.com>> >>>> wrote: >>>> >>>> Hi, >>>> I am trying to run >>>> https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-greeter-example >>>> >>>> <https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-greeter-example> >>>> locally >>>> using >>>> https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-flink-harness-example >>>> >>>> <https://github.com/apache/flink-statefun/tree/master/statefun-examples/statefun-flink-harness-example> >>>> >>>> And have several questions. >>>> 1. It seems fairly straightforward to use it with in memory message >>>> generators, but I can’t figure out how to add Kafka ingress/Egress so that >>>> I can use it with Kafk >>>> 2. GreetingModule already creates StatefulFunctionUniverse and so does >>>> Harness. Is there a way to short circuit it and have Harness get >>>> StatefulFunctionUniverse directly >>>> 3. Is there an example on how to write Flink main for stageful function? >>>> 4. Is there an example anywhere on how to run such examples in the IDE >>>> with Kafka? >>>> 5 There is a great stateful functions example >>>> https://github.com/ververica/flink-statefun-workshop >>>> <https://github.com/ververica/flink-statefun-workshop>, but its readme >>>> does not really describe implementation and neither does this article, >>>> referencing it >>>> https://dev.to/morsapaes/flink-stateful-functions-where-to-start-2j39 >>>> <https://dev.to/morsapaes/flink-stateful-functions-where-to-start-2j39>. >>>> Is there anything that describes this implementation? >>>> >>> >> >
