Thanks Jack! I've tried that Slack link but it requires an account with a @apache email
On Thu, Jun 22, 2023 at 10:08 AM Jack McCluskey via user < user@beam.apache.org> wrote: > Hey Joey, > > The best resource to look at, at the moment, is likely Robert Burke's > Prism runner that he is implementing ( > https://github.com/apache/beam/tree/master/sdks/go/pkg/beam/runners/prism). > Runners are pretty complicated and there are a number of primitives that > you need to have implemented on the runner side to get executing pipelines. > > The Slack has a link on the Beam Contact Us page > <https://beam.apache.org/community/contact-us/>, and I'd highly recommend > routing questions towards the developer mailing (d...@beam.apache.org) > list rather than the user one for runner implementation things. > > Thanks, > > Jack McCluskey > > On Thu, Jun 22, 2023 at 8:40 AM Joey Tran <joey.t...@schrodinger.com> > wrote: > >> Hi Beam community! >> >> I'm interested in trying to implement a runner with my company's >> execution environment but I'm struggling to get started. I've read the docs >> page >> <https://beam.apache.org/contribute/runner-guide/#testing-your-runner> >> on implementing a runner but it's quite high level. Anyone have any >> concrete suggestions on getting started? >> >> I've started by cloning and running the hello world example >> <https://github.com/apache/beam-starter-python>. I've then subclassed ` >> PipelineRunner >> <https://github.com/apache/beam/blob/9d0fc05d0042c2bb75ded511497e1def8c218c33/sdks/python/apache_beam/runners/runner.py#L103>` >> to create my own custom runner but at this point I'm a bit stuck. My custom >> runner just looks like >> >> class CustomRunner(runner.PipelineRunner): >> def run_pipeline(self, pipeline, >> options): >> self.visit_transforms(pipeline, options) >> >> And when using it I get an error about not having implemented "Impulse" >> >> NotImplementedError: Execution of [<Impulse(PTransform) label=[Impulse]>] >> not implemented in runner <my_app.app.CustomRunner object at 0x135d9ff40>. >> >> Am I going about this the right way? Are there tests I can run my custom >> runner against to validate it beyond just running the hello world example? >> I'm finding myself just digging through the beam source to try to piece >> together how a runner works and I'm struggling to get a foothold. Any >> guidance would be greatly appreciated, especially if anyone has any >> experience implementing their own python runner. >> >> Thanks in advance! Also, could I get a Slack invite? >> Cheers, >> Joey >> >
