Thanks Henry. issue is resolved now, it was a combination of somehow
importing multiple XyzOptions all with very similar argos..#
On Wed, Dec 4, 2024 at 9:26 PM Henry Tremblay via user <user@beam.apache.org>
wrote:
> You can’t have the same argument in the parse_know_args and XyzOptions
> (assuming you get these args in the same way).
>
>
>
> *From:* Sofia’s World <mmistr...@gmail.com>
> *Sent:* Tuesday, December 3, 2024 11:32 PM
> *To:* user@beam.apache.org
> *Subject:* Re: Problem running pipeline in unit tests
>
>
>
> Hello
>
> thanks for getting back.I tracked down the problem to one of the
> pipeline option classes i have been using for years.
>
> Somehow the argparse library is clashing, not sure if it is my IDE or
> something else.....
>
> I have replaced the option class at the bottom of this email with this
> method and i will see how it goes.. but i think it's entirely my issue
>
>
>
> Kind regards
>
>
>
> def parse_known_args(argv):
>
> *"""Parses args for the workflow.""" *parser = argparse.ArgumentParser()
> parser.add_argument('--recipients')
>
>
>
> class XyzOptions(PipelineOptions):
>
>
> @classmethod
> def _add_argparse_args(cls, parser):
> parser.add_argument('--recipients', default='mmistr...@gmail.com')
> parser.add_argument('--key')
> parser.add_argument('--sendgridkey')
>
>
>
> On Tue, Dec 3, 2024 at 10:05 PM XQ Hu via user <user@beam.apache.org>
> wrote:
>
> Can you share the full example to reproduce this?
>
>
>
> On Mon, Dec 2, 2024 at 4:30 PM Sofia’s World <mmistr...@gmail.com> wrote:
>
> HI all
>
> i am starting writing beam test in a new environment and suddenly i am
> finding this error
>
> while running a very simple test
>
>
>
> def test_word_count(self):
>
> with TestPipeline() as p:
>
> input_data = p | 'Start' >> beam.Create(['hello world', 'hello beam'])
>
> counts = (
>
> input_data
>
> | 'One' >>beam.FlatMap(lambda x: x.split(' '))
>
> | 'Two' >> beam.Map(lambda x: (x, 1))
>
> | 'Three' >> beam.CombinePerKey(sum)
>
> )
>
> debugSink = beam.Map(print)
>
> counts | debugSink
>
>
>
>
>
> the bot says it's an issue with argparse, and perhaps i have some lib
> clash... i fixed it long time ago but now i forgot
>
> Could anyone advise how can i address this problem?
>
>
>
> Kind regards
>
> Marco
>
>
>
>
>
> test_word_count
> (shareloader.test.test_stock_pick.TestEdgarUtils.test_word_count) ...
> Missing pipeline option (runner). Executing pipeline using the default
> runner: DirectRunner.
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> 'function' object has no attribute '__self__'
> ERROR
>
> ======================================================================
> ERROR: test_word_count
> (shareloader.test.test_stock_pick.TestEdgarUtils.test_word_count)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/workspaces/GCP_Experiments/dataflow/shareloader/test/test_stock_pick.py",
> line 63, in test_word_count
> with TestPipeline() as p:
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
> line 620, in __exit__
> self.result = self.run()
> ^^^^^^^^^^
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/testing/test_pipeline.py",
> line 115, in run
> result = super().run(
> ^^^^^^^^^^^^
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
> line 570, in run
> self._options).run(False)
> ^^^^^^^^^^
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
> line 594, in run
> return self.runner.run_pipeline(self, self._options)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/runners/direct/direct_runner.py",
> line 151, in run_pipeline
> all_options = options.get_all_options()
> ^^^^^^^^^^^^^^^^^^^^^^^^^
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/options/pipeline_options.py",
> line 351, in get_all_options
> cls._add_argparse_args(parser) # pylint: disable=protected-access
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/workspaces/GCP_Experiments/dataflow/samples/email_pipeline.py",
> line 46, in _add_argparse_args
> parser.add_value_provider_argument('--input_file', type=str,
> File
> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/options/pipeline_options.py",
> line 129, in add_value_provider_argument
> self.add_argument(*args, **kwargs)
> File "/home/codespace/.python/current/lib/python3.12/argparse.py", line
> 1472, in add_argument
> action = action_class(**kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^
> TypeError: _StoreAction.__init__() got an unexpected keyword argument
> 'default_value'
>
>
