thanks XQ.. must have been my mistake in altering the signature of the run() method i copied the example you provided and all is good
kind regards On Sun, Oct 6, 2024 at 12:41 AM XQ Hu via user <[email protected]> wrote: > This looks strange. It looks like add_argument is called twice. > > I am not sure it will work but can you put all your parser to a function > like parse_known_args and then call it from run? > > e.g., > https://github.com/apache/beam/blob/d4dd58b2c4c4b5867ace4bdd34e1bcc32de963cc/sdks/python/apache_beam/examples/per_entity_training.py#L110 > > On Sat, Oct 5, 2024 at 4:48 PM Sofia’s World <[email protected]> wrote: > >> HI all >> not quite sure what is happening, but i have a job that has been running >> with these parameters for the last month - from logs - >> >> Using launch args: [/template/dataflow_tester.py --limit=10 >> --project=datascience-projects >> --staging_location=gs://datascience-projects_cloudbuild >> --subnetwork=regions/us-central1/subnetworks/default >> --temp_location=gs://datascience-projects_cloudbuild --input= >> --template_location=gs://datascience-projects_cloudbuild/template_launches/2024-10-05_13_33_33-13068921462026318228/job_object >> --fmprepkey=abcdef --output=annual_fundamentals --pat=mypat >> --sdk_container_image=gcr.io/datascience-projects/dftester:latest3 >> --runner=DataflowRunner --job_name=dftester --region=us-central1 >> --service_account_email=682143946483 --period=10 --runtype=premarket] >> >> and suddenly i am getting this exception >> >> argparse.ArgumentError: argument --fmprepkey: conflicting option string: >> --fmprepkey >> >> this is the code that kickstart my pipeline... not sure why suddenly i am >> getting this error suddenly >> >> def run(argv: list[str] | None = None): >> """Main entry point; defines and runs the wordcount pipeline.""" >> >> logging.info(f'running with arguments:{argv}') >> parser = argparse.ArgumentParser() >> parser.add_argument('--fmprepkey') >> parser.add_argument('--input') >> parser.add_argument('--output') >> parser.add_argument('--period') >> parser.add_argument('--limit') >> parser.add_argument('--pat') >> parser.add_argument('--runtype') >> >> pipeline_options, other_args = parser.parse_known_args(argv) >> >> >> # We use the save_main_session option because one or more DoFn's in this >> # workflow rely on global context (e.g., a module imported at module >> level). >> logging.info(f'Starting tester pipeline:{pipeline_options.fmprepkey}') >> >> Could any one advice? did i pick up some recent changes? >> >> Kind regards >> Marco >> >> >> >> >> >> >> >> >> >> >>
