I ended up making this change[1] but it looks like Playground is still outputting all the log statements. Could someone help me with this? I suppose it's best to try and actually test this, but I had a hard time getting the environment set up.
[1]https://github.com/apache/beam/pull/29948 """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""ghnt [1] https://github.com/apache/beam/pull/29948#event-11504996254 On Thu, Nov 16, 2023 at 7:42 AM Joey Tran <joey.t...@schrodinger.com> wrote: > Good idea though it ended up being a shallow trace > ``` > File > "/opt/playground/backend/executable_files/91e3e49b-7197-4252-a8bd-93c5b252ed55/91e3e49b-7197-4252-a8bd-93c5b252ed55.py", > line 57, in <module> > assert False > ``` > I think I found where the log level is set anyways > ``` > > https://github.com/apache/beam/blob/master/playground/infrastructure/logger.py#L39 > ``` > > When I have some time, I'll try doing a local deployment of playground and > modifying those log levels > > On Wed, Nov 15, 2023 at 10:52 PM Valentyn Tymofieiev <valen...@google.com> > wrote: > >> I am also not familiar with Playground. I suspect you could try to make >> it crash and maybe find a stacktrace? Setting logging could like like so: >> https://github.com/apache/beam/blob/729c4de416b8252ec99f0a1253ac7af3023733df/sdks/python/apache_beam/examples/wordcount.py#L110 >> >> On Wed, Nov 15, 2023 at 12:06 PM Joey Tran <joey.t...@schrodinger.com> >> wrote: >> >>> The motivating example does not use LogElements, just Map(print) >>> >>> https://beam.apache.org/documentation/transforms/python/aggregation/combineglobally/#example-2-combining-with-a-lambda-function >>> >>> Some examples of the extraneous logging: >>> ``` >>> 2023-09-08 22:46:37,334 [INFO] ==================== <function >>> populate_data_channel_coders at 0x7ff2665e1a20> ==================== >>> 2023-09-08 22:46:37,336 [INFO] Creating state cache with size 104857600 >>> 2023-09-08 22:46:37,338 [INFO] Created Worker handler >>> <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler >>> object at 0x7ff2664c9870> for environment >>> ref_Environment_default_environment_2 (beam:env:embedded_python:v1, b'') >>> ``` >>> >>> The example code itself doesn't set the log level in some playground >>> code. Does anyone have a pointer to where? I'm not familiar >>> >>> On Wed, Nov 15, 2023 at 2:10 PM Valentyn Tymofieiev via dev < >>> dev@beam.apache.org> wrote: >>> >>>> Are the examples using LogElements? >>>> https://github.com/apache/beam/blob/2012107a0fa2bb3fedf1b5aedcb49445534b2dad/sdks/python/apache_beam/transforms/util.py#L1271 >>>> >>>> Note that LogElements by default prints to stdout, but can be >>>> configured to use a different logger. We could also change the default. >>>> >>>> On Tue, Nov 14, 2023 at 9:48 AM Robert Bradshaw via dev < >>>> dev@beam.apache.org> wrote: >>>> >>>>> +1 to at least setting the log level to higher than info. Some runner >>>>> logging (e.g. job started/done) may be useful. >>>>> >>>>> On Tue, Nov 14, 2023 at 9:37 AM Joey Tran <joey.t...@schrodinger.com> >>>>> wrote: >>>>> > >>>>> > Hi all, >>>>> > >>>>> > I just had a workshop to demo beam for people at my company and >>>>> there was a bit of confusion about whether the beam python playground >>>>> examples were even working and it turned out they just got confused by all >>>>> the runner logging that is output. >>>>> > >>>>> > Is this worth keeping? It seems like it'd be a common source of >>>>> confusion for new users >>>>> > >>>>> > Cheers, >>>>> > Joey >>>>> >>>>
