On Thu, Oct 29, 2026 at 10:09 AM Tarun Annapareddy <[email protected]> wrote:
> Unless there are objections. I am proposing the option-1 short-term fix to > unblock the 2.78 release cut. I will update the design doc for option-3, > once 2.78 is done, it also needs alignment from Runner. > Using option 1 to unblock the release sounds good to me. We can look into options 2 or 3 as the long term fix subsequently. > > Thank you, > Tarun. > > On Wed, Oct 28, 2026 at 4:45 PM Chamikara Jayalath <[email protected]> > wrote: > >> >> >> On Wed, Oct 28, 2026 at 4:35 PM Tarun Annapareddy < >> [email protected]> wrote: >> >>> Hi, >>> >>> ` I understand there is some urgency to have a quick fix as multi-env >>> pipelines might be blocked. If those pipelines are blocked already, do we >>> have a workaround for affected users, for example to use a custom container >>> instead of passing --requirements_file that collides in Python vs YAML >>> environment?` >>> >>> > I do not have a good workaround for these customers. Multi-env YAML >>> pipelines will fail. I am surprised there is no report, perhaps because the >>> regression started only in 2.76 and multi-env is not a regular use case. >>> >> >> This will include all Beam YAML users that use Python providers. >> >> https://beam.apache.org/documentation/sdks/yaml-providers/#python >> >> >>> >>> Thank you, >>> Tarun. >>> >>> On Wed, Oct 28, 2026 at 4:27 PM Valentyn Tymofieiev <[email protected]> >>> wrote: >>> >>>> Thanks, Tarun. >>>> >>>> In addition to boot.go, SDK is also looking for some portion of the >>>> staged files, like pickled_main_session >>>> <https://github.com/apache/beam/blob/0c5adeaff12d46edffa88adbb43fd4ea8fad0f26/sdks/python/apache_beam/runners/worker/sdk_worker_main.py#L405> >>>> (somehow >>>> we are still using it after cloudpickle migration for new purposes), in >>>> #1/#2 SDK would need to be aware that there is an assigned worker ID (and >>>> it is not the same ID anymore as the ID within boot.go, due to sibling >>>> SDK protocol >>>> <https://github.com/apache/beam/blob/0c5adeaff12d46edffa88adbb43fd4ea8fad0f26/sdks/python/container/boot.go#L278> >>>> ). >>>> >>>> We could do a targeted fix just for requirements.txt, which is even >>>> more narrow and reversible. Though I think the proper way to solve >>>> this is on the runner. I understand there is some urgency to have a >>>> quick fix as multi-env pipelines might be blocked. If those pipelines are >>>> blocked already, do we have a workaround for affected users, for example to >>>> use a custom container instead of passing --requirements_file that collides >>>> in Python vs YAML environment? >>>> >>>> If we have a workaround, I'd suggest to try to prototype a runner side >>>> fix before rushing in the SDK change. >>>> >>>> I think fixing the contract on the runner is a cleaner direction (see >>>> #A vs #B in my earlier email), though others are welcome to weigh in here. >>>> >>>> If there are reasons to believe #B is the way to go (SDK must >>>> deduplicate), then I'd explore if we can strengthen some guarantees for >>>> files supplied via --files_to_stage (perhaps via a proto change), so that >>>> we determine the location for these files in a more deterministic manner, >>>> possilbly irrespective of the runner. >>>> >>>
