This is your daily summary of Beam's current high priority issues that may need
attention.
See https://beam.apache.org/contribute/issue-priorities for the meaning and
expectations around issue priorities.
Unassigned P1 Issues:
https://github.com/apache/beam/issues/29973 [Failing Test]:
testReshufflePreservesMetadata not working for some runners
https://github.com/apache/beam/issues/29972 [Bug]:
testHotKeyCombineWithSideInputs permared on Spark SparkStructuredStreaming
runner
https://github.com/apache/beam/issues/29971 [Bug]: FixedWindows not working for
large Kafka topic
https://github.com/apache/beam/issues/29926 [Bug]: FileIO: lack of timeouts may
cause the pipeline to get stuck indefinitely
https://github.com/apache/beam/issues/29912 [Bug]: floatValueExtractor judge
float and double equality directly
https://github.com/apache/beam/issues/29902 [Bug]: Messages are not ACK on
Pubsub starting Beam 2.52.0 on Flink Runner in detached mode
https://github.com/apache/beam/issues/29825 [Bug]: Usage of logical types
breaks Beam YAML Sql
https://github.com/apache/beam/issues/29413 [Bug]: Can not use Avro over 1.8.2
with Beam 2.52.0
https://github.com/apache/beam/issues/29099 [Bug]: FnAPI Java SDK Harness
doesn't update user counters in OnTimer callback functions
https://github.com/apache/beam/issues/29022 [Failing Test]: Python Github
actions tests are failing due to update of pip
https://github.com/apache/beam/issues/28760 [Bug]: EFO Kinesis IO reader
provided by apache beam does not pick the event time for watermarking
https://github.com/apache/beam/issues/28715 [Bug]: Python WriteToBigtable get
stuck for large jobs due to client dead lock
https://github.com/apache/beam/issues/28383 [Failing Test]:
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorkerTest.testMaxThreadMetric
https://github.com/apache/beam/issues/28339 Fix failing
"beam_PostCommit_XVR_GoUsingJava_Dataflow" job
https://github.com/apache/beam/issues/28326 Bug:
apache_beam.io.gcp.pubsublite.ReadFromPubSubLite not working
https://github.com/apache/beam/issues/28142 [Bug]: [Go SDK] Memory seems to be
leaking on 2.49.0 with Dataflow
https://github.com/apache/beam/issues/27892 [Bug]: ignoreUnknownValues not
working when using CreateDisposition.CREATE_IF_NEEDED
https://github.com/apache/beam/issues/27648 [Bug]: Python SDFs (e.g.
PeriodicImpulse) running in Flink and polling using tracker.defer_remainder
have checkpoint size growing indefinitely
https://github.com/apache/beam/issues/27616 [Bug]: Unable to use
applyRowMutations() in bigquery IO apache beam java
https://github.com/apache/beam/issues/27486 [Bug]: Read from datastore with
inequality filters
https://github.com/apache/beam/issues/27314 [Failing Test]:
bigquery.StorageApiSinkCreateIfNeededIT.testCreateManyTables[1]
https://github.com/apache/beam/issues/27238 [Bug]: Window trigger has lag when
using Kafka and GroupByKey on Dataflow Runner
https://github.com/apache/beam/issues/26911 [Bug]: UNNEST ARRAY with a nested
ROW (described below)
https://github.com/apache/beam/issues/26343 [Bug]:
apache_beam.io.gcp.bigquery_read_it_test.ReadAllBQTests.test_read_queries is
flaky
https://github.com/apache/beam/issues/26329 [Bug]: BigQuerySourceBase does not
propagate a Coder to AvroSource
https://github.com/apache/beam/issues/26041 [Bug]: Unable to create
exactly-once Flink pipeline with stream source and file sink
https://github.com/apache/beam/issues/24776 [Bug]: Race condition in Python SDK
Harness ProcessBundleProgress
https://github.com/apache/beam/issues/24389 [Failing Test]:
HadoopFormatIOElasticTest.classMethod ExceptionInInitializerError
ContainerFetchException
https://github.com/apache/beam/issues/24313 [Flaky]:
apache_beam/runners/portability/portable_runner_test.py::PortableRunnerTestWithSubprocesses::test_pardo_state_with_custom_key_coder
https://github.com/apache/beam/issues/23944 beam_PreCommit_Python_Cron
regularily failing - test_pardo_large_input flaky
https://github.com/apache/beam/issues/23709 [Flake]: Spark batch flakes in
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElement and
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundle
https://github.com/apache/beam/issues/23525 [Bug]: Default PubsubMessage coder
will drop message id and orderingKey
https://github.com/apache/beam/issues/22913 [Bug]:
beam_PostCommit_Java_ValidatesRunner_Flink is flakes in
org.apache.beam.sdk.transforms.GroupByKeyTest$BasicTests.testAfterProcessingTimeContinuationTriggerUsingState
https://github.com/apache/beam/issues/22605 [Bug]: Beam Python failure for
dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest.test_metrics_it
https://github.com/apache/beam/issues/21714
PulsarIOTest.testReadFromSimpleTopic is very flaky
https://github.com/apache/beam/issues/21706 Flaky timeout in github Python unit
test action
StatefulDoFnOnDirectRunnerTest.test_dynamic_timer_clear_then_set_timer
https://github.com/apache/beam/issues/21643 FnRunnerTest with non-trivial
(order 1000 elements) numpy input flakes in non-cython environment
https://github.com/apache/beam/issues/21476 WriteToBigQuery Dynamic table
destinations returns wrong tableId
https://github.com/apache/beam/issues/21469 beam_PostCommit_XVR_Flink flaky:
Connection refused
https://github.com/apache/beam/issues/21424 Java VR (Dataflow, V2, Streaming)
failing: ParDoTest$TimestampTests/OnWindowExpirationTests
https://github.com/apache/beam/issues/21262 Python AfterAny, AfterAll do not
follow spec
https://github.com/apache/beam/issues/21260 Python DirectRunner does not emit
data at GC time
https://github.com/apache/beam/issues/21121
apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT.test_streaming_wordcount_it
flakey
https://github.com/apache/beam/issues/21104 Flaky:
apache_beam.runners.portability.fn_api_runner.fn_runner_test.FnApiRunnerTestWithGrpcAndMultiWorkers
https://github.com/apache/beam/issues/20976
apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized.test_flink_metrics
is flaky
https://github.com/apache/beam/issues/20108 Python direct runner doesn't emit
empty pane when it should
https://github.com/apache/beam/issues/19814 Flink streaming flakes in
ParDoLifecycleTest.testTeardownCalledAfterExceptionInStartBundleStateful and
ParDoLifecycleTest.testTeardownCalledAfterExceptionInProcessElementStateful
P1 Issues with no update in the last week:
https://github.com/apache/beam/issues/29818 [Failing Test]:
beam_PostCommit_Java_ValidatesRunner_ULR is perma-red
https://github.com/apache/beam/issues/29760 [Bug]: Go SDK Dataflow jobs fail on
DataSampling disabled
https://github.com/apache/beam/issues/29515 [Bug]: WriteToFiles in python leave
few records in temp directory when writing to large number (100+) of files
https://github.com/apache/beam/issues/28219 [Bug]: BigQuery IO Batch load using
File_load causing the same job id ignoring inserts as the job_id is already
completed
https://github.com/apache/beam/issues/27022 [Bug]: Possible data loss in
BigtableIO r/w with large number of row and column
https://github.com/apache/beam/issues/26354 [Bug]: BigQueryIO direct read not
reading all rows when set --setEnableBundling=true
https://github.com/apache/beam/issues/25975 [Bug]: KinesisIO processing-time
watermarking can cause data loss
