Re: Assistance required: Error Translating Pipeline with ReadFromKafka in Apache Beam

2025-02-25 Thread Jan Lukavský
Hi Utharsh, the cause of the error is the use_deprecated_read flag you pass to the expansion service. Dataflow v2 expands KafkaIO using SDF [1] and does not support (as the error says) the depreacted Read transform. Just remove the flag and it should work. Best,  Jan [1] https://beam.apach

Assistance required: Error Translating Pipeline with ReadFromKafka in Apache Beam

2025-02-25 Thread Utkarsh Parekh
Hello, I’m encountering the following error while working with ReadFromKafka with the Dataflow runner V2 in Apache Beam: 2025-02-25 23:14:19.435 PST Error translating pipeline. Unknown non-composite transform urn beam:transform:read:v1 for external_1Reading-messages-from-Kafka-KafkaIO-Read-Ka