Re: How to read data from bigtable

2021-07-12 Thread Kenneth Knowles
Hi Raja & all, I'm moving the thread to user@beam.apache.org since I think many users may be interested in the answer to the question, or have other experiences to share. Kenn On Mon, Jul 12, 2021 at 12:58 PM Chamikara Jayalath wrote: > There's also an external connector that us directly suppo

Re: Dataproc+Flink+Beam Portable Runner Tutorial

2021-07-12 Thread Joey Tran
Yeah I saw the 1.9 and tried it initially but the cluster flink version is actually updated and in the 2.0 image flink==1.12 (I confirmed by looking at the flink dashboard from the YARN ResourceManager web interface). I checked out beam==2.30.0 and built :runners:flink:1.12:job-server:shadowJar an

[Bug] error in FlinkRunner with pub/sub

2021-07-12 Thread swarna kamal Paul
Dear Team, I am facing the following error while running a streaming pipeline (python) in Apache Beam on Flinkrunner. The pipeline contains a GCP pub/sub io source and pub/sub target. WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. ERROR:root:java.la