This error can happen when the job server and sdk versions are mismatched (due to protobuf incompatibilities). The sdk and job server containers should use the same beam version.
On Mon, Dec 30, 2019 at 11:47 AM Yu Watanabe <yu.w.ten...@gmail.com> wrote: > Hello. > > I would like to get help with issue having in job-server. > > I have set up flink (session cluster) and job server everything in > docker , however, jobserver seems to reject requests from the beam > client as described in below error. > > ================================================== > jobserver_1 | [grpc-default-executor-11] WARN > org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService - > Encountered Unexpected Exception during validation > jobserver_1 | java.lang.RuntimeException: Failed to validate > transform ref_AppliedPTransform_Create/FlatMap(<lambda at > core.py:2468>)_4 > jobserver_1 | at > > org.apache.beam.runners.core.construction.graph.PipelineValidator.validateTransform(PipelineValidator.java:203) > jobserver_1 | at > > org.apache.beam.runners.core.construction.graph.PipelineValidator.validateComponents(PipelineValidator.java:112) > <TRIMMED> > jobserver_1 | Caused by: > > org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.InvalidProtocolBufferException: > Protocol message had invalid UTF-8. > jobserver_1 | at > > org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.InvalidProtocolBufferException.invalidUtf8(InvalidProtocolBufferException.java:141) > jobserver_1 | at > > org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.Utf8$DecodeUtil.handleTwoBytes(Utf8.java:1909) > jobserver_1 | at > > org.apache.beam.vendor.grpc.v1p21p0.com.google.protobuf.Utf8$DecodeUtil.access$700(Utf8.java:1883) > <TRIMMED> > ================================================== > > Below is description about my environment. > > 1. Combination of versions of each containers are, > > apache beam: 2.16.0 > sdk: python3.7_sdk: 2.19.0.dev > flink: flink1.9_job_server:latest > > 2. Set up containers using beam repository. > > https://github.com/yuwtennis/beam-deployment/wiki/Setting-up-flink-runner > > 3. Set up docker-compose as below. > > > https://github.com/yuwtennis/beam-deployment/blob/master/flink-session-cluster/docker-compose.yml > > 4. Kept source code very simple. > > > https://github.com/yuwtennis/beam-deployment/blob/master/flink-session-cluster/samples/src/sample.py > > Do I need any other options in job server to properly > serialize/deserialize the incoming message from beam client ? > > My current options are below. > ====================================================== > command: [ "--artifacts-dir", "${ARTIFACTS_DIR}", "--flink-master-url", > "${FLINK_MASTER_URL}", "--job-host", "${JOB_HOST}", "--job-port", > "${JOB_PORT}" ] > ====================================================== > > I appreciate if I could get some help. > > Thanks, > Yu Watanabe > > -- > Yu Watanabe > yu.w.ten...@gmail.com >
