Keshav created BEAM-10334:
-----------------------------
Summary: Message limit for grpc message is set to default 4 MB
Key: BEAM-10334
URL: https://issues.apache.org/jira/browse/BEAM-10334
Project: Beam
Issue Type: Bug
Components: sdk-py-harness
Affects Versions: 2.22.0, 2.18.0
Reporter: Keshav
The default limit of a grpc message is set low to 4 MB, and is not overridden
in the sdk_worker.py for the _control_channel. This was first discovered when
submitting a beam job via a [flink operator orchestrated by
kubernetes|[https://github.com/GoogleCloudPlatform/flink-on-k8s-operator]]. The
below error (for apache-beam 2.18.0) was seen.
{code:java}
2020/06/24 18:37:28 Executing: python -m
apache_beam.runners.worker.sdk_worker_main Traceback (most recent call last):
File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec) File "/usr/local/lib/python3.7/runpy.py", line 85, in
_run_code exec(code, run_globals) File
"/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
line 221, in <module> main(sys.argv) File
"/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker_main.py",
line 155, in main sdk_pipeline_options.view_as(ProfilingOptions)) File
"/usr/local/lib/python3.7/site-packages/apache_beam/runners/worker/sdk_worker.py",
line 117, in run for work_request in control_stub.Control(get_responses()):
File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 416, in
__next__ return self._next() File
"/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 706, in _next
raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of
RPC that terminated with: status = StatusCode.RESOURCE_EXHAUSTED details =
"Received message larger than max (4847139 vs. 4194304)" debug_error_string =
"{"created":"@1593023849.325990420","description":"Received message larger than
max (4847139 vs.
4194304)","file":"src/core/ext/filters/message_size/message_size_filter.cc","file_line":188,"grpc_status":8}"
{code}
This can be easily fixed by modifying the options parameter to allow any size
grpc message to be sent or received.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
