Joe McDonnell created IMPALA-14512:
--------------------------------------
Summary: impala-python3 needs a newer version of sh
Key: IMPALA-14512
URL: https://issues.apache.org/jira/browse/IMPALA-14512
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 5.0.0
Reporter: Joe McDonnell
bin/single_node_perf_run.py uses the sh package. When it switched to python3,
this line stopped working and instead produces this error:
{noformat}
14:15:59 Exception in thread Thread-30:
14:15:59 Traceback (most recent call last):
14:15:59 File
"/home/ubuntu/Impala/toolchain/toolchain-packages-gcc10.4.0/python-3.8.18/lib/python3.8/threading.py",
line 932, in _bootstrap_inner
14:15:59 self.run()
14:15:59 File
"/home/ubuntu/Impala/toolchain/toolchain-packages-gcc10.4.0/python-3.8.18/lib/python3.8/threading.py",
line 870, in run
14:15:59 self._target(*self._args, **self._kwargs)
14:15:59 File
"/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/sh.py",
line 1484, in output_thread
14:15:59 done = stream.read()
14:15:59 File
"/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/sh.py",
line 1974, in read
14:15:59 self.write_chunk(chunk)
14:15:59 File
"/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/sh.py",
line 1949, in write_chunk
14:15:59 self.should_quit = self.process_chunk(chunk)
14:15:59 File
"/home/ubuntu/Impala/infra/python/env-gcc10.4.0-py3/lib/python3.8/site-packages/sh.py",
line 1847, in process
14:15:59 handler.write(chunk)
14:15:59 TypeError: write() argument must be str, not bytes{noformat}
This is easy to reproduce locally by running impala-python3 from IMPALA_HOME:
{noformat}
import sh
import sys
sh.cat('README.md', _out=sys.stdout){noformat}
This will fail. We are currently only an old sh version, and sh=2.2.2 works
properly with our python3 version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
