[
https://issues.apache.org/jira/browse/KUDU-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17886535#comment-17886535
]
ASF subversion and git services commented on KUDU-3604:
-------------------------------------------------------
Commit b109bca3c13f09a06034f68c05096cb73d67e3bc in kudu's branch
refs/heads/branch-1.17.x from Marton Greber
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=b109bca3c ]
KUDU-3604: Pin Cython Version, Package Maintenance
Currently, users need to manually install Cython before installing the
Kudu Python package. By running '$ pip install Cython' on most
relatively new systems, they obtain Cython 3.x. However, our setup.py
and build process are not compatible with Cython 3.x. In this patch,
I've added a prerequisite step to setup.py that identifies and
installs the correct Cython version before all the package build/install
steps.
Additionally, the Kudu Python package pulled from PyPi includes all the
test files. This was resolved with a simple change in MANIFEST.in to
exclude the test source folder. Moreover, end-users were also getting
the test dependencies from setup.py because we had a single
requirements.txt file for both distribution and development. I have
now split the dependencies into two files: requirements.txt and
requirements_dev.txt. The dependencies in requirements.txt are
automatically pulled during the execution of setup.py. For development
tasks, developers need to manually install requirements_dev.txt before
running setup.py. The build-and-test.sh script was also updated to
use requirements_dev.txt.
Furthermore, there were several details and edge cases covered in the CI
script related to older OS versions (e.g., el6) and unsupported Python
versions (e.g., Python 2.6), as well as unpinned package versions. Since
all packages are now version-pinned and these limitations are no longer
relevant, I have cleaned up the package installation part in
build-and-test.sh for the Python packages.
As a follow-up patch, I plan to add more developer-facing documentation
on how to set up the development environment, run unit tests, debug,
etc.
Change-Id: I45ad45c56fc3f3ecf57a413edbd0866fccbd8f79
Reviewed-on: http://gerrit.cloudera.org:8080/21816
Reviewed-by: Abhishek Chennaka <achenn...@cloudera.com>
Tested-by: Abhishek Chennaka <achenn...@cloudera.com>
(cherry picked from commit 8d9b3574a12bb98b79e29375c76f98a1a48af5ad)
Reviewed-on: http://gerrit.cloudera.org:8080/21872
Reviewed-by: Alexey Serbin <ale...@apache.org>
> Fix Kudu Python PyPi installation
> ---------------------------------
>
> Key: KUDU-3604
> URL: https://issues.apache.org/jira/browse/KUDU-3604
> Project: Kudu
> Issue Type: Improvement
> Reporter: Marton Greber
> Assignee: Marton Greber
> Priority: Major
>
> Currently if installing Kudu from PyPi the following error is thrown:
> Ubuntu18 x86_64:
> {code:java}
> kudu/client.pyx:3453:16: 'KuduRangePartition' is not a type identifier
>
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> if not isinstance(range_partition.upper_bound, PartialRow):
> upper_bound =
> self._table.schema.new_row(range_partition.upper_bound)
> else:
> upper_bound = range_partition.upper_bound
> upper_bound._own = 0
> p = new KuduRangePartition(
> ^
> ------------------------------------------------------------
>
> kudu/client.pyx:3453:12: new operator can only be applied to a C++ class
> Compiling kudu/client.pyx because it changed.
> Compiling kudu/errors.pyx because it changed.
> Compiling kudu/schema.pyx because it changed.
> [1/3] Cythonizing kudu/client.pyx
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> File "/tmp/pip-install-ACxkFy/kudu-python/setup.py", line 159, in
> <module>
> extensions = cythonize(extensions)
> File
> "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line
> 1154, in cythonize
> cythonize_one(*args)
> File
> "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line
> 1321, in cythonize_one
> raise CompileError(None, pyx_file)
> Cython.Compiler.Errors.CompileError: kudu/client.pyx
> ----------------------------------------
> ERROR: Command errored out with exit status 1: python setup.py egg_info Check
> the logs for full command output.
> {code}
> The solution is to go into the Kudu Python source folder and install the
> requirements file. This shouldn't be necessary.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
