Hi Yunze, +1 for separating
And for the migration progress: 1. since the `pulsar-functions` lib doesn't have any code but just a list of dependencies, I think there is no need to setup a "functions" lib? 2. we should update docs about what dependencies are required for Python functions 3. the Dockerfile of Pulsar image should be updated to install the required python dependencies so it can start python function in process runtime mode 4. the python runner image's Dockerfile should be updated to install the required python dependencies so it can run in k8s runtime mode Regards, Pengcheng Jiang Yunze Xu <y...@streamnative.io.invalid> 于2023年4月20日周四 10:46写道： > Hi Neng, > > I think currently a simple solution is to document which dependencies > should users install to use the Python Functions. > > Before: > > ``` > # For version < 3.2.0 > pip install pulsar-client[functions] > ``` > > Now: > > ``` > # For version >= 3.2.0 > pip install pulsar-client > pip install grpc-io > # Other dependencies... > ``` > > In future, we can provide a separate PyPI package like `pulsar-functions`. > > BTW, currently the functions extra dependencies cannot be installed > for Python 3.10 and 3.11. I have tested the following images with the > `pip install 'pulsar-client[functions]==3.1.0'` command. > - python:3.11.2-bullseye > - python:3.10.11-bullseye > > The reason is the version incompatibility of grpcio. > > > ERROR: Failed building wheel for grpcio > > Though I only tested 3.1.0, since the dependencies never changed, they > should also fail for older Python client releases like 2.10.0. > > Thanks, > Yunze > > On Thu, Apr 20, 2023 at 12:26 AM Neng Lu <nl...@apache.org> wrote: > > > > Hi Yunze, > > > > +1 for separating Python client and Python Pulsar Functions pip > installation. > > On the Java side, the client lib and functions lib are also published > separately. > > > > My concern is how the migration progress should look like, > > 1. we need to set up functions lib so that users can install it using > `pip install pulsar-functions` > > 2. the current `pip install pulsar-client[functions]` should prompt user > to use the new way > > 3. all docs need to be updated > > 4. for historical versions, what can we do? > > > > > > > > On 2023/04/19 15:23:49 Yunze Xu wrote: > > > Hi all, > > > > > > The Python client has been separated since PIP-209 [1] and now the > > > Python client is maintained in a separated repository [2]. However, > > > the Python Function is still maintained in the main repo [3]. > > > > > > Currently, we can install the Python client with the following ways: > > > 1. pip install pulsar-client > > > 2. pip install pulsar-client[avro] > > > 3. pip install pulsar-client[functions] > > > 4. pip install pulsar-client[all] > > > > > > See [4] for the difference. However, for the 3rd and 4th ways, it > > > installs all the dependencies required by the Python Functions. > > > However, they are broken for the recent releases because of the > > > outdated dependencies [5]. However, these dependencies are from the > > > Python Functions [3], not the Python client library itself. Also, > > > there are no tests in the Python client repo [2] for these functions > > > so these dependencies cannot be verified. > > > > > > IMO, these dependencies should be maintained in the directory of the > > > Python Functions. We should not rely on the Python client to install > > > the dependencies for the Python Functions. > > > > > > Therefore, my suggestion is to drop the 3rd and 4th installation > > > methods in future releases of the Python client. After that, we should > > > update the scripts in the main repo to install the Python Functions in > > > [3]. > > > > > > I'm not familiar with the Pulsar Functions, so feel free to show your > > > suggestions if any of you have any concerns. > > > > > > [1] https://github.com/apache/pulsar/pull/17881 > > > [2] http://github.com/apache/pulsar-client-python > > > [3] > https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python > > > [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/ > > > [5] > https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88 > > > > > > Thanks, > > > Yunze > > > > -- <https://streamnative.io/> Pengcheng Jiang Software Engineer e: pengcheng.ji...@streamnative.io p: 13540631948 streamnative.io <http://github.com/streamnative> <https://www.linkedin.com/company/streamnative/> <https://twitter.com/streamnativeio/>
