Hi Pengcheng, > I think there is no need to setup a "functions" lib?
Yes. Even before 3.2.0 (not released yet), users can still set up the functions by installing `pulsar-client` without the `functions` extra component and installing the other dependencies manually. Thanks, Yunze On Thu, Apr 20, 2023 at 10:49 AM Pengcheng Jiang <pengcheng.ji...@streamnative.io.invalid> wrote: > > Hi Yunze, > > +1 for separating > > And for the migration progress: > 1. since the `pulsar-functions` lib doesn't have any code but just a list > of dependencies, I think there is no need to setup a "functions" lib? > 2. we should update docs about what dependencies are required for Python > functions > 3. the Dockerfile of Pulsar image should be updated to install the required > python dependencies so it can start python function in process runtime mode > 4. the python runner image's Dockerfile should be updated to install the > required python dependencies so it can run in k8s runtime mode > > Regards, > Pengcheng Jiang > > > Yunze Xu <y...@streamnative.io.invalid> 于2023年4月20日周四 10:46写道： > > > Hi Neng, > > > > I think currently a simple solution is to document which dependencies > > should users install to use the Python Functions. > > > > Before: > > > > ``` > > # For version < 3.2.0 > > pip install pulsar-client[functions] > > ``` > > > > Now: > > > > ``` > > # For version >= 3.2.0 > > pip install pulsar-client > > pip install grpc-io > > # Other dependencies... > > ``` > > > > In future, we can provide a separate PyPI package like `pulsar-functions`. > > > > BTW, currently the functions extra dependencies cannot be installed > > for Python 3.10 and 3.11. I have tested the following images with the > > `pip install 'pulsar-client[functions]==3.1.0'` command. > > - python:3.11.2-bullseye > > - python:3.10.11-bullseye > > > > The reason is the version incompatibility of grpcio. > > > > > ERROR: Failed building wheel for grpcio > > > > Though I only tested 3.1.0, since the dependencies never changed, they > > should also fail for older Python client releases like 2.10.0. > > > > Thanks, > > Yunze > > > > On Thu, Apr 20, 2023 at 12:26 AM Neng Lu <nl...@apache.org> wrote: > > > > > > Hi Yunze, > > > > > > +1 for separating Python client and Python Pulsar Functions pip > > installation. > > > On the Java side, the client lib and functions lib are also published > > separately. > > > > > > My concern is how the migration progress should look like, > > > 1. we need to set up functions lib so that users can install it using > > `pip install pulsar-functions` > > > 2. the current `pip install pulsar-client[functions]` should prompt user > > to use the new way > > > 3. all docs need to be updated > > > 4. for historical versions, what can we do? > > > > > > > > > > > > On 2023/04/19 15:23:49 Yunze Xu wrote: > > > > Hi all, > > > > > > > > The Python client has been separated since PIP-209 [1] and now the > > > > Python client is maintained in a separated repository [2]. However, > > > > the Python Function is still maintained in the main repo [3]. > > > > > > > > Currently, we can install the Python client with the following ways: > > > > 1. pip install pulsar-client > > > > 2. pip install pulsar-client[avro] > > > > 3. pip install pulsar-client[functions] > > > > 4. pip install pulsar-client[all] > > > > > > > > See [4] for the difference. However, for the 3rd and 4th ways, it > > > > installs all the dependencies required by the Python Functions. > > > > However, they are broken for the recent releases because of the > > > > outdated dependencies [5]. However, these dependencies are from the > > > > Python Functions [3], not the Python client library itself. Also, > > > > there are no tests in the Python client repo [2] for these functions > > > > so these dependencies cannot be verified. > > > > > > > > IMO, these dependencies should be maintained in the directory of the > > > > Python Functions. We should not rely on the Python client to install > > > > the dependencies for the Python Functions. > > > > > > > > Therefore, my suggestion is to drop the 3rd and 4th installation > > > > methods in future releases of the Python client. After that, we should > > > > update the scripts in the main repo to install the Python Functions in > > > > [3]. > > > > > > > > I'm not familiar with the Pulsar Functions, so feel free to show your > > > > suggestions if any of you have any concerns. > > > > > > > > [1] https://github.com/apache/pulsar/pull/17881 > > > > [2] http://github.com/apache/pulsar-client-python > > > > [3] > > https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python > > > > [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/ > > > > [5] > > https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88 > > > > > > > > Thanks, > > > > Yunze > > > > > > > > > -- > > <https://streamnative.io/> > > Pengcheng Jiang > > Software Engineer > > e: pengcheng.ji...@streamnative.io > > p: 13540631948 > > streamnative.io > > <http://github.com/streamnative> > <https://www.linkedin.com/company/streamnative/> > <https://twitter.com/streamnativeio/>
