Hi Yong, Thank you for putting this together. The overall approach looks good to me. Since we already use bookkeeper for package management, this proposal just abstracts the package management into a common facility. That seems totally reasonable.
I have two suggestions: 1) make the package storage pluggable. it makes sense to use bookkeeper as the package storage for bare mental deployments. however it makes more sense to use cloud storage for this purpose when running Pulsar in cloud. 2) regarding changes in function runtime, there are many if-else in handling different package protocols (e.g. http, file). we should consider improving that into a better abstraction to make it easier to extend to different protocol in the future. Thanks, Sijie On Thu, Oct 31, 2019 at 5:03 PM Yong Zhang <zhangyong1025...@gmail.com> wrote: > Hi all, > > I am starting a proposal about providing a package management to manage > user functions and connectors. > > *motivation* > > Apache Pulsar provides Functions for users transforming their data as they > want. > If there are many users who use the same function in different namespaces, > they > need to upload the same function multiple times. The same problem exists > in Sinks > and Sources as well. > > Also, there is no version management in Functions, Sinks, and Sources. If > a > new version of a Function is developed, users have to update the function > with a new package. If bugs are found in the new version of the Function, > users have to rollback the function by updating the function with an old > package. > That means users have to manage their packages in their own services. > > The detail proposal is in > > https://docs.google.com/document/d/1FDSNhc8YB1yUMFBUrx8p-1nH1s5Kftp04PkFKWec-2U/edit?usp=sharing > < > https://docs.google.com/document/d/1FDSNhc8YB1yUMFBUrx8p-1nH1s5Kftp04PkFKWec-2U/edit?usp=sharing > > > > If you have any issues feel free to let me know. > > Thanks, > Yong
