Dear Kota, On Mon, Sep 17, 2018 at 11:43 PM Kota TSUYUZAKI <tsuyuzaki.k...@lab.ntt.co.jp> wrote: > > Hi Quio, > > > I know Storlets can provide user-defined computation functionalities, > > but I guess some capabilities can only be achieved using middleware. > > For example, a user may want such a feature: upon each PUT request, it > > creates a compressed copy of the object and stores both the original > > copy and compressed copy. It's feasible using middlware but I don't > > think Storlets provide such capability. > > Interesting, exactly currently it's not supported to write to multi objects > for a PUT request but as well as other middlewares we could adopt the > feasibility into Storlets if you prefer. > Right now, the multi read (i.e. GET from multi sources) is only available and > I think we would be able to expand the logic to PUT requests too. IIRC, in > those days, we had discussion on sort of the > multi-out use cases and I'm sure the data structure inside Storlets are > designed to be capable to that expantion. At that time, we called them "Tee" > application on Storlets, I could not find the > historical discussion logs about how to implement tho, sorry. I believe that > would be an use case for storlets if you prefer the user-defined application > flexibilities rather than operator defined > Swift middleware. > > The example of multi-read (GET from multi sources) are here: > https://github.com/openstack/storlets/blob/master/tests/functional/python/test_multiinput_storlet.py > > And if you like to try to write multi write, please join us, I'm happy to > help you anytime. >
Thanks! I'm interested and would like to join, as well as contribute! Another potential use case: imagine I want to compress objects upon PUTs using two different algorithms X and Y, and use the future 'multi-write' feature to store three objects upon any single PUT (original copy, X-compressed copy and Y-compressed copy). I can install two Storlets which implement X and Y respectively. However, seems Storlets engine can only invoke one per PUT, so this is still not feasible. Is that correct? > > > Another example is that a user may want to install a Swift3-like > > middleware to provide APIs to a 3rd party, but she doesn't want other > > users to see this middleware. > > > > If the definition can be made by operators, perhaps one possible solution > that preparing different proxy-server endpoint for different users is > available. i.e. an user uses no-s3api available proxy, > then the others use a different proxy-server endpoint that has the s3api in > the pipeline. > > Or, it sounds like kinda defaulter middleware[1], I don't think it has the > scope turning on/off the middlewares for now. > > 1: https://review.openstack.org/#/c/342857/ I see, thanks for pointing out the defaulter project! Best, Qiao > > Best, > Kota > > (2018/09/18 11:34), Qiao Kang wrote: > > Kota, > > > > Thanks for your reply, very helpful! > > > > I know Storlets can provide user-defined computation functionalities, > > but I guess some capabilities can only be achieved using middleware. > > For example, a user may want such a feature: upon each PUT request, it > > creates a compressed copy of the object and stores both the original > > copy and compressed copy. It's feasible using middlware but I don't > > think Storlets provide such capability. > > > > Another example is that a user may want to install a Swift3-like > > middleware to provide APIs to a 3rd party, but she doesn't want other > > users to see this middleware. > > > > Regards, > > Qiao > > > > On Mon, Sep 17, 2018 at 9:19 PM Kota TSUYUZAKI > > <tsuyuzaki.k...@lab.ntt.co.jp> wrote: > >> > >> With Storlets, users will be able to create their own applications that > >> are able to run like as a Swift middeleware. The application (currently > >> Python and Java are supported as the language but the > >> apps can calls any binaries in the workspace) can be uploaded as a Swift > >> object, then, users can invoke them with just an extra header that > >> specifies your apps. > >> > >> To fit your own use case, we may have to consider to invole or to > >> integrate the system for you but I believe Storlets could be a choice for > >> you. > >> > >> In detail, Storlets documantation is around there, > >> > >> Top Level Index: https://docs.openstack.org/storlets/latest/index.html > >> System Overview: > >> https://docs.openstack.org/storlets/latest/storlet_engine_overview.html > >> APIs: https://docs.openstack.org/storlets/latest/api/overview_api.html > >> > >> Thanks, > >> > >> Kota > >> > >> (2018/09/17 8:59), John Dickinson wrote: > >>> You may be interested in Storlets. It's another OpenStack project, > >>> maintained by a Swift core reviewer, that provides this sort of > >>> user-defined middleware functionality. > >>> > >>> You can also ask about it in #openstack-swift > >>> > >>> --John > >>> > >>> > >>> > >>> On 16 Sep 2018, at 9:25, Qiao Kang wrote: > >>> > >>>> Hi, > >>>> > >>>> I'm wondering whether Swift allows any user (not the administrator) to > >>>> specify which middleware that she/he wants his data object to go > >>>> throught. > >>>> For instance, Alice wants to install a middleware but doesn't want Bob to > >>>> use it, where Alice and Bob are two accounts in a single Swift cluster. > >>>> > >>>> Or maybe all middlewares are pre-installed globally and cannot be > >>>> customized on a per-account basis? > >>>> > >>>> Thanks, > >>>> Qiao > >>>> _______________________________________________ > >>>> Mailing list: > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >>>> Post to : openstack@lists.openstack.org > >>>> Unsubscribe : > >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >>> > >>> _______________________________________________ > >>> Mailing list: > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >>> Post to : openstack@lists.openstack.org > >>> Unsubscribe : > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >> > >> > >> -- > >> ---------------------------------------------------------- > >> Kota Tsuyuzaki(露﨑 浩太) <tsuyuzaki.k...@lab.ntt.co.jp> > >> NTT Software Innovation Center > >> Distributed Computing Technology Project > >> Phone 0422-59-2837 > >> Fax 0422-59-2965 > >> ----------------------------------------------------------- > >> > >> > >> _______________________________________________ > >> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >> Post to : openstack@lists.openstack.org > >> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > > > > > > > -- > ---------------------------------------------------------- > Kota Tsuyuzaki(露﨑 浩太) <tsuyuzaki.k...@lab.ntt.co.jp> > NTT Software Innovation Center > Distributed Computing Technology Project > Phone 0422-59-2837 > Fax 0422-59-2965 > ----------------------------------------------------------- > _______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
