Quick summary of our call: Daryl will soon start with a front end, build against a very simple mock-backend. Congxian will start implementing the Spring-based backend early April.
As soon as the first prototype of the UI is ready, we'll share it here for feedback. On Thu, Mar 21, 2019 at 10:08 AM Robert Metzger <rmetz...@apache.org> wrote: > Okay, great. > > Congxian Qiu, Daryl and I have a kick-off call later today at 2pm CET, 9pm > China time about the design of the ecosystem page (see: > https://github.com/rmetzger/flink-community-tools/issues/4) > Please let me know if others want to join as well, I can add them to the > invite. > > On Wed, Mar 20, 2019 at 4:10 AM Becket Qin <becket....@gmail.com> wrote: > >> I agree. We can start with english-only and see how it goes. The comments >> and descriptions can always be multi-lingual but that is up to the package >> owners. >> >> On Tue, Mar 19, 2019 at 6:07 PM Robert Metzger <rmetz...@apache.org> >> wrote: >> >>> Thanks. >>> >>> Do we actually want this page to be multi-language? >>> >>> I propose to make the website english-only, but maybe consider allowing >>> comments in different languages. >>> If we would make it multi-language, then we might have problems with >>> people submitting packages in non-english languages. >>> >>> >>> >>> On Tue, Mar 19, 2019 at 2:42 AM Becket Qin <becket....@gmail.com> wrote: >>> >>>> Done. The writeup looks great! >>>> >>>> On Mon, Mar 18, 2019 at 9:09 PM Robert Metzger <rmetz...@apache.org> >>>> wrote: >>>> >>>>> Nice, really good news on the INFRA front! >>>>> I think the hardware specs sound reasonable. And a periodic backup of >>>>> the website's database to Infra's backup solution sounds reasonable too. >>>>> >>>>> Can you accept and review my proposal for the website? >>>>> >>>>> >>>>> On Sat, Mar 16, 2019 at 3:47 PM Becket Qin <becket....@gmail.com> >>>>> wrote: >>>>> >>>>>> > >>>>>> > I have a very capable and motivated frontend developer who would be >>>>>> > willing to implement what I've mocked in my proposal. >>>>>> >>>>>> >>>>>> That is awesome! >>>>>> >>>>>> I created a Jira ticket[1] to Apache Infra and got the reply. It >>>>>> looks that >>>>>> Apache infra team could provide a decent VM. The last piece is how to >>>>>> ensure the data is persisted so we won't lose the project info / user >>>>>> feedbacks when the VM is down. If Apache infra does not provide a >>>>>> persistent storage for DB backup, we can always ask for multiple VMs >>>>>> and do >>>>>> the fault tolerance by ourselves. It seems we can almost say the >>>>>> hardware >>>>>> side is also ready. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jiangjie (Becket) Qin >>>>>> >>>>>> [1] https://issues.apache.org/jira/browse/INFRA-18010 >>>>>> >>>>>> On Fri, Mar 15, 2019 at 5:39 PM Robert Metzger <rmetz...@apache.org> >>>>>> wrote: >>>>>> >>>>>> > Thank you for reaching out to Infra and the ember client. >>>>>> > When I first saw the Ember repository, I thought it is the whole >>>>>> thing >>>>>> > (frontend and backend), but while testing it, I realized it is >>>>>> "only" the >>>>>> > frontend. I'm not sure if it makes sense to adjust the Ember >>>>>> observer >>>>>> > client, or just write a simple UI from scratch. >>>>>> > I have a very capable and motivated frontend developer who would be >>>>>> > willing to implement what I've mocked in my proposal. >>>>>> > In addition, I found somebody (Congxian Qiu) who seems to be eager >>>>>> to help >>>>>> > with this project for the backend: >>>>>> > https://github.com/rmetzger/flink-community-tools/issues/4 >>>>>> > >>>>>> > For Infra: I made the same experience when asking for more GitHub >>>>>> > permissions for "flinkbot": They didn't respond on their mailing >>>>>> list, only >>>>>> > on Jira. >>>>>> > >>>>>> > >>>>>> > >>>>>> > On Thu, Mar 14, 2019 at 2:45 PM Becket Qin <becket....@gmail.com> >>>>>> wrote: >>>>>> > >>>>>> >> Thanks for writing up the specifications. >>>>>> >> >>>>>> >> Regarding the website source code, Austin found a website[1] whose >>>>>> >> frontend code[2] is available publicly. It lacks some support (e.g >>>>>> login), >>>>>> >> but it is still a good starting point. One thing is that I did not >>>>>> find a >>>>>> >> License statement for that source code. I'll reach out to the >>>>>> author to see >>>>>> >> if they have any concern over our usage. >>>>>> >> >>>>>> >> Apache Infra has not replied to my email regarding some details >>>>>> about the >>>>>> >> VM. I'll open an infra Jira ticket tomorrow if there is still no >>>>>> response. >>>>>> >> >>>>>> >> Thanks, >>>>>> >> >>>>>> >> Jiangjie (Becket) Qin >>>>>> >> >>>>>> >> [1] https://emberobserver.com/ >>>>>> >> [2] https://github.com/emberobserver/client >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> On Thu, Mar 14, 2019 at 1:35 AM Robert Metzger < >>>>>> rmetz...@apache.org> >>>>>> >> wrote: >>>>>> >> >>>>>> >>> @Bowen: I agree. Confluent Hub looks nicer, but it is on their >>>>>> company >>>>>> >>> website. I guess the likelihood that they give out code from >>>>>> their company >>>>>> >>> website is fairly low. >>>>>> >>> @Nils: Beam's page is similar to our Ecosystem page, which we'll >>>>>> >>> reactivate as part of this PR: >>>>>> >>> https://github.com/apache/flink-web/pull/187 >>>>>> >>> >>>>>> >>> Spark-packages.org did not respond to my request. >>>>>> >>> I will propose a short specification in Becket's initial document. >>>>>> >>> >>>>>> >>> >>>>>> >>> On Mon, Mar 11, 2019 at 11:38 AM Niels Basjes <ni...@basjes.nl> >>>>>> wrote: >>>>>> >>> >>>>>> >>>> Hi, >>>>>> >>>> >>>>>> >>>> The Beam project has something in this area that is simply a page >>>>>> >>>> within their documentation website: >>>>>> >>>> https://beam.apache.org/documentation/sdks/java-thirdparty/ >>>>>> >>>> >>>>>> >>>> Niels Basjes >>>>>> >>>> >>>>>> >>>> On Fri, Mar 8, 2019 at 11:39 PM Bowen Li <bowenl...@gmail.com> >>>>>> wrote: >>>>>> >>>> > >>>>>> >>>> > Confluent hub for Kafka is another good example of this kind. I >>>>>> >>>> personally like it over the spark site. May worth checking it >>>>>> out with >>>>>> >>>> Kafka folks >>>>>> >>>> > >>>>>> >>>> > On Thu, Mar 7, 2019 at 6:06 AM Becket Qin < >>>>>> becket....@gmail.com> >>>>>> >>>> wrote: >>>>>> >>>> >> >>>>>> >>>> >> Absolutely! Thanks for the pointer. I'll submit a PR to >>>>>> update the >>>>>> >>>> >> ecosystem page and the navigation. >>>>>> >>>> >> >>>>>> >>>> >> Thanks, >>>>>> >>>> >> >>>>>> >>>> >> Jiangjie (Becket) Qin >>>>>> >>>> >> >>>>>> >>>> >> On Thu, Mar 7, 2019 at 8:47 PM Robert Metzger < >>>>>> rmetz...@apache.org> >>>>>> >>>> wrote: >>>>>> >>>> >> >>>>>> >>>> >> > Okay. I will reach out to spark-packages.org and see if >>>>>> they are >>>>>> >>>> willing >>>>>> >>>> >> > to share. >>>>>> >>>> >> > >>>>>> >>>> >> > Do you want to raise a PR to update the ecosystem page >>>>>> (maybe sync >>>>>> >>>> with >>>>>> >>>> >> > the "Software Projects" listed here: >>>>>> >>>> >> > >>>>>> https://cwiki.apache.org/confluence/display/FLINK/Powered+by+Flink) >>>>>> >>>> and >>>>>> >>>> >> > link it in the navigation? >>>>>> >>>> >> > >>>>>> >>>> >> > Best, >>>>>> >>>> >> > Robert >>>>>> >>>> >> > >>>>>> >>>> >> > >>>>>> >>>> >> > On Thu, Mar 7, 2019 at 10:13 AM Becket Qin < >>>>>> becket....@gmail.com> >>>>>> >>>> wrote: >>>>>> >>>> >> > >>>>>> >>>> >> >> Hi Robert, >>>>>> >>>> >> >> >>>>>> >>>> >> >> I think it at least worths checking if spark-packages.org >>>>>> owners >>>>>> >>>> are >>>>>> >>>> >> >> willing to share. Thanks for volunteering to write the >>>>>> requirement >>>>>> >>>> >> >> descriptions! In any case, that will be very helpful. >>>>>> >>>> >> >> >>>>>> >>>> >> >> Since a static page has almost no cost, and we will need >>>>>> it to >>>>>> >>>> redirect >>>>>> >>>> >> >> to the dynamic site anyways, how about we first do that >>>>>> while >>>>>> >>>> working on >>>>>> >>>> >> >> the dynamic website? >>>>>> >>>> >> >> >>>>>> >>>> >> >> Thanks, >>>>>> >>>> >> >> >>>>>> >>>> >> >> Jiangjie (Becket) Qin >>>>>> >>>> >> >> >>>>>> >>>> >> >> On Thu, Mar 7, 2019 at 4:59 AM Ufuk Celebi <u...@apache.org >>>>>> > >>>>>> >>>> wrote: >>>>>> >>>> >> >> >>>>>> >>>> >> >>> I like Shaoxuan's idea to keep this a static site first. >>>>>> We >>>>>> >>>> could then >>>>>> >>>> >> >>> iterate on this and make it a dynamic thing. Of course, >>>>>> if we >>>>>> >>>> have the >>>>>> >>>> >> >>> resources in the community to quickly start with a >>>>>> dynamic site, >>>>>> >>>> I'm >>>>>> >>>> >> >>> not apposed. >>>>>> >>>> >> >>> >>>>>> >>>> >> >>> – Ufuk >>>>>> >>>> >> >>> >>>>>> >>>> >> >>> On Wed, Mar 6, 2019 at 2:31 PM Robert Metzger < >>>>>> >>>> rmetz...@apache.org> >>>>>> >>>> >> >>> wrote: >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > Awesome! Thanks a lot for looking into this Becket! The >>>>>> VMs >>>>>> >>>> hosted by >>>>>> >>>> >> >>> Infra >>>>>> >>>> >> >>> > look suitable. >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > @Shaoxuan: There is actually already a static page. It >>>>>> used to >>>>>> >>>> be >>>>>> >>>> >> >>> linked, >>>>>> >>>> >> >>> > but has been removed from the navigation bar for some >>>>>> reason. >>>>>> >>>> This is >>>>>> >>>> >> >>> the >>>>>> >>>> >> >>> > page: https://flink.apache.org/ecosystem.html >>>>>> >>>> >> >>> > We could update the page and add it back to the >>>>>> navigation bar >>>>>> >>>> for the >>>>>> >>>> >> >>> > coming weeks. What do you think? >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > I would actually like to push for a dynamic page right >>>>>> away. >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > I know it's kind of a bold move, but how do you feel >>>>>> about >>>>>> >>>> sending the >>>>>> >>>> >> >>> > owners of spark-packages.org a short note, if they are >>>>>> >>>> interested in >>>>>> >>>> >> >>> > sharing the source? We could maintain the code together >>>>>> in a >>>>>> >>>> public >>>>>> >>>> >> >>> repo. >>>>>> >>>> >> >>> > If they are not interested in sharing, or we decide not >>>>>> to ask >>>>>> >>>> in the >>>>>> >>>> >> >>> first >>>>>> >>>> >> >>> > place, I'm happy to write down a short description of >>>>>> the >>>>>> >>>> requirements, >>>>>> >>>> >> >>> > maybe some mockups. We could then see if we find >>>>>> somebody here >>>>>> >>>> in the >>>>>> >>>> >> >>> > community who's willing to implement it. >>>>>> >>>> >> >>> > Given the number of people who are eager to contribute, >>>>>> I >>>>>> >>>> believe we >>>>>> >>>> >> >>> will >>>>>> >>>> >> >>> > be able to find somebody pretty soon. >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > On Wed, Mar 6, 2019 at 3:49 AM Becket Qin < >>>>>> >>>> becket....@gmail.com> >>>>>> >>>> >> >>> wrote: >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > > Forgot to provide the link... >>>>>> >>>> >> >>> > > >>>>>> >>>> >> >>> > > [1] https://www.apache.org/dev/services.html#blogs >>>>>> (Apache >>>>>> >>>> infra >>>>>> >>>> >> >>> services) >>>>>> >>>> >> >>> > > [2] https://www.apache.org/dev/freebsd-jails >>>>>> (FreeBSD Jail >>>>>> >>>> provided >>>>>> >>>> >> >>> by >>>>>> >>>> >> >>> > > Apache Infra) >>>>>> >>>> >> >>> > > >>>>>> >>>> >> >>> > > On Wed, Mar 6, 2019 at 10:46 AM Becket Qin < >>>>>> >>>> becket....@gmail.com> >>>>>> >>>> >> >>> wrote: >>>>>> >>>> >> >>> > > >>>>>> >>>> >> >>> > >> Hi Robert, >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >> Thanks for the feedback. These are good points. We >>>>>> should >>>>>> >>>> absolutely >>>>>> >>>> >> >>> > >> shoot for a dynamic website to support more >>>>>> interactions in >>>>>> >>>> the >>>>>> >>>> >> >>> community. >>>>>> >>>> >> >>> > >> There might be a few things to solve: >>>>>> >>>> >> >>> > >> 1. The website code itself. An open source solution >>>>>> would >>>>>> >>>> be great. >>>>>> >>>> >> >>> TBH, >>>>>> >>>> >> >>> > >> I do not have much experience on building a website. >>>>>> It'll >>>>>> >>>> be great >>>>>> >>>> >> >>> if >>>>>> >>>> >> >>> > >> someone could help comment on the solution here. >>>>>> >>>> >> >>> > >> 2. The hardware to host the website. Apache Infra >>>>>> provides >>>>>> >>>> a few >>>>>> >>>> >> >>> > >> services[1] that Apache projects can leverage. I did >>>>>> not see >>>>>> >>>> >> >>> database >>>>>> >>>> >> >>> > >> service, but maybe we can run a simple MySQL db in >>>>>> FreeBSD >>>>>> >>>> jail[2]. >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >> @Bowen & vino, thanks for the positive feedback! >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >> @Shaoxuan Wang <wshaox...@gmail.com> >>>>>> >>>> >> >>> > >> Thanks for the suggestion. That sounds reasonable to >>>>>> me. We >>>>>> >>>> >> >>> probably need >>>>>> >>>> >> >>> > >> a page in the Flink official site anyways, even just >>>>>> >>>> provide links >>>>>> >>>> >> >>> it to >>>>>> >>>> >> >>> > >> the ecosystem website. So listing the connectors in >>>>>> that >>>>>> >>>> static >>>>>> >>>> >> >>> page seems >>>>>> >>>> >> >>> > >> something we could start with while we are working >>>>>> on the >>>>>> >>>> dynamic >>>>>> >>>> >> >>> pages. >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >> Thanks, >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >> Jiangjie (Becket) Qin >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >> On Wed, Mar 6, 2019 at 10:40 AM Shaoxuan Wang < >>>>>> >>>> wshaox...@gmail.com> >>>>>> >>>> >> >>> > >> wrote: >>>>>> >>>> >> >>> > >> >>>>>> >>>> >> >>> > >>> Hi Becket and Robert, >>>>>> >>>> >> >>> > >>> >>>>>> >>>> >> >>> > >>> I like this idea! Let us roll this out with Flink >>>>>> >>>> connectors at >>>>>> >>>> >> >>> the >>>>>> >>>> >> >>> > >>> first beginning. We can start with a static page, >>>>>> and >>>>>> >>>> upgrade it >>>>>> >>>> >> >>> when we >>>>>> >>>> >> >>> > >>> find a better solution for dynamic one with rich >>>>>> functions. >>>>>> >>>> >> >>> > >>> >>>>>> >>>> >> >>> > >>> Regards, >>>>>> >>>> >> >>> > >>> Shaoxuan >>>>>> >>>> >> >>> > >>> >>>>>> >>>> >> >>> > >>> >>>>>> >>>> >> >>> > >>> On Wed, Mar 6, 2019 at 1:36 AM Robert Metzger < >>>>>> >>>> rmetz...@apache.org >>>>>> >>>> >> >>> > >>>>>> >>>> >> >>> > >>> wrote: >>>>>> >>>> >> >>> > >>> >>>>>> >>>> >> >>> > >>>> Hey Becket, >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>> This is a great idea! >>>>>> >>>> >> >>> > >>>> For this to be successful, we need to make sure >>>>>> the page >>>>>> >>>> is placed >>>>>> >>>> >> >>> > >>>> prominently so that the people submitting >>>>>> something will >>>>>> >>>> get >>>>>> >>>> >> >>> attention for >>>>>> >>>> >> >>> > >>>> their contributions. >>>>>> >>>> >> >>> > >>>> I think a dynamic site would probably be better, >>>>>> if we >>>>>> >>>> want >>>>>> >>>> >> >>> features >>>>>> >>>> >> >>> > >>>> such as up and downvoting or comments. >>>>>> >>>> >> >>> > >>>> I would also like this to be hosted on Apache >>>>>> infra, and >>>>>> >>>> endorsed >>>>>> >>>> >> >>> by >>>>>> >>>> >> >>> > >>>> the community. >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>> Does anybody here know any existing software that >>>>>> we >>>>>> >>>> could use? >>>>>> >>>> >> >>> > >>>> The only think I was able to find is AUR: >>>>>> >>>> >> >>> https://aur.archlinux.org/ >>>>>> >>>> >> >>> > >>>> (which is a community packages site for Arch >>>>>> Linux. The >>>>>> >>>> source >>>>>> >>>> >> >>> code of this >>>>>> >>>> >> >>> > >>>> portal is open source, but the layout and >>>>>> structure is >>>>>> >>>> not an >>>>>> >>>> >> >>> ideal fit for >>>>>> >>>> >> >>> > >>>> our requirements) >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>> Best, >>>>>> >>>> >> >>> > >>>> Robert >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>> On Tue, Mar 5, 2019 at 12:03 PM Becket Qin < >>>>>> >>>> becket....@gmail.com> >>>>>> >>>> >> >>> > >>>> wrote: >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> > >>>>> Hi folks, >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> > >>>>> I would like to start a discussion thread about >>>>>> creating >>>>>> >>>> a Flink >>>>>> >>>> >> >>> > >>>>> ecosystem website. The website aims to help >>>>>> contributors >>>>>> >>>> who >>>>>> >>>> >> >>> have developed >>>>>> >>>> >> >>> > >>>>> projects around Flink share their work with the >>>>>> >>>> community. >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> > >>>>> Please see the following doc for more details. >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> >>>>>> >>>> >>>>>> https://docs.google.com/document/d/12oCItoLbKrLGuwEUFcCfigezIR2hW3925j1hh3kGp4A/edit# >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> > >>>>> Thanks, >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> > >>>>> Jiangjie (Becket) Qin >>>>>> >>>> >> >>> > >>>>> >>>>>> >>>> >> >>> > >>>> >>>>>> >>>> >> >>> >>>>>> >>>> >> >> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> -- >>>>>> >>>> Best regards / Met vriendelijke groeten, >>>>>> >>>> >>>>>> >>>> Niels Basjes >>>>>> >>>> >>>>>> >>> >>>>>> >>>>>
