Hi - I’ll point out that your champion and mentors do have access and they could easily do that for you.
Regards, Dave > On Nov 1, 2019, at 12:48 AM, Dominik Riemer <rie...@fzi.de> wrote: > > Thank you very much Kenn, that's awesome! > I'll add you as a mentor once we got access to the wiki, afterwards I'll also > start a separate DISCUSS thread to discuss our proposal. > > Dominik > > -----Original Message----- > From: Kenneth Knowles <k...@apache.org> > Sent: Thursday, October 31, 2019 1:49 AM > To: general <general@incubator.apache.org> > Subject: Re: Write Access to Incubator Wiki > > Cool! > > I see you have mentors lined up. I would love to also be involved as mentor. > > The list of initial committers seems a little small. Browsing the commit > history it does seem to be dominated by just a couple of contributors. I see > this is acknowledged (at least implicitly) in the proposal. I expect a > principal challenge will be attracting diverse contributors, committers, and > PMC to ensure this project survives independent of any particular entity > (individual or corporate). > > Kenn > > On Wed, Oct 30, 2019 at 1:16 PM Dominik Riemer <rie...@fzi.de> wrote: > >> Hi all, >> >> >> >> a short introduction to myself: My name is Dominik Riemer and I'm a >> co-initiator of StreamPipes, an open source self-service toolbox for >> analyzing IoT data streams. After presenting the tool at this year's >> ApacheCon NA, followed by very friendly and fruitful discussions with >> many people from the Apache community, we are sure that we'd like >> continue the development of StreamPipes as an Apache community >> project. Before we start a discussion process, I'd like to ask to get >> write access to the Incubator wiki (username: riemer). An initial draft of >> the proposal is attached below. >> >> >> >> Thanks for your help! >> >> Dominik >> >> >> >> >> >> -------------------------------- >> >> >> >> StreamPipes - Apache Incubator Proposal, Proposal Draft >> >> >> >> == Abstract == >> >> StreamPipes is a self-service (Industrial) IoT toolbox to enable >> non-technical users to connect, analyze and explore (Industrial) IoT >> data streams. >> >> >> >> = Proposal = >> >> >> >> The goal of StreamPipes >> (www.streampipes.org<http://www.streampipes.org>) >> is to provide an easy-to-use toolbox for non-technical users, e.g., >> domain experts, to exploit data streams coming from (Industrial) IoT >> devices. Such users are provided with an intuitive graphical user >> interface with the Pipeline Editor at its core. Users are able to >> graphically model processing pipelines based on data sources (streams), data >> processors and data sinks. >> Data processors and sinks are self-contained microservices, which >> implement either stateful or stateless processing logic (e.g., a trend >> detection or image classifier). Their processing logic is implemented >> using one of several provided wrappers (we currently have wrappers for >> standalone/Edge-based processing, Apache Flink, Siddhi and working >> wrapper prototypes for Apache Kafka Streams and Spark, in the future >> we also plan to integrate with Apache Beam). An SDK allows to easily >> create new pipeline elements. Pipeline elements can be installed at >> runtime. To support users in creating pipelines, an underlying >> semantics-based data model enables pipeline elements to express >> requirements on incoming data streams that need to be fulfilled, thus >> reducing modeling errors. >> >> Data streams are integrated by using StreamPipes Connect, which allows >> to connect data sources (based on standard protocols, such as MQTT, >> Kafka, Pulsar, OPC-UA and further PLC4X-supported protocols) without >> further programming using a graphical wizard. Additional user-faced >> modules of StreamPipes are a Live dashboard to quickly explore IoT >> data streams and a wizard that generates code templates for new >> pipeline elements, a Pipeline Element Installer used to extend the algorithm >> feature set at runtime. >> >> >> >> === Background === >> >> StreamPipes was started in 2014 by researchers from FZI Research >> Center for Information Technology in Karlsruhe, Germany. The original >> prototype was funded by an EU project centered around predictive >> analytics for the manufacturing domain. Since then, StreamPipes was >> constantly improved and extended by public funding mainly from federal >> German ministries. In early 2018, the source code was officially released >> under the Apache License 2.0. >> At the same time, while we focused on bringing the research prototype >> to a production-grade tool, the first companies started to use StreamPipes. >> Currently, the primary goal is to widen the user and developer base. >> At ApacheCon NA 2019, after having talked to many people from the >> Apache Community, we finally decided that we would like to bring >> StreamPipes to the Apache Incubator. >> >> >> >> === Rationale === >> >> The (Industrial) IoT domain is a highly relevant and emerging sector. >> Currently, IoT platforms are offered by many vendors ranging from SMEs >> up to large enterprises. We believe that open source alternatives are >> an important cornerstone for manufacturing companies to easily adopt >> data-driven decision making. From our point of view, StreamPipes fits >> very well into the existing (I)IoT ecosystem within the ASF, with >> projects such as Apache PLC4X focusing on connecting machine data from >> PLCs, or other tools we are also using either in the core of >> StreamPipes or with integrations (Apache Kafka, Apache IoTDB, Apache >> Pulsar). StreamPipes itself focuses on enabling self-service IoT data >> analytics for non-technical users. >> >> The whole StreamPipes code is currently on Github. To get a rough >> estimate of the project size: >> >> * streampipes: Backend and core modules, ~3300 commits >> >> * streampipes-ui: User Interface, ~1300 commits >> >> * streampipes-pipeline-elements: ~100 Pipeline Elements (data >> processors/algorithms and sinks), ~500 Commits >> >> * streampipes-connect-adapters: ~20 Adapters to connect data, ~100 >> commits To achieve our goal to further extend the code base with new >> features, new connectors and new algorithms and to grow both the user >> and developer community, we believe that a community-driven >> development process is the best way to further develop StreamPipes. >> Finally, after having talked to committers from various Apache >> IoT-related projects and participation in spontaneous hacking sessions >> and being impressed by the collaboration among individual projects, we >> decided that (from our point of view) the ASF is the ideal place to be the >> future home of StreamPipes. >> >> >> >> === Initial Goals === >> >> * Move the existing codebase to Apache >> >> * Fully align with Apache development- and release processes >> >> * Perform name search and do a thorough review of existing licenses >> >> * First Apache release >> >> >> >> === Current Status === >> >> ** Meritocracy ** >> >> We are absolutely committed to strengthen StreamPipes as a real >> community-driven open source project. The existing committer base is >> highly motivated to foster the open source way in the industrial IoT >> sector and, together with existing Apache communities focused on this >> domain, provide open source tooling for Industrial IoT projects in the >> same way Apache offers in the Big Data space, for instance. >> >> The development philosophy behind StreamPipes has always followed the >> principles of meritocracy - although most committers are still active >> in the project, we managed to onboard new, committed developers >> regularly. 2 people, who are today core of the developer team, have >> joined during the past year. Therefore, we would aim to continuously >> expand the PMC and committer base based on merit. >> >> >> >> ** Community ** >> >> Since being open-sourced in 2018, the public interest in StreamPipes >> has steadily grown. Several companies, mainly from the manufacturing >> domain, have tested StreamPipes in form of proof-of-concept projects. >> First companies have started to use StreamPipes in production. This >> was due to a high number of events from meetups, research conferences, >> demo sessions up to hackathons we participated or organized during the >> past two years. After having generated a general interest in >> StreamPipes, our next focus will be to find more committers to diversify the >> contributor base. >> >> >> >> ** Core Developers ** >> >> The core developers of the system are Dominik Riemer, Philipp Zehnder, >> Patrick Wiener and Johannes Tex. All core developers are initial >> committers in the current proposal. Some former students who recently >> started to work at companies and who have also worked on the project >> with great commitment, will be asked to further contribute to the project. >> >> >> >> ** Alignment ** >> >> StreamPipes has dependencies to a lot of existing Apache projects - >> this is one reason why we think that the ASF is the best future home >> for StreamPipes. The messaging layer is based on Apache Kafka (and >> also Apache Pulsar as a future option), and runtime wrappers exist for >> Apache Flink, Apache Spark and Apache Kafka Streams. StreamPipes >> Connect already includes adapters for several Apache projects. Most >> importantly, we integrate (and plan to deepen the integration) with >> IIoT-focused projects such as Apache PLC4X. Also, several data sinks >> exist to send messages to tools from other Apache projects (e.g., Apache >> Kafka, Apache Pulsar, and Apache IoTDB). >> Together with these tools (and also after having talked to the core >> developers after this year's ApacheCon) we are absolutely convinced >> that a tight integration between these tools will strengthen the open >> source IoT ecosystem. >> >> >> >> === Known Risks === >> >> ** Orphaned Products ** >> >> We don't expect the risk of an orphaned product. The initial >> committers have worked on the project for years and are absolutely >> committed to making this open source tool a great success. All initial >> committers are committed to work on StreamPipes in their free time. >> >> >> >> ** Inexperience with Open Source ** >> >> All initial committers have years of expertise related to open source >> development and understand what open source means. However, none of >> the initial committers are currently committers to Apache projects, >> although some have already contributed to some projects. From a >> variety of events and from intensively studying Apache mailing lists, >> we are sure that the Apache Way is the way we'd like the project to >> move into the future. We expect to benefit from the experiences from >> the ASF in building successful open source projects. >> >> >> >> ** Length of Incubation ** >> >> We are aware that incubation is a process that is focused on building >> the community, learning the Apache Way and other important things such >> as learning the release process and handling licensing and trademark issues. >> We are also aware that, although there is a steadily increasing >> interest in StreamPipes, a major challenge we would need (and are >> willing) to work on during the incubation phase is widening the >> committer base. We look forward to that as a large developer base is exactly >> what we are striving for. >> >> >> >> ** Homogeneous Developers ** >> >> Most current developers work for the same institution (FZI). The >> motivation of all developers goes beyond their commitment to work and >> all current committers work on StreamPipes in their free time. >> Recently, we have received first pull requests from external >> contributors and a growing interest from users and companies outside >> of FZI. First manufacturing companies have already evaluated and >> adopted StreamPipes. To attract external developers, we've created an >> extensive documentation, have a Slack channel to quickly answer >> questions, and provide help via mail. Therefore, we believe that >> making the developer community more heterogeneous is not only mandatory, but >> something that can be achieved during the next months. >> >> >> >> ** Reliance on salaried developers ** >> >> Currently, StreamPipes receives support from salaried developers, >> mainly research scientists from FZI. However, all core developers >> substantially work on StreamPipes in their spare time. As this has >> been the case from the beginning in early 2014, it can be expected >> that a substantial portion of volunteers will continue to be working >> on the project and we aim at strengthening the base of non-paid >> committers and paid committers of other companies. At the same time, >> funding of the initial StreamPipes team is secured by public funding >> for the next few years, making sure that there will be also enough >> commitment from developers during their work time. >> >> >> >> ** Relationships with other Apache products ** StreamPipes is often >> compared to tools such as Node-Red and Apache Nifi. This is mainly >> based on a similar UI concept (dataflow approach). Despite some >> technological differences (e.g., the microservice analytics approach >> vs. single-host runtime of Node-Red, the wrapper architecture and the >> underlying semantics-based model), we believe the target audience >> differs. We aim to collaborate with the Apache Nifi community in terms >> of exchanging best practices and also integrating both projects (e.g., by >> building connectors). >> >> As mentioned above, quite a few adapters and data sinks are already >> available that link to existing Apache projects. >> >> >> >> ** An excessive fascination with the Apache Brand ** Although we >> recognize the Apache brand as the most visible brand in the open >> source domain, the primary goal of this proposal is not to create >> publicity, but to widen the developer base. We believe that successful >> projects have broad and diverse communities. We expect that an Apache >> project, with a clear and proven way to develop open source software, >> helps in finding new committers. As the core development team has >> already worked on StreamPipes for the past few years and is fully >> committed to the software and its benefit for the industrial IoT >> domain, we would also continue development without being an Apache project. >> >> >> >> === Documentation === >> >> Currently, we host a website at https://www.streampipes.org More >> technical info (user + developer guide) can be found in the documentation: >> https://docs.streampipes.org, where users can find tutorials and >> manuals on how to extend StreamPipes using the SDK. >> >> >> >> === Initial Source === >> >> Currently, the following Github repositories exist, all licensed under >> the Apache Software License 2.0: >> >> * streampipes (https://www.github.com /streampipes/streampipes, the >> backend & pipeline management module) >> >> * streampipes-ui (https://www.github.com/streampipes/streampipes-ui, >> the UI module) >> >> * streampipes-pipeline-elements ( >> https://www.github.com/streampipes/streampipes-pipeline-elements, >> library of data processors and sinks) >> >> * streampipes-connect-adapters ( >> https://www.github.com/streampipes/streampipes-connect-adapters, >> StreamPipes connect adapters) >> >> * streampipes-docs >> (https://www.github.com/streampipes/streampipes-docs, >> the abovementioned documentation) >> >> >> >> === Source and intellectual property submission plan === All initial >> committers will sign a ICLA with the ASF. FZI, as the organizational >> body that has employed the main contributors of StreamPipes, will sign >> a CCLA and donate the codebase to the ASF (both subject to formal >> approval). All major contributors are still active in the project. >> >> >> >> === External Dependencies === >> >> We did an initial review of all dependencies used in the various projects. >> No critical libraries that depend on category X licenses were found, >> some minor issues have already been resolved (e.g., removing >> dependencies to org.json libraries). Most external dependencies used >> by the Java-based (backend, pipeline-elements and connect) modules are >> licensed under the Apache License 2.0, whereas some licenses are Cat B >> (e.g., CDDL). Most external dependencies the UI requires on are licensed >> under the MIT license. >> >> Once we are moving to the Incubator, we would do a complete check of >> all transitive dependencies. We don't expect any surprises here. >> >> >> >> === Cryptography === >> >> (not applicable) >> >> >> >> === Required Resources === >> >> ** Mailing Lists ** >> >> We plan to use the following mailing lists: >> >> * us...@streampipes.incubator.apache.org<mailto: >> us...@streampipes.incubator.apache.org> >> >> * d...@streampipes.incubator.apache.org<mailto: >> d...@streampipes.incubator.apache.org> >> >> * priv...@streampipes.incubator.apache.org<mailto: >> priv...@streampipes.incubator.apache.org> >> >> * comm...@streampipes.incubator.apache.org<mailto: >> comm...@streampipes.incubator.apache.org> >> >> As StreamPipes is targeted to a non-technical audience, we see a >> dedicated user mailing list as an important requirement to help users. >> >> >> >> ** Subversion directory ** >> >> (not applicable) >> >> >> >> ** Git repositories ** >> >> We would like to use Git for source code management and enable Github >> mirroring functionality. >> >> >> >> As we plan to merge some of the repos described above to simplify the >> release process we ask to create the following source repositories: >> >> * streampipes (containing backend + UI) >> >> * streampipes-extensions (containing modules that can be dynamically >> installed at runtime: pipeline elements and connect adapters) >> >> * streampipes-website (containing docs + website) >> >> >> >> ** Issue tracking ** >> >> JIRA ID: StreamPipes >> >> >> >> === Initial Committers === >> >> List of initial committers in alphabetical order: >> >> Christofer Dutz (christofer.dutz at c-ware dot de) Dominik Riemer >> (dominik dot riemer at gmail dot com) Johannes Tex (tex at fzi dot de) >> Patrick Wiener (wiener at fzi dot de) Philipp Zehnder (zehnder at fzi >> dot de) >> >> >> >> === Sponsors === >> >> ** Champion ** >> >> * Christofer Dutz (christofer.dutz at c-ware dot de) >> >> >> >> ** Mentors ** >> >> * Christofer Dutz (christofer.dutz at c-ware dot de) >> >> * Julian Feinauer (Jfeinauer at apache dot org) >> >> * Justin Mclean (justin at classsoftware dot com) >> >> >> >> ** Sponsoring Entity ** >> >> The Apache Incubator >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
