On 23 Apr 2011, at 23:16, "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov> wrote:
> Hi Guys, > > Looks great and sounds very complimentary to OODT. Happy that you guys > mention alignment -- might be nice to expose data from an OODT file > management+workflow management+resource management system as a Airavata > gateway. Just a quick thought after skimming the proposal. Yes, I felt such an alignment may well make sense. However, I'm only the champion, not a committer so I'll not commit the team to such an opportunity. I look forward to learning more once we are in the incubator. Sent from my mobile device. > > Good stuff! > > Cheers, > Chris > > On Apr 23, 2011, at 2:53 PM, Ross Gardler wrote: > >> I would like to propose Airavata for entry into the Apache Incubator. >> >> The full proposal can be found at [1] and is copied at the end of this >> mail. For those in a hurry here's a quick summary: >> >> Airavata is a software toolkit currently used to build science gateways >> but that has a much wider potential use. It provides features to >> compose, manage, execute, and monitor large scale applications and >> workflows on computational resources ranging from local clusters to >> national grids and computing clouds. Users can use Airavata back end >> services and build gadgets to deploy in open social containers such as >> Apache Rave and modify them to suite their needs. Airavata builds on >> general concepts of service oriented computing, distributed messaging, >> and workflow composition and orchestration. >> >> Airavata will provide web interfaces and scalable Service Oriented >> Architecture based backend services to build or enhance Science Gateway >> (see https://www.teragrid.org/web/science-gateways/) and similar >> environments. Airavata will specifically focus on: >> >> 1. sophisticated server-side tools for registering and managing large >> scale applications on computational resources. >> >> 2. graphical user interfaces to construct, execute, control, manage and >> reuse of scientific workflows. >> >> 3. interfacing and interoperability with with various external (third >> party) data and provenance management tools >> >> The project team consists of a number of existing Apache Committers and >> the code comes from the same stable as some of the code donated to >> Apache Rave (Incubating). >> >> We welcome your questions, suggestions, observations and support. >> >> Ross >> >> [1] http://wiki.apache.org/incubator/AiravataProposal >> >> FULL PROPOSAL TEXT >> ================== >> >> = Airavata Proposal for Apache Incubator = >> >> == Abstract == >> Airavata is a software toolkit currently used to build science >> gateways but that has a much wider potential use. It provides features >> to compose, manage, execute, and monitor large scale applications and >> workflows on computational resources ranging from local clusters to >> national grids and computing clouds. Users can use Airavata back end >> services and build gadgets to deploy in open social containers such as >> Apache Rave and modify them to suite their needs. Airavata builds on >> general concepts of service oriented computing, distributed messaging, >> and workflow composition and orchestration. >> >> >> == Proposal == >> >> Airavata will provide web interfaces and scalable Service Oriented >> Architecture based backend services to build or enhance Science >> Gateway (see https://www.teragrid.org/web/science-gateways/) >> and similar environments. Airavata will specifically focus on: >> >> 1. sophisticated server-side tools for registering and managing large >> scale applications on computational resources. >> 2. graphical user interfaces to construct, execute, control, manage >> and reuse of scientific workflows. >> 3. interfacing and interoperability with with various external (third >> party) data and provenance management tools. >> >> == Background == >> Working in close quarters with Apache Axis2 committers and inspired by >> the true open source community driven software development of ASF, >> Suresh Marru and Marlon Pierce have been pioneering the idea of a >> Science Gateways software-based Apache project since late 2008. Many >> Apache members have fostered these ideas and guided them to arrive at >> this proposal. >> >> Currently the software is a actively used in various science >> gateways. But the tools are general purpose and build upon widely used >> Apache tools like Axis2, ODE engine. The core team is motivated to >> expand the community and build a community welcoming both synergistic >> software components and also new usage scenarios. >> >> It is perhaps worth noting that one of the three seed projects that >> make up the Apache Rave (Incubating) project is also the product of >> this same team and is derived from the same Science Gateways >> community. >> >> == Rationale == >> >> The nature of computational problems has evolved from simple desktop >> calculations to complex, multidisciplinary activities that require the >> monitoring and analysis of remote data streams, database and web >> search and large ensembles of simulations. In the academic domain >> Science Gateways have emerged to address these needs and have built >> software platforms that provide a community of users with the ability >> to easily solve computational problems within a specific domain. The >> tools developed to support these gateways are potentially of value to >> any organisation needing to perform complex computations. Gateways >> provide a convenient interface to the underlying infrastrucure without >> the need for a deep understanding of the intricacies that >> infrastructure. >> >> We summarize the rationale for choosing The Apache Software Foundation >> (ASF) below. This is what we hope to gain from participating in the >> ASF. >> >> 1. '''Broader impact''': our science gateway tool set is based on >> Service Oriented Architecture principles, and it has always been our >> goal to align our software with broader trends in the development of >> software for distributed systems. Participating in the ASF provides a >> concrete way to implement this idea. In particular, we have done >> extensive work on the workflow systems, messaging, and application >> management as Web services from the perspective of computational science >> use cases (i.e., high failure rates, very long running jobs, dynamic >> service creation, workflows not expressible as directed acyclic graphs, >> etc). These requirements and our work to implement them have already had >> direct impact on the Apache Axis 2 and Apache ODE projects. As an Apache >> project, it is hoped that our community will have an enhanced >> opportunity for collaboration and complementary development with Apache >> Hadoop (for scientific application management), Apache QPID (for >> messaging), Apache Rave (incubator - Open Social Container) and others. >> It is our goal to expand our software’s usage beyond just science >> gateways to the broader enterprise community. >> 2. '''Sustainability''': Science gateway software development (and >> cyberinfrastructure software generally) is primarily funded in the US by >> the National Science Foundation (NSF), so the long term sustainability >> of software across funding cycles is a longstanding problem. The NSF is >> attempting to solve this problem, and its vision for sustainable >> software is described here: >> http://www.nsf.gov/pubs/2010/nsf10015/nsf10015.jsp. Participating in the >> ASF is our project’s vision for reaching software sustainability that >> underpins the NSF CF21 vision. As a successful ASF project (after >> incubation), we will have created a community led, rather than funding >> led, environment for the development of our sotware. This community, >> through our community engagement work and adoption of meritocratic >> principles, will expand beyond our current core team and existing >> project collaborations. This will greatly increase the chances that our >> software will continue to grow and improve beyond the participation of >> any individuals. >> 3. '''Maturity''': much of the software included in this proposal was >> developed initially by graduate students as part of their Ph. D. work. >> The Open Grid Computing Environment has devoted significant effort >> (through salaried staff and volunteers from collaborating institutions) >> to convert these research projects into mature, reliable, well-written, >> packaged components. The code is currently hosted at SourceForge, but >> we recognize the need to go beyond just the SourceForge support tools to >> participate in a real community of software engineering experts. It is >> our desire, through the Apache Incubator, to take our software >> engineering efforts to a higher level by learning from the substantial >> experience of appropraite Apache Committers. Apache mentors will provide >> initial guidance, as will the attraction of additional committers from >> the relevant Apache projects. >> >> == Initial Goals == >> >> * Implement a standalone version of the code base with a simple hello >> world service, workflow and gadget(s) to access the examples. >> * Migration of documentation and design knowledge from existing SF >> project >> * Re-architect Grid based security (GSI) dependencies and adopt more >> general purpose security implementations. >> * Make sure Cloud (including hadoop) support is more first class. >> * Aim to have the first Apache release within the first 6 months >> * Verify with Apache Legal that some of the more esoteric licences in >> our dependencies are acceptable, or replace them as appropriate >> >> == Current Status == >> >> The proposed tools are currently hosted on SourceForge at >> http://sourceforge.net/projects/ogce/ (source at >> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/) and are >> described at http://www.collab-ogce.org. >> >> == Meritocracy == >> >> A significant portion of initial committers are already ASF >> Committers/Members, >> and the entire team is well experienced with open source software >> development. The existing code base has resulted from >> multi-institutional collaborative projects. The developers are well >> aware of the Apache way and will honor the meritocracy policy of ASF >> foundation. >> >> == Community == >> >> To date our focus has been serving our immediate partners needs rather >> than looking outwards in order to build a broader community with >> diverse needs. Whilst the core team area likely to remain focussed on >> the Science Gateways communities we are keen to welcome community >> members from other disciplines. >> >> == Core Developers == >> Our core developers consist of participants from academic, >> not-for-profit and for-profit organisations. Many are already well >> versed in The Apache Way. >> >> Amongst our initial team we have one or more committers on the >> following Apache top level projects; axis, geronimo, synapse, ws, >> ws-pmc, ws-woden as well as Apache Rave (Incubating). >> >> == Alignment == >> Airavata software is built upon Apache Projects like Axis2, ODE, >> Rampart, Tomcat and Maven. We will try to closely align the project >> with ODE to ensure BPEL workflow compatibility. We will align with >> metadata management projects like Apache OODT. Web interfaces within >> the Airavata software will be synergistically developed with Apache >> Rave. >> >> == Known Risks == >> === Orphaned products === >> We acknowledge the need to seek project contributions outside the current >> developers. The core team actively travels and conducts workshops and >> tutorials at relevant academic conferences like Supercomputing, TeraGrid, >> Collaborative Technologies Systems and SciDAC. Previous experiences >> have showed that these tutorials and outreach efforts will bring in >> community participation. The general strategy will be to encourage >> users to be active in the community and develop patches and >> contribute. Also, the core developers use the Airavata software in >> multiple projects with a life span ranging from 2 to 10 years, so the >> risk of orphaned products is very minimal. >> >> Furthermore, by opening our doors to non-academic organisations >> already adopting large scale computation related projects in the ASF >> we hope to be able to build community beyond the proposing teams >> Science Gateway interests. >> >> === Inexperience with Open Source === >> The core team is very familiar with open source practices. The >> developers include existing Apache members who have long term experience >> with >> the Apache Way. The OGCE project has been an active >> open source project in SourceForge since November 2006. We welcome the >> new directions and are well prepared to follow the Apache way. >> >> === Homogenous Developers === >> We have a semi-distributed development environment distributed among >> Indiana University and Lanka Software Foundation. We fully expect >> contributions from the partnering science gateways adding to the >> heterogeneous development. >> >> === Reliance on Salaried Developers === >> The core developers are self motivated on the project and also are >> funded through various federal, state and endowment research >> grants. Participation in these research efforts based on Airavata >> software is mostly voluntary and above and beyond the requirements of >> the salaried jobs. >> >> The Open Gateway Computing project, from which the initial code >> donation is sourced, is funded for the next 3 years and is mandated by >> the funding guidelines to open source software development - >> http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=1032742. We >> believe in the Airavata software capabilities and its vital role in >> providing sustainable middleware for Science Gateways. Nevertheless, >> the core team will actively build upon Airavata software and foster >> developer community outside the current core. >> >> === Relationships with Other Apache Products === >> See “Alignment” above. Airavata is based on the concepts of Service >> Oriented Architecture and all services run within Tomcat >> container. The web services are based on Axis2. The orchestration of >> the scientific workflows uses Orchestration Director Engine. The >> software is built using Apache Maven. >> >> === An Excessive Fascination with the Apache Brand === >> The Apache brand would certainly help promote the software suite, but >> gaining the brand is not the motivation for this project. Airavata is >> being proposed to Apache because of the belief in Apache’s meritocracy >> model for mentored, community-driven, open source software is the best >> way to develop sustainable software. See “Rational” above. Most >> importantly, The Apache Software Foundation will help us create an >> institution-neutral contribution venue and will help us build a >> long-standing community around Airavata to sustain and improve it >> beyond the span of specific, targeted research grants. >> >> == Documentation == >> Existing documentation is available from the OGCE wiki, >> http://www.collab-ogce.org/ogce/index.php/Main_Page. In addition, >> there is abundance of presentation and self guided video tutorial >> material. Effort will be put in to collect all this information into >> meaningful documentation on the Apache websites. >> >> == Initial Source == >> The initial source of the project is in SourceForge. The source is >> available for anonymous check out from svn at >> https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-xbaya-gui/ >> >> == Source and Intellectual Property Submission Plan == >> Indiana University is the current holder of Intellectual Property >> rights for the software. The university has approved the code donation >> and signed trustees approval, Corporate Contributor Licence Agreement >> and Software Grant Agreement have been emailed to ASF secretary and >> received acknowledgement. >> >> Specifically Indiana University will donate 4 components into Airavata >> project. >> >> 1. XBaya Scientific Workflow Suite - includes a GUI for workflow >> composition and monitoring. The composed workflow can be exported to >> various workflow languages like BPEL, SCUFL, Condor DAG, Jython and >> Java. The defacto workflow enacting engine used is Apache ODE. >> 2. GFac - an application wrapper service that can be used to wrap >> command line-driven science applications and make them into robust, >> network- accessible services. This component is build on Axis2 web >> service stack. >> 3. XRegistry - a registry service for storing deployment information >> about wrapped application services and constructed workflows. >> 4. WS-Messenger - a “publish-subscribe” based message broker >> implemented on top of Apache Axis2 web services stack. It implements the >> WS-Eventing and WS-Notifications specifications and incorporates a >> message box component that facilities communications with clients behind >> firewalls and overcomes network glitches. >> >> == External Dependencies == >> >> Following the guideline -http://www.apache.org/legal/resolved.html, the >> following are the dependent software and all of them are in binary >> format in java archive (jar files). >> >> * CDDL license - Javax activation, JSR311, Portlet-API, Servlet-API >> * Apache V2: cog-jglobus, globus, caster, gridsphere, Woodstox, >> xmpp, xsul, sigiri, atomixmiser, weps-beans. >> * BSD: puretls, >> * MIT: bcporv, hsqldb, dom4j, slf4j >> * PSFL: Jython >> * GPL 2.0: mysql-connector-java >> * Other: >> * cryptix32, cryptix-asn1 (http://www.cryptix.org/LICENSE.TXT) >> * backport (public domain) >> * jaxen (http://jaxen.codehaus.org/license.html) >> >> Licence incompatibilities (GPL) will be resolved during incubation. >> >> == Cryptography == >> The software does not implement any cryptographic algorithms. However, >> to perform secured messaging and data movement and SSL communications, >> the software depends upon third party security libraries. These >> external libraries depend in turn on Java Security, Puretls, Cryptix >> and Bounce Castle libraries. Apache Cryptographic steps will be >> followed to register the use of these libraries. >> >> == Required Resources == >> >> === Mailing lists === >> 1. airavata-dev >> 2. airavata-commits >> 3. airavata-private >> >> === Subversion Directory === >> https://svn.apache.org/repos/asf/incubator/airavata >> >> === Issue Tracking === >> We intend to make use of Jira for issue tracking. Proposed key: AIRAVATA >> >> === Other Resources === >> >> We intend to manage our website using the Apache CMS. >> >> == Initial Committers == >> Names of initial committers with affiliation and current ASF status: >> >> || '''Name''' || '''Email''' || '''Affiliation''' >> || '''ICLA''' || '''ASF Status''' || '''Apache Id''' || >> || Suresh Marru || sma...@cs.indiana.edu || Indiana University >> || On File || Apache Commiter || smarru || >> || Marlon Pierce || mpie...@cs.indiana.edu || Indiana University >> || On File || Apache Commiter || mpierce || >> || Srinath Perera || hemap...@apache.org || Lanka Software >> Foundation || On File || Apache Member || hemapani || >> || Aleksander Slominski || aslom at us.ibm.com || IBM >> || On File || Apache Member || aslom || >> || Raminderjeet Singh || rami...@indiana.edu || Indiana University >> || On File || Apache Commiter || raminder || >> || Archit Kulshrestha || akuls...@indiana.edu || Indiana University >> || On File || N/A || N/A || >> || Chathura Herath || chath...@apache.org || Indiana University >> || On File || Apache Commiter || chathura || >> || Eran Chinthaka || chinth...@apache.org || Indiana University >> || On File || Apache Member || chinthaka || >> || Thilina Gunaratne || thil...@apache.org || Indiana University >> || On File || Apache Commiter || thilina || >> || Wathsala Vithanage || waths...@opensource.lk || Lanka Software >> Foundation || On File || N/A || N/A || >> >> All the parties are affiliated with companies and organizations that >> are familiar with the development of open source. We expect that the >> amount of volunteer work will increase, and more developers will come >> on board. >> >> == Champion == >> Ross Gardler, Apache Software Foundation >> >> == Nominated Mentors == >> * Ross Gardler, Member, Apache Software Foundation >> * Alek Slominski, Member, Apache Software Foundation >> * Ate Douma, Member, Apache Software Foundation >> * Sanjiva Weerawarna, Member, Apache Software Foundation >> * Paul Fremantle, Member, Apache Software Foundation >> >> == Sponsoring Entity == >> Apache Incubator Project. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> For additional commands, e-mail: general-h...@incubator.apache.org >> > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Senior Computer Scientist > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 171-266B, Mailstop: 171-246 > Email: chris.a.mattm...@nasa.gov > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Assistant Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
