+1 (binding)
On Thu, Feb 14, 2013 at 9:06 PM, Mattmann, Chris A (388J) < chris.a.mattm...@jpl.nasa.gov> wrote: > Hi Everyone, > > This VOTE has passed with the following tallies: > > +1 > > Chris Mattmann* > Andrew Hart* > Daniel Gruno > Paul Ramirez* > Gary Martin > Ross Gardler* > Ted Dunning* > Alexei Fedotov > Dave Fisher* > Suresh Marru* > Tommaso Teofili* > Andrea Pescetti > Chris Douglas* > Emmanuel Lécharny* > Tsengdar Lee > > * - indicates IPMC > > I'll get started creating the infrastructure tickets, and thanks to > everyone for VOTE'ing! > > Cheers, > Chris > > > On 2/5/13 8:18 AM, "Mattmann, Chris A (388J)" > <chris.a.mattm...@jpl.nasa.gov> wrote: > > >Hi Folks, > > > >OK, now that discussion has settled down, I'd like to call a VOTE for > >acceptance of Apache Open Climate Workbench into the Incubator. > >I'll leave the VOTE open the rest of the week and close it out next > >Monday, February 11th early am PT. > > > >[ ] +1 Accept Apache Open Climate Workbench into the Incubator > >[ ] +0 Don't care. > >[ ] -1 Don't accept Apache Open Climate Workbench into the Incubator > >because... > > > >Full proposal is pasted at the bottom of this email. Only VOTEs from > >Incubator PMC members are binding, but all are welcome to express their > >thoughts. > > > >Thank you! > > > >Cheers, > >Chris > > > >P.S. Here's my +1 (binding) > > > >------------- > >= Apache Open Climate Workbench, tool for scalable comparison of remote > >sensing observations to climate model outputs, regionally and globally. = > >=== Abstract === > >The Apache Open Climate Workbench proposal desires to contribute an > >existing community of software related to the analysis and evaluation of > >climate models, and related to the use of remote sensing data in that > >process. > > > >Specifically, we will bring a fundamental software toolkit for analysis > >and evaluation of climate model output against remote sensing data. The > >toolkit is called the [[http://rcmes.jpl.nasa.gov|Regional Climate Model > >Evaluation System (RCMES)]]. RCMES provides two fundamental components for > >the easy, intuitive comparison of climate model output against remote > >sensing data. The first component called RCMED (for "Regional Climate > >Model Evaluation Database") is a scalable cloud database that decimates > >remote sensing data and renalysis data related to climate using Apache > >OODT extractors, Apache Tika, etc. These transformations make > >traditionally heterogeneous upstream remote sensing data and climate model > >output homogeneous and unify them into a data point model of the form > >(lat, lng, time, value, height) on a per parameter basis. Latitude (lat) > >and Longitude (lng) are in WGS84 format, but can be reformatted on the > >fly. time is in ISO 8601 format, a string sortable format independent of > >underlying store. value carries with it units, related to interpretation > >and height allows for different values for different atmospheric vertical > >levels. All of RCMES is built on Apache OODT, Apache Sqoop/Apache Hadoop > >and Apache Hive, along with hooks to PostGIS and MySQL (traditional > >relational databases). The second component of the system, RCMET (for > >"Regional Climate Model Evaluation Toolkit") provides facilities for > >connecting to RCMED, dynamically obtaining remote sensing data for a > >space/time region of interest, grabbing associated model output (that the > >user brings, or from the Earth System Grid Federation) of the same form, > >and then regridding the remote sensing data to be on the model output > >grid, or the model output to be on the remote sensing data grid. The > >regridded data spatially is then temporally regridded using techniques > >including seasonal cycle compositing (e.g., all summer months, all > >Januaries, etc.), or by daily, monthly, etc. The uniform model output and > >remote sensing data are then analyzed using pluggable metrics, e.g., > >Probability Distribution Functions (PDFs), Root Mean Squared Error (RMSE), > >Bias, and other (possibly user-defined) techniques, computing an analyzed > >comparison or evaluation. This evaluation is then visualized by plugging > >in to the NCAR NCL library for producing static plots (histograms, time > >series, etc.) > > > >We also have performed a great deal of work in packaging RCMES to make the > >system easy to deploy. We have working Virtual Machines (VMWare VMX and > >Virtual Box OVA compatible formats) and we also have an installer built on > >Python Buildout (http://buildout.org/) called "Easy RCMET" for > dynamically > >constructing the RCMET toolkit. > > > >RCMES is currently supporting a number of recognized climate projects of > >(inter-)national significance. In particular, RCMES is supporting the > >[[http://www.globalchange.gov/what-we-do/assessment|U.S. National Climate > >Assessment (NCA) activities]] on behalf of NASA's contribution to the NCA; > >is working with the [[http://www.narccap.ucar.edu/|North American > Regional > >Climate Change Assessment Program (NARCCAP)]]; and is also working with > >the International [[http://wcrp-cordex.ipsl.jussieu.fr/|Coordinated > >Regional Downscaling Experiment (CORDEX)]]. > > > >=== Proposal === > >We propose to transition the RCMES software community, which includes > >developers of the RCMET and RCMED software, along with users of RCMES in > >the CORDEX project across a variety of academic institutions, scientists > >helping to improve the RCMES metrics, and visualizations, and regridding > >algorithms, packagers making RCMES easier to install, and scientists > >helping to lead some of these international projects that are already > >using RCMES. > > > >We have been working on the RCMES project since 2009 funded initially by > >the American Recovery and Reinvestment Act (ARRA) project out at NASA, and > >then branching out into other sources of support and sustainability (NASA; > >NSF, etc. -- see the > >[[http://rcmes.jpl.nasa.gov/about/overview|acknowledgements]] section on > >the RCMES website for a full list of supporting U.S. and international > >partners). > > > >With the existing RCMES community at Apache, we will also work to > >encourage other climate software projects e.g., Open Climate GIS, elements > >of the Earth System Grid Federation, other NASA climate projects funded > >under the Computational Modeling, Algorithms and Cyberinfrastructure > >(CMAC) to contribute to the Open Climate Workbench here at Apache. > > > >RCMED is a Big Data project that combines several underlying Apache > >software -- OODT, Tika, Hadoop, HIVE, and Sqoop -- and other related data > >management software. Its primary language is Java; RCMET, on the other > >hand, is a Python API, associated set of classes (framework), set of > >Python Bottle Web services, and a PHP "Wizard"-based User Interface that > >leverages Apache OODT Balance. > > > >=== Background === > >Bringing RCMES to Apache was the brain-child of Chris Mattmann, based on > >his solid experience with Apache OODT and bringing it to the ASF. Chris > >worked for a year to get the support of the JPL community including > >approvals from the Software Release authority at JPL to release the > >software. > > > >The initial code drop will include the RCMES SVN repository from JPL > >including prior revisions. We anticipate also including a smaller package, > >CDX, which contains some useful facilities for regridding, and command > >line tools for manipulating large datasets, and working with OPeNDAP, etc. > > > >After the code drop, we will work with our developers, users, documentors, > >and other members of the team to teach those unfamiliar with the Apache > >way how it works around here at Apache. 30% of the community from RCMES > >includes those intimately familiar with Apache including 6 ASF members -- > >the other 70% include a range of scientific code developers, climate > >scientists that use RCMES, program officers that will help make > >documentation and slides for the code, and advocate for it in the > >community. Their experience with Apache ranges from using various ASF > >products, to contributing patches to them, to not using any ASF software > >at all. > > > >With this diversity, we anticipate that while everything may not just work > >turnkey out of the box, this represents a unique opportunity to > >demonstrate Apache to the international community and to show the benefits > >of its community and social models. That said, we also have a lot of ASF > >experience to make sure everyone learns the Apache way. > > > >=== Rationale === > >We are bringing RCMES to Apache for a few reasons. First, we feel that it > >will immediately enable our collaborators across a number of institutions > >both nationally and internationally have the opportunity to work on a > >common software base, and to improve it with contributions from their own > >sites. Currently these are difficult to negotiate now because of varied > >legal and contribution frameworks -- Apache allows us to simplify this to > >a unified one. Second, using the ASF's world-wide mirroring system, we > >will be able to deliver climate software broadly to the community as we > >release it, rather than sneaker netting the software around or > >establishing our own point release infrastructure. > > > >Bringing this project to Apache also immediately thrusts the ASF into the > >thriving ecosystem of the Coordinated Regional Downscaling Experiment > >(CORDEX), the US National Cimate Assessment, the North American Regional > >Climate Change Assessment Program (the US contribution to CORDEX) and into > >relevance for upcoming Intergovernmental Panel on Climate Change (IPCC) > >assessment activities at a number of different institutions. We also seek > >to help lead and encourage du jour standard development rather than top > >down level dictating of standards for climate software and the ASF will > >provide us a means for that. > > > >=== Initial Goals === > >The initial goals of the proposed project are: > > > > * Stand up a sustaining Apache-based community around the JPL RCMES > >codebase. > > * Active relationships and possible cooperation with related projects and > >communities, including end user and scientific communities, CORDEX, > >NARCCAP, US NCA, IPCC, ESG, etc. > > * Active relationships and possible cooperation with existing Apache > >communities, e.g., OODT, Hadoop/HIVE, Sqoop, Tika, SIS, etc. > > * Initial Apache release. > > * Leverage Apache Open Climate Workbench in climate activities at NASA, > >in the international community as mentioned above, and beyond. > > * Vetting all software licenses and making sure IP is clear (software > >grant from JPL forthcoming). > > > >== Current Status == > >=== Meritocracy === > >30% of the proposed initial committers are familiar with the meritocracy > >principles of Apache. As stated above this includes 6 ASF members. Of the > >mentorship list, we have included Chris Douglas, a PMC member from Hadoop > >and ASF member to help guide the community. Chris M. and Chris D. have > >guided a number of projects through the Incubator over the years. The > >other mentor includes Paul Ramirez, who has experience with the Incubator > >-- he was a mentor for Apache Any23, and also was one of the PPMC members > >and eventual mentor for Apache SIS. The 70% of proposed initial committers > >that aren't as familiar with Apache have a broad range of experience in > >other open source projects, and have a deep respect and affinity for the > >foundation and the work that gets done here. The more experience ASF > >mentors and project members will help to guide them. > >=== Community === > >There is an existing, established community of developers and users of > >this projet. This includes established communities including the > >Coordinated Regional Downscaling Experiment, the U.S. National Climate > >Assessment (NCA), the North American Regional Climate Change Assessment > >Program (NARCCAP), and more. The Coordinated Regional Climate Downscaling > >Experiment (CORDEX, http://wcrp.ipsl.jussieu.fr/cordex/about.html) is a > >world wide effort of coordination of regional climate downscaling (RCD) > >experiments driven by the World Climate Research Program (WRCP, > >http://www.wcrp-climate.org/index.shtml). Recently, a large number of RCD > >projects have been carried out on a large parts of the world. To maximize > >the benefits of these research activities the WCRP designed a framework > >(Giorgi, WMO-Bulletin, 2009) focused on "quality-control [of] data sets of > >RCD-based information for the recent historical past and 21st century > >projections, covering the majority of populated land regions on the > >globe". CORDEX defined different control domains (up to 10, > >http://cordex.dmi.dk/joomla/) for almost all the populated regions of the > >world in a way to standardize the experiments and make them comparable. A > >key region focused on Africa was also designated as the top priority by > >WRCP. CORDEX also provides a a series of conventions and list of variables > >that have to be followed by any project that wants to contribute to the > >experiment. Each CORDEX region has a coordinator and regional and > >international periodic meetings are scheduled in a way to ensure the > >global well being. NARCCAP is the U.S. contribution to CORDEX. From the > >[[http://www.globalchange.gov/what-we-do/assessment/nca-overview|US > >National Climate Assessment]] site, work is "being conducted under the > >auspices of the Global Change Research Act of 1990. The GCRA requires a > >report to the President and the Congress every four years that integrates, > >evaluates, and interprets the findings of the U.S. Global Change Research > >Program (USGCRP); analyzes the effects of global change on the natural > >environment, agriculture, energy production and use, land and water > >resources, transportation, human health and welfare, human social systems, > >and biological diversity; and analyzes current trends in global change, > >both human-induced and natural, and projects major trends for the > >subsequent 25 to 100 years." > > > >Apache Open Climate Workbench will support all of these communities above, > >with an eye towards being a general purpose climate evaluation toolkit for > >model output and remote sensing data. > > > >=== Core Developers === > >The initial set of developers comes from various NASA centers (JPL, and > >Goddard Space Flight Center), NASA HQ, various Universities participating > >in CORDEX (Cape Town, University of New South Wales), the Indian Institute > >of Tropical Meteorology, the Free Univ. Berlin), the University of > >California Los Angeles, and Howard University. As mentioned previously > >several of our developers are Apache veterans and understand how it works > >around here and for those that don't, they will have great mentorship. > > > >=== Alignment === > >Our proposed effort aligns with the U.S. National Climate Assessment, the > >CORDEX effort, other efforts, including the Earth System Grid Federation, > >other climate software including the Open Climate GIS toolkit, other > >science portals for climate including the Climate Information Portal (CIP) > >at the University of Cape Town, and other related projects. > > > >There are also a number of related Apache projects and dependencies, that > >will be mentioned in the Relationships with Other Apache products section. > > > >== Known Risks == > >=== Orphaned products === > >Our project has a history of funding support from JPL, NASA (Applications > >program/ARRA, NCA, AIST), NSF (ExArch project), international investment > >from collaborators, and from other funding sources. The funding sources > >are all target future deliverables and activities, so there is little > >chance this software and community will be orphaned. > > > >=== Inexperience with Open Source === > >All the initial developers have worked on open source before -- 30% of the > >proposed initial community are experience with the ASF, and are PMC > >members and committers on ASF project including 6 ASF members. Our mentors > >are all ASF members, and we welcome any interest from additional Apache > >mentors in the effort. Those 70% of our project that aren't Apache > >committers, PMC members, or members will benefit from the leadership of > >the other 30% of the project. > > > >=== Homogenous Developers === > >The initial developers come from a variety of backgrounds and with a > >variety of needs for the proposed framework. Everyone is used to > >communicating on mailing lists as the project spans timezones, > >international institutions and centers of excellence for climate science. > > > >=== Reliance on Salaried Developers === > >All of the proposed initial developers are paid to work on this or related > >projects, but the proposed project is not the primary task for anyone. > > > >=== Relationships with Other Apache Products === > >As mentioned above, RCMES and the Apache Open Climate Workbench already > >depend on Apache OODT for facade interfaces to underlying data warehouses > >for storing remote sensing data; and for metadata extraction and > >transformation. The software also uses Apache Tika for this (through a > >transitive dependency from OODT). In addition, we have hooks to Apache > >Hadoop/HIVE, as well as dependencies on Apache Sqoop for dumping out > >remote sensing data from MySQL and into HIVE. > > > >=== A Excessive Fascination with the Apache Brand === > >All of us are familiar with Apache and have a respect for its brand and > >community. Chris Mattmann is a big proponent of Apache's sustainability > >factor -- and it's ability to grow software communities, in an > >institution, or funding source neutral manner. All of the community have > >an extreme respect for Apache, including those in our communities who > >aren't necessarily trained computer scientists, but are Scientists (big > >"S", e.g., land, physical, Earth/Climate scientists). > > > >== Documentation == > >The initial RCMES code base will come from the internal JPL Subversion > >repository. The [[Regional Climate Model Evaluation System (RCMES) > >project|http://rcmes.jpl.nasa.gov]] at [[JPL|http://www.jpl.nasa.gov/]] > >has documentation on the existing software, including links to funding > >support, communities, and other projects. We will continue to maintain > >that site at JPL, which is part of the reason for rebranding the project > >here at Apache with a new name to not interfere with the existing RCMES > >one that has a following. In addition, we hope to evolve RCMES@JPL to > have > >increasing levels of dependency on Apache Open Climate Workbench, so that > >we can incrementally transition with little impact to existing customers. > > > >In addition, JPL's [[http://cdx.jpl.nasa.gov|Climate Data eXchange > (CDX)]] > >website also has documentation on the existing software. > > > >== Initial Source == > >The project will start with seed code donated by NASA JPL via Mattmann and > >the rest of the initial committers, which consists of the Regional Climate > >Model Evaluation System (RCMES) toolkit, and the Climate Data eXchange > >(CDX) software. This will include the core Python API for RCMET, the RCMED > >OODT catalog project (which stores remote sensing data to MySQL/PostGIS, > >and HIVE), and the RCMED extractors for various climate formats. The > >source will also include Easy-RCMET, the Python Buildout for RCMET. In > >addition, we will bring along the CDX toolkit, which includes a CDX client > >package that performs subsetting, access, regridding of climate data; and > >also includes a Python Buildout installer of its own called Uber CDX. > > > >== Source and Intellectual Property Submission Plan == > >All seed code and other contributions will be handled through the normal > >Apache contribution process. Mattmann has been authorized by NASA JPL to > >lead the contribution of RCMES and CDX into the Incubator via his existing > >Apache CLA, and a Software Grant to be provided. > > > >We will also contact other related efforts for possible cooperation and > >contributions. > > > >== External Dependencies == > >Our project depends on a number of external libraries with various > >licensing conditions. An initial list of such dependencies is shown below. > >||<tableclass="bodyTable"rowclass="b">'''Library''' ||'''License''' || > >||<rowclass="b">[[http://www.ncl.ucar.edu/|NCAR NCL]]||MIT compat|| > >||<rowclass="a">[[http://www.pyngl.ucar.edu/Nio.shtml|PyNIO]]||MIT > >compat|| > >||<rowclass="b">[[http://www.pyngl.ucar.edu/|PyNGL]]||MIT compat|| > >||<rowclass="a">[[ > http://matplotlib.sourceforge.net/|Matplotlib]]||Modifie > >d > > PSF license|| > >||<rowclass="b">[[http://scipy.org/|Scipy]]||MIT compat|| > >||<rowclass="a">[[http://numpy.scipy.org/|NumPy]]||MIT compat|| > >||<rowclass="b">[[http://www.hdfgroup.org/HDF5/|HDF5]]||BSD|| > >||<rowclass="a">[[ > http://www.unidata.ucar.edu/software/netcdf/|NetCDF]]||M > >I > >T|| > > > > > >== Cryptography == > >The project itself will not use cryptography, but it is possible that some > >of the external software libraries will include cryptographic code to > >handle features present in various science data formats. If we need to > >provide an export control statement regarding cryptographic code per > >Apache policy, we will follow a similar approach by Mattmann in > >[[http://lucene.apache.org/nutch/|Apache Nutch]] and by Jukka Zitting > lead > >this effort in Apache Tika. Mattmann is familiar with this process. > > > >== Required Resources == > >Mailing lists > > > > * d...@climate.incubator.apache.org > > * comm...@climate.incubator.apache.org > > * priv...@climate.incubator.apache.org > > > >Subversion Directory > > > > * https://svn.apache.org/repos/asf/incubator/climate > > > >Issue Tracking > > > > * JIRA CLIMATE (CLIMATE) > > > >Other Resources > > > > * CLIMATE Wiki http://cwiki.apache.org/CLIMATE > > * Review Board instance - CLIMATE > > * Jenkins instance - CLIMATE > > > >== Initial Committers == > >||'''Name''' ||'''Email''' ||'''Affiliation''' ||'''CLA''' || > >||Chris A. Mattmann ||mattmann at apache dot org > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||yes || > >||Cameron E. Goodale ||goodale at apache dot org > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||yes || > >||Paul Ramirez ||pramirez at apache dog org > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||yes || > >||Andrew F. Hart ||ahart at apache dot org > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||yes || > >||Jinwon Kim||jkim at atmos dot ucla dot edu > >||[[http://jifresse.ucla.edu|UCLA Joint Institute for Regional Earth > >System Science and Engineering]] || no|| > >||Duane Waliser||duane dot waliser at jpl dot nasa dot gov > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] || no || > >||Huikyo Lee||Huikyo dot Lee at jpl dot nasa dot > >gov||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] || no || > >||Paul Loikith|| Paul dot C dot Loikith at jpl dot nasa dot gov > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] || no || > >||Daniel J. Crichton||crichton at apache dot > >org||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] || yes > || > >||Kim Whitehall||Kim dot D dot Whitehall at jpl dot nasa dot gov > >||[[ > http://www.physics1.howard.edu/~pmisra/HUPAS/HUPAS/HUPAS%20Jenkins.htm > >l > >|Howard University]] || no || > >||Paul Zimdars||pzimdars at apache dot > >org||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] || yes > || > >||Chris Jack||cjack at csag dot uct dot ac dot > >za||[[http://www.csag.uct.ac.za/|University of Cape Town]] || no || > >||Bruce Hewitson||hewitson at csag dot uct dot ac dot > >za||[[http://www.csag.uct.ac.za/|University of Cape Town]] || no || > >||Lluis Fita Borrell||l dot fitaborrell at unsw dot edu dot > >au||[[http://unsw.edu.au/|University of New South Wales]] || yes || > >||Jason Evans||jason dot evans at unsw dot edu dot > >au||[[http://unsw.edu.au/|University of New South Wales]] || no || > >||Estani Gonzalez||estanislao dot gonzalez at met dot fu-berlin dot de > >||[[http://www.geo.fu-berlin.de/met/|Free University Berlin]] || yes || > >||Luca Cinquini||luca dot cinquini at jpl dot nasa dot gov > >||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] || yes || > >||J. Sanjay||sanjay at tropmet dot res dot in || > >[[http://tropmet.res.in/|Indian Institute of Tropical Meteorology]] || > yes > >|| > >||M. V. S. Rama Rao||ramarao at tropmet dot res dot in > >||[[http://tropmet.res.in/|Indian Institute of Tropical Meteorology]] || > >yes || > >||Tsengdar Lee||tsengdar dot j dot lee at nasa dot gov || > >[[http://hq.nassa.gov/|NASA HQ]] || no || > >||Laura Carriere||laura dot carriere at nasa dot gov > >||[[http://www.gsfc.nasa.gov/|NASA Goddard Space Flight Center]] || no || > >||Denis Nadeau|| denis dot nadeau at nasa dot > >gov||[[http://www.gsfc.nasa.gov/|NASA Goddard Space Flight Center]] || no > >|| > >||Michael Joyce|| Michael dot J dot Joyce at jpl dot nasa dot > >gov||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||yes|| > >||Shakeh Khudikyan||Shakeh dot E dot Khudikyan at jpl dot nasa dot > >gov||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||yes|| > >||Maziyar Boustani||Maziyar dot Boustani at jpl dot nasa dot > >gov||[[http://www.jpl.nasa.gov/|NASA Jet Propulsion Laboratory]] ||no|| > >||Suresh Marru||smarru at apache dot org||[[http://pti.iu.edu/|Indiana > >University]] ||yes|| > > > > > >== Sponsors == > >Champion > > > > * Chris Mattmann (mattmann at apache dot org) > > > >Nominated Mentors > > > > * Chris A. Mattmann (mattmann at apache dot org) > > * Chris Douglas (cdouglas at apache dot org) > > * Paul Ramirez (pramirez at apache dot org) > > > >Sponsoring Entity > > > > * Apache Incubator > > > > > > > >--------------------------------------------------------------------- > >To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > >For additional commands, e-mail: general-h...@incubator.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > >
