Thank you Massimo for your detailed reply.
Some comments inline:
On 03/09/2016 09:50 AM, massimo di stefano wrote:
Hi,
I’ll try to address some of the issues raised in the last couple of mails:
1) concern about adding extra materials to the live
2) Include or drop jupyter from 9.5 release
3) contributing new notebooks to the live
4) which notebook to include on Osgeo-Live 9.5
1) About the concern in adding extra materials to the live:
To address:
Cameron:
I concerned about how much new material we are attempting add related to
Jupyter notebooks, all at the last moment.
Angelos:
Furthermore, as part of the Jupyter setup, a big number [2] [3] of Python
projects/libraries were also added to OSGeoLive, as soft dependencies to
support Notebooks, without official review or decision by the community,
From the dependencies listed by Angelos and linked here:
[2] https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L30
<https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L30>
[3] https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L43
<https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L43>
My proposed PR —> https://github.com/OSGeo/OSGeoLive-Notebooks/pull/5
<https://github.com/OSGeo/OSGeoLive-Notebooks/pull/5>
uses the following packages:
###
python-matplotlib
python-scipy
goal-bin
python-geographiclib
python-geocoder
###
From which only the last 2 are new to the osgeolive.
The other packages where already installed in the previous version of the
OSGeo-live
and are pretty common in the OSGeo software ecosystem.
In the PR comments, I added notes on its footprint, which is in the order of 10
MB.
Thank you for the detailed assessment of the dependencies brought in by
your work.
Not including gdal-bin was actually a bug ;)
You are correct that the other packages were already present in the
previous release, I just wanted to point out that we are being strict to
the Notebook itself, but we include other projects without even
discussing about it...
2) Include jupyter YES or NO
I’ll leave this to the community, but i should clarify that:
- ’there is a tremendous amount of work behind the finally complete jupiter
packaging’ that makes the live a better product.
- all the notebooks developed for the GSoC are working just fine with ipython
notebook as well as with jupiter notebook.
And I should point out that:
- ipython notebook is not something new to the OSGeo-live, it is more than one
year that is shipped within the OSGeo-live.
- the notebook has been successful adopted by some of the main OSGeo projects
during OSGeo-conference workshops
(see [1] [2] as example)
- to have the GSoC merged into the current OSGeo-live, there is no need of
jupiter ... if you really want to drop it :( .
For the record, I am in favor of including the Python stack in
OSGeo-Live, including Jupyter.
We DO need a working Jupyter quickstart to be able to include it though,
and it has to be ready within the next 48h
3) contributing new notebooks to the live:
Although the notebook can be used as a “scripting tool” or like a simple ide
IMHO the notebook is not just that, the notebook adds the capabilities to
include rich descriptive narrative to a data processing workflow including the
possibilities to interact with data through a nicely done widget interactive
system and finally enabling the printing of publication-ready reports (see
latex inline rendering and pdf export).
+1
Also, on the OSGeo-live, we are aiming (AFAIK) to use the same `common dataset`
among projects (or, at least, we attempt to) this way different software
projects can be more easily compared, say for complexity of their usage or for
output quality/performance comparison.
Very valid point raised here regarding the use of the common dataset in
the Notebooks. This should be a requirement for selection.
Having said that,
from a first look at all the other notebooks (or I should say, the only ones if
the PR will be not accepted ) shipped within the OSGeo-live I noticed that
almost all the notebooks under the “projects” folder [3] are not the result of
original work. Some of those notebooks are just a carbon copy of the examples
.py (script) code from the relative `project src code` on GitHub and the
potential offered by the notebook platform is not exploited at all. … no
problem from my side in having them included on the OSGeo-live, but I hope the
contributors of those notebook will make a more effective use of the tool.
Moreover, almost none of those notebooks make use of the data we already ship
within the live…
Hint: the notebook developer (I said developer, right … not just contributors)
can make use of one of the power of the notebook which is the capability of
mixing python and bash in the same document . ...
eg. you need a geoJSON file? just use ‘ogr2ogr' to convert one of our ’shape
file' to the format you need … or make a numpy loop generate user defined novel
data .. perhaps from a nice query to PostGIS (via psycopg) … it will take few
lines of code and a nicely done HTML description for each step)
I agree on the term developer vs contributor.
4) which notebook to include on OSGeo-Live 9.5
trying to address this:
I'm proposing that we release just a few of the Notebooks first, seek community
feedback on this small subset, adapt if required. But most importantly build an
OSGeo-Live notebook community and buy in before going too wide.
while I’d love to see more people contributing to the notebook ecosystem on the
OSGeo-live as well. In regards to the GSoC notebooks, I can’t see how possibly
can be to apply this:
‘'' release just a few of the Notebooks ‘’’
To the GSoC’s notebooks.
If you have an idea of what the GSoC-2015 was all about (and as principal
mentor you should!) you will notice that those notebooks are linked together,
they are a whole .. what you propose in the sentence above (if applied to the
GSoC’s notebooks), sounds like publishing a book without all the pages….
As a co-mentor of the GSoC project, I have fully reviewed the Notebooks
you provided and reproduced your results back then.
You recently opened a PR about including them in OSGeoLive. Wearing my
OSGeoLive maintainer hat, I had to re-evaluate your Notebooks in case of
missing libraries, dead links etc. This process/review did not imply
that your code was not working.
I also have to report that the Notebooks from GSoC are working as
expected in nightly builds of OSGeoLive.
In the proposed PR I already reduced the number of notebooks, I removed extra
features or incomplete, placeholders and other notebook based on python3 (which
is not shipped for this release)
Thank you for removing the placeholders, they were sending out the wrong
impression of an incomplete work. I hope the interesting topics that got
removed can be added again later with a full content.
as per vote .. I want to see my GSoC merged and that’s why I’m still working on
the OSGeo-live.
Of course, I’m biased and I let the community decide.
Thank you for opening the motion on a different thread.
Cheers,
Massimo.
Best,
Angelos
[1] https://github.com/wenzeslaus/python-grass-addon
<https://github.com/wenzeslaus/python-grass-addon>
[2] https://github.com/zarch/workshop-pygrass
<https://github.com/zarch/workshop-pygrass>
[3] https://github.com/OSGeo/OSGeoLive-Notebooks/tree/master/projects
<https://github.com/OSGeo/OSGeoLive-Notebooks/tree/master/projects>
On Mar 7, 2016, at 8:15 PM, Angelos Tzotsos <gcpp.kal...@gmail.com> wrote:
Hi,
This discussion is not only related to the work made by Massimo and other
contributors, but has further implications on how we include new projects in
OSGeoLive:
So far, Jupyter has been included in the development builds as a natural
evolution of the IPython project (which was also recently included in
OSGeoLive), so Jupyter never followed the path to be officially included [1].
As we speak (far past the feature freeze date) only the overview doc is
committed so one could argue in favor of dropping Jupyter from this release
completely...
Furthermore, as part of the Jupyter setup, a big number [2] [3] of Python
projects/libraries were also added to OSGeoLive, as soft dependencies to
support Notebooks, without official review or decision by the community, just
by following volunteers' vision of what functionality should be demonstrated by
the Notebooks. Part of this vision was to keep OSGeoLive relevant in terms of
tools and trends in the open source geospatial world.
Now the proposal is to select a subset of Notebooks to present to the community
in order to get feedback and build a notebook community to support further
adoption. A natural follow-up question is: should we also select which
supporting projects will be included? If we drop some of the Notebooks, their
dependencies should also be dropped, right?
So how do we decide which notebooks/supporting libraries to keep? And how do we
do that without being disrespectful to all volunteers who contributed their
time to maintain the Jupyter/Python stack for some months now (including
hacking notebooks, creating debian packages, mentoring or participating in the
GSoC project)?
Here is my proposal:
1. Massimo and Brian are named official maintainers of Jupyter, as the only
actual contributors of Notebooks [4]
2. Jupyter maintainers have to provide all the needed documentation (overview
and quickstart) ASAP, else Jupyter is dropped from 9.5
3. Jupyter maintainers get to decide which Notebooks to include in the final
iso.
4. If they disagree, the community votes weather to include all notebooks OR
drop Jupyter from 9.5 release and re-evaluate for 10.0
Regards,
Angelos
[1] https://wiki.osgeo.org/wiki/Live_GIS_Disc#How_to_add_a_project_to_OSGeoLive
<https://wiki.osgeo.org/wiki/Live_GIS_Disc#How_to_add_a_project_to_OSGeoLive>
[2] https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L30
<https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L30>
[3] https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L43
<https://github.com/OSGeo/OSGeoLive/blob/master/bin/install_jupyter.sh#L43>
[4] https://github.com/OSGeo/OSGeoLive-Notebooks/graphs/contributors
<https://github.com/OSGeo/OSGeoLive-Notebooks/graphs/contributors>
On 03/08/2016 12:17 AM, Cameron Shorter wrote:
Hi all,
I should clarify my statement below, (as has been to me off list), as it might
appear that I'm implying a lack of future, or quality of notebooks.
My comments below relate to level of external testing and size of community who
have reviewed Massimo's notebooks.
I think that Massimo has done an excellent job pioneering notebooks within the
OSGeo-Live framework, and these notebooks provide a great platform from which
to demonstrate OSGeo functionality.
I think our next step is to work toward bringing a groundswell of community
behind the development of these notebooks.
My suggested approach differs a little with that proposed by Massimo, although
I think we are aiming toward the same long term goal (of wide adoption and
community maintenance of Notebooks within the OSGeo-Live framework).
I'm proposing that we release just a few of the Notebooks first, seek community
feedback on this small subset, adapt if required. But most importantly build an
OSGeo-Live notebook community and buy in before going too wide.
This question is still unresolved within the core OSGeo-Live team, and we need
to make a decision fast, as our Release Candidate is due next Monday 14 March.
Opinions from our OSGeo-Live community would be greatly appreciated so we can
make a wise decision moving forward.
Warm regards, Cameron
On 7/03/2016 10:54 pm, Cameron Shorter wrote:
Angelos, all,
I concerned about how much new material we are attempting add related to
Jupyter notebooks, all at the last moment.
With OSGeo-Live, we have built our reputation around quality and stability, and
I think we should be careful not to compromise that. We will attract more users
to Jupyter notebooks if they try one excellent notebook, and look elsewhere for
more, than if they try 10 notebooks which almost work.
So before adding a new Notebook, I suggest that it should be tested start to
finish, and then thoroughly reviewed by the author, and then at least one
other person, preferably 2.
Am I right in understanding that we are currently proposing to add ~ 30 new
notebooks? I'd be inclined to pick out 2 to 5 of these and focus on getting
just these working.
(The remainder can be included on OSGeo-Live for testing and workshops, just
ensure that you can only find it if provided with the correct URL)
That said, who do we have available to help test notebooks? If you can help
out, please reply to this email, volunteering your services.
On 7/03/2016 10:34 pm, Angelos Tzotsos wrote:
2. Jupyter Notebooks: We currently have a git repository with notebooks to
include in the final release and we also have an open pull request to merge the
work from GSoC 2015 [5].
There is a special nightly build [6][7] including the GSoC notebooks.
We need to evaluate all our notebooks for this release and make a decision on
the notebooks to be included.
Perhaps we need a team of volunteers to go through all notebooks and review
them? Perhaps we need a spreadsheet listing all notebooks and their status?
Thoughts?
--
Angelos Tzotsos, PhD
OSGeo Charter Member
http://users.ntua.gr/tzotsos <http://users.ntua.gr/tzotsos>
_______________________________________________
Live-demo mailing list
Live-demo@lists.osgeo.org <mailto:Live-demo@lists.osgeo.org>
http://lists.osgeo.org/mailman/listinfo/live-demo
<http://lists.osgeo.org/mailman/listinfo/live-demo>
http://live.osgeo.org <http://live.osgeo.org/>
http://wiki.osgeo.org/wiki/Live_GIS_Disc
<http://wiki.osgeo.org/wiki/Live_GIS_Disc>
--
Angelos Tzotsos, PhD
OSGeo Charter Member
http://users.ntua.gr/tzotsos
_______________________________________________
Live-demo mailing list
Live-demo@lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/live-demo
http://live.osgeo.org
http://wiki.osgeo.org/wiki/Live_GIS_Disc
