I agree that there are many scenarios small numbers of features, and for these it is "fast enough" to do something like make a call to GEOS for all pair-wise combinations of features. A layer or collection of scripts on top of OGR would work fine here.
For scenarios involving large numbers of features, I suspect it is much harder to do it fast and within available memory. It is probably necessary to do a multi-pass approach, where the first step operates only on the spatial indexes of the layers involved. It is probably too slow--or at least very suboptimal--to even read all of the features into memory, much less call GEOS on them; all of that would happen in a later pass, when a subset of features was identified via the spatial index pass. I suspect all of this must be implemented in the core OGR code, where it is possible to quickly and selectively probe spatial indexes, unless OGR wants to expose an abstraction allowing higher level code to do that. I could definitely appreciate the view that either of these approaches is significant scope creep. In any case, I appreciate that OGR developers are considering this functionality, even if it is ultimately only for the small case. I hope you are able to develop something. Jason -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Ari Jolma Sent: Wednesday, April 18, 2012 12:45 PM To: [email protected] Subject: Re: [gdal-dev] Layer operations, a proposal On 04/18/2012 03:07 AM, Jason Roberts wrote: > As I have mentioned previously > (http://lists.osgeo.org/pipermail/gdal-dev/2010-January/023089.html) Thanks for pointing this out. > I think > something like this would be a great addition to OGR. No doubt it > would be a huge job--basically turning OGR into a GIS--but if it could > be done well, with sufficient performance and in the core API, then > many systems could potentially leverage it, instead of everybody > implementing their own versions of Ari's pseudocode. I don't think it would be a huge job. Maybe I'm missing something but I see the basic implementation very simple and generic. Perhaps there is a performance problem somewhere (looping through 1 million features within a loop of 1 million features and using shapefiles) but for common cases it should be ok, especially if the layers fit into memory. Once I get something done (currently testing the 2nd algorithm), it would be interesting to do some benchmarking with available datasets. Best regards, Ari > > Best, > Jason > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Frank Warmerdam > Sent: Tuesday, April 17, 2012 7:32 PM > To: Ari Jolma > Cc: [email protected] > Subject: Re: [gdal-dev] Layer operations, a proposal > > Ari, > > I think this would be an interesting addition. Would you be willing > to write up an RFC? > > I think the layer creation step should be a different call than the > feature processing step to maximize the chance that folks who need to > do something very format specific at layer creation can do so and > still use the generic algorithm for processing the features. > > Your operations appear to be focused on spatial relationships, but > I've also heard a lot of interest in feature joins on attribute > fields. One case is just merging features from a layer with a common > attribute value. Another case would be some sort of attribute based > join between layers. Hopefully we can use a reasonably similar approach for a variety of operations. > > I'm curious how you plan to handle the spatial relationships efficiently? > An in memory spatial index? I'd like have the API overly constrain > how we implement this as I can imagine the internal operation > implementation evolving over time, and perhaps taking different > strategies for different styles of input data. For instance an > in-memory spatial index might not make much sense for large numbers of > very simple point features while it would be good for heavy polygon features. > > > Best regards, > Frank > > On Tue, Apr 17, 2012 at 2:42 PM, Ari Jolma<[email protected]> wrote: >> Folks, >> >> I propose a set of new methods for OGR layers (see the PDF). I took >> the basic ideas from page >> http://courses.washington.edu/gis250/lessons/Model_Builder/ which >> seems rather comprehensive. I wrote the pseudo code myself quickly >> (thus it may not be optimal and may contain bugs - improvement >> suggestions are of course welcome). >> >> I was considering to write these in Perl (on top of the Perl >> bindings) but probably they could also be in OGR core. Code-wise they >> would probably go into ogrlayer.cpp. >> >> If there is support/interest, I assume I should submit this as a RFC >> (with a patch)? >> >> Best regards, >> >> Ari >> >> >> _______________________________________________ >> gdal-dev mailing list >> [email protected] >> http://lists.osgeo.org/mailman/listinfo/gdal-dev > > _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
