Yes, I have looked at the various newer versions of the GeoTools Shapefile readers. They seemed to be too complicated to be a drop in replacement for our older GeoTools version (which also has the capability to randomly read, but that is useless in OJ's implementation).
Larry On 8/30/07, Sunburned Surveyor <[EMAIL PROTECTED]> wrote: > Larry, > > I don't know if it is of interest to you, but GeoTools has a streaming > ESRI Shapefile parser. They actually have the ability to randomly > access an indexed Shapefile. I plan on using their Shapefile code in > my FeatureCache. The idea is to use the GeoTools code to access one > Feature at a time from the Shapefile, storing this Feature in the > FeatureCache. > > SS > > On 8/30/07, Larry Becker <[EMAIL PROTECTED]> wrote: > > >At one point I actually rewrote the Shapefile parser to be streaming as > > >well... > > > > I guess I hadn't actually realized it wasn't until now. Do you > > remember what happened to the code, or why you didn't stay with the > > streaming version? A very large shape file seems like a more likely > > scenario that I actually care about. > > > > Larry > > On 8/30/07, Martin Davis <[EMAIL PROTECTED]> wrote: > > > This would be especially important if you had multiple > > > FeatureCollections stored in one file, and you wanted to load only one > > > of them. > > > > > > Sunburned Surveyor wrote: > > > > Paul is correct. The pull parser does not reduce the memory of the > > > > parsing results, but it does reduce the memory used during the parsing > > > > process. That is because an in-memory representation of the entire XML > > > > document is not constructed. > > > > > > > > One advantage of this is using the parser to select only data within > > > > the XML file that meets specific criteria. For example, if we had a > > > > 50MB SGF file representing the city of Stockton, I could parse the > > > > file and create only building features, even thought the file might > > > > contain road features, landmark features, park features. etc. > > > > In fact, I could even parse the file and only create features for > > > > buildings whose "building type" attribute was set to "Public". This > > > > allows me to extract the information I want without reading all 50 MB > > > > into memory. > > > > > > > > The Sunburned Surveyor > > > > > > > > On 8/30/07, Paul Austin <[EMAIL PROTECTED]> wrote: > > > > > > > >> Hi Larry, > > > >> > > > >> You are correct that the resulting data set will take up a lot of > > > >> memory > > > >> at the end, the advantage with the pull parser is that you don't take > > > >> up > > > >> a whole bunch of extra memory for the XML DOM structures which > > > >> typically > > > >> get loaded into memory for the whole document. So with the pull parser > > > >> there is little memory overhead where as for DOM you have probably at > > > >> least 2x memory required to load if not more > > > >> > > > >> Paul > > > >> > > > >> Larry Becker wrote: > > > >> > > > >>> It isn't the parser that takes up the memory except temporarily), but > > > >>> the memory resident dataset after loading. This will still limit the > > > >>> size. > > > >>> > > > >>> Larry > > > >>> > > > >>> On 8/30/07, Sunburned Surveyor <[EMAIL PROTECTED]> wrote: > > > >>> > > > >>> > > > >>>> Yup. It makes you wonder why they didn't use pull parsers from the > > > >>>> very beginning, doesn't it. > > > >>>> > > > >>>> SS > > > >>>> > > > >>>> On 8/30/07, Paul Austin <[EMAIL PROTECTED]> wrote: > > > >>>> > > > >>>> > > > >>>>> Agreed the pull parser is the only way to go for large XML files > > > >>>>> > > > >>>>> Paul > > > >>>>> > > > >>>>> Sunburned Surveyor wrote: > > > >>>>> > > > >>>>> > > > >>>>>> Martin, > > > >>>>>> > > > >>>>>> If we decide to support a restricted form of GML 2 we could build > > > >>>>>> our > > > >>>>>> reader and writer on top of the XML Pull Parser from Sun. This > > > >>>>>> would > > > >>>>>> help us to avoid memory problems when reading in large files. > > > >>>>>> > > > >>>>>> https://sjsxp.dev.java.net/ > > > >>>>>> > > > >>>>>> Just a thought. > > > >>>>>> > > > >>>>>> The Sunburned Surveyor > > > >>>>>> > > > >>>>>> ------------------------------------------------------------------------- > > > >>>>>> This SF.net email is sponsored by: Splunk Inc. > > > >>>>>> Still grepping through log files to find problems? Stop. > > > >>>>>> Now Search log events and configuration files using AJAX and a > > > >>>>>> browser. > > > >>>>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > >>>>>> _______________________________________________ > > > >>>>>> Jump-pilot-devel mailing list > > > >>>>>> Jump-pilot-devel@lists.sourceforge.net > > > >>>>>> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > >>>>>> > > > >>>>>> > > > >>>>>> > > > >>>>> ------------------------------------------------------------------------- > > > >>>>> This SF.net email is sponsored by: Splunk Inc. > > > >>>>> Still grepping through log files to find problems? Stop. > > > >>>>> Now Search log events and configuration files using AJAX and a > > > >>>>> browser. > > > >>>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > >>>>> _______________________________________________ > > > >>>>> Jump-pilot-devel mailing list > > > >>>>> Jump-pilot-devel@lists.sourceforge.net > > > >>>>> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>> ------------------------------------------------------------------------- > > > >>>> This SF.net email is sponsored by: Splunk Inc. > > > >>>> Still grepping through log files to find problems? Stop. > > > >>>> Now Search log events and configuration files using AJAX and a > > > >>>> browser. > > > >>>> Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > >>>> _______________________________________________ > > > >>>> Jump-pilot-devel mailing list > > > >>>> Jump-pilot-devel@lists.sourceforge.net > > > >>>> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > >>>> > > > >>>> > > > >>>> > > > >>> > > > >>> > > > >> ------------------------------------------------------------------------- > > > >> This SF.net email is sponsored by: Splunk Inc. > > > >> Still grepping through log files to find problems? Stop. > > > >> Now Search log events and configuration files using AJAX and a browser. > > > >> Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > >> _______________________________________________ > > > >> Jump-pilot-devel mailing list > > > >> Jump-pilot-devel@lists.sourceforge.net > > > >> https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > >> > > > >> > > > > > > > > ------------------------------------------------------------------------- > > > > This SF.net email is sponsored by: Splunk Inc. > > > > Still grepping through log files to find problems? Stop. > > > > Now Search log events and configuration files using AJAX and a browser. > > > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > > _______________________________________________ > > > > Jump-pilot-devel mailing list > > > > Jump-pilot-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > > > > > > > > > > > -- > > > Martin Davis > > > Senior Technical Architect > > > Refractions Research, Inc. > > > (250) 383-3022 > > > > > > > > > ------------------------------------------------------------------------- > > > This SF.net email is sponsored by: Splunk Inc. > > > Still grepping through log files to find problems? Stop. > > > Now Search log events and configuration files using AJAX and a browser. > > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > > _______________________________________________ > > > Jump-pilot-devel mailing list > > > Jump-pilot-devel@lists.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > > > > > > -- > > http://amusingprogrammer.blogspot.com/ > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Jump-pilot-devel mailing list > > Jump-pilot-devel@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Jump-pilot-devel mailing list > Jump-pilot-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > -- http://amusingprogrammer.blogspot.com/ ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
