Michaël, We are planning to include an API for writing to DataStores at some point in the future, but we do not have a target date yet.
I should also remind you, this API is not intended to feed a streaming renderer (like Geotools), so there will still be cases where the size of the data will preclude the user from viewing all the data at one time. David Zwiers Vivid Solutions Telephone : (250) 385-6040 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michaël Michaud Sent: August 22, 2006 4:03 PM To: List for discussion of JPP development and use. Subject: Re: [JPP-Devel] Refactoring JUMP fpr support oflarge spatialdatasets... >P.S. - If I can't get this figured out I will give you a call, but I >wanted to post this to the list for the benefit of the other >developers. > > I am very pleased to follow this discussion. I already tested shapefile-scalable from agile, unsuccessufully tried to implement a hsqldb-database connection to avoid the memory pb, successfully implemented the feature interface to read features on disk, and catched headaches trying to understand the GeoTools datastore api. So I am pleased to follow this interesting discussion on how will jump soon allow large dataset reading. The next question is... is the new datastore api planned for scalable readers only or does it include data modification capabilities for scalable disk-based readers/writers ? Michaël >On 8/22/06, David Zwiers <[EMAIL PROTECTED]> wrote: > > >>Sunburned, >> >>Most of the required components are readily available. I would look into >>leveraging the DataStore framework (Jump CVS) in conjunction with an >>indexed SHP reader (GeoTools port / interface?). You need only write the >>'glue code' between the DataStore framework (already does caching based >>on the viewing area) and your SHP reader of choice. >> >>If you need a few more directed hints to get started, feel free to drop >>me a line. >> >>David Zwiers >>Vivid Solutions >>Telephone : (250) 385-6040 >> >> >>-----Original Message----- >>From: [EMAIL PROTECTED] >>[mailto:[EMAIL PROTECTED] On Behalf Of >>Sunburned Surveyor >>Sent: August 22, 2006 8:26 AM >>To: List for discussion of JPP development and use. >>Subject: [JPP-Devel] Refactoring JUMP fpr support of large >>spatialdatasets... >> >>I've been taking a look at the OpenJUMP source code, as well as the >>work done by Alvaro on Agiles version of OpenJUMP, in an effort to >>come up with a relatively simple solution to the RAM restriction >>OpenJUMP has when reading large ESRI Shapefiles or other spatial >>datasets. I've hit an obstacle of sorts, and I was hoping some of the >>more experienced developers would have a solution. >> >>Alvaro overcame the RAM restriction by creating an object that >>implemented the FeatureCollection interface, but stored its features >>on disk, and not in RAM. I had the same idea, and was initially >>pleased when I saw Alvaro had pioneered this technique. I didn't >>understand why Alvaro had created his own version of the OpenJUMP >>core, instead of encapsulating his support for large spatial datasets >>in a plug-in. >> >>Then I read Alvaro's note: >> >>" In package org.agil.dao, you could see a FeatureCollection >>implementation >>(FeatureCollectionOnDemand) which read features directly from a >>database. >>After use it, you must change all JUMP code where get a List from a >>FeatureCollection, and change it for an Iterator." >> >>I took a look at the source code and found that Alvaro was indeed >>correct. The FeatureCollection interface has two methods that return a >>generic list of all the Feature objects in the FeatureCollection. This >>creates a problem, as all of the Feature objects then need to exist in >>RAM, which is what we are trying to avoid in the first place. >> >>One of these two methods, the getFeatures() method, is called 26 times >>in other parts of the source code. >> >>What does this mean? It means both JUMP and OpenJUMP are currently >>wired in a way that prevents RAM independent FeatureCollection >>implementations without some significant changes to the core source >>code. (It would require that all of the methods that expect a list of >>all the Features in a FeatureCollection be changed to work with an >>Iterator object, as Alvaro suggested.) >> >>There are a couple of ways to fix this. I want to know which way will >>be the simplest and most effective. >> >>[1] Refactor the OpenJUMP source code to use Iterators instead of >>lists when calling the two problem methods of the FeatureCollection >>interface. >> >>[2] Create a "RAM" independent object that implements the Feature >>interface instead of a RAM independent implementation of an object the >>Feature Collection interface. We could then return a list of these RAM >>independent Features. (I'm not sure how this would reduce memory usage >>though. Probably not as well as a RAM independent FeatureCollection >>implementation. There might also be a pretty severe performance >>penalty. ???) >> >>[3] Create a custom implementation of the Java.Util.List interface >>that doesn't require its member objects to be in RAM. The only real >>problem I saw with this technique initially is the toArray() method. I >>suppose we could return an empty array, because I don't know another >>way to implement this method without placing all the Feature Objects >>into the array, and thus into RAM. This technique will also ivolve a >>bit of work, as the List interface contains quite a few methods. >> >>I'm really looking for the simplest solution. I have planned a >>"spatial database" for OpenJUMP-Ex that will solve this problem in >>OpenJUMP's design. So I don't really want to invest a lot of energy >>into what I hope will be a temporary "band-ais" solution. >> >>Thank you for your thoughts. >> >>The Sunburned Surveyor >> >>------------------------------------------------------------------------ >>- >>Using Tomcat but need to do more? Need to support web services, >>security? >>Get stuff done quickly with pre-integrated technology to make your job >>easier >>Download IBM WebSphere Application Server v.1.0.1 based on Apache >>Geronimo >>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >>_______________________________________________ >>Jump-pilot-devel mailing list >>Jump-pilot-devel@lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> >> >> >>------------------------------------------------------------------------- >>Using Tomcat but need to do more? Need to support web services, security? >>Get stuff done quickly with pre-integrated technology to make your job easier >>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >>_______________________________________________ >>Jump-pilot-devel mailing list >>Jump-pilot-devel@lists.sourceforge.net >>https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel >> >> >> > >------------------------------------------------------------------------- >Using Tomcat but need to do more? Need to support web services, security? >Get stuff done quickly with pre-integrated technology to make your job easier >Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo >http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 >_______________________________________________ >Jump-pilot-devel mailing list >Jump-pilot-devel@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel > > > > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Jump-pilot-devel mailing list Jump-pilot-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel
