Re: [JPP-Devel] [jump-pilot:bugs] Re: #515 Raster display in low memory situation

[giuseppe . aruta]

Hi,
I agree with Michael that ij this step we can consider a good result to have a 
better RasterImageLayer framework for this three reasons: it if faster in 
opening images, it requires less ram, it can open more TIF versions than before.
Possibly we can postpone, as Michael suggests, to rewrite the code to make it 
simpler.
One idea could be to extend Referenced image framework to styling (only 
monoband raster) and give up with RasterImageLayer , even if this will require 
more and more job to rewrite plugins. Or Ede proposals which seems to be more 
interesting. 
We can wait after migration to Openjump V.2
I want to really thank Ede and Michael to all of the efforts, work, ideas and 
discussion. All your contribution that surely many user will benefit in their 
work or course activities.
Thanks again
Giuseppe
--
Inviato da myMail per Android domenica, 29 novembre 2020, 06:35PM +01:00 da 
michael michaud via Jump-pilot-devel  jump-pilot-devel@lists.sourceforge.net :

>
>
>Hi Ede,
>Thank you for the overview of the main raster classes.
>I don't know if your proposition of saving the displayed image to disk would 
>improve performance/memory usage. It would need tests. Anyway, let's postpone 
>this to v2. From my point of view, making raster code more robust and simpler 
>is more important, but making it faster or less greedy in memory would also be 
>nice.
>Michaël
> 
>>envoyé : 26 novembre 2020 à 14:43
>>de : ede < e...@users.sourceforge.net>
>>à : " [jump-pilot:bugs] " < 5...@bugs.jump-pilot.p.re.sourceforge.net>
>>objet : [jump-pilot:bugs] Re: #515 Raster display in low memory situation
>>On 11/25/2020 13:15, michael michaud wrote:
>>>Even after several bug fixes, I still not have a good understanding of the 
>>>image framework.
>>>To be able to imagine improvements, I would need a more precise idea about 
>>>when the image is read from disk and when/where/what is cached.
>>for ReferencedImage(imagery.geoimg) it's pretty easy, albeit the class naming 
>>is maybe not optimal.
>>GeoRaster - takes care of loading, provides a RenderedOp
>>GeoReferencedRaster - adds Referencing, vulgo Envelop
>>GeoImage - renders/paints a given GeoRaster (implements the whole rendering 
>>chain, the only place that caches)
>>the Sextante Raster Image framework is a convoluted mess, not sure what's 
>>going on there. just contributed it reusing GeoReferencedRaster for a way to 
>>enforce the tif loader and reusing the same instance for performance. my 
>>current guess is it creates a buffered image once and caches that in memory.
>>>It would deserve a page on the wiki if it does not exists so that we know 
>>>precisely what we are taking about.
>>probably would. just shying away creating documentation that'll probably 
>>become obsolete fast because nobody maintains it. am pretty wikilazy myself ;(
>>>Not sure I understand your proposition. Can you explain what would be the 
>>>benefit of reading a lossless compressed tiff image from a temp folder 
>>>compared to reading it from the original file ?
>>idea is to cache the visualization on disk instead in memory hence saving 
>>some.
>>>Or is it about saving the symbolization step ?
>>if symbolization is the creation of the visualization of the values in the 
>>monoband, then yes.
>>>I have no idea if transforming rough data to the displayed image is 
>>>cpu-intensive, but I think reading from disk is more time consuming.
>>probably not, creation of the the visualization involves two times iterating 
>>over all pixels of the whole image, one go to find the lower/upper limit, 
>>second time to create the visualization based on that information.
>>>Also it seems that the cached displayed image is just the part of the image 
>>>we want to display. It seems difficult to read/write on disk every time the 
>>>envelope of the visible part of the image changes.
>>it's what ReferencedImage(imagery.geoimg) does except for whole image 
>>overviews.
>>>There are also other ways to optimize like using tiled images or subsampled 
>>>overviews (at least in geotiff). I don't think we are ready to take 
>>>advantage of it.
>>agreed. that's why i suggest the caching on disk approach.
>>..ede
>>--
>>[bugs:#515] Raster display in low memory situation
>>Status: open
>>Milestone: OJ_future
>>Created: Tue Nov 24, 2020 10:20 PM UTC by michael michaud
>>Last Updated: Wed Nov 25, 2020 12:15 PM UTC
>>Owner: michael michaud
>>This ticket follows ticket #513.
>>After correction of the bug reported by Roberto Rossi in #513 two problems 
>>are left over.
>>Images are not loaded permanently in memory, but when they must be shown, 
>>RasterImageLayer evaluates if they fit in memory before trying to display. To 
>>evaluate if the image fits in memory it multiplies the number of pixels by 4 
>>bytes (usual pixel deep for a color image). This evaluation should be more 
>>precise and take into account 1 bit images (b&w) or 64 bits float values.

Re: [JPP-Devel] [jump-pilot:bugs] Re: #515 Raster display in low memory situation

[edgar . soldin]

the bug is already tagged OJ_future so no hurry there.

we still got 5 open OJ 1.16 bugs[1] which look like they might be solved 
already.. ede

[1] https://t1p.de/oix8

On 11/30/2020 10:40, giuseppe.ar...@gmail.com wrote:
> Hi,
> I agree with Michael that ij this step we can consider a good result to have 
> a better RasterImageLayer framework for this three reasons: it if faster in 
> opening images, it requires less ram, it can open more TIF versions than 
> before.
> Possibly we can postpone, as Michael suggests, to rewrite the code to make it 
> simpler.
> One idea could be to extend Referenced image framework to styling (only 
> monoband raster) and give up with RasterImageLayer , even if this will 
> require more and more job to rewrite plugins. Or Ede proposals which seems to 
> be more interesting.
> We can wait after migration to Openjump V.2
> I want to really thank Ede and Michael to all of the efforts, work, ideas and 
> discussion. All your contribution that surely many user will benefit in their 
> work or course activities.
> Thanks again
> Giuseppe
> 
> --
> Inviato da myMail per Android
> 
> domenica, 29 novembre 2020, 06:35PM +01:00 da michael michaud via 
> Jump-pilot-devel jump-pilot-devel@lists.sourceforge.net 
> :
> 
> 
> 
> Hi Ede,
> 
> Thank you for the overview of the main raster classes.
> 
> I don't know if your proposition of saving the displayed image to disk 
> would improve performance/memory usage. It would need tests. Anyway, let's 
> postpone this to v2. From my point of view, making raster code more robust 
> and simpler is more important, but making it faster or less greedy in memory 
> would also be nice.
> 
> Michaël
> 
>  
> 
>> *envoyé :* 26 novembre 2020 à 14:43
>> *de :* ede > >
>> *à :* "[jump-pilot:bugs] " <5...@bugs.jump-pilot.p.re.sourceforge.net 
>> >
>> *objet :* [jump-pilot:bugs] Re: #515 Raster display in low memory 
>> situation
>>
>>
>> On 11/25/2020 13:15, michael michaud wrote:
>>
>> Even after several bug fixes, I still not have a good understanding 
>> of the image framework.
>> To be able to imagine improvements, I would need a more precise idea 
>> about when the image is read from disk and when/where/what is cached.
>>
>> for ReferencedImage(imagery.geoimg) it's pretty easy, albeit the class 
>> naming is maybe not optimal.
>> GeoRaster - takes care of loading, provides a RenderedOp
>> GeoReferencedRaster - adds Referencing, vulgo Envelop
>> GeoImage - renders/paints a given GeoRaster (implements the whole 
>> rendering chain, the only place that caches)
>>
>> the Sextante Raster Image framework is a convoluted mess, not sure 
>> what's going on there. just contributed it reusing GeoReferencedRaster for a 
>> way to enforce the tif loader and reusing the same instance for performance. 
>> my current guess is it creates a buffered image once and caches that in 
>> memory.
>>
>> It would deserve a page on the wiki if it does not exists so that we 
>> know precisely what we are taking about.
>>
>> probably would. just shying away creating documentation that'll probably 
>> become obsolete fast because nobody maintains it. am pretty wikilazy myself 
>> ;(
>>
>> Not sure I understand your proposition. Can you explain what would 
>> be the benefit of reading a lossless compressed tiff image from a temp 
>> folder compared to reading it from the original file ?
>>
>> idea is to cache the visualization on disk instead in memory hence 
>> saving some.
>>
>> Or is it about saving the symbolization step ?
>>
>> if symbolization is the creation of the visualization of the values in 
>> the monoband, then yes.
>>
>> I have no idea if transforming rough data to the displayed image is 
>> cpu-intensive, but I think reading from disk is more time consuming.
>>
>> probably not, creation of the the visualization involves two times 
>> iterating over all pixels of the whole image, one go to find the lower/upper 
>> limit, second time to create the visualization based on that information.
>>
>> Also it seems that the cached displayed image is just the part of 
>> the image we want to display. It seems difficult to read/write on disk every 
>> time the envelope of the visible part of the image changes.
>>
>> it's what ReferencedImage(imagery.geoimg) does except for whole image 
>> overviews.
>>
>> There are also other ways to optimize like using tiled images or 
>> subsampled overviews (at least in geotiff). I don't think we are ready to 
>> take advantage of it.
>>
>> agreed. that's why i suggest the caching on disk approach.
>>
>> ..ede
>>
>> 
>> ---

[JPP-Devel] [jump-pilot:bugs] #513 displaying raster layers in a correct way

[Giuseppe Aruta via Jump-pilot-devel]

I am going to close this bug as it has already solved by Michael's last 
modification on RasterImageLayer framework. 


---

** [bugs:#513] displaying raster layers in a correct way**

**Status:** open
**Milestone:** OJ_1.16
**Labels:** raster display 
**Created:** Wed Nov 11, 2020 12:35 PM UTC by Giuseppe Aruta
**Last Updated:** Mon Nov 23, 2020 01:50 PM UTC
**Owner:** nobody
**Attachments:**

- 
[correct_display.png](https://sourceforge.net/p/jump-pilot/bugs/513/attachment/correct_display.png)
 (27.4 kB; image/png)
- 
[wrong_display.png](https://sourceforge.net/p/jump-pilot/bugs/513/attachment/wrong_display.png)
 (169.1 kB; image/png)


This bug has been described by Roberto Rossi on post:
[JPP-Devel] 6506 loading raster test
It comes out when the RAM  used by OpenJUMP is largely engaged to load and 
display many raster (Sextante) layers, together with big shape files.
It affects only continuous raster layer with a single band (like DTM or DEM)
Roberto provided some test files and a video that discribes the bug.

The link for the video is provided here: 
https://mediaspace.unipd.it/id/1_varzgomj
The files to test the bug are available here:
https://drive.google.com/file/d/1sWFahWBxJnaEherHy3ywGEI5b-CYweOP/view?usp=sharing
In the zip file there 2 shapefile and 26 raster images.

description of the test bug:

1) load the shapefiles
2) load the rasters and wait that they are all displayed
3) zoom to an area that covers a part of the raster set
4) deactivate visibility to all the vector and raster layers
5) start to activate visibility to the layers one by one, starting from vectors 
(which should be the last below in the pile of layers)
6) when you reach at one of the continuous raster layer (DTMxx and Depitxx, 
Aspect, Slope) it may be show a wrong display (see wrong_display.png) instead 
of the right one (see right_display.png).

Note that, depending to the quantity of RAM that your OS uses you may or may 
not see the bug:
a) Roberto Rossi who uses Windows 10 with 8 Gb got the bug after reactivating 
visibility of few rasters
b) I use Ubuntu Mate with 6 RAM and I had to (re)load other 11 raster layersto 
GOT the bug.

Usually the bug disapears if the user zoom in to another area withing the same 
raster (not sure about zoom out). The bug disapears if RAM is free removing 
some layers (shapefiles) from the OpenJUMP project.



---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #513 displaying raster layers in a correct way

[Giuseppe Aruta via Jump-pilot-devel]

- **status**: open --> closed-fixed



---

** [bugs:#513] displaying raster layers in a correct way**

**Status:** closed-fixed
**Milestone:** OJ_1.16
**Labels:** raster display 
**Created:** Wed Nov 11, 2020 12:35 PM UTC by Giuseppe Aruta
**Last Updated:** Mon Nov 30, 2020 07:52 PM UTC
**Owner:** nobody
**Attachments:**

- 
[correct_display.png](https://sourceforge.net/p/jump-pilot/bugs/513/attachment/correct_display.png)
 (27.4 kB; image/png)
- 
[wrong_display.png](https://sourceforge.net/p/jump-pilot/bugs/513/attachment/wrong_display.png)
 (169.1 kB; image/png)


This bug has been described by Roberto Rossi on post:
[JPP-Devel] 6506 loading raster test
It comes out when the RAM  used by OpenJUMP is largely engaged to load and 
display many raster (Sextante) layers, together with big shape files.
It affects only continuous raster layer with a single band (like DTM or DEM)
Roberto provided some test files and a video that discribes the bug.

The link for the video is provided here: 
https://mediaspace.unipd.it/id/1_varzgomj
The files to test the bug are available here:
https://drive.google.com/file/d/1sWFahWBxJnaEherHy3ywGEI5b-CYweOP/view?usp=sharing
In the zip file there 2 shapefile and 26 raster images.

description of the test bug:

1) load the shapefiles
2) load the rasters and wait that they are all displayed
3) zoom to an area that covers a part of the raster set
4) deactivate visibility to all the vector and raster layers
5) start to activate visibility to the layers one by one, starting from vectors 
(which should be the last below in the pile of layers)
6) when you reach at one of the continuous raster layer (DTMxx and Depitxx, 
Aspect, Slope) it may be show a wrong display (see wrong_display.png) instead 
of the right one (see right_display.png).

Note that, depending to the quantity of RAM that your OS uses you may or may 
not see the bug:
a) Roberto Rossi who uses Windows 10 with 8 Gb got the bug after reactivating 
visibility of few rasters
b) I use Ubuntu Mate with 6 RAM and I had to (re)load other 11 raster layersto 
GOT the bug.

Usually the bug disapears if the user zoom in to another area withing the same 
raster (not sure about zoom out). The bug disapears if RAM is free removing 
some layers (shapefiles) from the OpenJUMP project.



---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #512 Georeferenced RasterImage : useless dialog to create a worlfile

[Giuseppe Aruta via Jump-pilot-devel]

Michael- Will you close this ticket? Or shall I do?


---

** [bugs:#512] Georeferenced RasterImage : useless dialog to create a worlfile**

**Status:** open
**Milestone:** OJ_1.16
**Created:** Wed Nov 11, 2020 08:29 AM UTC by michael michaud
**Last Updated:** Mon Nov 23, 2020 01:47 PM UTC
**Owner:** nobody


Trying to open a georeferenced GeoTiff  as a (Sextante) RasterImage , I'm 
invited to define the worldfile through a dialog box. Whatever I enter in the 
diaog box, the image will be georeferenced using the geotags of the geotiff, 
which is fine, but the dialogbox is misleading and will create a tfw file which 
is inconsistant with the geotiff tags.


---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #512 Georeferenced RasterImage : useless dialog to create a worlfile

[Giuseppe Aruta via Jump-pilot-devel]

- **status**: open --> pending



---

** [bugs:#512] Georeferenced RasterImage : useless dialog to create a worlfile**

**Status:** pending
**Milestone:** OJ_1.16
**Created:** Wed Nov 11, 2020 08:29 AM UTC by michael michaud
**Last Updated:** Mon Nov 30, 2020 07:53 PM UTC
**Owner:** nobody


Trying to open a georeferenced GeoTiff  as a (Sextante) RasterImage , I'm 
invited to define the worldfile through a dialog box. Whatever I enter in the 
diaog box, the image will be georeferenced using the geotags of the geotiff, 
which is fine, but the dialogbox is misleading and will create a tfw file which 
is inconsistant with the geotiff tags.


---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #512 Georeferenced RasterImage : useless dialog to create a worlfile

[michael michaud via Jump-pilot-devel]

- **status**: pending --> closed-fixed



---

** [bugs:#512] Georeferenced RasterImage : useless dialog to create a worlfile**

**Status:** closed-fixed
**Milestone:** OJ_1.16
**Created:** Wed Nov 11, 2020 08:29 AM UTC by michael michaud
**Last Updated:** Mon Nov 30, 2020 07:56 PM UTC
**Owner:** nobody


Trying to open a georeferenced GeoTiff  as a (Sextante) RasterImage , I'm 
invited to define the worldfile through a dialog box. Whatever I enter in the 
diaog box, the image will be georeferenced using the geotags of the geotiff, 
which is fine, but the dialogbox is misleading and will create a tfw file which 
is inconsistant with the geotiff tags.


---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #511 CropWarp raster plugin can't overload existing file

[michael michaud via Jump-pilot-devel]

- **Milestone**: OJ_1.16 --> OJ_future
- **Comment**:

Thank Peppe,
I let it opened as it may probably be fixed in a better way, but it can be 
postponed to version 2. Currently, a first dialog ask the user if he wants to 
overwrite the existing file. If yes, the user is informed it is not possible.



---

** [bugs:#511] CropWarp raster plugin can't overload existing file**

**Status:** open
**Milestone:** OJ_future
**Created:** Tue Nov 10, 2020 10:01 PM UTC by michael michaud
**Last Updated:** Mon Nov 23, 2020 01:48 PM UTC
**Owner:** nobody


If one process an image with CropWarpPlugin and try to save the result with the 
name  of an existing image (want to replace the former image), there is no  
warning as in other plugins, and it produces a weird image (looks like a mixt 
between the old and the new image).


---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #382 Delete Incremental Warping Vector tool does not work

[michael michaud via Jump-pilot-devel]

- **status**: open --> closed-fixed



---

** [bugs:#382] Delete Incremental Warping Vector tool does not work**

**Status:** closed-fixed
**Milestone:** OJ_1.16
**Created:** Mon Nov 24, 2014 04:24 PM UTC by Jukka Rahkonen
**Last Updated:** Wed Oct 28, 2020 10:53 AM UTC
**Owner:** nobody


With OpenJUMP r.4108 PLUS it is possible to draw vectors with "Draw Incremental 
Warping Vector" and see the warping result immediately but the "Delete 
Incremental Warping Vector" does not do anything useful. It just flashes the 
symbol that is clicked with the tool.

This warping utility is probably not very popular because it does not work even 
in OpenJUMP 1.3.1 from year 2010. Or then it has to do with jre versions.

It would also be better to render the incremental warping vectors as lines like 
the normal warping vectors. Now the incremental vector has only a pin symbol in 
the end point of the vector.


---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel

[JPP-Devel] [jump-pilot:bugs] #516 RemodelerTool not correct

[michael michaud via Jump-pilot-devel]

- **status**: open --> pending
- **Comment**:

Should be fixed. Waiting for feedback from a known user.



---

** [bugs:#516] RemodelerTool not correct**

**Status:** pending
**Milestone:** OJ_1.16
**Created:** Wed Nov 25, 2020 11:29 AM UTC by michael michaud
**Last Updated:** Wed Nov 25, 2020 11:29 AM UTC
**Owner:** michael michaud


There are several bugs in ModelerTool :
Z should not be interpolated point before or point after has no z
Z interpolation is not correctly computed in the normal case
When the new geometry produced is invalid, there is a warning, but the 
transaction is not rolled back
Corrections are in progress


---

Sent from sourceforge.net because jump-pilot-devel@lists.sourceforge.net is 
subscribed to https://sourceforge.net/p/jump-pilot/bugs/

To unsubscribe from further messages, a project admin can change settings at 
https://sourceforge.net/p/jump-pilot/admin/bugs/options.  Or, if this is a 
mailing list, you can unsubscribe from the mailing list.___
Jump-pilot-devel mailing list
Jump-pilot-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jump-pilot-devel