Hi Barry I have recently done something similar and had a slightly different problem, but still related to too many files, with GeoServer performing very slowly when zoomed out. Our solution, which works very well, was to create a layer group with the original (TIFF) ImageMosaic store from scale 50000 - 2600000 and another top layer ImageMosaic store from scale 2600000 - 400000000.
This top layer was created by downsampling each DEM image to 6.25% with gdal_translate and then combining them into 16x16 degree tiles with gdalwarp. The same gdaldem hillshade command can then be run on these top layer tiles. By splitting the layer like this, the number of files being accessed was reduced to a level that GeoServer can very easily manage and it works perfectly for us. The top layer does not consume too much extra disk space. I'm not sure if this is the best way to do it, so I'm always willing to hear from others if there's a better solution. Regards Peter AfriGIS On Tue, 21 May 2019 at 04:18, Barry Carter <[email protected]> wrote: > I'm trying to provide SRTM1 (1 arcsecond resolution elevation) images and > data > using GeoServer, but can't figure out how to serve such a large dataset > efficiently. Here's what I've done so far: > > - Downloaded the 1 degree squared HGT files. The Earth is 360*180 = > 64800 > square degrees, but only 14295 HGT files since the other square > degrees > are 100% water. The HGT files are uncompressed and 25,934,402 bytes > each. > > - I then used commands like: > > gdal_translate -co TILED=YES -co COMPRESS=DEFLATE N58W122.hgt > N58W122.hgt.tif > > to convert these HGT files into TIFFs. The TIFFs vary in size, but are much > smaller. > > - I then imported the directory with the TIFFs as a store using > ImageMosaic. This took a while, but appears to have worked, since > "srtm1" > (the name I gave it) now appears in my list of stores. > > - I then created a layer using this source, but when I use "OpenLayers" > preview, the server hangs for a while and ultimately dies with "too > many > open files" (or similar). This seems odd, since my Linux OS had > fs.file-max set to 13063650. I even tried doubling it to 26127300, but > that didn't help. > > At this point, I suspect I need to create an image pyramid or something so > GeoServer can handle the data efficiently, but I'm not sure exactly what to > do. I've used 'gdal_retile.py' previously to create pyramids from one large > image into tiles at various zoom levels, but I think I want to do the > opposite > here (merge smaller images for lower zoom levels). The gdal_retile.py > option > -pyramidsonly seems to say it does that, but I can't get it work (it just > hangs and creates no files) > > > > _______________________________________________ > Geoserver-users mailing list > > Please make sure you read the following two resources before posting to > this list: > - Earning your support instead of buying it, but Ian Turton: > http://www.ianturton.com/talks/foss4g.html#/ > - The GeoServer user list posting guidelines: > http://geoserver.org/comm/userlist-guidelines.html > > If you want to request a feature or an improvement, also see this: > https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer > > > [email protected] > https://lists.sourceforge.net/lists/listinfo/geoserver-users >
_______________________________________________ Geoserver-users mailing list Please make sure you read the following two resources before posting to this list: - Earning your support instead of buying it, but Ian Turton: http://www.ianturton.com/talks/foss4g.html#/ - The GeoServer user list posting guidelines: http://geoserver.org/comm/userlist-guidelines.html If you want to request a feature or an improvement, also see this: https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer [email protected] https://lists.sourceforge.net/lists/listinfo/geoserver-users
