[gdal-dev] Zarr BLOCKSIZE in gdalmdimtranslate with variables of varying dimension
Hi list, gdalmdimtranslate is a very handy tool e.g. to translate netCDF files to Zarr as it can handle huge files that xarray just chokes on. I have a question regarding handling output blocksizes for the Zarr format for files with variables of varying dimensions though. For example I need to translate a netCDF file with time, z, lat, lon dimensions and variables which are either 4D (time, z, lat, lon), 3D (time, lat, lon) and 2D (lat, lon). I would like to remap chunksizes / blocksizes to better suit my use case. However, it seems that I can only specify blocksizes as e.g. -co ARRAY:BLOCKSIZE 2, 2, 128, 128 - which will throw an error and skip the variables that don't have 4 dimensions. Also, the output doesn't contain the dimension variables in this case. Likewise, for 4D variables if I only specify 3 blocksizes. Have tried this with gdal versions 3.8.4 and 3.4.1 with similar results. So the question is if there is any way to specify blocksizes e.g. by dimension name? Or if the approach should rather be to just create 3 output Zarr folders - one for each variable dimensionality (2D, 3D and 4D)? I can of course manually "merge" these folders afterwards. Best regards, Simon ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Motion: approve use of GDAL Sponsorship Program funds for 2 documentation contributors
+1 Tamas Even Rouault via gdal-dev ezt írta (időpont: 2025. márc. 12., Sze, 14:16): > Hi, > > Seth Girvin (https://github.com/geographika) and Harrissou Sant-anna > (DelazJ) are interested in contributing improvements to the GDAL > documentation, one of the most recurring item where enhancements are > wished from the feedback we got through the User Survey last November. > Both have experience on documentation of open source geospatial software > (respectively on Mapserver for Seth and QGIS for Harrissou) and are > excited to bring it to GDAL. This is understood this will be a small > part time activity for both of them. Dan Baston has kindly agreed to > supervise their work, regarding initial setup, review of pull requests > and filing in documentation tickets for planned tasks in the issue > tracker. We will also welcome Seth and Harrissou inputs. > > A non-exhaustive list of the potential different activities is: > > - Adding more examples to existing documentation (typically for command > line utilities) > - Systematic addition of docstrings for Python API > - Adding tutorial-like material (with graphics when appropriate), > illustrating workflows that can combine several tools to produce a > desired outcome > - Adding a user-focused, example-driven introduction to the project on > gdal.org and github.com/OSGeo/GDAL, ala README of > https://github.com/qgis/qgis > - Harmonization of presentation of existing documentation / making it > more easy to find content > - Hierarchisation of content in existing documentation. For popular > tools or drivers, it could probably be interesting to have "basic" and > "advanced" sections (typically for the GeoTIFF page that could/should > probably be split) > > Motion: approve use of up to 25,000 euros for Seth Girvin and up to > 25,000 euros for Harrissous Sant-anna from the GDAL Sponsorship Program, > for the period April 1st 2025 - March 31st 2026, on documentation > enhancements for GDAL. > > Starting with my +1, > > Even > > -- > http://www.spatialys.com > My software is free, but my time generally not. > > ___ > gdal-dev mailing list > gdal-dev@lists.osgeo.org > https://lists.osgeo.org/mailman/listinfo/gdal-dev > ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
Re: [gdal-dev] Zarr BLOCKSIZE in gdalmdimtranslate with variables of varying dimension
Simon, see the ARRAY:IF(DIM=2):BLOCKSIZE=256,256 example of GDALGroup::CopyFrom() mentioned at https://gdal.org/en/stable/doxygen/classGDALGroup.html#a28324579902842d7dbd1f93e9698b1ec Even Le 12/03/2025 à 09:23, Simon Lyngby Kokkendorff via gdal-dev a écrit : Hi list, gdalmdimtranslate is a very handy tool e.g. to translate netCDF files to Zarr as it can handle huge files that xarray just chokes on. I have a question regarding handling output blocksizes for the Zarr format for files with variables of varying dimensions though. For example I need to translate a netCDF file with time, z, lat, lon dimensions and variables which are either 4D (time, z, lat, lon), 3D (time, lat, lon) and 2D (lat, lon). I would like to remap chunksizes / blocksizes to better suit my use case. However, it seems that I can only specify blocksizes as e.g. -co ARRAY:BLOCKSIZE 2, 2, 128, 128 - which will throw an error and skip the variables that don't have 4 dimensions. Also, the output doesn't contain the dimension variables in this case. Likewise, for 4D variables if I only specify 3 blocksizes. Have tried this with gdal versions 3.8.4 and 3.4.1 with similar results. So the question is if there is any way to specify blocksizes e.g. by dimension name? Or if the approach should rather be to just create 3 output Zarr folders - one for each variable dimensionality (2D, 3D and 4D)? I can of course manually "merge" these folders afterwards. Best regards, Simon ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev -- http://www.spatialys.com My software is free, but my time generally not. ___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
