Re: [gdal-dev] GDAL mailing list email delays

2025-05-29 Thread Laurențiu Nicola via gdal-dev
r and their script would move on to another target. But if you have a real mail server (or a compromised account, or an open relay which accepts and forwards messages for anyone), it won't help at all. Laurentiu On Thu, May 29, 2025, at 20:34, Laurențiu Nicola via gdal-dev wrote: > Hi,

Re: [gdal-dev] GDAL mailing list email delays

2025-05-29 Thread Laurențiu Nicola via gdal-dev
Hi, I think it's at least in part an anti-spam measure. We reject messages from new users and their server will try later. Spammers are less likely to retry delivery of their messages. I'm personally not a fan of this, but.. Laurentiu On Thu, May 29, 2025, at 20:29, Daniel Evans via gdal-dev

Re: [gdal-dev] Performance Issue with VRT Pixel Function and Large Number of Source Rasters

2025-04-14 Thread Laurențiu Nicola via gdal-dev
Hi Abdul, Can you bump https://gdal.org/en/stable/user/configoptions.html#config-GDAL_MAX_DATASET_POOL_SIZE and see if it helps? And do all inputs have the same block size of 1048x7? Ideally, your inputs and output will all have the same block size or a multiple of it, so you could also add -c

Re: [gdal-dev] Fwd: Performance Variability with GDAL Caching and Multi-Threading for MODIS Data

2025-04-01 Thread Laurențiu Nicola via gdal-dev
Hi, ReprojectImage is an older API that doesn't support the full set of warping options. If you're asking about -multi and -wm NUM_THREADS, they don't get (at least in gdalwarp and IIRC) enabled automatically by GDAL_NUM_THREADS. Switching to gdal.Warp might be worthwhile. Note that pymodis is

Re: [gdal-dev] Fwd: Performance Variability with GDAL Caching and Multi-Threading for MODIS Data

2025-04-01 Thread Laurențiu Nicola via gdal-dev
Hi, Since it's not exactly clear from your description, what operations are you running, just the equivalent of gdal.Translate()? gdal.Warp()? GDAL can use threading in a couple of places: • to compress the output before writing it, e.g. the NUM_THREADS creation option of GTiff • to decompre

Re: [gdal-dev] Reading Layers In USGS Geospatial Topo Files:

2025-03-25 Thread Laurențiu Nicola via gdal-dev
Hi Carl, I randomly found https://opengislab.com/blog/tag/GeoPDF+to+GeoTiff which seems helpful, but I have no experience with those USGS PDFs. I tried to build one with their app, but it will take a day or so for them to send it. Laurentiu On Tue, Mar 25, 2025, at 22:40, Carl Carlson via gdal

Re: [gdal-dev] Mute HDF5-DIAG diagnostic output

2025-03-04 Thread Laurențiu Nicola via gdal-dev
I guess that's because you're enabling exceptions, but even without them I get no error logging at all, at least on the file I tried with. Laurentiu On Wed, Mar 5, 2025, at 08:55, Christian Beilschmidt via gdal-dev wrote: > I tried it with the Python bindings, but there I don't get the `HDF5-DIA

Re: [gdal-dev] Mute HDF5-DIAG diagnostic output

2025-03-04 Thread Laurențiu Nicola via gdal-dev
PS: the only reference to HDF5's error handling I found while skimming the GDAL code was https://github.com/OSGeo/gdal/blob/0caa252/frmts/hdf5/hdf5vfl.h#L236-L241. On Tue, Mar 4, 2025, at 10:09, Laurențiu Nicola via gdal-dev wrote: > Hi Christian, > > On Tue, Mar 4, 2025, at 10

Re: [gdal-dev] Mute HDF5-DIAG diagnostic output

2025-03-04 Thread Laurențiu Nicola via gdal-dev
Hi Christian, On Tue, Mar 4, 2025, at 10:05, Christian Beilschmidt via gdal-dev wrote: I tried sending GDAL's error logging to /dev/null but this does not change anything. I take it https://docs.rs/gdal/latest/gdal/config/fn.set_error_handler

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-27 Thread Laurențiu Nicola via gdal-dev
Agreed, I know I'm heavily biased myself towards a little corner of tech. PS: I raised the issue of the local headers in https://github.com/csaybar/ESA-zar-zip-decision/issues/5, if you want to follow. On Thu, Feb 27, 2025, at 12:07, Michael Sumner wrote: > Agree with all this, unfortunately xar

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-27 Thread Laurențiu Nicola via gdal-dev
Hi, While I agree with the sentiment, I'm not sure I agree about some of the details. It assumes that on ZARR ZIPs, you need to make twice as many reads, for the local file headers and for the data. But the information in the local header is already available in the central directory, so you ca

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-24 Thread Laurențiu Nicola via gdal-dev
I suspect it won't make a lot of difference. SOZIP is designed to allow seeking within a compressed file, but Zarr is tile- (block) based, where each of those is stored in a different file. So you end up uncompressing a whole file anyway. Laurentiu On Mon, Feb 24, 2025, at 21:17, Scott via gdal

Re: [gdal-dev] How to wrap a zarr in a zip and read it with vsizip?

2025-02-24 Thread Laurențiu Nicola via gdal-dev
Hi Kurt, You'll want something like: cd nczarr_v2.zarr zip -r ../nczarr_v2.zarr.zip . gdalinfo /vsizip/../nczarr_v2.zarr.zip Note that tar makes TAR, not ZIP, archives. Laurentiu On Mon, Feb 24, 2025, at 20:09, Kurt Schwehr via gdal-dev wrote: > Hi all, > > I seem to be having trouble exactly

Re: [gdal-dev] unsetting configs vs empty values

2025-02-18 Thread Laurențiu Nicola via gdal-dev
Hi, To me, that's a bit unexpected and possibly a bug, but not a problem I've ever run into in practice. You're probably aware of this already, but instead of: > export AWS_S3_ENDPOINT=... > gdalmdiminfo ... > unset AWS_S3_ENDPOINT > gdalmdiminfo ... you can do: > AWS_S3_ENDPOINT=... gdalmdimi

Re: [gdal-dev] no autodetect for ZARR output

2025-02-04 Thread Laurențiu Nicola via gdal-dev
Hi, On Tue, Feb 4, 2025, at 13:56, Even Rouault via gdal-dev wrote: > A PR to add .zarr as an extension for the Zarr driver ? Is it recommended by > the Zarr spec (can't find anything) or considered a best practice ? This > wasn't my impression, although that would certainly be desirable. Their

Re: [gdal-dev] Fwd: GDAL open flags

2025-01-16 Thread Laurențiu Nicola via gdal-dev
Hi Michał, Without knowing a lot about the underlying implementation, I would expect those flags to make no difference in performance. They exist for safety reasons (so you don't modify a dataset by mistake), not as an optimization. As for the docs, that seems expected. The documentation can't

Re: [gdal-dev] Help with overlapping images in tiles generated by gdal2tiles.py

2024-12-04 Thread Laurențiu Nicola via gdal-dev
Hi Jose, I'm not sure if this is happening in your case, but I've seen something similar with rotated rasters that used 0 or a negative value for the outside pixels. If you're doing interpolation or reprojection, those values will be used in the computation and affect neighboring pixels. But if

Re: [gdal-dev] How gdal_translate scale the value, like from a larger range to a smaller range?

2024-12-03 Thread Laurențiu Nicola via gdal-dev
Hi Lucas, GDAL will keep the input values unchanged when possible, and saturate (use the maximum or minimum representable value) when not. So 10050 will be transformed to 255 when you're converting to Byte, and -4 will be saved as -32768 for Int16. NaN nodata values will be converted to 0 w

[gdal-dev] Should we call RollbackTransaction if CommitTransaction fails?

2024-11-28 Thread Laurențiu Nicola via gdal-dev
Hi, Is there any advantage in trying to roll back a transaction that failed to commit? Will it leave the dataset in a "more usable" state than not doing anything? Thanks, Laurentiu___ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.

[gdal-dev] NULL-returning functions and CPLGetLastErrorMsg

2024-11-27 Thread Laurențiu Nicola via gdal-dev
Hello, Many GDAL functions can return nullptr (e.g. OSRGetName, OSRGetAuthorityCode etc.), but it's not clear if they set an error message that can be retrieved using CPLGetLastErrorMsg. I tried to check the code, but CPLErrorSetState isn't called from many places, so I'm probably missing somet

Re: [gdal-dev] OGC services and credentials

2024-10-29 Thread Laurențiu Nicola via gdal-dev
Hi, It's unlikely to be a problem for you, but credential passing via command-line arguments is considered an anti-pattern because they're visible by other users who might be looking at the running process list. Environment variables are better because other users can't inspect them. Of course

Re: [gdal-dev] gdal_translate

2024-10-15 Thread Laurențiu Nicola via gdal-dev
Yes, you can use gdal_edit -gcp to add some GCPs, then gdalwarp -tps to straighten out the image. QGIS has a tool to do this interactively, but last time I tried it, it somewhat buggy and confusing. Laurentiu On Tue, Oct 15, 2024, at 16:54, Paul via gdal-dev wrote: > Is there a way to transfor

Re: [gdal-dev] Proposed RFC 8 amendment regarding (prohibited use of) generative AI tools

2024-10-08 Thread Laurențiu Nicola via gdal-dev
Hi, > At the moment I am using copilot as a cut-and-paste on steroids and it > has improved my efficiency, to me it is no different than copying and > pasting from another file in the GDAL source code or going to SO to > search how to write a complex recursive lambda implementation in C++11, >

Re: [gdal-dev] Call for review on RFC 101: Raster dataset read-only thread-safety

2024-09-07 Thread Laurențiu Nicola via gdal-dev
On Sat, Sep 7, 2024, at 17:49, Even Rouault via gdal-dev wrote: > It could be argued that coding it in C wouldn't be the best idea. I've > spotted a TIFF Rust library (https://github.com/image-rs/image-tiff) but it > is functionally limited (at least in terms of supported codecs. didn't check >

Re: [gdal-dev] Call for review on RFC 101: Raster dataset read-only thread-safety

2024-09-06 Thread Laurențiu Nicola via gdal-dev
Hi, On Sat, Sep 7, 2024, at 01:34, Even Rouault via gdal-dev wrote: > I just don't see how to do that. Even a mutex around file operations would > not be sufficient since we'd want both Seek()+Read() to be atomic per thread. > PRead() would solve that, but very few drivers or underlying librarie

Re: [gdal-dev] NEWS.md for last release

2024-07-17 Thread Laurențiu Nicola via gdal-dev
On Wed, Jul 17, 2024, at 10:49, Javier Jimenez Shaw wrote: > I find it strange. There is a long list of news ... but it covers only things > that were added exactly in the x.y.0 releases. All the changes that happened > in "bugfix releases" (that is x.y.z for z>0) is only on that branch. > You

Re: [gdal-dev] NEWS.md for last release

2024-07-16 Thread Laurențiu Nicola via gdal-dev
Try https://github.com/OSGeo/gdal/blob/release/3.9/NEWS.md, I think the patch release changes either get folded into the next minor changelogs or don't get included at all. But the release branch always has the full set of changes. Laurentiu On Tue, Jul 16, 2024, at 12:09, Javier Jimenez Shaw v

Re: [gdal-dev] [EXTERNAL] [BULK] Re: GetPoint API listing

2024-07-08 Thread Laurențiu Nicola via gdal-dev
Yeah, sorry, I must have ended up looking at the Doxygen docs: https://gdal.org/doxygen/classOGRGeometry.html. There's a link to them on https://gdal.org/api/index.html, you can use those if you run into missing classes again. Laurentiu On Mon, Jul 8, 2024, at 22:59, Meyer, Jesse R. (GSFC-618.

Re: [gdal-dev] [EXTERNAL] [BULK] Re: GetPoint API listing

2024-07-08 Thread Laurențiu Nicola via gdal-dev
“C++ API” landing page > under “Vector API”? > https://gdal.org/api/index.html > > Best, > Jesse > > *From: *gdal-dev on behalf of Laurențiu > Nicola via gdal-dev > *Date: *Monday, July 8, 2024 at 3:12 PM > *To: *gdal-dev@lists.osgeo.org > *Subject: *[EXTERNAL]

Re: [gdal-dev] GetPoint API listing

2024-07-08 Thread Laurențiu Nicola via gdal-dev
Hi, You can find methods like addPoint and getPoint in the derived geometry classes like https://gdal.org/doxygen/classOGRLineString.html#a49c349d567436a1101017b1e71771ba3. Laurentiu On Mon, Jul 8, 2024, at 21:57, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev

Re: [gdal-dev] GDALGetGeoTransform

2024-06-11 Thread Laurențiu Nicola via gdal-dev
Hi Johannes, You can use GDALGetGCPCount to check if the file has GCPs, then use gdalwarp -tps or the equivalent to uncrinkle it: https://stackoverflow.com/questions/48770002/project-raster-file-using-gcps. Laurentiu On Tue, Jun 11, 2024, at 17:02, Johannes Paul via gdal-dev wrote: > Hello, >

Re: [gdal-dev] Hungarian Notation

2024-04-17 Thread Laurențiu Nicola via gdal-dev
Hi Andrew, I think GDAL uses a mix of styles, but if you dislike Hungarian notation, it's still important to distinguish between "systems" (e.g. dwBytes) and "apps" (e.g. cchWideChar) notation. The first one just repeats the variable type, which is mostly redundant, but the second actually giv

Re: [gdal-dev] GDAL under cygwin doesn't read geotiff

2024-04-03 Thread Laurențiu Nicola via gdal-dev
Hi, Someone reported on IRC a similar crash about two weeks ago. As far as I can tell, that one is up to the Cygwin maintainers, so you should report it on their mailing list https://cygwin.com/lists.html#cygwin. Not sure about the Cygwin error and how well-tested GDAL is on that platform. The

Re: [gdal-dev] CPLCalloc() used for small memory allocations. And for big ones?

2024-02-14 Thread Laurențiu Nicola via gdal-dev
Hi, On Wed, Feb 14, 2024, at 10:53, Abel Pau via gdal-dev wrote: > Hi, > > I am currently working on a driver and have been utilizing CPL_calloc(). > However, when attempting to pass the Actions, I encounter errors such as: > > warning C4244: 'function': conversion from 'MM_INTERNAL_FID' to 'si

Re: [gdal-dev] linting test in Code Checks always fails

2024-02-13 Thread Laurențiu Nicola via gdal-dev
On Tue, Feb 13, 2024, at 14:25, Abel Pau via gdal-dev wrote: > Hi, > I am trying to make my code good for all Actions in the project. I'm guessing you should remove https://github.com/AbelPau/gdal/tree/master/out/build/x64-Debug from git (and ideally squash at the end).___

Re: [gdal-dev] Delete untagged containers

2024-02-12 Thread Laurențiu Nicola via gdal-dev
Hi, That's related to the ghcr.io container registry, it's failing on your fork because you're missing an access token. Don't worry about it. Laurentiu On Mon, Feb 12, 2024, at 12:40, Abel Pau via gdal-dev wrote: > Hi, > when passing “actions” there is one that I think it’s not depending on me:

Re: [gdal-dev] core dump on dir info

2024-02-06 Thread Laurențiu Nicola via gdal-dev
On Tue, Feb 6, 2024, at 16:04, Even Rouault via gdal-dev wrote: > Le 06/02/2024 à 14:46, Javier Jimenez Shaw a écrit : > I remember older VirtualBox versions indeed had issues to propage AVX to > guest, but I believed that has been fixed now. I'm not aware of issues > with SSEx not being propagat

Re: [gdal-dev] gdal_rasterize "burn count" feature

2024-02-01 Thread Laurențiu Nicola via gdal-dev
Hi, You can use -sql and -a to burn a field of the query. Laurentiu On Thu, Feb 1, 2024, at 17:11, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev wrote: > Hi, > > We’d like to use gdal_rasterize to burn in a value that is a simple numerical > function of the

Re: [gdal-dev] gdal_translate shifted values

2024-02-01 Thread Laurențiu Nicola via gdal-dev
the GRIB dataset with QGIS (/GDAL) on top of OSM shows a plausible > georeferencing. > > Even > > Le 01/02/2024 à 13:57, Laurențiu Nicola via gdal-dev a écrit : >> It's actually pretty easy to test: >> >> import numpy as np >> import xarray as xr >

Re: [gdal-dev] gdal_translate shifted values

2024-02-01 Thread Laurențiu Nicola via gdal-dev
tried to convert to NetCDF and open them in QGIS, but they look almost the same (not identical) to me after fixing up the coordinates, so there's probably more at play. Again, comparing the gdal_translate output with the GRIB data as read by GDAL can't show any issues. Laurentiu On

Re: [gdal-dev] gdal_translate shifted values

2024-02-01 Thread Laurențiu Nicola via gdal-dev
Hi Jukka, If GDAL shuffled around the pixels in that file, you wouldn't be able to notice it by comparing the GRIB and the TIFF, because both would be shuffled. My suggestion was to convert the GRIB to a NC or even CSV using xarray, then comparing the GDAL output against that. Laurentiu On Th

Re: [gdal-dev] Using VSIMEM properly

2024-01-17 Thread Laurențiu Nicola via gdal-dev
A pipe might prevent you from creating a spatial index, right? I didn't test it, but another method would be to use a join (you can query multiple data sources in the same ogr2ogr invocation, https://gdal.org/user/ogr_sql_dialect.html#joins), but they are pretty limited and you might lose the s

Re: [gdal-dev] Using VSIMEM properly

2024-01-17 Thread Laurențiu Nicola via gdal-dev
07872968?pid=nativeplacement&c=Global_Acquisition_YMktg_315_Internal_EmailSignature&af_sub1=Acquisition&af_sub2=Global_YMktg&af_sub3=&af_sub4=10604&af_sub5=EmailSignature__Static_> > >> On Wed, Jan 17, 2024 at 11:59 AM, Laurențiu Nicola via gdal-dev >> wrote: >

Re: [gdal-dev] Using VSIMEM properly

2024-01-17 Thread Laurențiu Nicola via gdal-dev
Hi Robert, The contents of /vsimem/ is only stored in memory and goes away when your application quits. You should use a real file instead, or something like a Python script that calls gdal.VectorTranslate. Laurentiu On Wed, Jan 17, 2024, at 12:52, nagyrobi_r--- via gdal-dev wrote: > > Hi all

Re: [gdal-dev] Any difference between VRT SimpleSource and no-op ComplexSource?

2023-12-21 Thread Laurențiu Nicola via gdal-dev
On Thu, Dec 21, 2023, at 15:51, Even Rouault wrote: > It only uses a ComplexSource when it needs to, that is to say when there is > nodata or a mask band. > I didn't realize this. I think I'm hitting a corner case with -separate, which is practically the only way I've ever used it: $ gdalbuildv

[gdal-dev] Any difference between VRT SimpleSource and no-op ComplexSource?

2023-12-21 Thread Laurențiu Nicola via gdal-dev
Hello, I noticed that gdalbuildvrt with no options produces VRTs that use ComplexSource, when SimpleSource would be sufficient. Is there any meaningful performance difference between the two? I know I should measure myself, I just didn't get to try it yet. Thanks, Laurentiu

Re: [gdal-dev] Under what (if any) conditions would GDALWriteBlock modify data buffer?

2023-12-17 Thread Laurențiu Nicola via gdal-dev
I don't think it does, but some parts (the older ones) of the public GDAL API don't use const. I admit I didn't check the implementation this time though. Laurentiu On Mon, Dec 18, 2023, at 00:09, Fitch, Simeon via gdal-dev wrote: > The last argument to function `CPLErr GDALWriteBlock(GDALRaster

Re: [gdal-dev] Is there a way to load a custom driver without a declaration in gdal_frmts.h?

2023-12-11 Thread Laurențiu Nicola via gdal-dev
Hi Jesse, https://gdal.org/api/gdaldriver_cpp.html#_CPPv4N17GDALDriverManager15AutoLoadDriversEv should answer your question. Laurentiu On Mon, Dec 11, 2023, at 18:11, Meyer, Jesse R. (GSFC-618.0)[SCIENCE SYSTEMS AND APPLICATIONS INC] via gdal-dev wrote: > Hi, > > I know drivers can be built

Re: [gdal-dev] GTiff bit shuffle compression feature request

2023-12-08 Thread Laurențiu Nicola via gdal-dev
On Fri, Dec 8, 2023, at 21:32, Even Rouault wrote: > yes, poor wording of mine. I meant that if using PREDICTOR=3, one should > compare with FILTER=DELTA. But looking more closely, they are not strictly > equivalent. PREDICTOR=3 applies the delta as b[0]-a[0], b[1]-a[1], b[2]-a[2], > b[3]-a[3] w

Re: [gdal-dev] GTiff bit shuffle compression feature request

2023-12-08 Thread Laurențiu Nicola via gdal-dev
On Fri, Dec 8, 2023, at 21:11, Even Rouault wrote: > >> >> - GeoTIFF DEFLATE 280 MB > Is it with PREDICTOR=3 ? (If not, then the ZARR datasets should also be > compressed with FILTER=DELTA to have a fair comparison) I would expect a fair comparison to be between PREDICTOR=3 and FILTER=DELTA

Re: [gdal-dev] GTiff bit shuffle compression feature request

2023-12-08 Thread Laurențiu Nicola via gdal-dev
Hi, In my (limited) experience, it can really make a difference for floating-point rasters. Testing with a small one I have on hand (10980x10980x1, Float32), I get: - GeoTIFF DEFLATE 280 MB - Zarr BLOSC zlib NONE 281 MB - Zarr BLOSC zlib BIT 253 MB - Zarr BLOSC zlib BYTE 249 MB Laurentiu

Re: [gdal-dev] Requiring numpy for the Python bindings

2023-12-06 Thread Laurențiu Nicola via gdal-dev
On Wed, Dec 6, 2023, at 16:23, Greg Troxel via gdal-dev wrote: > Keeping rust going has been an ongoing source of work and problems. > This is partly because of not having a reasonable bootstrap story from > C/C++, and mostly because the singleton compiler (rust is technically a > langauge but in p

Re: [gdal-dev] Requiring numpy for the Python bindings

2023-12-05 Thread Laurențiu Nicola via gdal-dev
Hi Greg, I'd like to better understand your concerns, so I have a couple of questions and remarks: 1. do the platforms you care about package Firefox, librsvg, or any other popular software that's using Rust? 2. do you have any reasons to believe that numpy will require Rust in the future? I s

Re: [gdal-dev] GDAL PDF Export: Support for different page sizes and multi-paged PDF

2023-11-14 Thread Laurențiu Nicola via gdal-dev
On Tue, Nov 14, 2023, at 19:56, Mandeep Singh wrote: > Thanks, i tried the same but for some reason the vector data was not > displayed in the pdf i guess due to georeferenced control points > > 1) How do i set the control points based on page size and dpi, eg if i have > to use EPSG 3857 syst

Re: [gdal-dev] GDAL PDF Export: Support for different page sizes and multi-paged PDF

2023-11-14 Thread Laurențiu Nicola via gdal-dev
Hi Mandeep, I've never used it, but perhaps you should try https://gdal.org/drivers/raster/pdf.html#creation-of-pdf-file-from-a-xml-composition-file-gdal-3-0 in combination with gdal_create? Laurențiu On Mon, Nov 13, 2023, at 14:02, Mandeep Singh via gdal-dev wrote: > Hi, > > I am currently u

Re: [gdal-dev] Fwd: oblique cuts on a raster using python GDAL

2023-11-09 Thread Laurențiu Nicola via gdal-dev
Yes, that's lossless. I was just offering a potential explanation for the issue you had with JPEG2000. Laurentiu On Thu, Nov 9, 2023, at 14:40, Naima Dambrine wrote: > Hi Laurentiu, > > Thank you, I think I'm not concerned because I use GTiff format as output, > then I specify a deflate compre

Re: [gdal-dev] Fwd: oblique cuts on a raster using python GDAL

2023-11-08 Thread Laurențiu Nicola via gdal-dev
Hi Naïma, I'm a bit late to the party, but note that the JP2OpenJPEG produces lossy output by default. You need to pass REVERSIBLE=YES and QUALITY=100 as creation options to make it use lossless compression. Laurentiu ___ gdal-dev mailing list gdal-de

Re: [gdal-dev] GDAL 3.8.0beta1 available for testing

2023-10-31 Thread Laurențiu Nicola via gdal-dev
Hi, I know it doesn't mean much (with them being unofficial and pretty incomplete), but the Rust bindings appear to work well with both 3.7.3 rc1 and 3.8.0 beta1. Laurentiu On Tue, Oct 31, 2023, at 16:33, Even Rouault via gdal-dev wrote: > Hi, > > I've prepared a beta1 of GDAL 3.8.0 to get feed

Re: [gdal-dev] Performance regression testing/benchmarking for CI

2023-10-10 Thread Laurențiu Nicola via gdal-dev
Hi, No experience with pytest-benchmark, but I maintain an unrelated project that runs some benchmarks on CI, and here are some things worth mentioning: - we store the results as a newline-delimited JSON file in a different GitHub repository (https://raw.githubusercontent.com/rust-analyzer/me

Re: [gdal-dev] Using ogr2ogr with limited memory

2023-09-28 Thread Laurențiu Nicola via gdal-dev
Hello, You might want to say which drivers you're using. I can imagine PostGIS with PG_USE_COPY using more memory with a larger batch size (ogr2ogr -gt), for example. But most likely you're not going to be able to do anything about it. So how much memory are we talking about, and which formats

Re: [gdal-dev] GeoTIFF and concurrent block reads

2023-09-21 Thread Laurențiu Nicola via gdal-dev
On Thu, Sep 21, 2023, at 18:05, Even Rouault wrote: > >> Of course, but I'm asking if it's worth calling ReadBlock on multiple >> threads (if it always takes a lock, it's not, and I should use RasterIO >> instead). > Not on the same dataset object, otherwise you'll get crashes as no lock is >

Re: [gdal-dev] GeoTIFF and concurrent block reads

2023-09-21 Thread Laurențiu Nicola via gdal-dev
Hi, On Thu, Sep 21, 2023, at 17:02, Even Rouault wrote: > Laurentiu, > >> >> GDAL 3.6 added support for multi-threaded reading using PRead, but I >> couldn't spot ReadBlock using the same code path. > If you read one single block at a time, the multi-threaded optimization > cannot kick in, sin

[gdal-dev] GeoTIFF and concurrent block reads

2023-09-21 Thread Laurențiu Nicola via gdal-dev
Hello, GDAL 3.6 added support for multi-threaded reading using PRead, but I couldn't spot ReadBlock using the same code path. On the other hand, I didn't see it taking a lock either, so I'm a little confused :-). My question is whether ReadBlock supports concurrent reads or not (*). (*) I thin

Re: [gdal-dev] Call for discussion/review of RFC95: Use standard C/C++ integer types

2023-09-15 Thread Laurențiu Nicola via gdal-dev
On Fri, Sep 15, 2023, at 19:21, Kurt Schwehr wrote: > - Q3: Yes to vsi_l_offset -> uint64_t. I'm 90% for it. For me: Yes offset has > meaning, but if the type is critical to understanding a function / method > interface, we should have a comment. Knowing that it's an unsigned 64 bit int > is usu

Re: [gdal-dev] A question of eficiency

2023-07-31 Thread Laurențiu Nicola via gdal-dev
Hi, GetFeatureCount() exists for GDAL users, not for driver writers. For example, in QGIS, you can right-click on a layer, choose "Show Feature Count" and you'll see the number of features next to the layer name. Or, to give another example, a hypothetical application could use it to display a

Re: [gdal-dev] Instructions for a new writing driver?

2023-07-06 Thread Laurențiu Nicola via gdal-dev
PS: Geoconcept might be a better example than AAIGrid, since it's a vector driver. On Thu, Jul 6, 2023, at 16:08, Laurențiu Nicola via gdal-dev wrote: > Hi, > > I think some existing drivers already contain some C files, AIGRID for > example. Others have a sdk subdirectory wit

Re: [gdal-dev] Instructions for a new writing driver?

2023-07-06 Thread Laurențiu Nicola via gdal-dev
Hi, I think some existing drivers already contain some C files, AIGRID for example. Others have a sdk subdirectory with library code in there. I don't think including your code as C will be a problem. On the other hand, offering a C API for new drivers might be an interesting idea. Using C++ m

Re: [gdal-dev] Instructions for a new writing driver?

2023-07-05 Thread Laurențiu Nicola via gdal-dev
Hi, I think your best bet would be to take inspiration from an existing driver, like the CSV one (which shouldn't be too different from the hypothetical SPF driver in the docs). Laurentiu On Wed, Jul 5, 2023, at 14:17, Abel Pau wrote: > Hi, following instructions about creating a new driver I

Re: [gdal-dev] Java API: how to find dimensions of MDArray

2023-06-05 Thread Laurențiu Nicola via gdal-dev
Hi Barry, I haven't used Java in a while, but those are type signatures as used in JNI. If I recall correctly, (DLjava/lang/Long;Ljava/lang/Boolean;)V is void f(double p1, double p2, double p3, double p4, double p5, long p6, boolean p6). They're used to disambiguate overloaded methods. Lau

Re: [gdal-dev] Java API: how to find dimensions of MDArray

2023-06-03 Thread Laurențiu Nicola via gdal-dev
Hi, On Sat, Jun 3, 2023, at 19:03, Even Rouault wrote: > Yes the multidim API is probably currently hardly usable outside C, C++ and > Python due to the lack of a few critical typemaps for the other languages. > These are unofficial and much less mature, but the Rust bindings have some basic m

Re: [gdal-dev] Want to work with MDArrays

2023-06-02 Thread Laurențiu Nicola via gdal-dev
Hi Barry, I think you want GDAL_OF_MULTIDIM_RASTER. There's an example here: https://gdal.org/tutorials/multidimensional_api_tut.html. Laurentiu On Fri, Jun 2, 2023, at 19:25, Barry DeZonia wrote: > Hello, > > I am a software developer that has written a numerical analysis software > package

[gdal-dev] Can block offsets ever be negative?

2023-06-01 Thread Laurențiu Nicola via gdal-dev
Hello, Just a quick question: can block offsets (such as those in in GDALReadBlock or IRasterIO) ever be negative? Is there any format that supports that? GDAL uses signed integers everywhere, which is fine, but the intention isn't always clear at times. Thanks, Laurentiu Nicola___

Re: [gdal-dev] How to decrease file size of colored raster bands?

2023-06-01 Thread Laurențiu Nicola via gdal-dev
Hi, Try saving the output into a new file. Those are "creation options". Laurentiu On Thu, Jun 1, 2023, at 20:34, afernandez wrote: > Hello Laurentiu, > The color palette is working fine. The issues seem to be caused by using > UInt16 because I changed to GDT_Byte per your suggestion and everyt

Re: [gdal-dev] How to decrease file size of colored raster bands?

2023-06-01 Thread Laurențiu Nicola via gdal-dev
Hi Arturo, On Thu, Jun 1, 2023, at 20:41, afernandez wrote: > Hi Scott, > As I mentioned to Laurentiu, I'm still not getting the handle on how to use > overviews but will keep working at it. You don't need to do anything except create them. QGIS will use them automatically, for example (make s

Re: [gdal-dev] How to decrease file size of colored raster bands?

2023-06-01 Thread Laurențiu Nicola via gdal-dev
Hi, There are a couple of things you can try: • you seem to be scaling the data to 8-bit, but saving it as UInt16; GDT_Byte should work, yielding a 50% reduction. I didn't quite understand the comment about the color, does the palette not work with 8-bit images? • you can enable and tune compr

Re: [gdal-dev] ogr.VectorTranslate to json appears to be incomplete

2023-05-26 Thread Laurențiu Nicola via gdal-dev
On Fri, May 26, 2023, at 20:48, Even Rouault wrote: > yes, the issue must come from the fact that gdal.VectorTranslate() > returns a gdal.Dataset handle that is still active, and you could That's what I originally tried and it didn't work, but it does now. Maybe I made a typo. Seems all right,

Re: [gdal-dev] ogr.VectorTranslate to json appears to be incomplete

2023-05-26 Thread Laurențiu Nicola via gdal-dev
It's a bit strange, I can reproduce it when pasting into the REPL, but not when running from a file. Laurentiu On Fri, May 26, 2023, at 20:27, Michael Smith wrote: > Well, a new install of gdal from conda-forge and now it works fine. > Sorry for the noise. > > Mike > > > -- > > Michael Smith

Re: [gdal-dev] Space optimization of OSM XYZ tiles

2023-05-26 Thread Laurențiu Nicola via gdal-dev
Hi Eric, Yes, you can transcode them on the fly with a server like https://github.com/developmentseed/titiler. I haven't used that one, but wrote a prototype of something like that in the past. You might want to have the original in EPSG:3857, though, to avoid the need to reproject the tiles w

Re: [gdal-dev] How to read tiles size of a tiff

2023-05-17 Thread Laurențiu Nicola via gdal-dev
Hi Andy, gdalinfo is the tool that displays information about rasters. Yours is 337x217 pixels, with a 337x24 block size, which is completely unrelated to the OL tiling grid, and there's no -256 here. The only unusual thing about your image is that instead of the customary positive X, negative

Re: [gdal-dev] Question About OGR

2023-05-10 Thread Laurențiu Nicola via gdal-dev
Hi, You no longer have the GDAL tools installed. I don't know how you acquired them before, but one option is something like !apt install gdal-bin (untested). Laurențiu On Thu, May 11, 2023, at 09:03, Arzu ÇUBUKÇU wrote: > Hello All, > > I have been working on file conversion in Python for a

Re: [gdal-dev] Space Disk/Memory of Image

2023-05-09 Thread Laurențiu Nicola via gdal-dev
Hello, The GDAL API actually has a couple of methods that return the size of a file (not dataset) on disk: VSIStatL , VSIStatExL . Since th

Re: [gdal-dev] Java-GDAL binding:How to convert gdal dataset to byte array

2023-04-13 Thread Laurențiu Nicola via gdal-dev
Hi, VSIUnlink is https://gdal.org/java/org/gdal/gdal/gdal.html#Unlink(java.lang.String), but I can't find VSIGetMemFileBuffer or VSIFOpenEx2L and VSIFReadL, which you could use instead. FileFromMemBuffer is there, but it goes in the wrong direction. So sorry, I have no idea. I've never used th

Re: [gdal-dev] Java-GDAL binding:How to convert gdal dataset to byte array

2023-04-13 Thread Laurențiu Nicola via gdal-dev
Hi, I don't think ReadAsArray is the API you're looking for. If you do that, you'll end up with a buffer of pixel values, not with a proper TIFF or PNG tile. Instead, you can either use the MEM driver or the /vsimem/ prefix (like in https://gdal.org/api/cpl.html#_CPPv424VSIInstallMemFileHandler

Re: [gdal-dev] VRT file: Recursion detected

2023-04-13 Thread Laurențiu Nicola via gdal-dev
Hi, Just a guess, but maybe your VRT is including itself? Laurentiu On Thu, Apr 13, 2023, at 11:45, Ari Jolma wrote: > What can cause "Recursion detected" error when opening a VRT-file? > > Is there something I can do in gdal.BuildVRT (I'm using gdal python > bindings) to avoid that? > > Ari >

Re: [gdal-dev] Terrain Analysis

2023-02-21 Thread Laurențiu Nicola via gdal-dev
Hi, For the first three you can look into gdaldem (https://linux.die.net/man/1/gdaldem). Regards, Laurentiu On Tue, Feb 21, 2023, at 18:38, Roa Carvajal, Camila via gdal-dev wrote: > Hi, > > Recently I used gdaldem to compute slope and aspect from a Digital Elevation > Model. In the past I'v

Re: [gdal-dev] Error when trying to build Docker Images

2023-02-15 Thread Laurențiu Nicola via gdal-dev
Hi, I think libmysqlclient-dev is called default-libmysqlclient-dev on Debian. Unfortunately, you might have to experiment a little with the package names and repository lists. Laurentiu On Wed, Feb 15, 2023, at 12:51, Dirk Stenger wrote: > Thanks for all the help. Now, the build of ubuntu-ful

Re: [gdal-dev] Error when trying to build Docker Images

2023-02-14 Thread Laurențiu Nicola via gdal-dev
Hi, Is that an arm64 system? Laurentiu On Tue, Feb 14, 2023, at 20:00, Dirk Stenger wrote: > All, > > Unfortunately, when trying to build the GDAL Docker Image on a local machine > (Ubuntu 22.04), the build fails with an error: > > Executed commands and logging: > > ~/git/gdal/docker/ubuntu-

Re: [gdal-dev] GDAL Overestimating Physical Memory

2023-01-26 Thread Laurențiu Nicola via gdal-dev
On Thu, Jan 26, 2023, at 16:52, Even Rouault wrote: > Laurențiu, > > are you 100% positive you've tested the updated version of the pull request? > I've just given a try to running gdallimits under Docker from a Ubuntu 22.04 > host and it successfully takes into account the /sys/fs/cgroup/memo

Re: [gdal-dev] GDAL Overestimating Physical Memory

2023-01-25 Thread Laurențiu Nicola via gdal-dev
Hello, I also managed to reproduce this in Docker (4 GB limit): # cat /proc/self/cgroup 0::/ # cat /sys/fs/cgroup/memory.max 4294967296 # autotest/cpp/gdallimits CPLGetNumCPUs = 32 CPLGetUsablePhysicalRAM = 62 GB (podman behaves exactly the same) Laurentiu On Thu, Jan 26, 2023, at 03:13, Ang

Re: [gdal-dev] Any trick to run SELECT gdal_get_pixel_value for lots of points?

2023-01-19 Thread Laurențiu Nicola via gdal-dev
Hi, I think I've mentioned it on the mailing list before, but https://www.orfeo-toolbox.org/CookBook/Applications/app_SampleExtraction.html has a pretty good (threaded) implementation of this. Of course, it's an extra piece of software to install. If you want to do it in pure SQL, you could al

Re: [gdal-dev] Interaction between no data and resampling

2023-01-16 Thread Laurențiu Nicola via gdal-dev
Laurențiu Nicola via gdal-dev wrote: > PS: > > On Mon, Jan 16, 2023, at 13:58, Laurențiu Nicola wrote: >> >> * the `-srcnodata` docs and `gdalwarpkernel.cpp` suggest that masked pixels >> are not included in the computation (unlike those of no data?) > The remark in

Re: [gdal-dev] Interaction between no data and resampling

2023-01-16 Thread Laurențiu Nicola via gdal-dev
PS: On Mon, Jan 16, 2023, at 13:58, Laurențiu Nicola wrote: > > * the `-srcnodata` docs and `gdalwarpkernel.cpp` suggest that masked pixels > are not included in the computation (unlike those of no data?) The remark in the brackets is wrong, of course, since the docs are specifically about `-s

[gdal-dev] Interaction between no data and resampling

2023-01-16 Thread Laurențiu Nicola via gdal-dev
Hello, I've been looking over the docs and the code, but I'm still a little confused about how masking and/or no data are handled during resampling in GDAL. I'm interested mostly in the `cubic` and `average` resamples, but I assume the answers will apply to the other modes just as well. In my

Re: [gdal-dev] SQL query to set column to GPKG raster pixel value of point intersection result

2022-12-08 Thread Laurențiu Nicola via gdal-dev
https://www.orfeo-toolbox.org/ has an app for something like this (SampleExtraction?), but I think it writes the values into a new file and last time I tried, it think it only worked with shapefiles did some reason. Laurentiu On Thu, Dec 8, 2022, at 20:07, Meyer, Jesse R. (GSFC-618.0)[SCIENCE S

Re: [gdal-dev] Tips on Speeding up OGR2OGR GeoPackage Creation

2022-10-21 Thread Laurențiu Nicola via gdal-dev
It is single-threaded (AFAIK), but as you said, loading the data doesn't take too long, so you won't win more than that by pipelining it. And SQLite doesn't support concurrent writes, not even in WAL mode, so there's not much use for threads there. I would have expected OGR_SQLITE_SYNCHRONOUS=N

Re: [gdal-dev] gdal_calc output to multiband nedcdf ?

2022-10-20 Thread Laurențiu Nicola via gdal-dev
Hello, I've never tried it, but I believe you can describe some computations in a VRT. But gdal_calc can't do this, so you'd need to write the VRT directly as XML. That's probably not much easier than writing some Python or C++ to perform the same computation. Laurentiu On Thu, Oct 20, 2022,

Re: [gdal-dev] Tips on Speeding up OGR2OGR GeoPackage Creation

2022-10-20 Thread Laurențiu Nicola via gdal-dev
Hi, https://gdal.org/drivers/vector/gpkg.html#configuration-options has a couple of options that can help with performance. See also https://gdal.org/drivers/vector/sqlite.html#target-drivers-vector-sqlite-performance-hints. But I don't recall them helping that much. YMMV. Regards, Laurentiu

Re: [gdal-dev] [EXT] Re: [Doc] gdal_calc.py extent option

2022-10-18 Thread Laurențiu Nicola via gdal-dev
Hi Matt, The definitions are correct. For the union you want a rectangle (or polygon) that's larger than the input, indeed. But you want the smallest of those rectangles, because there's an infinity of them. The larger ones would cover an arbitrary area of the plane (or the globe, if you want).

Re: [gdal-dev] Problem with coordinate transformation

2022-10-18 Thread Laurențiu Nicola via gdal-dev
Lelong wrote: > Thanks very much, I would never have figured that out myself. > > > > Just to make it clear I should add this flag on all OGRSpatialReference or > only when it is loaded with importFromWkt ? > > > > --Philippe > > > > > *From:* g

Re: [gdal-dev] Problem with coordinate transformation

2022-10-18 Thread Laurențiu Nicola via gdal-dev
Hi Philippe, The new behaviour is correct. See https://gdal.org/development/rfc/rfc73_proj6_wkt2_srsbarn.html#axis-order-issues for the axis order issue, a change made in GDAL 3.0. You can find a solution in https://github.com/OSGeo/gdal/blob/v3.5.0/MIGRATION_GUIDE.TXT#L67 (calling SetAxisMapp

  1   2   >