Perhaps another hapless user question: Using ogr2ogr, I can write very small parquet files to /vsiaz/<container>/<file>.parquet without issue. But with larger files (that ogr2ogr tries to write in chunks), Azure does not accept the first PUT request and ogr2ogr fails with:
ERROR 1: PUT of /vsiaz/<container>/<file>.parquet failed ERROR 1: WriteColumnChunk() failed for <column>: Error while writing ERROR 1: FileWriter::Close() failed with Only 28 out of 84 columns are initialized ARROW: Memory pool (writer layer): bytes_allocated = 0 ARROW: Memory pool (writer layer): max_memory = 56286976 GDAL: GDALClose(/vsiaz/<container>/<file>.parquet, this=0x5910350ebb10) GDAL: In GDALDestroy - unloading GDAL shared library. Error: Process completed with exit code 1. with --config CPL_CURL_VERBOSE=YES, this looks to be the issue: CURL_INFO_HEADER_OUT: PUT /<container>/<file>.parquet?<token> User-Agent: GDAL/3.10.2 Accept: */* Content-Length: 0 x-ms-blob-type: AppendBlob x-ms-date: Thu, 03 Apr 2025 23:02:46 GMT CURL_INFO_TEXT: Request completely sent off CURL_INFO_HEADER_IN: HTTP/1.1 409 The blob type is invalid for this operation. CURL_INFO_HEADER_IN: Content-Length: 228 CURL_INFO_HEADER_IN: Content-Type: application/xml CURL_INFO_HEADER_IN: Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0 CURL_INFO_HEADER_IN: x-ms-request-id: c24de328-b01e-003a-1eec-a49ba5000000 CURL_INFO_HEADER_IN: x-ms-version: 2022-11-02 CURL_INFO_HEADER_IN: x-ms-error-code: InvalidBlobType CURL_INFO_HEADER_IN: Date: Thu, 03 Apr 2025 23:02:45 GMT - no issue with ogr2ogr write of same data as flat geobuf to /vsiaz/ (and resulting blob type is AppendBlob) - no issue with `az copy` write of the same parquet to azure blob - no issue with ogr2ogr write of the same parquet to /vsis3/ - same issue on macos/gdal 10.2 and ghcr.io/osgeo/gdal:ubuntu-full-3.10.0 I'd file an issue but if it was a bug I'd think it would have already have been found by other users? thanks Simon _______________________________________________ gdal-dev mailing list gdal-dev@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/gdal-dev
