Yes apologies John for hijacking the thread, I thought it could be related to your issue but looking less and less likely. Steve's suggestion to use the commandline mapserv -nh "QUERY..." approach should rule out a few more issues.
Even - the same zip cannot be opened on Linux either (using Ark and unzip): "9487 extra bytes at begging or within zipfile". The GDAL within the build (3.3.3) creates a zip of the WFS just fine for the same layer/WFS service using ogr2ogr: ogr2ogr -f "ESRI Shapefile" "H:/Temp/test.shp.zip" WFS:"https://server/mapserver/?service=WFS&REQUEST=GetFeature&version=2.0.0 <https://plmonaghandev.compass.ie/mapserver/?service=WFS&REQUEST=GetFeature&version=2.0.0>" LayerName -overwrite -skipfailures GDAL 2.4.4 doesn't create zips so it must be MapServer doing the zipping in this version at least? I'm guessing this points to a MapServer issue, so I'll carry on debugging. Seth -- web:http://geographika.co.uk twitter: @geographika On Fri, Nov 19, 2021, at 4:11 PM, Even Rouault wrote: > Ah, GDAL also changed... Zipping is done by GDAL, so could be a GDAL related > change (there were adjustments in the past regarding ZIP64 that could result > in some bytes being different). Did you try opening the zip with another > utility (like 'unzip' on Linux) ? > > This might be then a completely different issue than the one reported by John. > > Le 19/11/2021 à 16:01, Seth G a écrit : >> Thanks Even, I'll give that a go (the bisect worked well for another issue a >> few months ago). >> >> I presume the zipping issue is more likely to be in the MapServer codebase >> than in GDAL? >> The GISInternals did switch from GDAL 2.4.4 to GDAL 3.3.3 from the 7.4.3 to >> 7.6.4 branches. >> >> The zip exports work fine over a web browser rather than command line which >> is strange. Maybe its related to modifications to the -nh switch. >> >> Seth >> >> -- >> web:http://geographika.co.uk >> twitter: @geographika >> >> >> On Fri, Nov 19, 2021, at 2:07 PM, Even Rouault wrote: >>> Could be a msIO_needBinaryStdout() missing somewhere. Seth, if you've the >>> chance to run a git bisect session, that could probably give a strong hint >>> of how to fix that. >>> >>> Le 19/11/2021 à 09:43, Seth G a écrit : >>>> Possibly unrelated but I ran into a similar issue exporting WFS to zipped >>>> shapefiles. >>>> Working fine in 7-4-3 but broken in 7-6-4 (and current master) - the zip >>>> files are corrupt, although they have an identical size. >>>> >>>> 7-zip reports "Headers Error Unconfirmed start of archive" >>>> >>>> The same command is used for both versions: >>>> >>>> mapserv -nh >>>> "QUERY_STRING=map=my.map&service=WFS&REQUEST=GetFeature&TYPENAME=LayerName&version=2.0.0&outputformat=shapezip&srsName=EPSG:3857" >>>> > output.zip >>>> >>>> I had a look with a hex editor but the start of the working and corrupt >>>> zips seem identical. >>>> >>>> On the other hand all PNGs / WMS services are fine. >>>> >>>> Seth >>>> >>>> >>>> -- >>>> web:http://geographika.co.uk >>>> twitter: @geographika >>>> >>>> >>>> On Thu, Nov 18, 2021, at 3:53 PM, Steve Lime wrote: >>>>> Hmmm... I've not run into or heard of this before although I'm not a >>>>> windows user. I did a quick sanity check with a mapfile here and the >>>>> latest 7.4 and 7.6 versions. While not exactly the same setup you have in >>>>> terms of versions, they produce the exact same png image. >>>>> >>>>> What do you get for output if you use mapserv.exe at the command line? So >>>>> something like: >>>>> >>>>> mapserv.exe -nh "QUERY_STRING= >>>>> map=dynamic\5708d96b-c606-4c35-95e7-085fedc1dcce.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&FORMAT=image%2Fpng&TRANSPARENT=true&LAYERS=MAP&TILED=true&WIDTH=512&HEIGHT=512&CRS=EPSG%3A3857&STYLES=&BBOX=-10877294.873093722%2C5536486.832751887%2C-10876071.880641159%2C5537709.82520445" >>>>> > test.png >>>>> >>>>> That would take PostMan and the web server out of the picture. Is there >>>>> any chance different versions of libpng are being used? What are you >>>>> using to manage the tiles? >>>>> >>>>> --Steve >>>>> >>>>> On Wed, Nov 17, 2021 at 4:57 PM John Huotari via MapServer-users >>>>> <[email protected]> wrote: >>>>>> I’m attempting to upgrade from MapServer 7.6.1 to 7.6.4 via compiled >>>>>> packages obtained from GISInternals. I’m running it on Windows/IIS and >>>>>> whereas 7.6.1 was generating .PNG tiles perfectly for me, after >>>>>> upgrading to 7.6.4, the .PNGs being created appear to be corrupt. I can >>>>>> replace the 7.6.4 exe and dlls with 7.6.1 versions and the PNG images >>>>>> generate fine again, so while there are quite a few places that could >>>>>> introduce an issue, with the exception of a change to MapServer >>>>>> everything would be identical in my stack between having the issue in >>>>>> 7.6.4 and not in 7.6.1. >>>>>> >>>>>> The good headers from 7.6.1 look like this >>>>>> >>>>>> 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 >>>>>> >>>>>> and the corrupted ones from 7.6.4 look like this >>>>>> >>>>>> 89 50 4e 47 0d 0d 0a 1a 0d 0a 00 00 00 0d 49 48 44 52 >>>>>> >>>>>> It appears that the 0a values from the valid header are being converted >>>>>> to 0d 0a. I might be wrong here, but it appears to me that something is >>>>>> interpreting the 0a as a line feed and given the code is running on >>>>>> Windows, is converting that LF into a CR LF. The replacement doesn’t >>>>>> seem to be limited to the file header as I see the 7.6.4 version of the >>>>>> file is slightly larger (18,571 bytes instead of 18,407 bytes) and in >>>>>> spot checking, I’ve verified some additional 0d’s exist precede 0a >>>>>> within the data blocks of the .PNG. Has anyone experienced anything >>>>>> like this or know of any fixes? >>>>>> >>>>>> PNG Images produced on the server with shp2img are just fine, it’s only >>>>>> images produced by making a WMS request to mapserv.exe that have the >>>>>> issue. An example WMS request would be >>>>>> >>>>>> https://<Server Name >>>>>> Removed>/mapserv.exe?map=dynamic\5708d96b-c606-4c35-95e7-085fedc1dcce.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&FORMAT=image%2Fpng&TRANSPARENT=true&LAYERS=MAP&TILED=true&WIDTH=512&HEIGHT=512&CRS=EPSG%3A3857&STYLES=&BBOX=-10877294.873093722%2C5536486.832751887%2C-10876071.880641159%2C5537709.82520445 >>>>>> >>>>>> Maybe I’m completely misdiagnosing the problem as these PNG image files >>>>>> just show up as corrupted within a browser – for example FireFox reports >>>>>> “The image <full URL here> cannot be displayed because it contains >>>>>> errors.” The way I obtained the actual .PNG images to view in a binary >>>>>> editor was to use PostMan and save the body of the results. Perhaps >>>>>> PostMan introduced the extra bytes when saving an unrecognizable format >>>>>> file to disk whereas it did not when saving a file it recognized as a >>>>>> valid PNG. I can’t find anything different between the valid and >>>>>> invalid files beyond the extra 0d’s that have been added though, so I >>>>>> don’t think PostMan or anything else in the chain introduced them. >>>>>> >>>>>> ******************* PLEASE NOTE ******************* >>>>>> This message, along with any attachments, is for the designated >>>>>> recipient(s) only and may contain privileged, proprietary, or otherwise >>>>>> confidential information. If this message has reached you in error, >>>>>> kindly destroy it without review and notify the sender immediately. Any >>>>>> other use of such misdirected e-mail by you is prohibited. Where allowed >>>>>> by local law, electronic communications with Zurich and its affiliates, >>>>>> including e-mail and instant messaging (including content), may be >>>>>> scanned for the purposes of information security and assessment of >>>>>> internal compliance with company policy. >>>>>> _______________________________________________ >>>>>> MapServer-users mailing list >>>>>> [email protected] >>>>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users >>>>> _______________________________________________ >>>>> MapServer-users mailing list >>>>> [email protected] >>>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users >>>>> >>>> >>>> >>>> _______________________________________________ >>>> MapServer-users mailing list >>>> [email protected] >>>> https://lists.osgeo.org/mailman/listinfo/mapserver-users >>>> >>>> >>> -- >>> http://www.spatialys.com >>> My software is free, but my time generally not. >>> >>> _______________________________________________ >>> MapServer-users mailing list >>> [email protected] >>> https://lists.osgeo.org/mailman/listinfo/mapserver-users >>> >> > -- > http://www.spatialys.com > My software is free, but my time generally not.
_______________________________________________ MapServer-users mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/mapserver-users
