Hi Jukka,
yes it would make sense for the driver to use a larger page size. You
can file an issue about that
Actually we could reuse the same logic as the QGIS OAPIF provider that
determines the page size from the limit.schema.maximum and
limit.schema.default values of the /api response:
if ( apiRequest.defaultLimit() > 0 && apiRequest.maxLimit() > 0 )
{
// Use the default, but if it is below 1000, aim for 1000
// but clamp to the maximum limit
mShared->mPageSize = std::min( std::max( 1000,
apiRequest.defaultLimit() ), apiRequest.maxLimit() );
}
else if ( apiRequest.defaultLimit() > 0 )
mShared->mPageSize = std::max( 1000, apiRequest.defaultLimit() );
else if ( apiRequest.maxLimit() > 0 )
mShared->mPageSize = apiRequest.maxLimit();
else
mShared->mPageSize = 100; // fallback to arbitrary page size
Even
Le 06/10/2023 à 16:39, Rahkonen Jukka via gdal-dev a écrit :
Hi,
The OAPIF driver is using page size of 10 features by default
https://gdal.org/drivers/vector/oapif.html#open-options.
“PAGE_SIZE=<integer>: Defaults to 10. Number of features to retrieve
per request. Minimum is 1, maximum 10000.”
I know that 10 is the default page size (“limit”) also in the standard
but I think that this is too small page size, both for the users and
especially for the service providers. Most users run GDAL utilities
with the defaults. The buildings collection in our OGC API Features
service has about 5 million features, which means that GDAL users are
fetching the data by sending half a million request. That is much
slower for the users than making 500 request, 10000 features each. And
think about our log files.
The better default page size would be the biggest that the OAPIF
server supports. By the standard the maximum is 10000 but service
providers may use bigger or smaller maximum page sizes. By the
standard the server can always return less features than client is
asking with “limit=” and the client must be prepared to follow the
next links.
My suggestion for the new page size default is 10000. If that feels
too big for some reason, then would 1000 feel better?
-Jukka Rahkonen-
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
