Yuvraj,
Having the spec publicly available seems to be a pre-condition before
submitting a pull request implementing it, otherwise this will make
review close to impossible.
Regarding a separate EOPF driver vs enhancing the Zarr one, I don't have
enough elements to make a judgement. There are other options:
- the EOPF driver uses internally the Zarr one. It seems to be the most
attractive option assuming that EOPF Zarr is valid Zarr (which I hope it is)
- if the above hypothesis isn't completely true, the Zarr driver code is
restructured so that relevant parts can be reused by EOPF Zarr.
In any case, we want to avoid copy&paste of code between the 2 drivers
to ease maintenance.
Even
Le 02/04/2025 à 13:50, Adagale Yuvraj Bhagwan a écrit :
Hi Even,
1. Maturity of the EOPF Enhancements
EOPF (Earth Observation Processing Framework) is still in a relatively
early stage of development. While the core concepts have been tested
in prototype systems, EOPF’s final production readiness is not yet
established. We are currently refining specifications and partial read
logic, with an aim to gradually move toward production use in the
coming months.
2. Location of the Format Specification
The EOPF format specification is outlined in internal documentation
maintained by the EOPF development team (originally used in ESA’s EOPF
Sample Service activities). We plan to release a public-facing version
of this spec once it stabilizes, so that interested developers can
reference it directly.
3. Why Not Enhance the Existing Zarr Driver?
While the EOPF data layout leverages Zarr-like chunking, it carries
additional, domain-specific metadata and conventions—such as
specialized Earth Observation fields, naming conventions, and possibly
integration with SAFE or STAC metadata. These specialized aspects
don’t neatly map onto the existing Zarr driver’s scope.
*
*Technical*: Enhancing the Zarr driver to interpret EOPF’s custom
logic and metadata might introduce significant complexity or break
typical Zarr assumptions.
*
*Maintainability*: A standalone EOPF driver is less fragile, as we
can manage EOPF’s domain rules independently of any future Zarr
driver changes.
*
*Flexibility*: This also allows the EOPF plugin to evolve at a
different pace than the generic Zarr driver.
4. Current Skeleton Status
Yes, the posted code is indeed just a skeleton right now. It
demonstrates driver registration, basic dataset creation, and minimal
chunk-read placeholders. This is not yet production-ready. Our
near-term roadmap includes fleshing out actual reading of Zarr chunks,
robust metadata handling, and multi-band support. The skeleton is a
starting point that will expand into a more complete EOPF driver in
subsequent development stages.
Thanks,
Yuvraj
------------------------------------------------------------------------
*From:* Even Rouault <even.roua...@spatialys.com>
*Sent:* Tuesday, April 1, 2025 9:41 PM
*To:* Adagale Yuvraj Bhagwan <yuvraj.adag...@eurac.edu>;
gdal-dev@lists.osgeo.org <gdal-dev@lists.osgeo.org>
*Subject:* Re: [gdal-dev] Proposal for GDAL Driver: EOPF Zarr (Earth
Observation Product Format)
You don't often get email from even.roua...@spatialys.com. Learn why
this is important <https://aka.ms/LearnAboutSenderIdentification>
Hi Yuvraj,
A few questions:
- what is the maturity of the EOPF enhancements ?
- where are they described ? (i.e. the format specification)
- why not enhancing the existing Zarr driver ?
- from what I can currently see in
https://github.com/EOPF-Sample-Service/GDAL-ZARR-EOPF/tree/main/src
<https://github.com/EOPF-Sample-Service/GDAL-ZARR-EOPF/tree/main/src>,
this seems more to be a skeleton of a driver that something production
ready
Even
Le 01/04/2025 à 18:45, Adagale Yuvraj Bhagwan via gdal-dev a écrit :
Hello GDAL Community,
We’re developing a GDAL driver for the Earth Observation Product
Format (EOPF), a cloud-optimized Zarr-based format tailored for
large-scale EO data.
This driver aims to enable seamless access to EOPF datasets and their
metadata through GDAL, supporting features like chunked I/O, and
compatibility with STAC metadata.
Key features:
- Support for Zarr V2/V3 structures with EOPF-specific enhancements.
- Integration with cloud storage (S3, GCS, etc.).
- Alignment with ESA/Copernicus data standards.
We’d appreciate your feedback on integration requirements and best
practices. The code is available at
EOPF-Sample-Service/GDAL-ZARR-EOPF
<https://github.com/EOPF-Sample-Service/GDAL-ZARR-EOPF>, and we plan
to submit a PR soon.
Best regards,
*Yuvraj Adagale*
*Eurac Research*
*Researcher*
Institute for Earth Observation
*T* +39 344 584 4031
yuvraj.adag...@eurac.edu <mailto:yuvraj.adag...@eurac.edu>
Drususallee/Viale Druso 1
I-39100 Bozen/Bolzano
Legal Seat
Drususallee/Viale Druso 1
I-39100 Bozen/Bolzano
*_www.eurac.edu <http://www.eurac.edu/>_*
*_Facebook <https://facebook.com/eurac.research>_ | _YouTube
<https://www.youtube.com/EURACtv>_ | _X
<https://twitter.com/eurac>_ | _LinkedIn
<https://www.linkedin.com/company/euracresearch>_ | _Instagram
<https://www.instagram.com/euracresearch/>_****| CV*
_signature_1401579056 <https://www.eurac.edu/en>_
According to regulation (EU) 2016/679 this transmission is intended only
for the use of the addressee and may contain confidential information.
If you receive this transmission in error, please notify the sender
immediately
by email and delete all copies of this message and any attachments.
Diese Nachricht ist im Sinne der Verordnung (EU) 2016/679
ausschließlich für
den Adressaten bestimmt und kann vertrauliche Informationen enthalten.
Sollten Sie diese Nachricht irrtümlich erhalten haben, bitten wir
Sie, den
Absender darüber unverzüglich per E-Mail in Kenntnis zu setzen sowie die
Nachricht und etwaige Kopien und Anlagen zu vernichten.
Ai sensi del Regolamento UE 679/2016 questo messaggio è ad uso esclusivo
del destinatario e può contenere informazioni riservate. Qualora Le fosse
pervenuto per errore, Le chiediamo gentilmente di comunicarcelo
immediatamente via e-mail ed eliminare qualsiasi copia e allegato.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org <mailto:gdal-dev@lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
<https://lists.osgeo.org/mailman/listinfo/gdal-dev>
--
http://www.spatialys.com <http://www.spatialys.com/>
My software is free, but my time generally not.
--
http://www.spatialys.com
My software is free, but my time generally not.
_______________________________________________
gdal-dev mailing list
gdal-dev@lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev
