Skip to content

feat: add zarr store#89

Draft
cauriol wants to merge 26 commits intomainfrom
zarr_store
Draft

feat: add zarr store#89
cauriol wants to merge 26 commits intomainfrom
zarr_store

Conversation

@cauriol
Copy link
Copy Markdown
Contributor

@cauriol cauriol commented Jan 28, 2026

This PR improves the way Zarr stores are exposed and accessed through the /data endpoint, allowing Zarr data to be accessed in a cloud-native way.

  • Zarr store exposure via HTTP filesystem:
    The endpoint /data/{federation_backend}/{collection_id}/{item_id}/zarr exposes the Zarr store as an HTTP filesystem, follows the standard Zarr + fsspec access pattern. Clients can access the data remotely using common libraries such as zarr, xarray, or dask without downloading the full store.
    Exemple usage:
# pip install requests xarray zarr fsspec requests aiohttp dask
import xarray as xr

ds = xr.open_zarr(
    "https://<api-url>/data/<backend>/<collection>/<item>/{asset_name}.zarr",
    engine="zarr",
    chunks={},  
    decode_times=False,
    storage_options={
        "client_kwargs": {"trust_env": False},
    }
)

print(ds)
  • Human-readable index:
    A dedicated endpoint: /data/{federation_backend}/{collection_id}/{item_id}/zarr/index returns a JSON listing of the files contained in the Zarr store and their paths.

  • File access inside the Zarr store:
    Individual files can be accessed via: /data/{federation_backend}/{collection_id}/{item_id}/zarr/{path_to_file}, where path_to_file can be found in the human-readable index.

@cauriol cauriol self-assigned this Jan 28, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 28, 2026

Test Results

  2 files   -   2    2 suites   - 2   24s ⏱️ -32s
114 tests +  3  113 ✅ +  2  0 💤 ±0  1 ❌ +1 
228 runs   - 216  226 ✅  - 218  0 💤 ±0  2 ❌ +2 

For more details on these failures, see this check.

Results for commit d74acfb. ± Comparison against base commit b73a3b0.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Jan 28, 2026

badge

Code Coverage

Details
Filename                                 Stmts    Miss  Cover    Missing
-------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------
eodag/__init__.py                            0       0  100.00%
eodag/app.py                                75      15  80.00%   153, 167-174, 185-187, 243-255, 259
eodag/client.py                             25       0  100.00%
eodag/config.py                             19       0  100.00%
eodag/constants.py                           5       0  100.00%
eodag/core.py                              309      27  91.26%   135-136, 146, 290-291, 381, 509-510, 529, 560, 646, 662-666, 680-684, 686-690, 695, 708, 718-720, 723-724, 734, 737, 753
eodag/cql_evaluate.py                       48      18  62.50%   48, 53, 58, 75, 78, 81, 86-99, 105
eodag/dag.py                                57       0  100.00%
eodag/errors.py                             94      12  87.23%   143, 160-163, 179, 202-212
eodag/logs.py                               62       9  85.48%   40-49, 105
eodag/middlewares.py                        32       2  93.75%   20, 24
eodag/telemetry.py                          44       0  100.00%
eodag/utils.py                              41      10  75.61%   44, 69, 72-73, 86, 90-98, 134
eodag/eodag_types/__init__.py                0       0  100.00%
eodag/eodag_types/queryables.py             42       8  80.95%   46, 50-51, 61, 77-81
eodag/eodag_types/search.py                 25      25  0.00%    20-69
eodag/extensions/collection_order.py        73       1  98.63%   106
eodag/extensions/data_download.py          147      46  68.71%   67-70, 80-92, 113, 144-145, 152, 167-174, 178-180, 183-184, 191-223, 235
eodag/extensions/ecmwf.py                   69       0  100.00%
eodag/extensions/filter.py                  96       9  90.62%   182-183, 275-276, 296, 304, 310, 328-330
eodag/extensions/offset_pagination.py       12       0  100.00%
eodag/extensions/stac.py                   167       3  98.20%   348-350
eodag/models/__init__.py                     0       0  100.00%
eodag/models/item.py                        85       6  92.94%   50-53, 55, 69
eodag/models/links.py                      147      13  91.16%   39, 93-95, 201, 219, 251, 298, 347-358
eodag/models/stac_metadata.py              104       6  94.23%   82, 150, 163-170
TOTAL                                     1778     210  88.19%

Diff against main

Filename                             Stmts    Miss  Cover
---------------------------------  -------  ------  -------
eodag/extensions/data_download.py      +26      +4  +3.42%
TOTAL                                  +26      +4  -0.05%

Results for commit: d74acfb

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@cauriol cauriol force-pushed the zarr_store branch 3 times, most recently from 6b4ae98 to ab0855e Compare February 5, 2026 14:22
@cauriol cauriol marked this pull request as ready for review February 5, 2026 15:33
@cauriol cauriol requested a review from jlahovnik February 5, 2026 15:33
@cauriol cauriol marked this pull request as draft February 9, 2026 17:02
@cauriol cauriol marked this pull request as ready for review February 10, 2026 15:59
@cauriol cauriol marked this pull request as draft February 11, 2026 11:37
@cauriol
Copy link
Copy Markdown
Contributor Author

cauriol commented Feb 18, 2026

Currently, we only have DanubeHis and no other provider supporting Zarr, so it is difficult to properly test this part.

I will continue once we add another provider that supports Zarr.

The Zarr endpoint still needs to be implemented, along with the related tests.

@alambare
Copy link
Copy Markdown
Collaborator

I do not understand why fsspec, aiohttp and requests dependencies are needed here.
All the logic must be handled by EODAG. stac-fastapi-eodag is only a wrapper to expose EODAG as a webserver.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants