Skip to content

Paging through GET /api/v3_0/assets without a sort order silently skips assets #2572

Open

Description

@Flix6x

What happens

GET /api/v3_0/assets applies an ORDER BY only when both sort-by and sort-dir are given. Without them the query has no ordering at all, so Postgres returns rows in whatever order the plan happens to produce. Paging through the result set then shows some assets twice and never shows others.

Sorting by owner has the same problem in a milder form: assets sharing an owner have no defined order among themselves, so they can shift across page boundaries.

How to reproduce

Seed 50 assets, each with its own account, two sensors and two child assets (155 assets in total, including the test fixtures), then walk every page with per_page=7 and collect the ids:

request assets reported distinct assets seen duplicates
no sort 155 131 24
sort-by=owner&sort-dir=asc 150 147 3
sort-by=name&sort-dir=asc 150 150 0
sort-by=id&sort-dir=asc 155 155 0

Every duplicate means another asset was never returned on any page. The exact numbers vary per run and per query plan: they are not stable, which is the point. This is long-standing behaviour, not a regression; PR #2515 changed the query plan and so changed which assets get lost, but the same walk loses assets before that PR too.

Suggested fix

In query_assets_by_search_terms (flexmeasures/data/queries/generic_assets.py), always append GenericAsset.id as the final ORDER BY column, both in the branch that has search terms and in the one that does not, so that every result set has a total order. That makes the unsorted case deterministic and gives ties under sort-by=owner a stable tie-breaker.

Worth deciding separately: sort-by is currently ignored unless sort-dir is given as well. Defaulting sort-dir to asc would be less surprising.

Impact

The UI's asset tables always send a sort column, so they are largely unaffected. This mainly hits API clients that page through the asset list without specifying a sort order, which silently gives them an incomplete catalog.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions