Automations - first roundtrip for forecasts

documentation/api/change_log.rst

@@ -7,6 +7,8 @@ API change log
 
 v3.0-32 | July XX, 2026
 """"""""""""""""""""""""
+- Added ``GET /api/v3_0/assets/<id>/automations`` and ``GET /api/v3_0/assets/<id>/automations/<automation_id>`` for listing and inspecting forecast automations, including the sensors an automation reads from and writes to. Asset job entries now include ``created_via`` provenance; automation identity is included only when the caller may read that automation.
+- Added ``GET /api/v3_0/sources/<id>`` to show the full record of one data source, including the attributes in which data generators store their configuration.
 - API endpoints are now rate-limited. A request which exceeds a limit is answered with a ``429 (Too Many Requests)`` status code and a ``Retry-After`` header stating how many seconds to wait. Responses also carry ``X-RateLimit-*`` headers, describing the limit that applied, how much of it is left, and when it resets. A stricter limit applies to ``POST /assets/<id>/schedules/trigger``, ``POST /sensors/<id>/schedules/trigger`` and ``POST /sensors/<id>/forecasts/trigger`` than to other endpoints; the health endpoints are exempt. Per-account overrides are set by assigning the account a plan (a ``Plan`` database row), rather than through an account attribute.
 - Introduced the ``inflexible-consumption`` and ``inflexible-production`` flex-context fields, which make explicit how the sign of each inflexible device's power data should be read: positive values denote consumption resp. production. Each entry is a sensor reference (``{"sensor": <id>}``), optionally with source filters (``source-types``, ``exclude-source-types``, ``sources``, ``source-account``). Deprecated the ``inflexible-device-sensors`` field (a list of bare sensor IDs, whose sign convention is read from each sensor's ``consumption_is_positive`` attribute); it remains supported, but cannot be combined with the new fields in one flex-context.
 - Added a ``role`` query parameter to ``GET /api/v3_0/accounts`` for filtering accessible organisations by account role.

documentation/changelog.rst

@@ -14,7 +14,8 @@ v1.0.0 | July XX, 2026
 
 New features
 -------------
-
+* Automations - first roundtrip for forecasts: recurring tasks defined per asset, managed with new CLI commands (``flexmeasures add|edit|delete automation``), run by ``flexmeasures jobs run-automations``, and viewable in a new UI page and API endpoints (``[GET] /assets/(id)/automations``); an automation's details link to the sensors it reads from and writes to, and a sensor's page lists the automations feeding it; jobs now also record whether they were created via the CLI, the API or an automation [see `PR #2290 <https://www.github.com/FlexMeasures/flexmeasures/pull/2290>`_]
+* In the UI, the full record of the data source selected on a sensor page can be inspected, backed by a new API endpoint (``[GET] /sources/(id)``) [see `PR #2290 <https://www.github.com/FlexMeasures/flexmeasures/pull/2290>`_]
 * New ``inflexible-consumption`` and ``inflexible-production`` flex-context fields make explicit how the sign of each inflexible device's power data should be read (positive values denote consumption resp. production), accepting sensor references with optional source filters; they replace the now-deprecated ``inflexible-device-sensors`` field (bare sensor IDs, sign read from each sensor's ``consumption_is_positive`` attribute), which remains supported [see `PR #2358 <https://www.github.com/FlexMeasures/flexmeasures/pull/2358>`_]
 * An inflexible (unschedulable) device can be modelled as its own asset by giving its flex-model entry a single ``inflexible-consumption`` or ``inflexible-production`` sensor reference; such a device joins a ``group`` like any other member, so its fixed (measured) load counts towards the group's intermediate power constraint [see `PR #2374 <https://www.github.com/FlexMeasures/flexmeasures/pull/2374>`_]
 * The API is now rate-limited, with a generous default limit on all endpoints and a stricter limit on triggering schedules and forecasts (which only counts triggers we accepted). Limits are configurable, and can be set per organisation by putting its account on a plan, which hosts create with ``flexmeasures add plan`` and platform admins assign from the organisation's page in the UI; play servers are exempt, as they run simulations. See :ref:`plans-and-rate-limiting` [see `PR #2306 <https://www.github.com/FlexMeasures/flexmeasures/pull/2306>`_]

documentation/cli/change_log.rst

@@ -12,6 +12,8 @@ since v1.0.0 | July XX, 2026
 * Add ``flexmeasures add plan``, ``flexmeasures show plans`` and ``flexmeasures edit plan``, to manage the rate limits and quotas which apply to the accounts on a plan.
 * Add ``flexmeasures edit secret`` to store an encrypted secret on an account or asset.
 * Add ``flexmeasures delete secret`` to remove an encrypted secret from an account or asset.
+* Add ``flexmeasures add automation``, ``flexmeasures edit automation`` and ``flexmeasures delete automation`` to manage automations (recurring tasks on an asset; for now, computing forecasts).
+* Add ``flexmeasures jobs run-automations`` to queue jobs for all automations that are due to run this minute from standard five-field cron expressions. Run this command once per minute. It makes at most one queueing attempt per automation per minute, including when an attempt fails after partially queueing jobs.
 
 since v0.33.0 | June 01, 2026
 =================================

documentation/cli/commands.rst

@@ -41,6 +41,7 @@ of which some are referred to in this documentation.
 ``flexmeasures add annotation``                   Add annotation to accounts, assets and/or sensors.
 ``flexmeasures add toy-account``                  Create a toy account, for tutorials and trying things.
 ``flexmeasures add report``                       Create a report.
+``flexmeasures add automation``                   Add an automation: a recurring task (for now, computing forecasts) on an asset.
 ================================================= =======================================
 
 
@@ -75,6 +76,7 @@ of which some are referred to in this documentation.
 ``flexmeasures edit resample-data``               Assign a new event resolution to an existing sensor and resample its data accordingly.
 ``flexmeasures edit transfer-parenthood``         (Re)assign parent assets.
 ``flexmeasures edit transfer-ownership``          Transfer the ownership of an asset and its children to a different account.
+``flexmeasures edit automation``                  Edit the name, recurrence or activation status of an automation.
 ================================================= =======================================
 
 ``delete`` - Delete data
@@ -93,6 +95,7 @@ of which some are referred to in this documentation.
 ``flexmeasures delete prognoses``                 Delete forecasts and schedules (forecasts > 0).
 ``flexmeasures delete unchanged-beliefs``         Delete unchanged beliefs.
 ``flexmeasures delete nan-beliefs``               Delete NaN beliefs.
+``flexmeasures delete automation``                Delete an automation.
 ================================================= =======================================
 
 
@@ -117,6 +120,7 @@ of which some are referred to in this documentation.
 ``flexmeasures jobs run-job``                     Run a single job (useful for debugging it)
 ``flexmeasures jobs inspect-job``                 Inspect a background job and print its current status, result and metadata.
 ``flexmeasures jobs stats``                       Show estimated live statistics of the queueing system.
+``flexmeasures jobs run-automations``             Queue jobs for all automations that are due to run this minute (run this once per minute, e.g. via cron).
 ================================================= =======================================
 
 

documentation/features/forecasting.rst

@@ -213,3 +213,46 @@ Usage:
    Create the annotations you want to use as regressors before running the forecast.
    For holidays, use ``flexmeasures add holidays``, which supports both ``workalendar``
    and ``holidays``. See :ref:`annotations` for details.
+
+.. _automating_forecasts:
+
+Automating forecasts
+--------------------
+
+Instead of asking for forecasts one at a time, you can set up an *automation*: a recurring task defined on an asset.
+On each run, the automation queues forecasting jobs (so make sure a worker is processing the ``forecasting`` queue, see :ref:`redis-queue`).
+When the automation was created, its forecast parameters (see above) were stored, and validated with the same schema that the CLI and API use.
+Timing parameters are resolved on each run — for instance, the forecast start defaults to the time the automation runs, so each run produces fresh forecasts.
+The sensor on which forecasts are saved (``sensor-to-save``, falling back to ``sensor``) must belong to the automation asset or one of its descendants.
+This relationship is checked both when the automation is created and immediately before each run.
+
+Here is how you create an automation in the CLI, asking for daily (at 6 AM) forecasts of sensor 12:
+
+.. code-block:: bash
+
+    flexmeasures add automation --asset 3 --name "Daily PV forecasts" --cron "0 6 * * *" --sensor 12
+
+The recurrence is defined by a standard five-field cron string (minute, hour, day of month, month, and day of week), interpreted in the ``FLEXMEASURES_TIMEZONE``, which defaults to ``"0 0 * * *"`` (daily at midnight).
+Cron aliases and optional seconds or year fields are not supported.
+Automations are active by default (use ``--inactive`` to create them in deactivated state).
+Use ``flexmeasures edit automation`` to rename, re-schedule (``--cron``), activate or deactivate an automation, and ``flexmeasures delete automation`` to remove one.
+These changes are recorded in the asset's audit log.
+The stored data generator is required while the automation exists, so its data source cannot be deleted until the automation is removed.
+
+The forecaster and its configuration are stored on a data source.
+Pass ``--source`` to reuse the data source of an existing forecaster, in which case ``--forecaster`` and ``--config`` (and the individual configuration options) are not needed — the data source already determines them.
+
+For automations to actually run, let a cron job execute the following command once per minute:
+
+.. code-block:: bash
+
+    * * * * * flexmeasures jobs run-automations
+
+Each due automation then queues its forecasting jobs.
+Only one queueing attempt is made per automation per minute.
+If an attempt fails after queueing some jobs, it is not retried automatically, because a retry could duplicate that partial work.
+The jobs record how they were created, which is shown on the asset's status page (UI), where recent jobs are listed.
+
+Automations defined on an asset can be viewed on the asset's *Automations* page in the UI, and listed with the API endpoint `[GET] /assets/(id)/automations <../api/v3_0.html#get--api-v3_0-assets-id-automations>`_.
+An automation's details show the sensors it reads from and writes to, linking to each sensor's page.
+Conversely, a sensor's page lists the automations that write data to it.

flexmeasures/api/common/utils/api_utils.py

@@ -26,6 +26,7 @@
 )
 from flexmeasures.data.models.generic_assets import GenericAsset
 from flexmeasures.data.models.time_series import Sensor
+from flexmeasures.data.queries.generic_assets import asset_is_in_subtree
 from flexmeasures.data.utils import (
     SAVE_TO_DB_SUCCESS,
     SAVE_TO_DB_SUCCESS_BUT_NOTHING_NEW,
@@ -570,23 +571,6 @@ def _determine_copy_name(
     return f"{source_name} (Copy {max_index + 1})"
 
 
-def _asset_is_in_subtree(root_asset_id: int, candidate_asset_id: int) -> bool:
-    """Return True if candidate_asset_id is root or a descendant of root_asset_id."""
-    current_asset_id = candidate_asset_id
-    visited: set[int] = set()
-
-    while current_asset_id is not None and current_asset_id not in visited:
-        if current_asset_id == root_asset_id:
-            return True
-        visited.add(current_asset_id)
-        current_asset = db.session.get(GenericAsset, current_asset_id)
-        if current_asset is None:
-            return False
-        current_asset_id = current_asset.parent_asset_id
-
-    return False
-
-
 def copy_asset(
     asset: GenericAsset,
     account=None,
@@ -634,7 +618,7 @@ def copy_asset(
             target_account_id = int(account.id)
             target_parent_asset_id = int(parent_asset.id)
 
-        if target_parent_asset_id is not None and _asset_is_in_subtree(
+        if target_parent_asset_id is not None and asset_is_in_subtree(
             root_asset_id=asset.id,
             candidate_asset_id=target_parent_asset_id,
         ):