Upgrade python packaging and tooling by RobLevv · Pull Request #136 · AlexandreDecan/sismic

RobLevv · 2026-07-27T21:52:58Z

Hi, following this discussion #135 (comment)

I would be happy to upgrade the python tooling of sismic.

So here is the configuration I would like to apply:

transition setup.[py|cfg] to pyproject.toml
add ruff and mypy to a pre-commit configuration
add a github action to run those pre-commit hooks

This configuration will raise around 700+ linting/typing issues so it would be a good idea to let #135 be merged first to not apply twice some fixes and build on some improvements included in that PR.

I would like to let you discuss this configuration before applying all the fixes to the linting/typing issues as there will be many and review might be exhausting.

If you have any question or would like to have a different or be even more ambitious with the upgrade please let me know :)

have a nice night,
Rob.

note: do not look at the pipeline yet, there is many things to do
you can check RobLevv#1 if you want to know what it would look like

Morwenn · 2026-07-28T08:07:30Z

+  push:
+  pull_request:
+  schedule:
+    - cron: "0 6 * * 1"


Is that really useful? The project hardly ever moves and I don't see it failing in-between modifications.

Morwenn · 2026-07-28T08:12:22Z

+      matrix:
+        python-version: ["3.9"]
+    steps:
+      - uses: actions/checkout@v3


We're at actions/checkout@v7 nowadays, otherwise you get various warnings from GitHub.

Morwenn · 2026-07-28T08:13:28Z

+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9"]


It might be valuable to have at least the minimum supported version and the latest available Python version in the matrix. Intermediary versions could be nice too, but less valuable.

Oh wait, that's the tooling, not the tests 🤔

Scratch that.

AlexandreDecan · 2026-07-28T08:15:29Z

Thanks! :-)

Can you move the linting stuff in the github actions workflow to the other (test) workflow so we only have one checking everything?

And since you asked, would you agree to set the build system to hatchling? :-) I'm using uv for another project, I like their approach (and we can benefit from their python-version and uv.lock files as well) :-)

I'll try to find some time to merge the other pr today. No promise :-)

AlexandreDecan · 2026-07-28T09:17:18Z

I've just merged the other PR. Thanks @Morwenn for the changes!

@RobLevv if we migrate to uv, we should also update the .readthedocs.yaml file that currently refers to requirements.txt. I found this: https://about.readthedocs.com/blog/2026/04/uv-native-support/ so it should be quite easy to adapt it :-)
Also, it would be nice if, for sismic, we can stay close to (or at least not too far from) what I did for portion (https://github.com/alexandredecan/portion) so that I do not need to learn another toolchain for maintaining sismic :-)

Oh, and when you have time and if you're willing to, could you tell me what is your use case with sismic? (academia?) :-)

Morwenn · 2026-07-30T10:34:56Z

Oh, and when you have time and if you're willing to, could you tell me what is your use case with sismic? (academia?) :-)

We're actually from a small company making solar power plant control software, and more generally an Energy Management System. We use sismic at the heart of our controller to orchestrate the different parts of the plant.

Unfortunately I don't think I can tell much more about the component in which we use it exactly. I suppose that it's the kind of specific information that would fall under industrial property secrets. I can tell you that we've been using it a lot for around 5 years, without any major issue.

The recent pull requests were not done on company time; we individually decided that it was time to give back to the FOSS commons, and smooth out the library's experience a bit without fundamentally changing it.

AlexandreDecan · 2026-07-30T10:50:03Z

Thanks a lot! I'm really glad to see that what started as a kind of toy/scientific project is actually used and useful in production :-)

What drove you to use sismic compared to, e.g., Rhapsody or Yakindu? Its support for testing/validating statecharts? (that's something missing from "compellers" :-))

Morwenn · 2026-07-30T18:48:11Z

I have no idea to be frank. It was chosen before I joined the company, and we don't make use of advanced features.

I suppose that it's the ability to inject Python locals into the statechart and execute code there. But I haven't checked alternatives myself so I don't even know what more they have to offer.

We don't really use the validation tools even though the extra validation statecharts look cool. We do perform statuc analysis to ensure that our statecharts always follow some patterns that make reasoning easier (such as avoiding that two parallel statecharts access the same resource).

RobLevv · 2026-07-30T20:40:25Z

Hi again,
I have finished the setup update:

sadly github actions were a bit more painful that anticipated, I am more used to gitlab

updated all actions to the latest version
added a job for code quality (ruff + mypy essentially)
added a pre-commit config with python basics I like to use
added uv with hatchling build backend
configured all tools in pyproject.toml
I kept the lint job a separate job but defined in the same file as tests, because only 1 job on python latest is enough, there is no need to rerun it for each version, the tests already handle the version compatibility

Some choices that can still be made:

we could use uv_build instead of hatchling

as we use uv it would be logic to use there own build backend directly, as per their documentation, they recommend using hatchling only when we are doing some cursed custom build in the package, but for pure-python I think it straightforward.

we could use any new rust-based type checker instead of mypy

I have a lot of good reason to not use mypy anymore but it is a great starting point as it is better than nothing. There are pyrefly (from meta), ty (from astral who made ruff and uv) and zuban (from the guy who made jedi).
My personnal favorite is pyrefly right now, it is the most feature rich and most advanced but they are all in the "alpha" phase more or less.

Finally the technical direct choices I need your decision on:

what config for ruff ?

There is the basic one that finds all of classic errors
There is the all config that is very strict and pedantic

what config for mypy ?

Same here there is the normal and strict one

How do I fix all the errors found by those tools ?

For the straightforward ones I just apply what the standard/ruff think is best
For the more complex ones, do I ignore them with a small ruff:ignore ? or do I try to fix it to the best of my understanding of sismic with potentially small refactos ?
For mypy there are quite a few false positives, do you want me to be very strict or as soon as there is something not straightforward I skip them ?

Side note:
I personnally love tools like sismic and portion, I found myself re-implementing them quite a few times during my young career. I studied robotics and automatics and started as a robotics engineer. And sismic would be the base of all state machines I was using, if only I knew it at that time 😢 it would have saved me a lot of time. Same thing about portion, I implemented something like that 3 times already in various contexts.
Also, I think sismic is very useful to allow non-dev to agree on a statechart contract that is implemented exactly as described.
I think your works are very educational in the way that it applies engineering concepts in code quite beautifully :)

AlexandreDecan · 2026-08-01T06:54:55Z

Thanks! I'll get back to you soon for the various choices we have to make :-)

RobLevv · 2026-08-01T11:11:23Z

Thanks! I'll get back to you soon for the various choices we have to make :-)

Actually, I might have been a bit over zealous with all the questions and propositions of improvements I was just thinking since we upgrade some we could upgrade everything at once.
But we can stay on a more conservative approach for now with only configuring ruff and mypy as I just did, fixing all non-breaking errors and ignoring the other ones. That is safer and easier to handle.

I can always follow up later with more :) depending on what you want for sismic

AlexandreDecan · 2026-08-03T08:30:45Z

(Sorry, I had to edit this message, I don't know what happened but only parts of my initial answer were saved by GitHub ^^)

Hi,

Some choices that can still be made:
1. we could use uv_build instead of hatchling

I have no strong opinion on this. I'm using hatchling in other projects, but mostly because it is "fast and lightweight" compared to other build backend. I wasn't aware that uv released its own build backend and, for what I see, it seems even faster and more lightweight, so I'm fine with using it :-)

2. we could use any new rust-based type checker instead of mypy
I have a lot of good reason to not use mypy anymore but it is a great starting point as it is better than nothing. There are pyrefly (from meta), ty (from astral who made ruff and uv) and zuban (from the guy who made jedi).
My personnal favorite is pyrefly right now, it is the most feature rich and most advanced but they are all in the "alpha" phase more or less.

I'll have a look at zuban, I don't know about it. For now, mypy is enough. Let's consider it as the "de facto" standard for type checking in Python. If we need something more powerful (but I don't think it's the case with Sismic) we may decide to go for another tool.

Finally the technical direct choices I need your decision on:
1. what config for ruff ?
There is the basic one that finds all of classic errors
There is the all config that is very strict and pedantic

The basic one should be enough, just executing ruff without additional parameters :-) We may decide to disable some very specific rules if needed, though, through the pyproject.toml tool section.

2. what config for mypy ?
Same here there is the normal and strict one

I'm not a user of mypy so I don't know what are the differences between the "normal" and "strict" mode. Again, my suggestion is to "just support mypy" without additional parameters as much as possible.

3. How do I fix all the errors found by those tools ?
For the straightforward ones I just apply what the standard/ruff think is best
For the more complex ones, do I ignore them with a small ruff:ignore ? or do I try to fix it to the best of my understanding of sismic with potentially small refactos ?

I prefer avoiding ruff: ignore since that's quite specific to ruff. A noqa is more generic, but I would like to avoid them as much as possible. Can we move the task of "fixing pending errors" to another PR? (or can you pinpoint these errors so I can see what would be needed to fix them?)

For mypy there are quite a few false positives, do you want me to be very strict or as soon as there is something not straightforward I skip them ?

Could you pinpoint these cases so I can see what the problem is?

Side note: I personnally love tools like sismic and portion, I found myself re-implementing them quite a few times during my young career. I studied robotics and automatics and started as a robotics engineer. And sismic would be the base of all state machines I was using, if only I knew it at that time 😢 it would have saved me a lot of time. Same thing about portion, I implemented something like that 3 times already in various contexts. Also, I think sismic is very useful to allow non-dev to agree on a statechart contract that is implemented exactly as described. I think your works are very educational in the way that it applies engineering concepts in code quite beautifully :)

Thank you very much :-)

RobLevv · 2026-08-03T16:33:11Z

Ok great, thank you for your answers.
1 last question: Are you okay with pre-commit or would you like to only rely on running commands manually/in workflows ?

I am quite used to using pre-commit everytime now but I don't want to impose a new workflow for you.
Either way, I can update the contributing section to include how to use uv, ruff, mypy etc...

I have used the basic config for ruff and mypy, like you said.

I will clean a bit my work and the commits and will pinpoint the tricky places, then you will be good for review (in a few days maybe)

AlexandreDecan · 2026-08-05T06:26:13Z

Hi,

I'm not used to precommits (I have used them only one tbh) but I don't mind having them for portion. It is doesn't hurt those who don't want to use them :-)

I'm not sure we need to update the documentation (I don't remember having provided instructions for contributing so far and, in any case, the current/future setup remains quite common and quite visible even if not explicit in the doc).

RobLevv

every end of file/ end of line is fixed automatically, if some places feels odd, there may be some configurations to improve the behavior of the formatters.

Overall almost no ignore, and some got removed.

RobLevv · 2026-08-05T22:09:19Z

+    ):
        try:
-            import ipdb as pdb
+            import ipdb as pdb  # type: ignore # noqa


ipdb is not installed, I suppose it is only possible to use when it is installed by some other package (so the try ... except)
Anyway, mypy complains import is not possible (which is logic), type: ignore is mandatory here

RobLevv · 2026-08-05T22:10:24Z

+            import ipdb as pdb  # type: ignore # noqa
        except ImportError:
-            import pdb
+            import pdb  # noqa


ruff complain about pdb and ipdb https://docs.astral.sh/ruff/rules/debugger/
But I am pretty sure the goal here is to drop into the debugger so noqa

RobLevv · 2026-08-05T22:16:17Z

@@ -1,4 +1,6 @@
-from behave import given, when, then  # type: ignore
+# mypy: disable-error-code="operator"


here a tricky one:
mypy complains error: "_StepDecorator" not callable [operator] on all @given, @when, @then which is correct https://github.com/behave/behave/blob/4387315f12299d4b0b2ec5bf0b399e2366a99f5e/behave/step_registry.py#L224

I check with ty and zuban, they both infer the same.

only pyrefly bundles a typeshed for behave which infers def given(step_text: str, **kwargs) -> Callable[[_F], _F]: ... https://github.com/python/typeshed/blob/a2811326ffe3c0e69c4624de1d6b199a3c1f4ace/stubs/behave/behave/step_registry.pyi#L9

this line disables the error for the entire file, no other error was found.

RobLevv · 2026-08-05T22:29:19Z

@@ -1,22 +1,24 @@
-import ruamel.yaml as yaml
-import schema
+# mypy: disable-error-code="arg-type"


Another tricky one:

mypy raises "Or" has incompatible type "str"; expected "Callable[..., Any]" on every schema.Or
because it is waiting on a callable. already flagged and fixed keleshev/schema#342
waiting for a new tag

RobLevv · 2026-08-05T22:41:54Z

-            interpreter = statechart
+                DeprecationWarning,
+            )
+            interpreter: Interpreter = statechart


here I add a type hint because otherwise mypy infer this as Statechart which created an error at line 260 with interpreter = interpreter_klass() was erroring cannot assign Statechart to Interpreter

RobLevv · 2026-08-05T22:49:25Z

I have finished applying all the ruff + mypy fixes. I think it is ready for review, don't hesitate to challenge a bit the changes. I explained the mypy errors that are unfixable.

As you said, pre-commit is entirely opt-in (the lint action will sometimes complain more for people who do not use it, same for ruff and mypy if someone does not use it)

AlexandreDecan · 2026-08-07T15:47:15Z

Thanks! I'll review "ASAP" :-) I'm sorry to take that much time for every step ;-)

RobLevv · 2026-08-07T15:54:27Z

Take all the time you need :) I am on vacation.

Morwenn

I had filed a few review comments some time ago, but only realized today that I had to click "Submit review" for them to show -_-

Morwenn · 2026-08-01T11:43:28Z



-def map_assertion(step_text: str, existing_step_or_steps: Union[str, list[str]]) -> None:
+def map_assertion(step_text: str, existing_step_or_steps: str | list[str]) -> None:


X | Y for type unions is only available in Python 3.10, but sismic claims Python 3.9 support.

Actually, with using from __future__ import annotations this syntax becomes available in the file.
This is only possible for type hints because the evaluation is deferred but does not work for runtime types where I still used Union.
https://docs.python.org/3.7/whatsnew/3.7.html#pep-563-postponed-evaluation-of-annotations

Oh wow, so we're assuming that only type checkers will only read it? That's pretty cool :o

Another possibility would be to drop python 3.9 officially... That version is eol.

That's true. The main risk is if other libraries relying on sismic still guarantee 3.9 compatibility, or if some companies using the project are stuck on 3.9 for some reason. I don't know how likely it is though, I guess that somebody who really cares could open an issue later anyway.

Morwenn · 2026-08-11T17:26:54Z

+          cd examples/microwave/
+          pytest


Suggested change

cd examples/microwave/

pytest

pytest examples/microwave

I actually tried that at one point, but the tests still did not pass because the statechart used in the test is not found.
I tried many commands to try and run it from the root of the repo like you suggest but did not succeed, I might give it another go.

Oh, so there are test files that depend on where the command is run instead of where the executed tests are? Sounds like something that ought to be fixed. Whether that's for the current PR or not depends on how complicated it is I guess 🤔

I'm surprised by the behaviour. Could you provide more details? Is this something new due to the change or some old stuff impacting the recent change?

Morwenn · 2026-08-11T17:30:51Z

+        uses: actions/setup-python@v7
+        with:
+          python-version-file: ".python-version"
+      - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9


Why use a commit reference here?

I struggled a bit to find the correct version of this action, when looking on the marketplace, they declare that this action is at v9 but I tried it and this version did not exist.
The only non-hash version was v7 and it seemed quite old so I just used the value recommended on the marketplace page.

Looking at the project's tags, it would be v9.0.0 instead of v7. It looks like they stopped maintained "major version" tags. That's an odd choice though.

Ah, I found the marketplace description, which corroborates what you said. I guess it's to avoid supply chain attacks, which is fine. Could you add least add a comment to specify the target version?

Morwenn · 2026-08-11T17:39:29Z

    def __init__(self) -> None:
-        self._entry_time = dict()  # type: dict[str, float]
-        self._idle_time = dict()  # type: dict[str, float]
+        self._entry_time = {}  # type: dict[str, float]


Here we use comments for type hints, and below level-language type hints. It would be nice to choose just one and stick to it.

Just to remind the pros and cons since it's mostly the same thing:

Comments are parsed as comments and entirely discarded by CPython, so they don't end up in the AST. I guess it makes loading very slightly faster.

Proper Python construct has fewer risks of pointing an incorrect line due to an overzealous code formatter, and reduces the chances of import that end up "unused" for linters because they're only used in comments.

Morwenn · 2026-08-11T17:47:43Z

+    def _select_transitions(
+        self,
+        event: Event | None,
+        states: Iterable[str],


I think states would be better typed as set[str], the only type with which it is ever called: Iterable accepts a generator, and this function would fail with a generator because states would be consumed several times.

Morwenn · 2026-08-11T17:55:12Z

+        '--show-description',
+        dest='statechart_description',
+        action='store_true',
+        default=False,


The actions store_true and store_false already have the defaults False and True respectively, so default is redundant for them, and the only difference is that it's more verbose and we have the opportunity to get it wrong.

I propose to remove it for all such actions, which will also save us a few lines :D

Explicit is better than implicit. Let's keep the default values explicit :-)

Morwenn · 2026-08-11T20:18:16Z

@@ -0,0 +1 @@
+3.14


Isn't that the file you need to change to make sure that it targets Python 3.9 instead?

I think the minimal supported version is already defined in the pyproject.toml as 3.9
The maximal is not defined anymore so here seems a good place to define the "recommended python version to use, also the one you might use as a default when developing on this project"

Okay, I didn't know that convention, cool 👍

Morwenn · 2026-08-11T20:19:16Z

+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",


Python 3.15 has been out for some time, maybe it should be added here and tested in the CI too.

We could add 3.14t too, I am not sure it adds much though

Assuming that 3.14t is the multithreaded build of Python: there's at least the runner that uses multithreading, so maybe it's worth having at least one such threaded CI test.

Let's ignore 3.14t at the moment. We don't know which of dependencies are compatible and not explicitly mentioning "t" doesn't mean it cannot be used :-) it's just we do not pretend everything will work fine :-)

AlexandreDecan · 2026-08-19T13:17:41Z

Is there anything i/we should do before going further with this PR? (apart from updating the changelog and adding you as a contributor :-))

RobLevv force-pushed the upgrade-tooling-packaging-stuff branch from 7aec32c to ac0b574 Compare July 27, 2026 22:03

Morwenn reviewed Jul 28, 2026

View reviewed changes

RobLevv force-pushed the upgrade-tooling-packaging-stuff branch from 3052f35 to e17e289 Compare July 29, 2026 20:59

RobLevv force-pushed the upgrade-tooling-packaging-stuff branch from 7b12814 to f5d0144 Compare July 31, 2026 23:15

Upgrade python packaging and tooling

d2bcdca

RobLevv force-pushed the upgrade-tooling-packaging-stuff branch 2 times, most recently from 7cb0dac to eabed94 Compare August 5, 2026 22:39

Fix all ruff and mypy errors

db8fa25

RobLevv force-pushed the upgrade-tooling-packaging-stuff branch from eabed94 to db8fa25 Compare August 5, 2026 22:43

RobLevv commented Aug 5, 2026

View reviewed changes

RobLevv marked this pull request as ready for review August 5, 2026 22:47

Morwenn reviewed Aug 11, 2026

View reviewed changes

		@@ -1,4 +1,6 @@
		from behave import given, when, then # type: ignore
		# mypy: disable-error-code="operator"



		def map_assertion(step_text: str, existing_step_or_steps: Union[str, list[str]]) -> None:
		def map_assertion(step_text: str, existing_step_or_steps: str \| list[str]) -> None:

		@@ -0,0 +1 @@
		3.14

Uh oh!

Conversation

RobLevv commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlexandreDecan commented Jul 28, 2026

Uh oh!

AlexandreDecan commented Jul 28, 2026

Uh oh!

Morwenn commented Jul 30, 2026

Uh oh!

AlexandreDecan commented Jul 30, 2026

Uh oh!

Morwenn commented Jul 30, 2026

Uh oh!

RobLevv commented Jul 30, 2026

Uh oh!

AlexandreDecan commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RobLevv commented Aug 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexandreDecan commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RobLevv commented Aug 3, 2026

Uh oh!

AlexandreDecan commented Aug 5, 2026

Uh oh!

RobLevv left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RobLevv commented Aug 5, 2026

Uh oh!

AlexandreDecan commented Aug 7, 2026

Uh oh!

RobLevv commented Aug 7, 2026

Uh oh!

Morwenn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RobLevv commented Jul 27, 2026 •

edited

Loading

AlexandreDecan commented Aug 1, 2026 •

edited

Loading

RobLevv commented Aug 1, 2026 •

edited

Loading

AlexandreDecan commented Aug 3, 2026 •

edited

Loading