feat: add updateAddon.py automated tool to complement existing

[abdel792]

Link to issue number:

Close #37.

Summary of the issue:

Upgrading an existing add-on to follow the modern addonTemplate structure manually is a repetitive and error-prone process. Developers frequently encounter syntax issues or merge conflicts when manually porting old metadata into the new pyproject.toml structure.

Description of developer facing changes:

Introduces an automated update workflow. Developers can now run a single command (uv run updateAddon.py) from any directory on their PC to automatically migrate metadata, clean template placeholder data (like removing nvaccess from third-party authors lists), and merge project configurations while preserving custom formatting.

Description of development approach

• Script implementation: Added updateAddon.py at the repository root. The script leverages an AST-aware (Abstract Syntax Tree) approach to perform safe, robust parsing and merging of TOML structures.
• Dependency Update: Added tomlkit==0.13.0 under the # Update machinery section in pyproject.toml to support programmatic, comment-preserving TOML modifications.
• Linter Exclusion: Added updateAddon.py to the exclude lists of both [tool.ruff] and [tool.pyright] within pyproject.toml to avoid unnecessary environmental or strict typing alerts during standard repository validation.
• Documentation: Updated updatingExistingAddons.md to cleanly integrate prerequisites (Python 3.11+, Git available in PATH) and step-by-step instructions on how to use this new companion tool alongside the existing manual instructions.

Testing strategy:

The script and documentation workflow have been tested locally:

1. Initialized a legacy NVDA add-on workspace using old metadata configurations.
2. Ran git fetch template to ensure local tracking branches were up to date.
3. Executed uv run updateAddon.py from the repository root, and verified that a _bak_ backup directory was safely generated.
4. Checked the newly mutated pyproject.toml to ensure old custom metadata was correctly merged, template placeholder defaults were stripped, and file formatting/comments remained completely intact.
5. Ran uv sync followed by uv run scons to ensure the final environment synchronized properly and the .nvda-addon bundle compiled successfully with zero errors.
6. Repeated the execution from a completely different working directory outside the repository to confirm path flexibility.

Known issues with pull request:

None.

---

Code Review Checklist

• Testing: Manually tested across diverse repository paths to confirm robustness. Steps to reproduce are detailed in the testing strategy.
• API is compatible with existing add-ons: This is an infrastructure update tool for developers; it does not introduce any breaking API changes or modifications to the NVDA core ecosystem.
• Documentation: Technical developer documentation (updatingExistingAddons.md) has been explicitly updated to reflect these changes.
• UX of all users considered: N/A (Developer-facing infrastructure tool only).
• Security precautions taken: N/A (The script executes locally during development and does not interact with secure screens or lock screen states).

[abdel792]

Hi @nvdaes,

I have just added you as a collaborator to this repository.

If you have some time and interest, feel free to directly review and update the docstrings in updateAddon.py to ensure they perfectly align with the project's official documentation conventions.

Thank you so much for your help and guidance on this!

[nvdaes]

Thanks @abdel792 . I've accepted your invitation as a collaborator, and I'll try to update the docstrings.

[nvdaes]

@abdel792 , can we add a way to ignore files which shouldn't be merged?
I think that @CyrilleB79 may be interested, and i'd like to have this possibility. For example, I don't want to use the build_addon.yml workflow since it's failing for me, and I prefer to remove it.
Also, I think that instructions for the script should be placed at the start of the file, and manual instructions maybe removed or placed at the end. BTW, they contain an error of a typo. With your permission, I'll fix it.

[abdel792]

Thanks a lot @nvdaes for your quick commits and for improving the docstrings and file headers!

Regarding your suggestions:

1. Ignoring files: That's a great idea. Having a way to skip specific files (like workflows or custom configs) during the merge would make the tool much more flexible. I'll think about the best way to implement this (maybe via an ignore list/parameter) and update the script.
2. Documentation structure: Completely agree. The automated tool should definitely be the primary option at the top of the file, with the manual steps as a fallback at the end. Please feel free to reorganize the file and fix the typo you found, you have my full permission!

I'll let you know once I've drafted the file-ignoring feature so we can test it. Thanks again for your amazing collaboration!

[nvdaes]

@abdel792 , I've reorganized the documentation so that info about the script is at the start. Perhaps the information about merge should be removed=
Looking forward to see a way to ignore files/folders.
Thanks for all your amazing work.

[abdel792]

Hi @nvdaes,

Thank you so much for updating the documentation! It looks great and makes using the automated tool very clear.

Could you please do me a favor and also integrate the new feature requested by @CyrilleB79 that I just implemented in my latest commit? It allows developers to exclude specific template files during synchronization.

Here is the markdown section that needs to be added to the documentation (you can place it right before the build verification step):

### Excluding specific template files

By default, the script synchronizes every infrastructure file provided by AddonTemplate.

If you want to preserve specific files from your repository (for example, a customized GitHub workflow), you can exclude them from synchronization.

Open `updateAddon.py` and locate the `IGNORED_FILES` set near the beginning of the `main()` function:

```python
IGNORED_FILES = {
    os.path.join(".github", "workflows", "build_addon.yml").lower(),
}

Add any relative path from the template root to this set to prevent that file from being synchronized.

---

[abdel792]

Hi @nvdaes,

While looking closely at our documentation, I am considering a complete overhaul and restructuring of the documentation files to make everything cleaner, more cohesive, and easier to navigate for developers.

Before diving into this, I wanted to open a discussion here to get your thoughts, ideas, or any suggestions you might have regarding how we should reorganize the content.

Let me know what you think!

[abdel792]

I have just committed a complete overhaul and reorganization of the update documentation (updatingExistingAddons.md).

In this update, I have:

• Restructured the document to separate the recommended automated method from the manual workflow.
• Added the full comprehensive guide for the automated update script (updateAddon.py).
• Added a dedicated section explaining how to exclude specific template files using the IGNORED_FILES set, as requested by @CyrilleB79.

Please let me know if you have any feedback or further suggestions.

@nvdaes, please feel free to add, modify, or correct anything you see fit—you have the green light, of course!

[Copilot]

Pull request overview

This PR introduces a developer-facing automation tool (updateAddon.py) intended to help migrate/update existing NVDA add-ons to the modern AddonTemplate structure, alongside documentation and dependency updates to support that workflow.

Changes:

• Added updateAddon.py to clone the latest AddonTemplate, sync infrastructure files, and merge buildVars.py / pyproject.toml.
• Updated pyproject.toml dependencies (pyright bump + new tomlkit dependency) and excluded updateAddon.py from ruff/pyright checks.
• Expanded docs/managementFromGit/updatingExistingAddons.md with an “automated update” path plus revised manual instructions.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 11 comments.

File	Description
updateAddon.py	New CLI tool that syncs template infrastructure and performs AST/TOML-based merges.
pyproject.toml	Adds tomlkit dependency and excludes the new script from lint/type-checking; bumps pyright.
docs/managementFromGit/updatingExistingAddons.md	Documents the new automated update workflow and revises the manual update steps.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[wmhn1872265132]

I suggest renaming updateAddon.py to updateTemplate.py to make the purpose of the file clearer.

[abdel792]

Hi @wmhn1872265132,

Thanks for the suggestion! However, I'm concerned that renaming it to updateTemplate.py might be a bit misleading.

In this context, the "template" refers specifically to the upstream nvaccess/AddonTemplate repository. We aren't actually updating the template itself; rather, we are updating the local addon repository's infrastructure to align with the latest template standards.

Keeping updateAddon.py helps clarify that the target of the update is the addon's repository, not the template.

[CyrilleB79]

Hi all! You are discussing and working a lot and I'm not able to follow all this as would be needed, sorry.

It's not clear to me if this tool is needed:

1. once for add-on X based on an old version of the template to catch up with more recent template evolution
2. or each time that add-on X need to be updated from the template

@abdel792 you write:

Upgrading an existing add-on to follow the modern addonTemplate structure manually is a repetitive and error-prone process. Developers frequently encounter syntax issues or merge conflicts when manually porting old metadata into the new pyproject.toml structure.

Do you mean in case 1 or case 2? If you meant case 2, it's not true at all. I too have had merge difficulties to update my add-ons from template, e.g. when the structure of the buildVar.py changed. But now that my add-ons are synchronized, I update quite easily from the template. I just merge from (without --squash flag) my local templateMaster branch (which follows template/master) and usually have only conflict on the readme.md, which is easy to solve since I juste need to choose my add-on's version and discard all changes from the template on this file.

To me, upgrades from template now work correctly and I do not ask personally a new tool to perform this work, and won't probably need to use it. Sorry for the confusion.

Of course, I have a lot of commits coming from the template in my add-on's history. This may be considered undesirable by some. To me, it's not a problem. I can easily ignore them doing git log --first-parent.

Also small suggestion regarding the name: what about updateAddonFromTemplate.py?

[nvdaes]

@CyrilleB79 wrote:

It's not clear to me if this tool is needed

Imo, it's a good addition:

1. It avoids the necessity of add the template as a remote, which may be tricky for some people.
2. Imagine that the template evolves and buildVars.py is replaced with something completely different, or that we want to replace .precommit.yaml with prek.toml, which seems to be easier to manage since it doesn't require to add each repo to precommit.ci. I think that this feature may manage this in a consistent way avoiding, from the start, resolving conflicts.
   I'm using it to update the template for all my add-ons now.

[CyrilleB79]

Sorry @nvdaes, I do not question the usefulness of tis tool. My question is if this tool is meant to address case 1 (first update of add-on from template), or case 2 (regular updates from template).

[nvdaes]

@CyrilleB79 wrote:

My question is if this tool is meant to address case 1 (first update of add-on from template), or case 2 (regular updates from template).

My understanding is that this covers both cases, but I may be wrong. Let's wait for Abdel's reply.

[abdel792]

Hi @CyrilleB79, thank you for sharing your thoughts! And thanks @nvdaes for jumping in with those excellent points.

To answer your question directly: as @nvdaes suspected, this tool is designed to address both cases.

1. For Case 1 (Initial Catch-up from very old versions): In older versions of the template, addon_info was defined as a standard dictionary, and pyproject.toml didn't even exist yet. In this scenario, the script does some heavy lifting: it migrates the legacy dictionary structure in buildVars.py into the modern AddonInfo object now in use, and automatically generates a brand new, fully populated pyproject.toml file to seamlessly align the add-on with modern template standards.

2. For Case 2 (Regular Updates / Maintenance):
   I completely understand that maintaining a templateMaster branch and doing standard git merges works smoothly for you. However, for many developers, dealing with Git remotes and tracking upstream configuration changes can be intimidating.
   Even for synchronized repositories, the script adds value by safely checking if new keys have been introduced in buildVars.py or if updates/new dependencies have been added to pyproject.toml. It merges these programmatically without triggering layout corruption or unexpected syntax conflicts. This ensures all other infrastructure components of the workspace stay perfectly aligned with the latest template evolution with a single command.

It's completely optional, of course! Your manual Git merge workflow remains fully supported and valid (and is still detailed in the documentation). This tool simply offers a streamlined alternative.

Regarding the name suggestion: updateAddonFromTemplate.py is a fantastic idea. It perfectly clarifies the script's intent without being misleading (unlike updateTemplate.py, which sounded like we were modifying the template itself). I will rename the script and update the documentation to reflect this right away!

[abdel792]

Hi everyone,

I have just pushed a new update to address all the valuable feedback received from Copilot and the community:

1. Script Renaming: Following @CyrilleB79's excellent suggestion, I have renamed the script from updateAddon.py to updateAddonFromTemplate.py. This explicitly reflects that the tool targets the local add-on repository rather than the template itself.
2. Configuration Alignment: I've fully updated pyproject.toml to reflect this new filename, ensuring it remains excluded from ruff and pyright checks.
3. Documentation Overhaul: The updatingExistingAddons.md documentation has been thoroughly updated to adopt the new name and incorporate Copilot's recommendations. It now accurately reflects the modern prerequisites (Python 3.13), the correct workspace path cloning sequence, and precise details about the timestamped external backup directories.

Thank you all for this amazing collaboration! I'll mark the corresponding review conversations as resolved.

[abdel792]

Hi everyone,

I have just pushed a new commit to update the documentation (updatingExistingAddons.md) so that it perfectly aligns with Copilot's review and reflects the actual behavior of the script:

1. Cloning Paths: Corrected the initial setup guide by adding the missing cd {repoName} step after cloning.
2. Execution Modes: Explicitly documented the two execution modes of updateAddonFromTemplate.py (Standard Mode without arguments and Target Directory Mode by providing the optional addonDir argument).
3. Python Version: Bumped the minimum required Python version to 3.13 in the docs to match the template's requirements.
4. Backup System: Clarified that the script performs a full copy of the project into a dynamically named timestamped backup folder (<addon>_bak_<timestamp>) before editing any file.
5. Add-on Infrastructures: Added detailed sections highlighting how the script supports both legacy dictionary-based structures (generating a fresh pyproject.toml) and modern AddonInfo-based configurations.
6. Ignored Components: Clarified that the IGNORED_FILES set can exclude both files and entire directories.

@seanbudd: In my opinion, the PR is now in excellent shape and ready for review/merge. Please feel free to trigger a new Copilot code review cycle if you would like to verify everything.

@nvdaes, @CyrilleB79, and everyone else: please let me know if you have any additional feedback or final suggestions before we move forward!