MegaLinter v10 - Regression on exclude directory feature

[llaville]

Describe the bug
Excluding directories feature has behaviour changed with v10 and introduced a regression since v9.6

Summary by image

Note

1. First I though it was a performance issue (see previous screenshot) , before I realized that it came from excluding directory feature
2. I found it while I'm working on next version 9.8 of PHPLint, and wanted to check regression with MegaLinter.
3. My first attempt with MegaLinter v10 was on project https://github.com/llaville/sarif-php-converters when I relased a version 1.7.0 (compatible with future PHPLint 9.8), before I decided to step back to 9.6.0 (stable version)

To Reproduce
Steps to reproduce the behavior:

1. Use a project with PHP Source Code: For example https://github.com/llaville/sarif-php-converters/tree/1.7
2. Run Composer Installation (composer update) to install all vendor dependencies
3. Pull down Docker PHP flavor images X.Y.Z: 9.6.0 and 10.1.0 (to compare and check diffs) : docker pull ghcr.io/oxsecurity/megalinter-php:vX.Y.Z
4. Run MegaLinter Docker following command:

sudo rm -rf megalinter-reports/ && docker run --rm -v /var/run/docker.sock:/var/run/docker.sock:rw -v $(pwd):/tmp/lint:rw -e PARALLEL_PROCESS_NUMBER=11 -e FORWARD_EXCLUDED_DIRECTORIES=false ghcr.io/oxsecurity/megalinter-php:vX.Y.Z

Tip

I've limited usage of my CPU (12 cores) to have only 1 free; I think MegaLinter should follows this recommandation, rather than using all core available.

Both versions accept the same command, but 9.6.0 did not accept my WorkAround I found to fix temporary this regression issue. See after !

Expected behavior
A quick run results with all linters returning none error (like first screenshot); see PHP Linters time elapsed !

Screenshots
Version 9.6.0 :

BTW: mentionned MegaLinter 9.5.0 while it's running 9.6.0 (a minor typo error)

Version 10.1 with workaround (tips):

EditorConfig errors has the same issue (and I think all linters should have the same problem !)

My Opinion

Origin came from the "dot" added on command line (see cli_lint_mode_project_extra_args_after line 69-70) that did not respect the project config folders limitation.

The "dot" run as a current directory and check all folders, even if they were ignored previously.
In my project context : https://github.com/llaville/sarif-php-converters/blob/1.7/.gitignore

Additional context

The WorkAround I found that works find with only v10, because v9.6 raise the following error

Adds in your .mega-linter.yml config file, these directives :

PHP_PHPCS_COMMAND_REMOVE_ARGUMENTS: "."
PHP_PHPLINT_COMMAND_REMOVE_ARGUMENTS: "."
PHP_PHPSTAN_COMMAND_REMOVE_ARGUMENTS: "."

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ~~~~^^^^^^^^^^^^^^^
  File "/megalinter/MegaLinter.py", line 78, in run_linters
    linter.run(
    ~~~~~~~~~~^
        run_commands_before_linters=False,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        run_commands_after_linters=False,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        skip_console_reporter=True,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/megalinter/Linter.py", line 901, in run
    return_code, stdout = self.process_linter()
                          ~~~~~~~~~~~~~~~~~~~^^
  File "/megalinter/Linter.py", line 1126, in process_linter
    command = self.build_lint_command(file)
  File "/megalinter/Linter.py", line 1497, in build_lint_command
    cmd.remove(arg)
    ~~~~~~~~~~^^^^^
ValueError: list.remove(x): x not in list
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 203, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/megalinter/run.py", line 22, in <module>
    main()
    ~~~~^^
  File "/megalinter/run.py", line 17, in main
    linter.run()
    ~~~~~~~~~~^^
  File "/megalinter/MegaLinter.py", line 292, in run
    self.process_linters_parallel(self.active_linters, linters_do_fixes)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/megalinter/MegaLinter.py", line 462, in process_linters_parallel
    updated_linters = pool_result.get()
  File "/usr/local/lib/python3.14/multiprocessing/pool.py", line 774, in get
    raise self._value
ValueError: list.remove(x): x not in list

[llaville]

Issue probably introduced by #7150 : @bdovaz can you confirm ?

[llaville]

Quick summary results of what I get while running v10 on PHP Linters

⚠️ Linted [PHP] files with [phpstan]: Found 1 non blocking error(s) and 0 non blocking warning(s) - (300.2s)
- Using [phpstan v2.2.9] https://megalinter.io/10.1.0/descriptors/php_phpstan
- MegaLinter key: [PHP_PHPSTAN]
- Rules config: [/.github/linters/phpstan.neon.dist]
- CWD: [/tmp/lint]
- Command: [phpstan analyse --no-progress --no-ansi --memory-limit 1G -c /tmp/lint/.github/linters/phpstan.neon.dist .]
--Error detail:
[phpstan] Timed out after 300 seconds and was killed (LINTER_TIMEOUT_SECONDS (default)=300). Partial output above if any. If this repository legitimately needs more time, increase the value with PHP_PHPSTAN_TIMEOUT_SECONDS or LINTER_TIMEOUT_SECONDS (0 disables the timeout); a timeout with very low CPU usage usually means an I/O stall (e.g. workspace bind-mounted from Windows into a WSL2-backed container engine).

❌ Linted [PHP] files with [phplint]: Found 1 error(s) and 0 warning(s) - (4.85s)
- Using [phplint v9.7.2] https://megalinter.io/10.1.0/descriptors/php_phplint
- MegaLinter key: [PHP_PHPLINT]
- Rules config: [/.github/linters/.phplint.yml]
- CWD: [/tmp/lint]
- Command: [phplint --no-cache -c /tmp/lint/.github/linters/.phplint.yml --exclude .changes --exclude .git --exclude megalinter-reports .]
[Excluded directories] Forwarded EXCLUDED_DIRECTORIES to phplint through --exclude (disable with PHP_PHPLINT_FORWARD_EXCLUDED_DIRECTORIES: false)
--Error detail:
PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 8192 bytes) in /usr/local/composer/vendor/overtrue/phplint/src/Cache.php on line 172

❌ Linted [PHP] files with [phpcs]: Found 162390 error(s) and 162390 warning(s) - (714.31s)
- Using [phpcs v4.0.4] https://megalinter.io/10.1.0/descriptors/php_phpcs
- MegaLinter key: [PHP_PHPCS]
- Rules config: [/.github/linters/.phpcs.xml.dist]
- CWD: [/tmp/lint]
- Command: [phpcs -d memory_limit=1G --standard=/tmp/lint/.github/linters/.phpcs.xml.dist --ignore=*/.changes/*,*/.git/*,*/megalinter-reports/* .]
[Excluded directories] Forwarded EXCLUDED_DIRECTORIES to phpcs through --ignore= (disable with PHP_PHPCS_FORWARD_EXCLUDED_DIRECTORIES: false)

[bdovaz]

Issue probably introduced by #7150 : @bdovaz can you confirm ?

Well, to be honest, I'm not sure, but in my PR, 95% of the changes are to descriptors and tests. I didn't have to change anything in the core other than Linter.py (to add the new code, not modify the existing one) and very little else.

Could it be one of @nvuillam's PRs?