feat: add FunASR speech recognition backend

[LauraGPT]

Summary

Adds FunASR as a new speech recognition backend for LocalAI.

FunASR is an industrial-grade ASR toolkit (16K+ GitHub stars) from Alibaba DAMO Academy, featuring:

• SenseVoice: 170x realtime speed, emotion & audio event detection, 50+ languages
• Paraformer: Highest accuracy for Chinese ASR, non-autoregressive
• Built-in VAD, punctuation restoration, and speaker diarization

Changes

• backend/python/funasr/ — New Python gRPC backend wrapping FunASR's AutoModel API
• backend/index.yaml — Meta definition and image entries for cpu/cuda12/cuda13/rocm/metal/l4t platforms

Usage

name: funasr-sensevoice
backend: funasr
parameters:
  model: iic/SenseVoiceSmall

The backend supports the standard /v1/audio/transcriptions endpoint with optional language parameter.

Test plan

• Verify pip install funasr installs cleanly in the backend venv
• Test SenseVoice model loading and transcription via gRPC
• Test with CPU and CUDA backends
• Verify existing backends are unaffected (index.yaml meta anchor is additive)

[LauraGPT]

Hi @mudler, friendly ping — this PR adds FunASR as a speech recognition backend for LocalAI. Ready for review whenever convenient. Thanks!

[LauraGPT]

Hi @mudler, friendly follow-up — this PR adds FunASR as a speech recognition backend for LocalAI. FunASR recently added an agent-friendly CLI (funasr audio.wav --output-format json) and supports OpenAI-compatible API. Ready for review. Thanks!

[LauraGPT]

Rebased onto the latest master (resolved the backend/index.yaml conflict — the FunASR and supertonic entries now coexist) and signed off the commit, so DCO passes and the PR is mergeable again. Ready for review whenever you have a chance, @mudler — thanks! 🙏

[LauraGPT]

Rebased this PR onto the latest master and resolved the backend/index.yaml conflict again. The upstream supertonic Metal entries and the FunASR backend entries now coexist in the registry.

Current state:

• PR is mergeable again
• DCO is passing on the refreshed commit
• changed files are still limited to the FunASR backend and backend/index.yaml

Local checks run:

ruby -e 'require "yaml"; data=YAML.load_file("backend/index.yaml"); names=data.select { |x| x.is_a?(Hash) }.map { |x| x["name"] }; abort("missing funasr") unless names.include?("funasr"); abort("missing cpu-funasr") unless names.include?("cpu-funasr"); abort("missing metal-supertonic") unless names.include?("metal-supertonic"); puts "index ok: #{data.length} entries"'
python3 -m py_compile backend/python/funasr/backend.py
rg -n "<<<<<<<|=======|>>>>>>>" backend/index.yaml backend/python/funasr || true
git diff --check upstream/master...HEAD

[LauraGPT]

Quick maintainer-side validation pass after the latest rebase:

• Checked out 8105d38293320388ca18773bffca9507432b7e39 locally.
• git diff --check origin/master...HEAD passes.
• python3 -m py_compile backend/python/funasr/backend.py passes.
• bash -n passes for backend/python/funasr/install.sh, run.sh, and test.sh.
• Parsed all backend/python/funasr/requirements*.txt package lines; pip option lines such as --extra-index-url are present where expected.
• Parsed backend/index.yaml with PyYAML and confirmed the expected 15 FunASR backend names are present, including CPU, CUDA 12/13, ROCm, L4T, and Metal variants.
• make -n -C backend/python/funasr resolves the default target to bash install.sh as expected.

I did not run a full dependency install or model transcription in this bare ops environment, but the registry wiring and backend entrypoint syntax look clean from this pass.

[LauraGPT]

One caveat from a deeper pass so the validation note above is not read as a ready-to-merge sign-off: backend/python/funasr/test.sh currently runs python -m pytest -x -s test.py, but backend/python/funasr/test.py is not part of this PR.

So the registry wiring, syntax, and requirement-file checks are clean, but the make test path still needs either a small backend smoke test or an updated/removed test target before I would call this fully ready from the FunASR side.

[LauraGPT]

Follow-up on the test caveat I noted earlier: I pushed signed head f258f5f4 to add backend/python/funasr/test.py.

What it covers without downloading FunASR weights:

• Health() returns OK.
• LoadModel() uses the default iic/SenseVoiceSmall, fsmn-vad, CPU fallback, and disable_update=True.
• AudioTranscription() forwards a trimmed language parameter to model.generate() and builds transcript text/segments.
• Missing audio input returns an empty transcript without calling the model.

Validation on the refreshed branch:

python3 -m pytest -x -s backend/python/funasr/test.py
# 4 passed
python3 -m py_compile backend/python/funasr/backend.py backend/python/funasr/test.py
bash -n backend/python/funasr/install.sh backend/python/funasr/run.sh backend/python/funasr/test.sh
git diff --check

Note: running bash backend/python/funasr/test.sh directly on my remote ops host still stops before pytest because that host does not have uv installed (../common/libbackend.sh: uv: command not found). The test target itself is now present and passes when invoked with pytest.

[LauraGPT]

Rebased this PR onto current master (d521608e), so it now includes the upstream transcription resolver fix from #10731 (cd65a1f6), including the language / translate handling added there.

Current head is bc412813 on LauraGPT:feat/funasr-backend.

Validation run locally on the rebased branch:

• python3 -m pytest -x -s backend/python/funasr/test.py -> 4 passed
• python3 -m py_compile backend/python/funasr/backend.py backend/python/funasr/test.py
• bash -n backend/python/funasr/install.sh backend/python/funasr/run.sh backend/python/funasr/test.sh
• git diff --check upstream/master...HEAD

GitHub DCO is passing on the new head.

[LauraGPT]

I pulled the failing fork-side tests job log for current head bc412813 (LauraGPT/LocalAI, Actions job 85751160435) so this failure is not read as a FunASR backend regression.

The job reaches the CodeQL/code-scanning upload step and fails because the fork token cannot access the CodeQL API endpoints:

GITHUB_TOKEN Permissions:
  Contents: read
  Metadata: read
  Packages: read
...
This run of the CodeQL Action does not have permission to access the CodeQL Action API endpoints. This could be because the Action is running on a pull request from a fork.
...
Resource not accessible by integration - https://docs.github.com/rest

So the visible failing tests check on the fork is a fork-permission/code-scanning upload gate, not a failure in backend/python/funasr. The upstream DCO check on mudler/LocalAI is still passing for the same head.

For the FunASR backend itself, the latest local validation remains:

python3 -m pytest -x -s backend/python/funasr/test.py  # 4 passed
python3 -m py_compile backend/python/funasr/backend.py backend/python/funasr/test.py
bash -n backend/python/funasr/install.sh backend/python/funasr/run.sh backend/python/funasr/test.sh
git diff --check upstream/master...HEAD