Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ Contributors add user-facing entries under `[Unreleased]` in the same PR. Mainta

- **Docs:** Document `issuer.org` design-ownership policy; align ARPA-driven registry skills (`prompt_injection_firewall`, `bg_remover`, `novelty_extractor`) and catalog Issuer lines (#295).

### Fixed

- **Tests:** Isolate `pytest tests/` from the operator's global `config.yaml` via autouse `SKILLWARE_CONFIG_DIR` in `tests/conftest.py`; add configured-mode discovery and loader coverage alongside legacy-order tests (#302).

## [0.5.2] - 2026-08-27

### Added
Expand Down
10 changes: 10 additions & 0 deletions docs/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Tests fall into four layers: **bundle**, **framework**, **maintainer**, and **ex
| Optional extras sync (`scripts/sync_extras.py`, `tests/test_extras_sync.py`) | Done |
| Card UI schema vs execute output (`tests/test_card_ui_schema.py`) | Done |
| Local-execute example smoke tests in CI (`tests/test_examples_smoke.py`) | Done |
| Framework tests isolated from operator global config (`tests/conftest.py`, #302) | Done |

Every pull request runs `black --check`, `flake8`, `pytest skills/`, `pytest tests/`, and a **wheel-smoke** job that builds a wheel, installs it in a fresh venv (base install only — no `[all]` or per-skill extras), and verifies every bundled registry skill is present and loadable. Bundle tests gate merge the same as framework and maintainer tests.

Expand Down Expand Up @@ -69,6 +70,7 @@ pip install -r requirements.txt
- `tests/test_registry_docs.py` enforces doc-drift parity: skill catalog index matches manifests, examples README matches scripts on disk, and agent-loops.md references every registered skill.
- `tests/test_registry_identity.py` enforces manifest identity parity: every registry-layout skill's `manifest.name` matches its path-derived registry ID, and all manifest names are globally unique (#280).
- `tests/test_examples_smoke.py` provides an automated regression net for local-execute demo scripts under `examples/` without making network requests or requiring API keys (#237).
- `tests/conftest.py` isolates every test from the operator's real global `config.yaml` via `SKILLWARE_CONFIG_DIR` so local `pytest tests/` matches CI even after CLI mail/config init (#302). Legacy vs configured discovery order is covered in `tests/test_discovery.py` and `tests/test_loader.py`.
- Lives at the **root of `tests/`** only (`tests/test_loader.py`, `tests/test_cli.py`, …).
- Clone-repo only; runs in CI via `pytest tests/` together with maintainer tests below.

Expand Down Expand Up @@ -217,6 +219,14 @@ python -m pytest tests/skills/<category>/test_<skill_name>.py

Pytest is configured to collect from `tests/` and `skills/` only (`examples/` is ignored). See `[tool.pytest.ini_options]` in `pyproject.toml`.

### Operator global config and pytest (#302)

After normal CLI setup (for example `skillware mail signature init`), a user-level `config.yaml` may exist under your Skillware config directory. That switches skill discovery to **configured** mode (`project → external → bundled`) instead of **legacy** mode (`SKILLWARE_SKILL_PATH → cwd ./skills/ → bundled`).

Framework tests must not depend on your machine's operator config. An autouse fixture in `tests/conftest.py` points `SKILLWARE_CONFIG_DIR` at an empty temporary directory for every test run, so `pytest tests/` matches CI on a clean home directory.

If you add tests that exercise merged YAML behavior, write explicit project or global config files under `tmp_path` and call `clear_config_cache()` after changes — the autouse fixture already isolates the global layer.

### Writing tests

- **Bundle test:** `skills/<category>/<name>/test_skill.py` — required for new skills; copy from `templates/python_skill/test_skill.py`.
Expand Down
2 changes: 2 additions & 0 deletions docs/contributing/ai_native_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ pytest tests/test_registry_docs.py

These checks are part of `pytest tests/` and will run in CI regardless, but an early local run saves a round-trip.

Framework tests are isolated from your operator global `config.yaml` automatically (`tests/conftest.py`, #302). A local full suite should pass even after `skillware mail signature init`; CI remains authoritative.

Before Stage 5, scan your diff for:

- Unrelated files
Expand Down
19 changes: 19 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,25 @@
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))


@pytest.fixture(autouse=True)
def isolate_skillware_config(monkeypatch, tmp_path):
"""
Point global config at an empty temp directory for every test (#302).

Without this, a developer's real ~/.config/skillware/config.yaml (for example
after ``skillware mail signature init``) switches discovery to configured
mode and breaks legacy-order assertions in discovery/loader tests.
"""
from skillware.core.config import GLOBAL_CONFIG_DIR_ENV, clear_config_cache

isolated = tmp_path / "skillware-global-config"
isolated.mkdir()
monkeypatch.setenv(GLOBAL_CONFIG_DIR_ENV, str(isolated))
clear_config_cache()
yield
clear_config_cache()


@pytest.fixture
def mock_anthropic():
"""Mocks the Anthropic client."""
Expand Down
32 changes: 31 additions & 1 deletion tests/test_discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ def _write_registry_skill(root: Path, category: str, name: str) -> None:
)


def test_get_skill_roots_order_env_project_bundled(tmp_path, monkeypatch):
def test_get_skill_roots_order_env_project_bundled_legacy(tmp_path, monkeypatch):
"""Legacy mode (no YAML config): external → project → bundled."""
env_root = tmp_path / "external"
env_root.mkdir()
project_root = tmp_path / "project" / "skills"
Expand All @@ -49,6 +50,35 @@ def test_get_skill_roots_order_env_project_bundled(tmp_path, monkeypatch):
assert roots[-1].path == bundled_skills_root()


def test_get_skill_roots_order_project_external_bundled_configured(
tmp_path, monkeypatch
):
"""Configured mode (#246 default order): project → external → bundled."""
from skillware.core.config import PROJECT_CONFIG_FILENAME, clear_config_cache

env_root = tmp_path / "external"
env_root.mkdir()
project_dir = tmp_path / "project"
project_skills = project_dir / "skills"
project_skills.mkdir(parents=True)
(project_dir / PROJECT_CONFIG_FILENAME).write_text(
"paths:\n project: auto\n"
"resolution:\n order:\n - project\n - external\n - bundled\n",
encoding="utf-8",
)
monkeypatch.chdir(project_dir)
monkeypatch.setenv(SKILLWARE_SKILL_PATH_ENV, str(env_root))
clear_config_cache()

roots = get_skill_roots()
tiers = [root.tier for root in roots]

assert tiers[0] == SkillRootTier.PROJECT
assert tiers[1] == SkillRootTier.EXTERNAL
assert tiers[-1] == SkillRootTier.BUNDLED
assert roots[-1].path == bundled_skills_root()


def test_get_skill_roots_override_single_root(tmp_path):
override = tmp_path / "only"
override.mkdir()
Expand Down
50 changes: 49 additions & 1 deletion tests/test_loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,8 @@ def test_resolve_skill_from_env_path(tmp_path, monkeypatch):
assert bundle["registry_id"] is None


def test_resolve_skill_prefers_env_over_cwd(tmp_path, monkeypatch):
def test_resolve_skill_prefers_env_over_cwd_legacy(tmp_path, monkeypatch):
"""Legacy mode (no YAML config): SKILLWARE_SKILL_PATH wins over cwd ./skills/."""
env_root = tmp_path / "env_root"
env_skill = env_root / "shared_id"
env_skill.mkdir(parents=True)
Expand Down Expand Up @@ -398,6 +399,53 @@ def test_resolve_skill_prefers_env_over_cwd(tmp_path, monkeypatch):
assert bundle["manifest"]["name"] == "from_env"


def test_resolve_skill_prefers_cwd_over_env_in_configured_mode(tmp_path, monkeypatch):
"""Configured mode (#246 default order): project ./skills/ wins over env path."""
from skillware.core.config import PROJECT_CONFIG_FILENAME, clear_config_cache

env_root = tmp_path / "env_root"
env_skill = env_root / "shared_id"
env_skill.mkdir(parents=True)
(env_skill / "manifest.yaml").write_text(
"name: from_env\nversion: 0.1.0\ndescription: test\n"
"parameters:\n type: object\n properties: {}\n",
encoding="utf-8",
)
(env_skill / "skill.py").write_text(
"from skillware.core.base_skill import BaseSkill\n"
"class EnvSkill(BaseSkill):\n"
" def execute(self, **kwargs):\n"
" return {'source': 'env'}\n",
encoding="utf-8",
)

project_dir = tmp_path / "repo"
cwd_skill = project_dir / "skills" / "shared_id"
cwd_skill.mkdir(parents=True)
(cwd_skill / "manifest.yaml").write_text(
"name: from_cwd\nversion: 0.1.0\ndescription: test\n"
"parameters:\n type: object\n properties: {}\n",
encoding="utf-8",
)
(cwd_skill / "skill.py").write_text(
"from skillware.core.base_skill import BaseSkill\n"
"class CwdSkill(BaseSkill):\n"
" def execute(self, **kwargs):\n"
" return {'source': 'cwd'}\n",
encoding="utf-8",
)
(project_dir / PROJECT_CONFIG_FILENAME).write_text(
"paths:\n project: auto\n",
encoding="utf-8",
)

monkeypatch.setenv(SKILLWARE_SKILL_PATH_ENV, str(env_root))
monkeypatch.chdir(project_dir)
clear_config_cache()
bundle = SkillLoader.load_skill("shared_id")
assert bundle["manifest"]["name"] == "from_cwd"


def test_wheel_includes_skill_manifest(tmp_path):
wheel_dir = tmp_path / "wheels"
wheel_dir.mkdir()
Expand Down
Loading