Integration tests on DaprClient responses#981
Conversation
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #981 +/- ##
==========================================
- Coverage 86.63% 80.53% -6.10%
==========================================
Files 84 136 +52
Lines 4473 12678 +8205
==========================================
+ Hits 3875 10210 +6335
- Misses 598 2468 +1870 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR migrates example validation away from mechanical-markdown by introducing two pytest suites: (1) stdout-based “documentation” tests that run the examples and assert expected output, and (2) SDK-based integration tests that assert directly on DaprClient responses.
Changes:
- Replace mechanical-markdown validation with
pytestintox -e examples(stdout assertions). - Add
tox -e integrationsuite with a Dapr sidecar test harness and SDK-level assertions. - Update CI workflow and contributor docs to reflect the new testing approach.
Reviewed changes
Copilot reviewed 49 out of 49 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| tox.ini | Switch examples validation to pytest; add integration env |
| .github/workflows/validate_examples.yaml | Run both examples + integration in CI |
| pyproject.toml | Add pytest marker registration |
| examples/validate.sh | Remove mechanical-markdown entrypoint |
| README.md | Update developer instructions (examples/integration) |
| AGENTS.md | Update guidance for example validation workflow |
| examples/AGENTS.md | Update example validation documentation |
| CLAUDE.md | Add repo coding/style guidance |
| examples/conversation/real_llm_providers_example.py | Update output text |
| tests/clients/test_conversation_helpers.py | Docstring wording tweak |
| tests/examples/conftest.py | Add DaprRunner helper + example_dir fixture |
| tests/examples/test_configuration.py | Stdout-based example test |
| tests/examples/test_conversation.py | Stdout-based example test |
| tests/examples/test_crypto.py | Stdout-based example test |
| tests/examples/test_demo_actor.py | Stdout-based example test |
| tests/examples/test_distributed_lock.py | Stdout-based example test |
| tests/examples/test_error_handling.py | Stdout-based example test |
| tests/examples/test_grpc_proxying.py | Stdout-based example test |
| tests/examples/test_invoke_binding.py | Stdout-based example test |
| tests/examples/test_invoke_custom_data.py | Stdout-based example test |
| tests/examples/test_invoke_http.py | Stdout-based example test |
| tests/examples/test_invoke_simple.py | Stdout-based example test |
| tests/examples/test_jobs.py | Stdout-based example test |
| tests/examples/test_langgraph_checkpointer.py | Stdout-based example test |
| tests/examples/test_metadata.py | Stdout-based example test |
| tests/examples/test_pubsub_simple.py | Stdout-based example test |
| tests/examples/test_pubsub_streaming.py | Stdout-based example test |
| tests/examples/test_pubsub_streaming_async.py | Stdout-based example test |
| tests/examples/test_secret_store.py | Stdout-based example test |
| tests/examples/test_state_store.py | Stdout-based example test |
| tests/examples/test_state_store_query.py | Stdout-based example test |
| tests/examples/test_w3c_tracing.py | Stdout-based example test |
| tests/examples/test_workflow.py | Stdout-based example test |
| tests/integration/conftest.py | Add DaprTestEnvironment harness for SDK tests |
| tests/integration/secrets.json | Local secret store test data |
| tests/integration/test_configuration.py | SDK-based configuration tests |
| tests/integration/test_distributed_lock.py | SDK-based lock tests |
| tests/integration/test_invoke.py | SDK-based invocation tests |
| tests/integration/test_metadata.py | SDK-based metadata tests |
| tests/integration/test_pubsub.py | SDK-based pub/sub delivery tests |
| tests/integration/test_secret_store.py | SDK-based secret store tests |
| tests/integration/test_state_store.py | SDK-based state store tests |
| tests/integration/apps/invoke_receiver.py | gRPC app for invoke tests |
| tests/integration/apps/pubsub_subscriber.py | Subscriber app persisting received events |
| tests/integration/components/configurationstore.yaml | Redis configuration component |
| tests/integration/components/localsecretstore.yaml | Local-file secret store component |
| tests/integration/components/lockstore.yaml | Redis lock component |
| tests/integration/components/pubsub.yaml | Redis pubsub component |
| tests/integration/components/statestore.yaml | Redis state component |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 48 out of 48 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 48 out of 48 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| commands_pre = | ||
| pip uninstall -y dapr dapr-ext-grpc dapr-ext-fastapi dapr-ext-langgraph dapr-ext-strands dapr-ext-flask dapr-ext-langgraph dapr-ext-strands | ||
| pip install -e {toxinidir}/ \ | ||
| pip install -r {toxinidir}/dev-requirements.txt \ |
There was a problem hiding this comment.
The pip uninstall command lists dapr-ext-langgraph and dapr-ext-strands twice. This is harmless but noisy; consider removing the duplicates to keep the tox env setup clear and easier to maintain.
| # Point the SDK health check at the actual sidecar HTTP port. | ||
| # DaprHealth.wait_for_sidecar() reads settings.DAPR_HTTP_PORT, which | ||
| # is initialized once at import time and won't reflect a non-default | ||
| # http_port unless we update it here. | ||
| settings.DAPR_HTTP_PORT = http_port | ||
|
|
||
| client = DaprClient(address=f'127.0.0.1:{grpc_port}') | ||
| self._clients.append(client) | ||
| return client |
There was a problem hiding this comment.
start_sidecar() mutates the global settings.DAPR_HTTP_PORT to point the SDK health check at the chosen sidecar. Because this is global process state, it can leak across tests/modules; consider storing the previous value and restoring it in cleanup() (or using a context manager) to keep tests isolated.
Description
Tests the gRPC channel created by
DaprClientdirectly, instead of asserting on the log outputs of the examples like in the PR that removes Mechanical Markdown(#977).The old log assertions are kept as "documentation tests".
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #972
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: