Skip to content

docs(testing): align validation guide with current frameworks - #520

Open
omribz156 wants to merge 7 commits into
microsoft:mainfrom
omribz156:docs/update-testing-validation-frameworks
Open

docs(testing): align validation guide with current frameworks#520
omribz156 wants to merge 7 commits into
microsoft:mainfrom
omribz156:docs/update-testing-validation-frameworks

Conversation

@omribz156

@omribz156 omribz156 commented May 15, 2026

Copy link
Copy Markdown

Pull Request

IMPORTANT: Before submitting, please remove all sensitive data, secrets, tokens, or confidential information. Ensure you've redacted any NDA-covered information, IP addresses, resource names, or security-related details that shouldn't be publicly disclosed.

Description

Updates docs/contributing/testing-validation.md so the testing guide reflects the current repository layout and tooling. The previous guide still referenced Terratest / go test examples, while the repo now uses native Terraform .tftest.hcl tests, Rust crate tests, and Docusaurus docs checks.

Related Issue

Fixes #144

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Blueprint modification or addition
  • Component modification or addition
  • Documentation update
  • CI/CD pipeline change
  • Other (please describe):

Implementation Details

  • Replaced stale Terratest / go test guidance with native terraform test examples.
  • Pointed Terraform examples at the existing .tftest.hcl layout under src/000-cloud/000-resource-group/terraform/tests/.
  • Added current Rust crate-level cargo test guidance.
  • Added Docusaurus Jest / TypeScript docs test commands.

Testing Performed

  • Terraform plan/apply
  • Blueprint deployment test
  • Unit tests
  • Integration tests
  • Bug fix includes regression test (see Test Policy)
  • Manual validation
  • Other: markdown, spelling, and diff checks listed below

Validation Steps

Reviewers can validate this documentation-only change with:

npx markdownlint --config .markdownlint.json docs/contributing/testing-validation.md
npx cspell docs/contributing/testing-validation.md --config .cspell.json
git diff --check

Checklist

  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have run terraform fmt on all Terraform code
  • I have run terraform validate on all Terraform code
  • I have run az bicep format on all Bicep code
  • I have run az bicep build to validate all Bicep code
  • I have checked for any sensitive data/tokens that should not be committed
  • Lint checks pass (run applicable linters for changed file types)

Security Review

  • No credentials, secrets, or tokens are hardcoded or logged
  • RBAC and identity changes follow least-privilege principles
  • No new network exposure or public endpoints introduced without justification
  • Dependency additions or updates have been reviewed for known vulnerabilities
  • Container image changes use pinned digests or SHA references

Additional Notes

This was prepared with Codex assistance, with the final diff reviewed before opening.

Screenshots (if applicable)

N/A; documentation-only change.

@omribz156
omribz156 marked this pull request as ready for review May 15, 2026 15:55
@omribz156
omribz156 requested a review from a team as a code owner May 15, 2026 15:55
@omribz156

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for cleaning up this guide and replacing the stale Terratest examples with native terraform test guidance.

The PR description does not follow .github/PULL_REQUEST_TEMPLATE.md. Please update it to use the required sections: Description, Related Issue (with Fixes #144), Type of Change, Implementation Details, Testing Performed, Validation Steps, Checklist, and Security Review.

Comment thread docs/contributing/testing-validation.md Outdated
Comment thread docs/contributing/testing-validation.md Outdated
Comment thread docs/contributing/testing-validation.md Outdated
Comment thread docs/contributing/testing-validation.md Outdated
Comment thread docs/contributing/testing-validation.md Outdated
@omribz156

Copy link
Copy Markdown
Author

Thanks, that makes sense. I updated the PR description to follow .github/PULL_REQUEST_TEMPLATE.md, including the requested related issue, change type, implementation details, testing, validation, checklist, and security review sections.

No code/docs diff changed in this follow-up; only the PR body was updated.

This follow-up was Codex-assisted, with the final text reviewed before posting.

@bindsi Marcel Bindseil (bindsi) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Automated review: Documentation update replacing stale Terratest references with native terraform test guidance. Content aligns with the repo's actual test patterns (command=plan, tests/ directory, setup modules). No functional issues found.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omribz156 please update the PR to revert/preserve the blueprint testing section (keep the Go/Terratest content and the full-single-node-cluster/tests/ + 904-test-utilities/ references) while keeping all the component-level native-test improvements. Once the blueprint section is preserved, we can approve and merge.

This change is on us, the issue was outdated but has a new section for you to take into account. Please let us know if you are able to take it, or would prefer one of us to pick it up.

Thank you!

Comment thread docs/contributing/testing-validation.md
@omribz156
omribz156 force-pushed the docs/update-testing-validation-frameworks branch from b2b687d to 8eb7c25 Compare June 13, 2026 19:17
@omribz156

Copy link
Copy Markdown
Author

Thanks, fixed in 8eb7c25.

I restored the Blueprint Testing section to preserve the Go/Terratest guidance, blueprints/full-single-node-cluster/tests/, and src/900-tools-utilities/904-test-utilities/ references. I also split the top requirements table so component Terraform remains native terraform test, while blueprint IaC is explicitly Go/Terratest.

Verified with:

  • npx -y markdownlint-cli --config .markdownlint.json docs/contributing/testing-validation.md
  • npx cspell docs/contributing/testing-validation.md --config .cspell.json
  • git diff --check

@bindsi Marcel Bindseil (bindsi) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: the blueprint testing concern is addressed while preserving native component testing guidance. No actionable issues found.

Comment thread docs/contributing/testing-validation.md

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution and for the patience you have shown through this review process. I apologize for the slow turnaround and will check in on your next follow-up sooner.

Comment thread docs/contributing/testing-validation.md Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omribz156 thank you so much for addressing all changes. As there was no new request for review, I believe I missed everything had been addressed.

Looks good to me, we just require one markdown fix, can you please run npm run format-tables to fix a table alignment issue, which will cause CI to fail.

Thanks!

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

📚 Documentation Health Report

Generated on: 2026-09-01 13:59:29 UTC

📈 Documentation Statistics

Category File Count
Main Documentation 222
Infrastructure Components 230
Blueprints 40
GitHub Resources 26
AI Assistant Guides (Copilot) 17
Total 535

🏗️ Three-Tree Architecture Status

  • ✅ Bicep Documentation Tree: Auto-generated navigation
  • ✅ Terraform Documentation Tree: Auto-generated navigation
  • ✅ README Documentation Tree: Manual README organization

🔍 Quality Metrics

  • Frontmatter Validation:
    success
  • Link Validation: success

This report is automatically generated by the Documentation Automation workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs(testing): update testing-validation.md to reflect actual test frameworks

4 participants