Refactor Python code to comply with Debian Python Policy

[HappyBasher]

Python code and package structure should be refactored to comply with Debian Python Policy. Better code quality and mantainability should be achieved.
Related branch: https://github.com/linuxmuster/linuxmuster-base7/tree/python-refactoring
Refactoring is being implemented with the help of Claude AI.

[HappyBasher]

Complete Commit List for Python Refactoring Branch

This branch contains 106 commits with comprehensive Python refactoring work. All commits are now tracked under this issue.

Phase 1: Debian Python Policy Compliance (Commits 1-30)

• 2c7c3ef Refactor Python code to comply with Debian Python Policy (Phase 1)
• 6ed9b78 Add linuxmuster-common as build dependency
• cbe0f8c Add Uploaders field to debian/control
• 0e66ff9 Add python3-paramiko to Build-Depends
• 0730f14 Add python3-urllib3 to Build-Depends and Depends
• 1c009cf Fix linuxmuster-setup to load modules from Python package
• e1adb19 Add comprehensive test suite for linuxmuster-setup CLI
• b9c90ee Add integration test suite with snapshot/restore capability
• c951d63 Add snapshot deletion options to integration test suite
• 5fa11e5 Fix integration test execution and improve output
• 2f550b1 Fix test execution and improve progress output
• 76079d1 fix integration test execution issues
• 67a6692 add extensive debug output to integration tests
• 7ab7cc7 fix test execution by removing set -e
• a794c1e increase test timeout and improve success criteria
• e287374 add live output during setup execution
• d1c6b9a remove problematic set -e/+e toggles in test functions
• 0b72447 update imports in share/ scripts to use new package structure
• 99ca1bc add create-testusers test to integration test suite
• 0f309e0 restructure integration tests to run user creation after each setup test.
• 019fd50 improve interactive mode with back navigation in submenus.
• 6a40090 update tests README with comprehensive documentation and comparison.
• a3a7bfd remove deprecated readmes.
• e649b8e increase compat to 10.
• ddfeeb8 update all sbin scripts to use new package imports.
• bff4cc4 update debian changelog with comprehensive list of Phase 1 changes.
• 13d3af4 update changelog.
• 51d7fed Fix Python shebangs to comply with Debian Python Policy
• 1c0e9a8 Remove old lib/ and sbin/ directories
• b5c6649 Move dhcpd-update-samba-dns to share/ and update DHCP template

Security Fixes (Commits 31-40)

• dcbddc6 dhcpd.events.conf: update version.
• 8211c6d Improve code quality: Fix exception handling and remove os.system()
• 7f74178 Eliminate shell injection vulnerabilities: Replace subProc() with secure subprocess calls
• 757c974 create-testusers.py: replace subProc() with subprocess.run().
• dc3c2b1 init.py: replace subProc() with subprocess.run().
• 0795b52 Fix background process execution in i_linbo.py
• bf71428 Add /var/lib/samba to test snapshot paths
• b081928 Fix run_with_log() call signature in m_firewall.py
• a9080eb Fix subprocess.run() command format in import_devices.py
• 30335a9 Replace os.popen() and fix subprocess.run() in create-testusers.py

CLI Wrapper Fixes & Improvements (Commits 41-50)

• df3aa75 Add logging to create-testusers.py
• cc8ed08 Add comprehensive service restarts after snapshot restore
• 471b570 Move linuxmuster scripts from /usr/bin to /usr/sbin
• ffde413 Fix update_ntpconf.py: Replace broken wrapper with actual implementation
• 5b5a126 Fix holiday.py: Replace broken wrapper with actual implementation
• 14974e6 Fix remaining 4 CLI wrappers: Replace broken wrappers with actual implementations
• e1ca380 Fix snapshot restore: Skip restore for test_create_testusers
• 49c16f3 test_setup_integration.sh: check for already configured system.
• 02dc2ca Add CLI options for single test execution and restore control
• 32e0f81 Refactor: Extract service restart into reusable function

Test Infrastructure (Commits 51-60)

• 4705d5e test_setup_integration.sh: Add support for running multiple tests
• 97d1bb7 test_setup_integration.sh: Refactor snapshot/restore logic for test execution
• 9cc3fb2 test_setup_integration.sh: Allow -r option to work standalone
• 0f24b34 tests/README.md: Update documentation for new snapshot/restore behavior
• 2b65a37 import_devices.py: Add comprehensive logging to import-devices.log
• 9b4c0f8 docs: Update refactoring-phase-2.md with all recent changes
• 0345522 Code cleanup: Syntax corrections and code style improvements
• 57bdb69 import_subnets.py: fix beautifulsoup's xml features.
• c4d9abd opnsense_reset.py: Replace subProc() with subprocess.run()
• 30449b1 renew_certs.py: Replace subProc() with subprocess.run()

Module Import Fixes (Commits 61-70)

• 841c9b3 Fix opnsense_reset.py: Use importlib instead of subprocess for m_firewall
• 209fc98 docs: Add section 12 about fixing module import path in opnsense_reset.py
• b49fef2 Fix opnsense_reset.py: Correct return code logic for keytab operations
• a6e675d Remove duplicate firewall scripts from src/
• b46c9e3 Fix critical syntax error in k_samba-users.py
• 7700241 Phase 2: Create helpers.py and eliminate code duplication
• 2660143 Phase 3: Refactor d_templates, g_ssl, h_ssh to use helpers.py
• 9441652 Phase 3: Refactor j_samba-provisioning, k_samba-users, z_final to use helpers.py
• 7e0a406 Phase 3: Refactor create-testusers and m_firewall to use helpers.py
• 4f4db06 Phase 3: Refactor m_firewall.py - decompose long functions

Critical Security Fixes (Commits 71-80)

• 8cc0404 Fix: Prevent helpers.py from being executed during setup
• ba794b3 Add comprehensive docstrings and comments to all setup modules
• ca9226b Fix critical shell injection vulnerability in catFiles()
• 183cd32 Fix critical password leak in opnsense_reset.py
• 2bf5c68 Fix shell injection vulnerabilities in SSH functions
• d26eb58 Refactor renew_certs.py: Extract 308-line main() into CertificateRenewer class
• 87ad936 Create BaseCLI class to eliminate CLI code duplication
• e7353cc Create unified logging module to consolidate logging implementations
• e4c9cfb setup ntpsec fix.
• 8d33eaa Refactor getDevicesArray() into focused single-responsibility functions

Setup Fixes & Certificate Refactoring (Commits 81-90)

• e3c2d0b setup keyfile fix.
• 48b1c44 setup sshExec output fix.
• c96e65b test_setup_integration.sh check_configured fix.
• d98907a Refactor renew_certs.py: camelCase method names and consolidate with g_ssl.py
• 6bf526c Extract common certificate operations into shared functions
• ae640f2 Refactor g_ssl.py to use shared encodeCertToBase64() function
• 5b2f6ac Fix holiday_generate.py date parsing to handle both API formats
• a71914a Refactor import_devices.py: improve code quality and maintainability
• e03b9f9 Refactor import_subnets.py: introduce constants.
• 225a801 Refactor import_subnets.py for improved maintainability

CLI Script Refactoring (Commits 91-100)

• 91fea75 Add comprehensive docstrings and comments to modini.py
• 00360c8 Refactor opnsense_reset.py - split main() into helper functions
• 922c938 Refactor renew_certs.py - add comprehensive inline comments
• a72fc66 firewall setup interfaces fix.
• 1b00f18 Refactor import_devices.py: improve code quality and maintainability
• 36d9eda import_subnets.py gateway fix.
• 8890ab2 update changelog.
• 97c7755 fix linuxmuster.net version in setup dialog.
• 06d97d8 test_setup_integration.sh: add ability to run all tests with or without firewall setup (option -s).
• f04058f don't create keytab if firewall setup is skipped.

Final Improvements & Documentation (Commits 101-106)

• 8e8861f disable isc-dhcp-server6.service during setup.
• a909989 Refactor z_final.py: improve code quality and fix critical bug
• 4db8757 update changelog.
• e5301a5 update changelog.
• 960561e Restructure repository: flatten src/ directory and move examples
• c627af5 Add comprehensive Claude.md project documentation

Summary of Changes

Security Improvements

• Eliminated shell injection vulnerabilities (subProc() → subprocess.run())
• Fixed critical password leak in opnsense_reset.py
• Fixed shell injection in catFiles() and SSH functions
• Secure subprocess execution throughout codebase

Code Quality

• Comprehensive docstrings and inline comments
• Constants for magic numbers (NETPLAN_PERMISSIONS, FIREWALL_WAIT_TIMEOUT, etc.)
• Unified error handling with f-strings
• camelCase variables → snake_case (where appropriate)
• Split God functions into focused single-responsibility functions

Architecture

• Created setup/helpers.py for shared setup functionality
• Extracted certificate operations into shared functions
• Created BaseCLI class (in examples/)
• Created unified logging module (in examples/)
• Flattened directory structure (src/linuxmuster_base7/* → src/*)

Testing & Documentation

• Comprehensive integration test suite with VM snapshots
• Test execution improvements and debugging output
• Updated all documentation
• Added Claude.md for AI assistant guidance

Bug Fixes

• Fixed multiple syntax errors
• Fixed module import issues
• Fixed return code logic in various scripts
• Fixed date parsing in holiday_generate.py
• Fixed DHCP server6 disabling
• Fixed firewall setup interfaces

All 106 commits are now part of the python-refactoring branch and will be merged when this issue is resolved.