Skip to content

Bump v0.18.0 - #286

Merged
Xieyt merged 115 commits into
mainfrom
develop
Aug 22, 2025
Merged

Xieyt merged 115 commits into
mainfrom
develop

Conversation

@Xieyt

@Xieyt Xieyt commented Aug 22, 2025 •

Copy link
Copy Markdown
Collaborator

CHANGES

  • introduce new Python package fmx with rich/typer CLI in the container, more info at
  • add parallel supervisor control for start/stop/restart/status
  • support safe RQ worker suspension and migration orchestration
  • implement granular signal forwarding and process detachment in bench-wrapper
  • migrate bench-wrapper.sh to handle monitor/detach/kill for workers
  • refactor Dockerfile for workspace-centric pyenv/nvm management
  • update compose templates for Redis 8, remove redis-socketio
  • migrate install.sh for improved user/password arg parsing and logging
  • fix site config logic for redis_socketio to use redis-cache
  • consolidate supervisor config splitting and environment activation
  • enhance backup/migration routines for per-site pyenv/nvm detection

Xieyt and others added 30 commits May 28, 2025 16:09
- Update Linux build runner from ubuntu-20.04 to ubuntu-latest

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
Signed-off-by: Immanuel Raj <iamimmanuelraj@gmail.com>
Fix: Fix fm self update images command
- Add CLI tool for managing supervisord services
- Create supervisor utilities for service management
- Add worker monitoring functionality
- Implement parallel command execution support
- Add project configuration and installation scripts
- Create documentation for CLI usage and setup

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Split monolithic CLI into modular command files
- Add Nix flake configuration for package management
- Move start/stop/restart/status commands to own modules
- Update service name enum handling for better error cases
- Add explicit error handling for missing supervisor sockets
- Clean up command help text and documentation

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Create new supervisor package with dedicated modules
- Move supervisor-related code from utils into new package
- Add custom exceptions for supervisor operations
- Clean up command modules by removing redundant imports
- Restructure parallel execution handling in CLI module
- Add nullcontext for better progress handling

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Move supervisor utilities to dedicated supervisor module
- Improve error handling for supervisor connections
- Simplify restart logic by removing force flag
- Update imports across command modules
- Add connection checking and validation
- Enhance error messages and feedback
- Handle XML-RPC faults and protocol errors properly

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add force-timeout option to gracefully stop services
- Implement force kill logic with SIGKILL after timeout
- Enhance process state monitoring during stop operations
- Improve error handling and status messages
- Update help text for force restart option

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Move job waiting logic to dedicated workers module function
- Improve error handling and status reporting in wait_jobs
- Add graceful cleanup of database and Redis connections
- Update CLI interface for better job monitoring integration
- Enhance status messaging and progress feedback in restart
- Restructure restart flow to handle job waiting more robustly

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Consolidate restart steps into single unified flow
- Remove redundant code paths and conditions
- Simplify step numbering and progress messaging
- Remove conditional paths around job waiting and force restart
- Streamline restart sequence execution structure

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add pause_scheduler functionality to restart and stop commands
- Implement scheduler state management in config file
- Return scheduler state info from wait_for_jobs_to_finish
- Add new _update_site_config_scheduler helper function
- Update CLI options and help messages for scheduler control

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add maintenance mode flag to restart command
- Add helper functions for config key management
- Refactor site config update logic for better reusability
- Add validation and error handling for maintenance mode
- Improve state restoration process for graceful restarts

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Change maintenance_mode from 1 to true in site_config
- Update pause_scheduler value from 1 to true
- Update help text and messages to reflect boolean values

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Split supervisor actions into dedicated files
- Create fault handler module for error handling
- Move process states to constants module
- Extract stop/kill logic into standalone functions
- Reorganize supervisor executor module

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add hybrid-restart command for blue/green deployments
- Add wait_workers option to stop and restart commands
- Move stop helper functions to dedicated module
- Refactor supervisor API for better modularity
- Add process state constants and worker detection

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Remove redundant variable reassignment in `file_name_prefix`
- Simplify extraction of `file_name_prefix` using direct indexing

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Split supervisor logic into separate modules and classes
- Add RollingState class for managing blue/green deployments
- Move supervisor utils into dedicated files
- Update imports and module organization
- Add graceful restart functionality and state handling
- Enhance error handling and logging
- Improve process status formatting and verbosity options

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Remove graceful-restart.py and hybrid-restart.py commands
- Merge restart logic into single restart.py implementation
- Remove blue/green deployment state management
- Remove worker suffix handling from supervisor actions
- Simplify supervisor constants and remove state tracking
- Update process start handling in actions.py

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Move worker control from SIGUSR2 to Redis-based suspension
- Introduce RQ controller module for worker suspend/resume
- Refactor config helpers to use common_site_config.json
- Simplify restart workflow using new suspension mechanism
- Update CLI documentation for new Redis-based approach

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add wait_for_rq_workers_suspended function for monitoring worker states
- Add noop job enqueuing to wake idle workers during suspension
- Add suspend-rq and wait-workers CLI options to restart command
- Implement Redis-based worker suspension with timeout and polling
- Add command line parameters for worker suspension configuration

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Delete job waiting helper functions from cli.py
- Remove scheduler pause and wait job options from restart.py
- Remove wait job parameters from stop.py
- Optimize noop job enqueueing in rq_controller.py
- Cleanup unused imports and functions from workers.py

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Enhance output messages with bold/colored formatting
- Remove redundant noop job enqueuing logic from control_rq_workers
- Update worker suspension flow and monitoring
- Consolidate verbose messages in wait_for_rq_workers_suspended
- Streamline status updates in live display panel

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add new DisplayManager class for centralized console output
- Replace direct rich.print calls with DisplayManager methods
- Update start and status commands to use new DisplayManager
- Split supervisor imports for better organization

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add DisplayManager class for centralized message display
- Update service commands to use DisplayManager for consistent output
- Refactor wait behavior for worker processes in stop/restart
- Enhance worker suspension and restart logic
- Improve help messages and option descriptions in CLI
- Add main callback for context initialization
- Update error handling to use DisplayManager consistently

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Replace direct rich.print with DisplayManager helpers
- Add display global instance imports
- Update error and warning message formatting
- Consolidate display message formatting across modules
- Standardize process status messaging with highlight method

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Create dedicated virtual env for bench installation
- Update bench command path from pyenv to venv
- Enhance bench wrapper with improved worker signal handling
- Simplify supervisor config split logic
- Update configuration files to use new bench path

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add Cache-Control header for assets with 1-year caching
- Remove redundant proxy timeout and redirect settings

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add worker signal handling to restart command flow
- Implement signal_service_workers API function
- Create _handle_signal_workers handler in supervisor actions
- Add SIGNAL_NUM_WORKER_GRACEFUL_EXIT constant
- Add return_raw_results option to execute_parallel_command
- Add wait_after_signal_timeout parameter to restart command
- Implement polling logic for worker shutdown status

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Implement comprehensive signal handling for bench worker process
- Add dynamic signal trapping for signals 1-64
- Improve logging and error handling for signal forwarding
- Add explicit cleanup for signal traps
- Enhance debug output with detailed status messages

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
- Add SupervisorProcessError class inheriting from SupervisorError
- Implement process-specific exception handling for supervisor operations
- Include service and process name details in error messages

Signed-off-by: Alok Singh <alokmilenium@gmail.com>
Xieyt added 29 commits August 19, 2025 11:49
- Add `use_cache` input to `workflow_dispatch` trigger
- Introduce conditional logic to set `USE_CACHE` environment variable
- Update `cache-from` and `cache-to` fields to respect `USE_CACHE` value

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add newline after shebang in scripts
- Replace `jinja2` binary with `jinja2-cli` in entrypoint
- Refactor AWK scripts to ensure proper formatting
- Update Dockerfile to remove unused builder stage
- Add Python 3 and pip dependencies in nginx Dockerfile
- Use formatted `bench-dev-server` script handling in Python
- Add `source` command to container-run commands in Python

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove redundant conditional logic in cache-to
- Streamline cache-to assignments for all workflows

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add sourcing of /etc/bash.bashrc to the script
- Leave cleanup function unchanged

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add `pyenv_activate` call in `setup_pyenv_and_virtualenv`
- Adjust flow to align with activation step

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add version check and pyenv install logic in setup_pyenv_and_virtualenv
- Move workspace initialization to configure_workspace
- Remove redundant setup calls in configure_workspace
- Fix sourcing of bashrc in launch_supervisor_service

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove unused imports for cleaner code
- Add `migrate_pyenv_and_nvm` for environment updates
- Rename and filter `.zshrc` to exclude pyenv/nvm references
- Integrate pyenv and nvm configuration with `host_run_cp`
- Migrate Python environment and recreate virtualenv
- Add supervisor config splitting logic for modular configs
- Update compose project logic for better error handling
```

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace hardcoded CPU fallback with default in `os.cpu_count`
- Remove redundant whitespace and improve readability
- Refactor `section_name_delimeter` initialization

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Update ZSH environment path to /workspace/.oh-my-zsh
- Adjust fm.zsh-theme copy path to custom themes directory
- Remove redundant PATH export and zsh configuration
- Add export PATH for /opt/user/.bin in bashrc and zshrc
- Add oh-my-zsh directory copy from prebake stage

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add `image` field to service image dictionary
- Add `user` parameter to DockerComposeWrapper

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Replace `.zshrc` migration with `.oh-my-zsh`
- Update source path in `host_run_cp` invocation

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add JSON handling for site config reading
- Implement `_compose_exec_or_run` for service command execution
- Add `import_from_bench` method to retrieve database info
- Handle database query access denied scenario
- Introduce `DatabaseServiceQueryAccessDenied` exception
- Modify database commands to use `_compose_exec_or_run`

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove redis-socketio references across codebase
- Update redis-cache and redis-queue images to version 8-alpine
- Modify database backup logic to use bench-specific db info
- Adjust nginx conf handling during migration
- Refactor redundant exception handling in MariaDBManager
- Update db and config settings to use redis-cache for redis-socketio

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove type annotation for compose_project parameter

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Check existence of nginx config file before unlinking
- Add `|| continue` to handle pip install errors in loop

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Reorder `richprint.change_head` calls for clarity
- Add `docker.compose.run` to run nginx migration commands
- Add live output streaming for nginx migration steps
- Remove redundant `richprint.change_head` duplication

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add heading for nginx config migration
- Update nginx container run command for templating

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add log for migrating nginx config
- Remove old nginx default.conf if it exists
- Run nginx jinja2 command using docker compose
- Add log for migrating bench compose

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Remove duplicate nginx config migration block
- Streamline migration code by eliminating unused calls

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Move richprint.change_head call earlier
- Reorder richprint.live_lines and print calls
- Remove redundant richprint.change_head call

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Extract get_pyvenv_version from setup_pyenv_and_virtualenv
- Update .nvm and .pyenv checks to validate directories
- Remove redundant debug statements (set -x)
- Source helper-function.sh in user-script.sh

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Removed worker management section from README
- Deleted safety notes and process handling details
- Retained other operational command examples

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add clarification on default site usage with bench doctor
- Include additional example for site-specific usage

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Add trailing spaces for markdown compatibility
- Remove unused section about `--suspend-rq`

Signed-off-by: xieyt <alokmilenium@gmail.com>
- Change default `bench_name` to an empty string

Signed-off-by: xieyt <alokmilenium@gmail.com>
rework container runtime, process management, and introduce fmx CLI
@Xieyt
Xieyt merged commit 32921b8 into main Aug 22, 2025
14 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants