Test

.github/workflows/nf-test.yml

@@ -0,0 +1,42 @@
+name: nf-test CI
+
+on:
+  push:
+    branches: [ main, develop ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  test:
+    name: Run nf-test in Docker
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check out pipeline code
+        uses: actions/checkout@v4
+
+      - name: Run nf-test in Docker
+      - name: Install nf-test
+        run: |
+          curl -fsSL https://code.askimed.com/install/nf-test | bash
+          sudo mv nf-test /usr/local/bin/
+
+      - name: Run nf-test
+        run: |
+          nf-test test --coverage --verbose
+
+      - name: Upload coverage reports
+        uses: codecov/codecov-action@v4
+        if: always()
+        with:
+          files: .nf-test/coverage/coverage.xml
+          fail_ci_if_error: false
+
+      - name: Upload test results
+        uses: actions/upload-artifact@v4
+        if: always()
+        with:
+          name: nf-test-results
+          path: |
+            .nf-test/
+            !.nf-test/cache/

.gitignore

@@ -0,0 +1,15 @@
+.DS_Store
+.nextflow*
+work/
+results/
+conda_cache/
+singularity_cache/
+*.log
+*.html
+*.svg
+*.txt
+*.pyc
+__pycache__/
+.vscode/
+.idea/
+answer

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Broken String Biosciences
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

NF_SCHEMA_INTEGRATION.md

@@ -0,0 +1,109 @@
+# nf-schema Integration Summary
+
+## ✅ What's Been Implemented
+
+### 1. **nf-schema Plugin Configuration**
+- Added `plugins { id 'nf-schema@2.0.0' }` to `nextflow.config`
+- Enabled automatic parameter validation on pipeline startup
+
+### 2. **JSON Schema Definition** (`nextflow_schema.json`)
+- Complete parameter schema with validation rules
+- **File validation**: Input files checked for existence and format
+- **Type validation**: String, integer, boolean types enforced
+- **Range validation**: Numeric parameters (merge_distance ≥ 0)
+- **Pattern validation**: File paths must match expected patterns
+- **Help text**: Detailed descriptions for all parameters
+- **Icons**: Font Awesome icons for better UX
+- **Hidden parameters**: Advanced options hidden by default
+
+### 3. **Enhanced Main Workflow** (`main.nf`)
+- Import nf-schema functions: `validateParameters`, `paramsHelp`, `paramsSummaryLog`
+- Automatic parameter validation before workflow execution
+- Schema-generated help text with `--help` flag
+- Parameter summary logging with validation status
+- Proper error handling for invalid parameters
+
+### 4. **Module Version Tracking**
+All modules now emit `versions.yml` files:
+- `MERGE_BREAKENDS`: bedtools version
+- `BEDTOOLS_INTERSECT`: bedtools version  
+- `COUNT_INTERSECTIONS`: coreutils version
+- `COLLATE_STATISTICS`: coreutils version
+- `COLLECT_STATISTICS`: coreutils version
+
+### 5. **Enhanced Workflow**
+- Version collection and tracking throughout pipeline
+- Named workflow (`INDUCESEQ_ANALYSIS`) for better organization
+- Proper channel handling and emit declarations
+
+## 🧪 **Testing Framework**
+
+### Schema Validation Tests (`test_schema.sh`)
+1. **Help message test**: Verify schema-generated help
+2. **Valid parameters test**: Confirm valid parameters pass validation
+3. **Invalid file test**: Ensure missing files are caught
+4. **Invalid range test**: Verify numeric range validation
+
+## 📊 **Validation Features**
+
+### **JSON Schema Properties**
+```json
+{
+  "asisi_sites": {
+    "type": "string",
+    "format": "file-path", 
+    "exists": true,
+    "pattern": "^\\S+\\.(bed|BED)$"
+  },
+  "merge_distance": {
+    "type": "integer",
+    "minimum": 0
+  }
+}
+```
+
+### **Validation Capabilities**
+✅ **File existence**: Input files verified before execution  
+✅ **File format**: BED file extension patterns enforced  
+✅ **Type checking**: Parameters must match declared types  
+✅ **Range validation**: Numeric bounds enforced  
+✅ **Required parameters**: Essential parameters must be provided  
+✅ **Pattern matching**: String patterns validated  
+✅ **Help generation**: Auto-generated help from schema  
+
+## 🚀 **Usage Examples**
+
+### Basic validation
+```bash
+# Show schema-generated help
+nextflow run main.nf --help
+
+# Validate without running
+nextflow run main.nf -preview \\
+    --asisi_sites data/chr21_AsiSI_sites.t2t.bed
+```
+
+### Validation testing
+```bash
+# Run validation tests
+./test_schema.sh
+
+# Test invalid parameters
+nextflow run main.nf --merge_distance -1  # Will fail validation
+```
+
+## 📈 **Benefits Achieved**
+
+1. **Robust validation**: Catch parameter errors before pipeline execution
+2. **Better UX**: Clear error messages and auto-generated help
+3. **Documentation**: Schema serves as parameter documentation
+4. **Consistency**: Standardized parameter handling across runs
+5. **Debugging**: Parameter summary helps troubleshoot issues
+6. **Compliance**: Follows Nextflow best practices for parameter handling
+
+## 🔄 **Version Update**
+- Updated to version 1.1.0 to reflect nf-schema integration
+- Updated CHANGELOG.md with new features
+- Enhanced documentation with validation examples
+
+The pipeline now provides enterprise-grade parameter validation while maintaining ease of use! 🎉

PROJECT_SETUP.md

@@ -0,0 +1,197 @@
+# Project Setup Summary
+
+## 📁 Project Structure
+
+```
+coding-test-nextflow/
+├── main.nf                     # Main pipeline workflow (DSL2)
+├── nextflow.config             # Main configuration file with nf-schema plugin
+├── nextflow_schema.json        # JSON schema for parameter validation
+├── nextflow_advanced.config    # Advanced configuration options
+├── modules/                    # DSL2 process modules
+│   ├── merge_breakends.nf      # Merge adjacent break sites
+│   ├── bedtools_intersect.nf   # Intersect with AsiSI sites
+│   ├── count_intersections.nf  # Count breaks per AsiSI site
+│   ├── collate_statistics.nf   # Generate per-sample stats
+│   └── collect_statistics.nf   # Combine all sample stats
+├── lib/                        # Helper utilities
+│   └── WorkflowUtils.groovy    # Utility functions
+├── data/                       # Input data (provided)
+│   ├── chr21_AsiSI_sites.t2t.bed
+│   └── breaks/
+│       ├── Sample1.breakends.bed
+│       └── ... (16 samples total)
+├── run_pipeline.sh             # Pipeline runner script
+├── test_pipeline.sh            # Testing script
+├── test_schema.sh              # nf-schema validation testing
+├── README_PIPELINE.md          # Complete pipeline documentation
+├── CHANGELOG.md                # Version history
+├── LICENSE                     # MIT License
+├── VERSION                     # Version number (1.0.0)
+└── .gitignore                  # Git ignore file
+```
+
+## 🚀 Quick Start
+
+### 1. Basic Usage
+```bash
+# Run with default settings
+nextflow run main.nf
+
+# Or use the helper script
+./run_pipeline.sh local
+```
+
+### 2. Docker Usage
+```bash
+nextflow run main.nf -profile docker
+# Or
+./run_pipeline.sh docker
+```
+
+### 3. Test Run
+```bash
+nextflow run main.nf -profile test
+# Or
+./run_pipeline.sh test
+```
+
+### 4. Custom Parameters
+```bash
+nextflow run main.nf \\
+    --asisi_sites data/chr21_AsiSI_sites.t2t.bed \\
+    --sample_beds "data/breaks/*.breakends.bed" \\
+    --merge_distance 10 \\
+    --outdir custom_results
+```
+
+## 🔧 Parameter Validation with nf-schema@2.2.1
+
+The pipeline now includes robust parameter validation using the nf-schema@2.2.1 plugin:
+
+### Validation Features
+```bash
+# Show help with custom help message
+nextflow run main.nf --help
+
+# Validate parameters without running (preview mode)
+nextflow run main.nf -preview --asisi_sites data/chr21_AsiSI_sites.t2t.bed
+
+# Test parameter validation
+./test_schema.sh
+
+# Show hidden parameters (nf-schema@2.2.1 style)
+nextflow run main.nf --showHidden --help
+```
+
+### Schema Features
+✅ **JSON Schema 2020-12** - Latest schema specification  
+✅ **File existence checks** - Input files verified before execution  
+✅ **Type enforcement** - String, integer, boolean types enforced  
+✅ **Range validation** - Numeric parameters checked against limits (merge_distance ≥ 0)  
+✅ **Pattern matching** - File paths validated against expected patterns (.bed files)  
+✅ **Custom help** - Clear usage examples and parameter descriptions  
+✅ **Error messages** - Detailed validation error reporting  
+
+### Example Validation Errors
+```bash
+# Invalid merge_distance
+nextflow run main.nf --merge_distance -1
+# ERROR: -1 is less than 0
+
+# Invalid file path  
+nextflow run main.nf --asisi_sites invalid_file.bed
+# ERROR: the file or directory 'invalid_file.bed' does not exist
+```
+
+## 🔧 Configuration Profiles
+
+- **local**: Run locally (default)
+- **docker**: Run with Docker containers
+- **slurm**: Run on SLURM cluster
+- **aws**: Run on AWS Batch
+- **test**: Run with subset of data for testing
+
+## 📊 Pipeline Workflow
+
+1. **MERGE_BREAKENDS**: Merge adjacent break sites using bedtools merge
+2. **BEDTOOLS_INTERSECT**: Find overlaps between breaks and AsiSI sites
+3. **COUNT_INTERSECTIONS**: Quantify breaks at each AsiSI site
+4. **COLLATE_STATISTICS**: Generate per-sample summary statistics
+5. **COLLECT_STATISTICS**: Combine all statistics into final table
+
+## 📈 Outputs
+
+```
+results/
+├── merged/                     # Per-sample merged BED files
+├── intersections/              # Per-sample intersection results
+├── counts/                     # Per-sample AsiSI site counts
+├── statistics/                 # Per-sample statistics
+└── all_samples_statistics.tsv  # Combined statistics
+```
+
+## 🧪 Testing
+
+```bash
+# Validate inputs
+./run_pipeline.sh validate
+
+# Run test suite
+./test_pipeline.sh
+
+# Test with subset
+nextflow run main.nf -profile test
+```
+
+## 🌐 Cloud/HPC Examples
+
+### SLURM Configuration
+```groovy
+slurm {
+    process.executor = 'slurm'
+    process.queue = 'normal'
+    process.clusterOptions = '--account=your_account'
+}
+```
+
+### AWS Batch Configuration
+```groovy
+aws {
+    process.executor = 'awsbatch'
+    process.queue = 'nextflow-queue'
+    workDir = 's3://your-bucket/work'
+}
+```
+
+## 📋 Requirements
+
+- Nextflow >=22.10.1
+- Singularity or Docker
+- Input data in correct format
+
+## 🐳 Container Dependencies
+
+- **bedtools**: `quay.io/biocontainers/bedtools:2.31.1--hf5e1c6e_1`
+- **coreutils**: `quay.io/biocontainers/coreutils:9.5`
+
+## 📝 Key Features
+
+✅ **DSL2 modular architecture**  
+✅ **Container support for reproducibility**  
+✅ **Multi-executor support (local, cluster, cloud)**  
+✅ **nf-schema@2.2.1 parameter validation**  
+✅ **JSON Schema 2020-12 specification**  
+✅ **Auto-generated help documentation**  
+✅ **Detailed execution reporting**  
+✅ **Flexible configuration profiles**  
+✅ **Automated testing framework**  
+✅ **Complete documentation**  
+
+## 🔄 Version Control
+
+- Initial version: 1.0.0
+- Git repository ready for version control
+- Comprehensive .gitignore configuration
+
+The Nextflow project is now fully set up and ready for use! 🎉