CommonUI

A collection of reusable Pluto.jl notebooks and Julia utility modules for managing computational simulation workflows — from parameter generation to HPC job submission and data visualisation.

Overview

CommonUI is designed to slot into a larger project structure and provides a common interactive layer for:

1. Running simulations — locally (CPU/GPU) or on a Slurm HPC cluster
2. Visualising results — interactive exploration of simulation output
3. Utility modules — SSH/SCP helpers, DataFrame generation, and UI parsing tools

Expected project layout

MyProject/
├── Notebook.pluto.jl          # Home/entry notebook (not part of CommonUI)
├── sim/
│   ├── main.jl                # Simulation entry point
│   ├── DF.csv                 # Parameter DataFrame (generated by GenInputParams)
│   └── MyProject.sh           # Generated Slurm batch script
├── CommonUI/
│   ├── RunSimulations.pluto.jl
│   ├── DataVisualisation.pluto.jl
│   └── utils/
│       ├── SSH_utils.jl
│       ├── UI_utils.jl
│       └── DF_utils.jl
└── GenInputParams.pluto.jl    # Parameter sweep notebook (not part of CommonUI)

---

Notebooks

RunSimulations.pluto.jl

Interactive notebook to launch simulation jobs.

Features:

• Select individual simulations or run all (all) from sim/DF.csv
• Local execution — runs sequentially via julia --optimize=3, supports CPU threads or GPU
• Cluster execution (Slurm) — targets the Baobab HPC cluster at UNIGE, with:
  • Configurable partition, time limit, RAM, and GPU type (H100, A100-40Gb, A100-80Gb)
  • Automatic Slurm batch script generation
  • Code upload via SCP using SSH_utils
  • Remote directory creation and verification
  • Single reusable SSH connection — a switch (macOS/Linux) that routes every submit / queue / download through one shared login (SSH_utils.ssh_open), avoiding the cluster's too many logins throttle
  • Live queue view — a squeue --me panel with a refresh button, shown just before the download step
• Download results — incrementally fetch simulation output from your scratch back to a local folder:
  • Only files that are missing locally or newer on the cluster are transferred (modification-time comparison), so it can be re-run while jobs are still producing output
  • Adjustable parallelism (1–10 concurrent scp transfers) via a slider
  • Cross-platform (Windows/macOS/Linux): relies only on ssh/scp, no rsync required

Dependencies: PlutoUI, PlutoTeachingTools, CSV, DataFrames, ProgressLogging, RemoteFiles, OpenSSH_jll

---

DataVisualisation.pluto.jl

Interactive notebook for exploring simulation output.

Features:

• Loads a DataWorkspace from a CSV parameter file and .jld/.jld2 result files
• Launches an interactive explor_app (from DataVisualisation.jl)
• Adjustable display scale and height via sliders

Dependencies: PlutoUI, PlutoTeachingTools, DataVisualisation.jl

---

Utility Modules

utils/SSH_utils.jl

Thin wrapper around ssh and scp for cluster operations.

Function	Description
ssh(usr, hst, cmd)	Run a remote command and return output as a string
print_ssh(usr, hst, cmd)	Run a remote command and print the output
squeue(usr, hst; opt="--me")	Query the Slurm scheduler and return the output as a string (default --me = your own jobs)
ssh_open(usr, hst)	Open a shared master SSH connection that later ssh/scp reuse (see multiplexing note); returns the remote user@host
ssh_close(usr, hst)	Close the shared master connection and revert to one login per call
up(usr, hst, cluster_dir, local_file)	Upload a file to the cluster
up_dir(usr, hst, cluster_dir, local_dir)	Upload a directory to the cluster
up_file(usr, hst, cluster_dir, local_file)	Upload a single file (no -r flag)
down(usr, hst, cluster_path, local_dir)	Download a file/directory from the cluster
sync(usr, hst, cluster_dir, local_dir; nparallel=4)	Download a remote tree, transferring only files missing locally or newer on the cluster, up to nparallel at a time
mkdir(usr, hst, cluster_dir)	Create a remote directory if it does not exist
rm_dir(usr, hst, cluster_dir)	Remove a remote directory recursively (rm -rf), with guards against unsafe paths
readdir(usr, hst, cluster_dir)	List files in a remote directory

Connection multiplexing (fewer logins). By default each ssh/scp opens its own login, so a busy submit/download session can trip the cluster's too many logins rate-limit. Call ssh_open(usr, hst) once to establish a shared master connection (OpenSSH ControlMaster); every subsequent ssh/scp — including the parallel downloads in sync — then reuses it as a single login. ssh_close tears it down. This is opt-in and additive: without ssh_open, behaviour is unchanged. Supported on macOS/Linux only (Windows OpenSSH has no ControlMaster, where calls stay one-login-each). Pair it with an ssh-agent (ssh-add your key once) so a passphrase-protected key is unlocked only once.

---

utils/UI_utils.jl

Helpers for parsing user input strings in Pluto TextField widgets.

Function / Macro	Description
parse_values(s)	Parse a comma-separated string of numbers or start:step:stop ranges into a flat array
parse_to_slurm_array(s)	Convert the same format to a Slurm --array string (e.g. "1,3:5" → "1,3-5")
@named_parse [a_str, b, c_str]	Batch-parse _str variables and return (values, names)
print_list(names, values)	Display parameter name/value pairs, or show an alert if any field is empty

Range syntax (used in parse_values and parse_to_slurm_array):

"1,3,5"         → [1, 3, 5]
"1:5"           → [1, 2, 3, 4, 5]
"0:0.5:2"       → [0.0, 0.5, 1.0, 1.5, 2.0]
"1,3:5,8"       → [1, 3, 4, 5, 8]

---

utils/DF_utils.jl

Generates a full-factorial (Cartesian product) DataFrame from parameter lists.

Function	Description
generate_dataframe(listname, listtab)	Build a DataFrame with one row per parameter combination

Example:

using DataFrames
include("utils/DF_utils.jl")

names = ["alpha", "beta"]
values = [[0.1, 0.2], [10, 20, 30]]

df = DF_utils.generate_dataframe(names, values)
# 6 rows: all combinations of alpha ∈ {0.1, 0.2} × beta ∈ {10, 20, 30}

The resulting DataFrame is typically saved to sim/DF.csv and consumed by RunSimulations.pluto.jl.

---

Usage

This repository is not meant to be used standalone. It is intended to be used as a submodule within a parent project. The primary entry point is HydraFluids, which sets up the expected directory structure and provides the project-specific notebooks (Notebook.pluto.jl, GenInputParams.pluto.jl, sim/main.jl, etc.) that CommonUI depends on.

Refer to the HydraFluids repository for setup instructions and getting started.