was: mktemp: failed to create directory via template ‘/local/tmp/tmp.XXXXXXXXXX’: No such file or directory; juliaup: command failed: mktemp -d RHEL 8, now "ERROR: add requires at least one package, ERROR: preferences file is missing or empty."

[LinuxPersonEC]

Does juliaup attempt to run a mktemp -d? I'm just setting TMP=/local which has 777 permissions.

./juhpc $JUHPC_SETUP_INSTALLDIR $JULIAUP_INSTALLDIR --verbose=2 


     ██╗  ██╗   ██╗  ██╗  ██╗██████╗  ██████╗
     ██║  ██║   ██║  ██║  ██║██╔══██╗██╔════╝
     ██║  ██║   ██║  ███████║██████╔╝██║     
██╗  ██║  ██║   ██║  ██╔══██║██╔═══╝ ██║     
╚█████╔╝  ╚██████╔╝  ██║  ██║██║     ╚██████╗
 ╚════╝    ╚═════╝   ╚═╝  ╚═╝╚═╝      ╚═════╝


[ JUHPC: Starting installation of HPC setup for juliaup, julia and HPC key packages requiring system libraries...
[ JUHPC: Installing juliaup and julia temporarily in /local/juliaup...
mktemp: failed to create directory via template ‘/local/tmp/tmp.XXXXXXXXXX’: No such file or directory
juliaup: command failed: mktemp -d
[ JUHPC: ERROR: failed to install Juliaup (and Julia) in /local/juliaup.
[ JUHPC: cleaning up temporary juliaup installation in /local/juliaup.

EDIT: Well there was a start up script setting $TMPDIR to/local/tmpwhich does not exist after a reboot. I changed it to /local, which got past this error.

However the script terminates with:

  2 dependencies successfully precompiled in 1 seconds. 3 already precompiled.
ERROR: add requires at least one package
Stacktrace:
 [1] pkgerror(msg::String)
   @ Pkg.Types /local/juliaup/depot/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/Types.jl:68
 [2] require_not_empty
   @ /local/juliaup/depot/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:140 [inlined]
 [3] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, target::Symbol, allow_autoprecomp::Bool, kwargs::@Kwargs{io::IOContext{IO}})
   @ Pkg.API /local/juliaup/depot/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:258
 [4] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{})
   @ Pkg.API /local/juliaup/depot/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:159
 [5] add(pkgs::Vector{Pkg.Types.PackageSpec})
   @ Pkg.API /local/juliaup/depot/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:148
 [6] add(pkgs::Vector{String})
   @ Pkg.API /local/juliaup/depot/juliaup/julia-1.11.2+0.x64.linux.gnu/share/julia/stdlib/v1.11/Pkg/src/API.jl:147
 [7] top-level scope
   @ none:1
[ JUHPC: ERROR: preferences file is missing or empty.
[ JUHPC: cleaning up temporary juliaup installation in /local/juliaup.

I didn't choose a preference nor--postinstallwhich the instructions say is optional.

What do folks do when only a few nodes have a GPU and different versions of OpenMPI or MPICH are loadable modules?

[LinuxPersonEC]

OK some observations and perhaps suggestions.:

I kept getting tripped up on the path to HDF5. I'd suggest not hard coding this or at least be more clear in the docs:

julia_pref 'using HDF5; HDF5.API.set_libraries!("$(ENV["JUHPC_HDF5_HOME"])/lib/libhdf5.so", "$(ENV["JUHPC_HDF5_HOME"])/lib/libhdf5_hl.so")'

Many Linux flavors don't have that, or are perhaps lib64.

I'd suggest clarifying this instruction:

preferences for HPC key packages that require system libraries

And be specific on which one(s) are needed and/or provide an example.

The generated activate script is nice but is not very Lmod/module file friendly and if one uses the Lmod-provided sh_to_modulefile script the module file is far from usable. The source-sh option only became available in Lmod 8.6

Here's a (partial) modulefile I created that appears to be working (edited after some experimenting got me the correct values) and note the leading colon in JULIA_LOAD_PATH:

prepend-path	PATH			/opt/julia-parallel/juliaup_wrapper:/home/$::env(USER)/juliaup/bin
setenv		JULIAUP_DEPOT_PATH	/home/$::env(USER)/juliaup/depot
setenv		JULIA_DEPOT_PATH	/home/$::env(USER)/juliaup/depot
setenv		JULIA_LOAD_PATH		:/opt/julia-parallel/julia_preferences

setenv		JUHPC_MPI_EXEC		/usr/mpi/gcc/openmpi-4.1/bin/mpicc
setenv		HDF5_DIR		/opt/hdf5p-1.14.2/lib

module load openmpi/gcc/64
module load hdf5p

We may end up adding CUDA but not all nodes have a GPU and we'd need to document the usage.

[omlins]

Please excuse the late reply; I have been in vacation for four weeks :)

ERROR: add requires at least one package

Looking at the error it seems to me that it appeared because you did not set any environment variable regarding the key packages for which we want to set preferences: https://github.com/JuliaParallel/JUHPC/tree/main?tab=readme-ov-file#1-export-environment-variables-for-the-installation-of-some-hpc-key-packages
(I opened an issue avoid an error in this case in future: #35)

What do folks do when only a few nodes have a GPU and different versions of OpenMPI or MPICH are loadable modules?

JUHPC is designed such that you can create a different setup for each case that you require. You can for example create a setup for MPI only and one for CUDA-aware MPI (see examples: https://github.com/JuliaParallel/JUHPC/tree/main/examples/cscs/daint/gpu)
etc.
If you create multiple setups they will not interfere with each other.

I kept getting tripped up on the path to HDF5. I'd suggest not hard coding this or at least be more clear in the docs

Good point, ideally one should search for the required libraries in the sub directories of JUHPC_HDF5_HOME. If you feel like doing a PR, please go ahead! (or probably it would make more sense to solve this in HDF5.jl)

The generated activate script is nice but is not very Lmod/module file friendly and if one uses the Lmod-provided sh_to_modulefile script the module file is far from usable. The source-sh option only became available in Lmod 8.6

If you have ideas on how to make it Lmod friendly, then feel free to do a PR. However, you mention the source-sh option became available in Lmod 8.6. Doesn't that solve then the issue you raised? Do we need to be compatible with old versions?

[LinuxPersonEC]

Good point, ideally one should search for the required libraries in the sub directories of JUHPC_HDF5_HOME. If you feel like doing a PR, please go ahead! (or probably it would make more sense to solve this in HDF5.jl)

Ah not my forté but yes the latter sounds more appropriate.

The generated activate script is nice but is not very Lmod/module file friendly and if one uses the Lmod-provided sh_to_modulefile script the module file is far from usable. The source-sh option only became available in Lmod 8.6

If you have ideas on how to make it Lmod friendly, then feel free to do a PR. However, you mention the source-sh option became available in Lmod 8.6. Doesn't that solve then the issue you raised? Do we need to be compatible with old versions?

Well, for older systems, of which there are many, it depends on your stance or tolerance for how far back to support old versions of anything. Even if it's as simple as a warning/tooltip if an older version is detected, that could be a quick/more graceful way to alert the user.

I'm just happy to see someone create this and put it all together!

[LinuxPersonEC]

Is there a reason why this error happens? Is this a specific env var that needs setting?

julia> Pkg.add("LinearAlgebra")
ERROR: UndefVarError: `Pkg` not defined in `Main`
Suggestion: check for spelling errors or missing imports.
Stacktrace:
 [1] top-level scope
   @ REPL[2]:1

julia> Pkg
ERROR: UndefVarError: `Pkg` not defined in `Main`
Suggestion: check for spelling errors or missing imports.
Hint: Pkg is loaded but not imported in the active module Main.

julia -e 'using Pkg; Pkg.status()'
ERROR: ArgumentError: Package Pkg not found in current path.
Stacktrace:
 [1] macro expansion
   @ ./loading.jl:2296 [inlined]
 [2] macro expansion
   @ ./lock.jl:273 [inlined]
 [3] __require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2271
 [4] #invoke_in_world#3
   @ ./essentials.jl:1089 [inlined]
 [5] invoke_in_world
   @ ./essentials.jl:1086 [inlined]
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:2260

julia> readdir(Base.load_path_expand("@stdlib"))
60-element Vector{String}:
 "ArgTools"
 "Artifacts"
 "Base64"
 "CRC32c"
 "CompilerSupportLibraries_jll"
 "Dates"
 "DelimitedFiles"
 "Distributed"
 "Downloads"
 ⋮
 "Test"
 "UUIDs"
 "Unicode"
 "Zlib_jll"
 "dSFMT_jll"
 "libLLVM_jll"
 "libblastrampoline_jll"
 "nghttp2_jll"
 "p7zip_jll"

UPDATE: I fixed this by adjusting the module file $PATH to have juliaup_wrapper before bin and also making sure the leading colon : in front of $JULIA_LOAD_PATH so :/opt/julia-parallel/julia_preferences

I do wonder should $JULIA_PROJECT be set?

[omlins]

Glad you could fix it!

In general I would set as little Julia environment variables as required for a consistent setup (to leave the user the freedom to set them). If you see any need please share!