DoIntegrate python strategies error #2

Activity

1. fspecque commented on Apr 7, 2025

   

   fspecque

   on Apr 7, 2025

   Collaborator

   More actions

   Hi!

   Thanks for your interest in the package and for reporting. I'm unable to reproduce this issue and I've never experienced it either, so I might be of little help here.

   After some search, my guess is that it's probably an error with R/reticulate and python/conda (but I cannot be sure).
   Let's make some tests (here with the conda environment for BBKNN). If you didn't use the default name for the environment, replace "SeuratIntegrate_bbknn" by the name you chose in the lines below.

   • first check if the error happens outside of RStudio (i.e. from a terminal).
   • if it does, check that reticulate can initialize the conda environment in an R session:

   reticulate::py_config()  # load default python just to check if it works
   
   # Close R and in a new session:
   reticulate::use_condaenv("SeuratIntegrate_bbknn", required = TRUE)
   reticulate::py_config()  # any error here?
   
   np <- reticulate::import("numpy", convert=FALSE)         # any error here?
   anndata <- reticulate::import("anndata", convert=FALSE)  # any error here?
   bbknn <- reticulate::import("bbknn", convert=FALSE)      # any error here?
   sc <- reticulate::import("scanpy", convert=FALSE)        # any error here?
   scipy <-  reticulate::import('scipy', convert = FALSE)   # any error here?

   • try to install numpy (and/or any of the faulty import()) with pip after conda activate SeuratIntegrate_bbknn and try the integration again in a fresh R session.

   Note that you can create a backup of a conda environment before making any change to it if you want to.

   Let me know if any of these suggestions works, and if not, tell me where it fails and I'll try to come up with some more.
   Florian
2. kingstonaj commented on Apr 9, 2025

   

   kingstonaj

   on Apr 9, 2025

   Author

   More actions

   Hi Florian,

   Thanks so much for the response! Unfortunately nothing much seems to be amiss with the Reticulate/Conda environments that I can spot.

   To address your suggestions:

   • The error is replicated when R is run in terminal, with a slightly different error message than previous (shown with bbkn for brevity but identical with scanorama and scanvi)

   Integration 1 in 2: integrating using 'bbknnIntegration'
   0%   10   20   30   40   50   60   70   80   90   100%
   [----|----|----|----|----|----|----|----|----|----|
   **************************************************|
   
    *** caught segfault ***
   address 0x540, cause 'invalid permissions'
   Warning: `bbknnIntegration`: failed, because "Failed to retrieve the result of MulticoreFuture (<none>) from the forked worker (on localhost; PID 74927).
   Post-mortem diagnostic: No process exists with this PID, i.e. the forked localhost worker is no longer alive. The total size of the 5 globals exported is 213.51 MiB. 
   The three largest globals are 'method_expr' (213.51 MiB of class 'call'), 'caller_env' (629 bytes of class 'function') and 'eval_tidy' (623 bytes of class 'function')"
    -> Skipping this integration

   • The Conda environment is initialised with no errors using reticulate in an R session (in both terminal and Rstudio) and all the modules seem to import fine. Outputs are as follows:

   > reticulate::py_config()
   python:         /Users/alex/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/7CuHLUQLhjpOEvByKlr3Q/bin/python3
   libpython:      /Users/alex/Library/Caches/org.R-project.R/R/reticulate/uv/python/cpython-3.11.11-macos-aarch64-none/lib/libpython3.11.dylib
   pythonhome:     /Users/alex/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/7CuHLUQLhjpOEvByKlr3Q:/Users/alex/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/7CuHLUQLhjpOEvByKlr3Q
   virtualenv:     /Users/alex/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/7CuHLUQLhjpOEvByKlr3Q/bin/activate_this.py
   version:        3.11.11 (main, Mar 17 2025, 21:33:08) [Clang 20.1.0 ]
   numpy:          /Users/alex/Library/Caches/org.R-project.R/R/reticulate/uv/cache/archive-v0/7CuHLUQLhjpOEvByKlr3Q/lib/python3.11/site-packages/numpy
   numpy_version:  2.2.4
   
   NOTE: Python version was forced by py_require()

   > reticulate::use_condaenv("SeuratIntegrate_bbknn", required = TRUE)
   > reticulate::py_config()  
   python:         /opt/anaconda3/envs/SeuratIntegrate_bbknn/bin/python
   libpython:      /opt/anaconda3/envs/SeuratIntegrate_bbknn/lib/libpython3.13.dylib
   pythonhome:     /opt/anaconda3/envs/SeuratIntegrate_bbknn:/opt/anaconda3/envs/SeuratIntegrate_bbknn
   version:        3.13.2 | packaged by conda-forge | (main, Feb 17 2025, 14:02:48) [Clang 18.1.8 ]
   numpy:          /opt/anaconda3/envs/SeuratIntegrate_bbknn/lib/python3.13/site-packages/numpy
   numpy_version:  2.1.3
   
   NOTE: Python version was forced by use_python() function

   I also tried the classic uninstall/reinstall of the packages and Conda environments through terminal, but the result was unchanged. Let me know if this sheds any light on the situation, v happy to provide more info if needed!

   Thanks,
   Alex
3. fspecque commented on Apr 10, 2025

   

   fspecque

   on Apr 10, 2025

   Collaborator

   More actions

   Hi!

   Thank you for the detailed feedback!

   Given that you get a segfault and that none of the conda environments works, I'm starting to wonder whether this is an error with conda binaries... (see here and there for instance).

   I also tried the classic uninstall/reinstall of the packages and Conda environments through terminal

   Did you try installing (at least numpy) with pip install rather than conda install inside "SeuratIntegrate_bbknn"? That was suggested in this issue

   Regarding the putative conda binaries incompatibility, here are a few suggestions:

   • it might be helpful to try something else with the same conda environment (let's stick to "SeuratIntegrate_bbknn"). For example, you could try to RunUMAP() with umap-learn package after you've generated a graph with FindNeighbours() to see if at least that works. Lunch it from a terminal and don't use future preferably, it provides more troubleshooting information.

   reticulate::use_condaenv("SeuratIntegrate_bbknn", required = TRUE)
   
   obj <- RunUMAP(obj, graph = "RNA_snn",   # or "SCT_snn" if normalised with SCTransform
                  umap.method = "umap-learn", n.epoch = 400L)

   • if my intuition is correct, a possible fix would be to create another conda environment with R and necessary packages, install SeuratIntegrate in the environment (using R, for now SeuratIntegrate is not installable with conda), and lunch the integration with R installed in the environment. Not ideal, but it might work according to an aforementioned issue

   • in the meantime (or alternatively), you can open an issue from the reticulate repository, they surely are more used to this kind of segmentation fault.

   Hope that helps! Please let me know how it goes, I'd be happy to hear what outcomes you get and assist further if I can.
   Florian