Skip to content

challenges with number of cores being used in tensorflow #625

@njtierney

Description

@njtierney

I've been running into some issues getting greta onto CRAN, essentially to a NOTE from CRAN saying that the vignettes take longer to run, suggesting that I'm using more than 2 cores.

I've tried to change the number of cores to be set to 2 - see e.g.,

greta-dev/greta@2f74868

greta-dev/greta@0ca5edf

greta-dev/greta@f96247d#diff-f59966e6d97fc8ef5e8462d139ea656d75a5570bcd82369147e848d925d5ddfe

And I've also tried to set the number of cores used in the vignettes: https://github.com/greta-dev/greta/blob/main/vignettes/get_started.Rmd#L10-L17

It does make me wonder if perhaps something in https://github.com/greta-dev/greta/blob/989b74670fce35e1802ae939da712d111aee114b/R/zzz.R#L30-L35 is triggering a large CPU load?

I also posted a bit about this here: https://forum.posit.co/t/cpu-time-xx-times-elapsed-time/210116/3

Below is an exchange from R Devel, in case this is useful

https://stat.ethz.ch/pipermail/r-package-devel/2026q1/012239.html

@t-kalinowski in a private email exchange said that the following might be a solution, to call the following after tf is loaded:

https://www.tensorflow.org/api_docs/python/tf/config/threading/set_intra_op_parallelism_threads
https://www.tensorflow.org/api_docs/python/tf/config/threading/set_inter_op_parallelism_threads

To have something like this triggered from .onLoad():

if (is_cran())
  reticulate::py_register_load_hook("tensorflow", function() {
    reticulate::py_run_string(
      local = TRUE,
      glue::trim(
        r"---(
        import tensorflow as tf
        tf.config.set_visible_devices(tf.config.get_visible_devices("CPU"))
        tf.config.threading.set_inter_op_parallelism_threads(1)
        tf.config.threading.set_intra_op_parallelism_threads(1)
        )---"
      )
    )
  })

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions