future (>= 1.24.0): Now future(…, seed = TRUE) forwards the RNG state

[HenrikBengtsson]

Hi. FYI, future 1.24.0 is now on CRAN. A signficant change is:

• Now future(…, seed = TRUE) forwards the RNG state in the calling R session. Previously, it would leave it intact.

So, your https://berkeley-scf.github.io/tutorial-dask-future/R-future#82-the-seed-when-using-futures-directly section can now be updated, because we now get:

library(future)
plan(multisession,workers=4)   # or some other plan
set.seed(1)
n <- 10
out <- list(); length(out) <- n
for(i in seq_len(n)) {
     out[[i]] <- future( {
       ## some code here as in foreach
       tmp <- rnorm(1e7)
       c(mean(tmp), sd(tmp))
     }, seed = TRUE)
}
sapply(out, value)
#>               [,1]          [,2]          [,3]         [,4]         [,5]
#> [1,] -7.323433e-05 -0.0001715487 -0.0005094992 0.0002571283 0.0003831295
#> [2,]  9.999894e-01  1.0001384289  1.0000917354 1.0001548200 0.9999650848
#>              [,6]         [,7]         [,8]         [,9]         [,10]
#> [1,] 2.627626e-05 2.848467e-06 0.0002893493 0.0004919797 -0.0002516645
#> [2,] 1.000183e+00 1.000202e+00 0.9996883582 1.0001180154  0.9998334065

[paciorek]

@HenrikBengtsson Thanks for letting me know, but I'm confused about a couple things:

1. In v. 1.24.0, using seed = <some_number> (or seed = <some_L'Ecuyer seed>) results in each worker giving back the same results (see reprex below), which seems to contract help(future):
   "If the latter, then a L'Ecuyer-CMRG seed will be automatically created based on the given seed."

plan(multisession,workers=4)   
n <- 5
out <- list(); length(out) <- n
for(i in seq_len(n)) {
     out[[i]] <- future( {
       rnorm(1)
     }, seed = 2)
}
sapply(out, value)
# [1] 0.9398208 0.9398208 0.9398208 0.9398208 0.9398208

2. When I wrote that tutorial, I thought one could safely use seed = TRUE with future() as I do in the code in the tutorial. So I'm confused as to what has changed in 1.24.0 and what you think I need to change in my example code in the tutorial.

[HenrikBengtsson]

1. In v. 1.24.0, using `seed = <some_number>` (or `seed = <some_L'Ecuyer seed>`) results in each worker giving back the same results (see reprex below), which seems to contract `help(future)`:
   "If the latter, then a L'Ecuyer-CMRG seed will be automatically created based on the given seed."

If you specify a constant seed (basic integer or L'Ecuyer-CMRG seed), you'll end up starting out with a (bijective) constant .Random.seed, when resolving the future. Thus, if all futures use the same seed, then they'll also use the same .Random.seed, which gives the same random numbers.

So, the results in your example above expected, and has always worked that way as far as I can tell.

2. When I wrote that tutorial, I thought one could safely use `seed = TRUE` with `future()` as I do in the code in the tutorial.

It was only "safe" to use seed = TRUE in the sense that you didn't get warning on using the RNG when you didn't intend to. However, statistically, it was not safe, because contrary to:

a <- rnorm(1)
b <- rnorm(1)

where you get two different random numbers, you didn't get that with:

a %<-% rnorm(1) %seed% TRUE
b %<-% rnorm(1) %seed% TRUE

I think most people would assume them to be different, not the same. That is what is fixed in future 1.24.0. No need for people to roll their own L’Ecuyer-CMRG seeds.

PS. From the way you phrased https://github.com/berkeley-scf/tutorial-dask-future, I assumed you interpreted the future (< 1.24.0) behavior this way.

[paciorek]

ok, thanks. I'm still a bit confused about what I was thinking previously, but how things work now seems clear and useful.