Skip to content

Commit d969dc9

Browse files
Merge branch 'release/1.80.0'
2 parents dc1e819 + 98dc9de commit d969dc9

52 files changed

Lines changed: 966 additions & 553 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/R-CMD-check.yaml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ jobs:
2828
- {os: ubuntu-latest, r: 'oldrel-1' }
2929
- {os: ubuntu-latest, r: 'oldrel-2' }
3030
- {os: ubuntu-latest, r: '4.0' , testme_exclude: 'mpi' }
31+
- {os: ubuntu-latest, r: '3.6' , testme_exclude: 'mpi' }
3132

3233
- {os: ubuntu-latest, r: 'release' , strict: true, label: 'strict' }
3334

@@ -50,11 +51,12 @@ jobs:
5051
_R_COMPARE_LANG_OBJECTS: eqonly
5152
## R CMD check
5253
_R_CHECK_CRAN_INCOMING_: false
54+
_R_CHECK_CODOC_FUNCTIONS_MISSING_FROM_USAGES_: true
5355
_R_CHECK_MATRIX_DATA_: true
5456
_R_CHECK_SUGGESTS_ONLY_: true
5557
_R_CHECK_THINGS_IN_TEMP_DIR_: true
5658
_R_CHECK_TESTS_NLINES_: 300
57-
RCMDCHECK_ERROR_ON: error
59+
RCMDCHECK_ERROR_ON: note
5860
NOT_CRAN: true
5961
R_TESTME_EXCLUDE_NAME: ${{ matrix.config.testme_exclude }}
6062
## Specific to futures

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,4 @@ docs/
99
revdep/data.sqlite
1010
revdep/checks/*
1111
revdep/library/*
12+
.osgrep

‎DESCRIPTION‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
Package: future
2-
Version: 1.76.0
2+
Version: 1.80.0
33
Title: Unified Parallel and Distributed Processing in R for Everyone
44
Depends:
55
R (>= 3.2.0)

‎NEWS.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,50 @@
1+
# Version 1.80.0 [2026-10-15]
2+
3+
## Significant Changes
4+
5+
* Memory use on parallel workers has gone down significantly in some
6+
cases. A side effect is also significantly faster processing in
7+
those cases. This improvement has an immediate effect on several
8+
**future.apply** functions: `future_lapply()`, `future_sapply()`,
9+
`future_apply()`, `future_tapply()`, `future_by()`,
10+
`future_Filter()`, `future_eapply()`, `future_mapply()`, and
11+
`future_Map()`.
12+
13+
## Bug Fixes
14+
15+
* `cluster` and `multisession` futures could leave a stray, unclosed
16+
worker connection behind if a worker was relaunched. This would
17+
eventually result in warnings on "closing unused connection" when
18+
the garbage collector ran.
19+
20+
* Explicitly specifying globals as a character vector via attributes
21+
`add` or `ignore` would be ignored silently, e.g. `globals =
22+
structure(c("a", "b"), add = "c")`.
23+
24+
* A future that uses a global `...` together with a global
25+
`...future.FUN()` function, which is what **future.apply** uses,
26+
could fail with "cycles in parent chains are not allowed". In R (<
27+
4.5.0), it ended up in an infinite loop. This bug was introduced in
28+
**future** 1.40.0 (2025-04-10).
29+
30+
* A sequential future that uses a global `...` together with a global
31+
`...future.FUN()` function, which is what **future.apply** uses,
32+
could permanently change the parent of the function's environment,
33+
if that environment inherits from the global environment. This bug
34+
was introduced in **future** 1.40.0 (2025-04-10).
35+
36+
* `makeClusterFuture(globals = ...)` would result in an future
37+
evaluation error.
38+
39+
* Futures failed on R (< 4.0.0). This bug was introduced in **future**
40+
1.68.0 (2025-11-17).
41+
42+
* Globals identified automatically (the default) could carry extra,
43+
unneeded load that was exported to the parallel workers. This could
44+
result in significant overhead in both total memory use and total
45+
processing time.
46+
47+
148
# Version 1.76.0 [2026-09-24]
249

350
## New Features

‎R/000.re-exports.R‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,9 @@
99
#' switch to use the ones from the \pkg{parallelly} package. Thank you!_
1010
#'
1111
#' * [parallelly::availableCores()]
12-
# Used by: ale (1), BatchGetSymbols (2), codalm (1), cft (1), deseats (3),
12+
# Used by: ale (2), BatchGetSymbols (2), codalm (1), cft (1), deseats (3),
1313
# elevatr (1), heterogen, PINstimation (1), powRICLPM (1),
14-
# readsdr (1), recforest (3), sigminer (2), skpr (1), smoots (3),
14+
# readsdr (1), recforest (3), skpr (1), smoots (3),
1515
# TriDimRegression (1), uci (1)
1616
# [2025-12-22, 2026-01-31, 2026-04-04]
1717
#' * [parallelly::availableWorkers()]

‎R/backend_api-11.ClusterFutureBackend-class.R‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -407,6 +407,20 @@ stopWorkers.ClusterFutureBackend <- function(backend, interrupt = TRUE, ...) {
407407

408408
## Stop workers
409409
mdebugf_push("Stop cluster workers ...")
410+
411+
## SANITY CHECK: Assert that the backend's worker list is in sync with
412+
## the memoized cluster that clusterRegistry holds.
413+
## See also https://github.com/futureverse/future/issues/820 and
414+
## https://github.com/futureverse/future/issues/830.
415+
workers <- backend[["workers"]]
416+
registered <- clusterRegistry$getCluster(debug = debug)
417+
if (!is.null(registered) && !identical(workers, registered)) {
418+
warning(FutureWarning(sprintf(
419+
"Detected the workers of %s to be out of sync with the memoized cluster. This leaves stray, unclosed connections behind that will triggers a 'closing unused connection' warnings when the garbage collect runs",
420+
class(backend)[1]
421+
)))
422+
}
423+
410424
clusterRegistry$stopCluster(debug = debug)
411425
env <- environment(ClusterFutureBackend)
412426
env[["last"]] <- NULL
@@ -1231,6 +1245,7 @@ requestNode <- local({
12311245

12321246
workers[[node_idx]] <- node2
12331247
backend[["workers"]] <- workers
1248+
clusterRegistry$setCluster(workers)
12341249
node <- node2
12351250

12361251
if (debug) {
@@ -1609,6 +1624,7 @@ handleInterruptedFuture <- local({
16091624

16101625
## Update backend
16111626
backend[["workers"]] <- workers
1627+
clusterRegistry$setCluster(workers)
16121628

16131629
node <- NULL
16141630

@@ -1743,6 +1759,10 @@ clusterRegistry <- local({
17431759
cluster
17441760
} ## startCluster()
17451761

1762+
setCluster <- function(cl) {
1763+
cluster <<- cl
1764+
}
1765+
17461766
stopCluster <- function(debug = FALSE) {
17471767
if (debug) {
17481768
mdebug_push("Stopping existing cluster ...")
@@ -1800,6 +1820,7 @@ clusterRegistry <- local({
18001820
list(
18011821
getCluster = getCluster,
18021822
startCluster = startCluster,
1823+
setCluster = setCluster,
18031824
stopCluster = stopCluster
18041825
)
18051826
}) ## clusterRegistry()

‎R/backend_api-13.MultisessionFutureBackend-class.R‎

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@
1212
#'
1313
#' @keywords internal
1414
#' @rdname FutureBackend-class
15-
#" @importFrom parallelly supportsMulticore
1615
#' @export
1716
MultisessionFutureBackend <- function(workers = availableCores(constraints = "connections-16"), rscript_libs = .libPaths(), interrupts = TRUE, gc = FALSE, earlySignal = FALSE, ...) {
1817
debug <- isTRUE(getOption("future.debug"))
@@ -23,18 +22,14 @@ MultisessionFutureBackend <- function(workers = availableCores(constraints = "co
2322
on.exit(mdebugf_pop())
2423
}
2524

26-
default_workers <- missing(workers)
2725
if (is.function(workers)) workers <- workers()
2826
stop_if_not(is.numeric(workers))
2927
workers <- structure(as.integer(workers), class = setdiff(class(workers), c("integer", "numeric")))
3028
stop_if_not(length(workers) == 1, is.finite(workers), workers >= 1)
3129

3230
## Fall back to sequential futures if only a single additional R process
3331
## can be spawned off, i.e. then use the current main R process.
34-
## Sequential futures best reflect how multicore futures handle globals.
3532
if (workers == 1L && !inherits(workers, "AsIs")) {
36-
## AD HOC: Make sure plan(multicore) also produces a warning, if needed
37-
if (default_workers) supportsMulticore(warn = TRUE)
3833
## covr: skip=1
3934
return(SequentialFutureBackend(...))
4035
}

‎R/backend_api-Future-class.R‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -548,6 +548,10 @@ assertOwner <- local({
548548

549549
#' Run a future
550550
#'
551+
#' @usage
552+
#' run(future, ...)
553+
#' \method{run}{Future}(future, ...)
554+
#'
551555
#' @param future A \link{Future}.
552556
#' @param \ldots Not used.
553557
#'
@@ -581,7 +585,7 @@ run.Future <- function(future, ...) {
581585
## Be conservative for now; don't allow lazy futures created in another R
582586
## session to be launched. This will hopefully change later, but we won't
583587
## open this door until we understand the ramifications. /HB 2020-12-21
584-
if (isTRUE(getOption("future.lazy.assertOwner"))) {
588+
if (isTRUE(getOption("future.lazy.assertOwner", TRUE))) {
585589
assertOwner(future)
586590
}
587591

‎R/backend_api-FutureRegistry.R‎

Lines changed: 8 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -53,28 +53,6 @@ FutureRegistry <- local({
5353
on.exit(mdebugf_pop())
5454
}
5555

56-
futuresDB <- db[[where]]
57-
drop <- integer(0L)
58-
59-
on.exit({
60-
## Clean out collected futures
61-
if (length(drop) > 0) {
62-
if (debug) {
63-
mdebug_push("Remove collected futures ...")
64-
mdebugf("Indices of futures to drop: [n=%d] %s", length(drop), commaq(drop))
65-
}
66-
futuresDB <- futuresDB[-drop]
67-
db[[where]] <<- futuresDB
68-
if (debug) mdebug_pop()
69-
} else {
70-
if (debug) mdebug("Indices of futures to drop: [n=0] <none>")
71-
}
72-
}, add = TRUE)
73-
74-
if (debug) {
75-
on.exit(mdebugf_pop(), add = TRUE)
76-
}
77-
7856
for (ii in seq_along(futures)) {
7957
future <- futures[[ii]]
8058

@@ -118,31 +96,10 @@ FutureRegistry <- local({
11896

11997
## (b) Make sure future is removed from registry, unless
12098
## already done via above value() call.
121-
idx <- indexOf(futuresDB, future = future)
99+
idx <- indexOf(db[[where]], future = future)
122100
if (!is.na(idx)) {
123-
drop <- c(drop, idx)
124-
125-
## Update backend
126-
backend <- future[["backend"]]
127-
if (!is.null(backend)) {
128-
## Update counters
129-
counters <- backend[["counters"]]
130-
counters["finished"] <- counters["finished"] + 1L
131-
backend[["counters"]] <- counters
132-
133-
## Update total runtime
134-
result <- future[["result"]]
135-
if (inherits(result, "FutureResult")) {
136-
times <- result[c("finished", "started")]
137-
if (length(times) == 2L) {
138-
dt <- times[["finished"]] - times[["started"]]
139-
runtime <- backend[["runtime"]]
140-
runtime <- runtime + dt
141-
backend[["runtime"]] <- runtime
142-
}
143-
}
144-
}
145-
} ## if (!is.na(idx))
101+
FutureRegistry(where, action = "remove", future = future, earlySignal = FALSE, debug = debug)
102+
}
146103

147104
## (c) Collect only the first resolved future?
148105
if (firstOnly) {
@@ -235,11 +192,14 @@ FutureRegistry <- local({
235192
}
236193
} else if (action == "collect-first") {
237194
collectValues(where, futures = futures, firstOnly = TRUE, debug = debug)
195+
futures <- db[[where]]
238196
} else if (action == "collect-all") {
239197
collectValues(where, futures = futures, firstOnly = FALSE, debug = debug)
198+
futures <- db[[where]]
240199
earlySignal <- FALSE ## Avoid second round of result collection
241200
} else if (action == "reset") {
242-
db[[where]] <<- list()
201+
futures <- list()
202+
db[[where]] <<- futures
243203
if (debug) mdebug("Erased registry")
244204
} else if (action == "list") {
245205
if (debug) mdebug("Listing all futures")
@@ -263,6 +223,7 @@ FutureRegistry <- local({
263223
## Collect values, which will trigger signaling during
264224
## calls to resolved().
265225
collectValues(where, futures = futures[idxs], firstOnly = FALSE)
226+
futures <- db[[where]]
266227
}
267228
if (debug) mdebugf("Early signaling of %d future candidates ... done", length(futures))
268229
}

‎R/core_api-cancel.R‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,28 @@
1-
#' Cancel a future
1+
#' Cancel one or more futures
22
#'
33
#' Cancels futures, with the option to interrupt running ones.
4+
#' `cancel()` is vectorized, i.e. accepts a collection of futures
5+
#' to be canceled.
46
#'
5-
#' @param x A Future.
7+
#' @param x A [Future], a list, an environment, or a list environment
8+
#' (\link[listenv:listenv]{listenv}). Lists and environments may hold
9+
#' a mix of futures and non-future elements, and may be nested, e.g.
10+
#' a list of lists of futures; every future found, at any depth, is
11+
#' canceled.
612
#'
713
#' @param interrupt If TRUE, running futures are interrupted, if the
814
#' future backend supports it.
915
#'
1016
#' @param \ldots All arguments used by the S3 methods.
1117
#'
1218
#' @return
13-
#' `cancel()` returns (invisibly) the canceled [Future]s after
14-
#' flagging them as "canceled" and possibly interrupting them as well.
19+
#' `cancel()` returns (invisibly) `x` after canceling every [Future]
20+
#' therein, that is, after flagging each one as "canceled" and
21+
#' possibly interrupting it as well.
1522
#'
16-
#' Canceling a lazy or a finished future has no effect.
23+
#' Canceling a lazy or a finished future has no effect. Elements of
24+
#' `x` that are neither futures nor collections of futures are left
25+
#' untouched.
1726
#'
1827
#' @examplesIf (interactive() || .Platform[["OS.type"]] != "windows")
1928
#' ## Set up two parallel workers
@@ -30,7 +39,7 @@
3039
#' ## Wait until at least one of the futures is resolved
3140
#' while (!any(resolved(fs))) Sys.sleep(0.1)
3241
#'
33-
#' ## Cancel the future that is not yet resolved
42+
#' ## Cancel futures that are not yet resolved
3443
#' r <- resolved(fs)
3544
#' cancel(fs[!r])
3645
#'

0 commit comments

Comments
 (0)