Error: Cannot find progress bar (req_perform_parallel)

[JBGruber]

When I run a req_perform_parallel with a progress bar and abort the operation, every new call to req_perform_parallel will error until the R session is restarted. Here is a minimal example:

library(httr2)
reqs <- list(
  request(example_url()) |>
    req_url_path("/delay") |> 
    req_url_path_append(sample(1:10, 1))
) |>
  rep(100)

resps <- req_perform_parallel(reqs, progress = TRUE)

### stop with Esc ##
resps <- req_perform_parallel(reqs, progress = TRUE)
#> Error in cli::cli_progress_update(..., id = id) : 
#>   Cannot find progress bar `cli-25573-7`
#> Error in cli::cli_progress_update(..., id = id) : 
#>   Cannot find progress bar `cli-25573-7`

This does not apply to req_perform_sequential.

[hadley]

Hmmm create_progress_bar() should probably add an exit handler to the calling environment that automatically closes the progress bar. But I'm a bit surprised that's necessary because I don't understand why it would try and reuse the previous progress bar.

[JBGruber]

I also have no idea. And here is some more weirdness. I just noticed that it does not happen every time! If I hit Esc too fast, it does not seem to reuse the id???

Screencast_20241211_183723.webm

[hadley]

@gaborcsardi any idea what's going on here? The progress bar is created in a helper function:

httr2/R/utils.R

Lines 242 to 284 in e972770

	create_progress_bar <- function(total,
	name,
	config,
	env = caller_env(),
	config_arg = caller_arg(config),
	error_call = caller_env()) {
	if (is_false(config)) {
	return(list(
	update = function(...) {},
	done = function() {}
	))
	}
	if (is.null(config) || is_bool(config)) {
	args <- list()
	} else if (is_scalar_character(config)) {
	args <- list(name = config)
	} else if (is.list(config)) {
	args <- config
	} else {
	stop_input_type(
	config,
	what = c("a bool", "a string", "a list"),
	arg = config_arg,
	call = error_call
	)
	}
	args$name <- args$name %||% name
	# Can be removed if https://github.com/r-lib/cli/issues/630 is fixed
	if (is.infinite(total)) {
	total <- NA
	}
	args$total <- total
	args$.envir <- env
	id <- exec(cli::cli_progress_bar, !!!args)
	list(
	update = function(...) cli::cli_progress_update(..., id = id),
	done = function() cli::cli_progress_done(id = id)
	)
	}

Maybe something is going wrong with how I set .envir?

[gaborcsardi]

I am not sure, but my guess is that the progress bar's env is already removed, but the curl multi-handle is still alive and you get a final update() call for it.

[hadley]

Ooooh I bet that's it.

[hadley]

I think we can fix this by using a separate pool for each req_perform_parallel() call, rather than relying on the default global pool.

[gaborcsardi]

I think you can also tryCatch() and ignore the error.

[hadley]

I can't reliably reproduce this. I tried what I thought would illustrate it more reliably:

library(httr2)
options(cli.progress_show_after = 0)

reqs <- list(
  request(example_url()) |> req_url_path("/delay/10"),
  request(example_url()) |> req_url_path("/delay/1"),
  request(example_url()) |> req_url_path("/delay/10"),
  request(example_url()) |> req_url_path("/delay/1"),
  request(example_url()) |> req_url_path("/delay/10"),
  request(example_url()) |> req_url_path("/delay/1")
) 
resps <- req_perform_parallel(reqs, progress = TRUE)

But I only manage to see the error once, so I suspect it might be more of a problem on windows.

[JBGruber]

I'm on Arch Linux (BTW)

It only seems to happen when I interrupt the process after a couple of seconds and not every time. Which makes this the weirdest behaviour that I've experienced so far in R.

Can you start another process after you see the error once? I need to restart R.

[hadley]

But my experiments suggest that there might be a bigger problem — I think terminating req_perform_parallel() doesn't actually terminate the other ongoing requests.

...

Hmmmm, some experiments suggest that maybe that's not the case?