`Code.eval_*` functions are not reentrant

[lukaszsamson]

Elixir and Erlang/OTP versions

Erlang/OTP 28 [erts-16.2] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [jit]

Elixir 1.20.0-rc.4 (808075d) (compiled with Erlang/OTP 28)

Operating system

any

Current behavior

Repro in iex:
Nested eval:

Code.eval_string("""
          Code.eval_string("1 + 1")
          IO.inspect(:erlang.get({:elixir, :eval_env}), label: :inside_after_inner)
          raise "boom"
        """, [], %{Code.env_for_eval([]) | file: "foo"})
inside_after_inner: :undefined
** (RuntimeError) boom
    (elixir 1.20.0-rc.4) src/elixir.erl:369: :elixir.eval_external_handler/3
    (stdlib 7.2) erl_eval.erl:924: :erl_eval.do_apply/7
    (elixir 1.20.0-rc.4) src/elixir.erl:355: :elixir.erl_eval/3
    (elixir 1.20.0-rc.4) src/elixir.erl:326: :elixir.eval_forms/4
    (elixir 1.20.0-rc.4) lib/module/parallel_checker.ex:156: Module.ParallelChecker.verify/1
    (elixir 1.20.0-rc.4) lib/code.ex:649: Code.validated_eval_string/4

No nested eval - works as expected

Code.eval_string("""
          IO.inspect(:erlang.get({:elixir, :eval_env}), label: :inside)
          raise "boom"
        """, [], %{Code.env_for_eval([]) | file: "foo"})
inside: #Macro.Env<...
** (RuntimeError) boom
    foo:3: (file)

Notice after nested eval eval_env gets cleaned up and stacktrace on error is wrong.

Reason: unconditional process dict erase in

elixir/lib/elixir/src/elixir.erl

Line 357 in 2d6e61c

erlang:erase(?elixir_eval_env)

The same behavior affects dbg_callback

elixir/lib/elixir/src/elixir.erl

Line 335 in 2d6e61c

erlang:erase({elixir, dbg_callback})

Expected behavior

Three alternatives:

1. The code fixed to store nested evals on stack
2. Code.eval_* documented that reentrancy is not supported and leads to broken behavior
3. Hard error/warning on nested eval in runtime

[josevalim]

Good find. We should make both reentrant by restoring the previous value. Please do send a PR?

[jonatanklosko]

We have the same issue with :dbg_callback:

elixir/lib/elixir/src/elixir.erl

Lines 334 to 336 in 2d6e61c

	after
	erlang:erase({elixir, dbg_callback})
	end.

It's less likely to run into, because it's used early during expansion, but it shows up with multiple inner Code.eval_* - the first one clears :dbg_callback, the second expands with the original callback. Example:

defmodule Test do
  def run() do
    opts = [dbg_callback: {__MODULE__, :dbg_callback_custom, []}]

    Code.eval_string(
      """
      dbg(:outer1) # OK
      IO.inspect(:erlang.get({:elixir, :dbg_callback}), label: :inside_before_inner)
      Code.eval_string("dbg(:inner1)") # OK
      IO.inspect(:erlang.get({:elixir, :dbg_callback}), label: :inside_after_inner)
      Code.eval_string("dbg(:inner2)") # WRONG - prints using the default :dbg_callback
      dbg(:outer2) # OK
      """,
      [],
      opts
    )
  end

  def dbg_callback_custom(code, _options, _caller) do
    quote do
      {unquote(code), :dbg_transformed}
    end
  end
end

Test.run()

In both cases we can store a stack of values and restore if there is anything to pop, clear otherwise.