Skip to content

function_rpc_interface_invoke is not thread-safe (shared invoke_curl handle causes data race) #693

@Codingisinmyblud

Description

@Codingisinmyblud

🐛 Bug Report

Expected Behavior

Concurrent RPC function invocations from multiple threads should execute independently without interfering with each other's request parameters.

Current Behavior

All synchronous invocations share a single CURL *invoke_curl handle (rpc_loader_impl.cpp:68). The setopt + perform sequence in function_rpc_interface_invoke (l200-205) is not atomic.

If two threads interleave there, then one thread's URL can end up paired with the other thread's post body which might send the wrong payload to the wrong endpoint.

SOmething I found interesting is that (function_rpc_interface_await L386) already creates a per-call CURL * easy handle, so the async path does not have this problem.

Also, while looking at this function, I noticed the error log on L212 uses [%] as a format specifier. Shouldn't that be [%s]? The async path on L277 uses %s.

Possible Solution

Mirror what function_rpc_interface_await already does i.e. create a fresh CURL * easy handle per invocation instead of reusing the shared invoke_curl one. This eliminates the race entirely without needing a mutex.

Steps to Reproduce

  1. Initialize the RPC loader and discover functions from a remote endpoint.
  2. From two or more threads, concurrently invoke different RPC functions
  3. Observe that requests may be sent to the wrong URL, with the wrong body or crash due to concurrent mutation of the shared CURL * handle's internal state.

Context (Environment)

This came up while I was reviewing the RPC loader code in the for the Function Mesh GSoC project. In a mesh topology, multiple languages and threads will be invoking different RPC endpoints concurrently, this race condition is a blocker for that use case. The async path (await) is already safe since it allocates per-call handles, so this inconsistency between invoke and await doesn't look like it was ntentional.

Detailed Description

The (loader_impl_rpc_type struct) holds a single CURL *invoke_curl that is initialized once during rpc_loader_impl_initialize (L536) and then reused by every synchronous call. The CURLOPT_URL, CURLOPT_POSTFIELDS, CURLOPT_POSTFIELDSIZE, and CURLOPT_WRITEDATA options are all set on this shared handle immediately before curl_easy_perform.

The discover_curl handle has the same pattern but is less likely to be called concurrently since discovery typically happens once at load time.

Possible Implementation

Replace the shared nvoke_curl approach with per-call handles.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions