Hey everyone,
I encountered an unexpected behavior while working with instrumentation-langchain.
When using @traceloop/instrumentation-langchain] alongside LangChain's [RunnableWithMessageHistory], [CallbackManager._configureSync] causes two issues:
- Drops all existing callback handlers when [inheritableHandlers] is a [CallbackManager] instance (not an array).
- Duplicates [TraceloopCallbackHandler] on child runnable invocations, resulting in duplicate spans/traces sent to the backend.
Root Cause
In patchCallbackManager, the patched [_configureSync] assumes [inheritableHandlers] is always null or an [Array]:
callbackManagerAny._configureSync = function (inheritableHandlers, localHandlers, ...) {
const updatedInheritableHandlers = inheritableHandlers && Array.isArray(inheritableHandlers)
? [...inheritableHandlers, callbackHandler]
: [callbackHandler]; // ← BUG: discards CallbackManager instance entirely
return originalConfigureSync.call(this, updatedInheritableHandlers, localHandlers, ...);
};
However, LangChain internally passes a CallbackManager instance (via runManager.getChild() → patchConfig()) as inheritableHandlers. When Array.isArray(CallbackManager) returns false, the patch replaces it with [callbackHandler], discarding all existing handlers — including RootListenersTracer used by RunnableWithMessageHistory to persist chat history messages.
Additionally, the handler is injected unconditionally without checking if a TraceloopCallbackHandler was already inherited from a parent runnable, causing duplicate traces.
Impact
- Chat history not persisted:
RunnableWithMessageHistory._exitHistory is never called because RootListenersTracer is dropped
- Duplicate telemetry data: Each child runnable invocation adds another
TraceloopCallbackHandler, multiplying spans sent to the observability backend
Steps to Reproduce
- Initialize traceloop with any exporter:
traceloop.initialize({ appName: 'test', exporter: someExporter });
- Create a chain wrapped with
RunnableWithMessageHistory:
const chainWithHistory = new RunnableWithMessageHistory({
runnable: prompt.pipe(llm).pipe(new StringOutputParser()),
getMessageHistory: (_) => chatHistory,
inputMessagesKey: 'question',
historyMessagesKey: 'history',
});
- Invoke or stream the chain:
await chainWithHistory.invoke({ question: 'Hello' }, { configurable: { sessionId: 'abc' } });
Expected: Messages are persisted to chat history; single set of traces sent to backend
Actual: Messages are NOT persisted (Bug 1); OR if workaround applied for Bug 1, duplicate traces are sent (Bug 2)
Suggested Fix
In patchCallbackManager, before injecting the handler:
- When
inheritableHandlers is a CallbackManager instance, extract its inheritableHandlers array instead of discarding it
- Filter out any existing
TraceloopCallbackHandler (by name === 'traceloop_callback_handler') before adding a new one to prevent duplication
callbackManagerAny._configureSync = function (inheritableHandlers, localHandlers, ...rest) {
let handlers;
if (inheritableHandlers && !Array.isArray(inheritableHandlers)) {
// Extract handlers from CallbackManager instance
handlers = [...(inheritableHandlers.inheritableHandlers || [])];
} else {
handlers = inheritableHandlers ? [...inheritableHandlers] : [];
}
// Deduplicate: remove inherited TraceloopCallbackHandler before adding a fresh one
handlers = handlers.filter(h => h?.name !== 'traceloop_callback_handler');
handlers.push(callbackHandler);
return originalConfigureSync.call(this, handlers, localHandlers, ...rest);
};
Environment
@traceloop/node-server-sdk: latest
@traceloop/instrumentation-langchain: 0.24.0
@langchain/core: >=1.0.0
Hey everyone,
I encountered an unexpected behavior while working with instrumentation-langchain.
When using @traceloop/instrumentation-langchain] alongside LangChain's [RunnableWithMessageHistory], [CallbackManager._configureSync] causes two issues:
Root Cause
In patchCallbackManager, the patched [_configureSync] assumes [inheritableHandlers] is always null or an [Array]:
However, LangChain internally passes a
CallbackManagerinstance (viarunManager.getChild()→patchConfig()) asinheritableHandlers. WhenArray.isArray(CallbackManager)returnsfalse, the patch replaces it with[callbackHandler], discarding all existing handlers — includingRootListenersTracerused byRunnableWithMessageHistoryto persist chat history messages.Additionally, the handler is injected unconditionally without checking if a
TraceloopCallbackHandlerwas already inherited from a parent runnable, causing duplicate traces.Impact
RunnableWithMessageHistory._exitHistoryis never called becauseRootListenersTraceris droppedTraceloopCallbackHandler, multiplying spans sent to the observability backendSteps to Reproduce
RunnableWithMessageHistory:Expected: Messages are persisted to chat history; single set of traces sent to backend
Actual: Messages are NOT persisted (Bug 1); OR if workaround applied for Bug 1, duplicate traces are sent (Bug 2)
Suggested Fix
In
patchCallbackManager, before injecting the handler:inheritableHandlersis aCallbackManagerinstance, extract itsinheritableHandlersarray instead of discarding itTraceloopCallbackHandler(byname === 'traceloop_callback_handler') before adding a new one to prevent duplicationEnvironment
@traceloop/node-server-sdk: latest@traceloop/instrumentation-langchain: 0.24.0@langchain/core: >=1.0.0