Skip to content

Commit daccec7

Browse files
feat: install jemalloc to replace glibc malloc for reduced memory fragmentation
Co-Authored-By: gl_anatolii.yatsuk <gl_anatolii.yatsuk@airbyte.io>
1 parent f40eca7 commit daccec7

1 file changed

Lines changed: 6 additions & 0 deletions

File tree

Dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,12 @@ RUN pip install dist/*.whl
2424
# This is a no-op unless DD_PROFILING_ENABLED or similar env vars are set at runtime.
2525
RUN pip install "ddtrace>=2.16,<3"
2626

27+
# Install jemalloc to replace glibc's malloc. jemalloc returns freed memory to the OS
28+
# much more aggressively than glibc, preventing unbounded RSS growth in long-running syncs
29+
# caused by native C allocation fragmentation (urllib3, OpenSSL, orjson, etc.).
30+
RUN apt-get update && apt-get install -y --no-install-recommends libjemalloc2 && rm -rf /var/lib/apt/lists/*
31+
ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2
32+
2733
# Recreate the original structure
2834
RUN mkdir -p source_declarative_manifest \
2935
&& echo 'from source_declarative_manifest.run import run\n\nif __name__ == "__main__":\n run()' > main.py \

0 commit comments

Comments
 (0)