ci: Response Cache memory growth test

[pskiran1]

What does the PR do?

• Added a memory growth test for when the response cache is enabled.
• The script generate_random_data.py creates random data to trigger cache misses, insertions, and evictions.
• The new test case checks if memory growth is within normal limits.

Checklist

• PR title reflects the change and is of format <commit_type>: <Title>
• Changes are described in the pull request.
• Related issues are referenced.
• Populated github labels field
• Added test plan and verified test passes.
• Verified that the PR passes existing CI.
• Verified copyright is correct on all changed files.
• Added succinct git squash message before merging ref.
• All template sections are filled out.
• Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

• build
• ci
• docs
• feat
• fix
• perf
• refactor
• revert
• style
• test

Related PRs: triton-inference-server/core#449

Where should the reviewer start?

Test plan:

• CI Pipeline ID: 33813824

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

Test script output - Without memory leak fix (Memory grows over time):

Test script output - With memory leak fix (Memory is stable):

[pskiran1]

I added this threshold value based on observations from example runs. Please let me know if you have any recommendations for configuring it.
Thank you.

[yinggeh]

What's the averge memory growth before your fix?

[pskiran1]

It exceeds 3MB on running this test. For reference, I attached a screenshot showing memory growth before the fix.
If the fix is included, it is less than 0.5 MB

[yinggeh]

Nice work