Skip to content

KB max batch size: apply insert limit globally#11905

Merged
ea-rus merged 2 commits into
releases/25.11.0from
max-insert-batch
Nov 19, 2025
Merged

KB max batch size: apply insert limit globally#11905
ea-rus merged 2 commits into
releases/25.11.0from
max-insert-batch

Conversation

@ea-rus
Copy link
Copy Markdown
Collaborator

@ea-rus ea-rus commented Nov 18, 2025

Description

Continue of #11463

Changes:

  • Check insert limit globally: SQL, rest
  • Use limit from env var: KB_MAX_INSERT_BATCH_SIZE
    • default value is 50k

Fixes https://linear.app/mindsdb/issue/FQE-1776/apply-knowledge-base-upload-limits-consistently

Type of change

  • 🐛 Bug fix (non-breaking change which fixes an issue)

Verification Process

To ensure the changes are working as expected:

  • Test Location: Specify the URL or path for testing.
  • Verification Steps: Outline the steps or queries needed to validate the change. Include any data, configurations, or actions required to reproduce or see the new functionality.

Additional Media:

  • I have attached a brief loom video or screenshots showcasing the new functionality or change.

Checklist:

  • My code follows the style guidelines(PEP 8) of MindsDB.
  • I have appropriately commented on my code, especially in complex areas.
  • Necessary documentation updates are either made or tracked in issues.
  • Relevant unit and integration tests are updated or added.

env var: KB_MAX_INSERT_BATCH_SIZE
@ea-rus ea-rus requested a review from StpMax November 18, 2025 11:38
@entelligence-ai-pr-reviews
Copy link
Copy Markdown
Contributor

Entelligence AI Vulnerability Scanner

Status: No security vulnerabilities found

Your code passed our comprehensive security analysis.

Analyzed 2 files in total

@entelligence-ai-pr-reviews
Copy link
Copy Markdown
Contributor

Review Summary

🏷️ Draft Comments (1)

Skipped posting 1 draft comments that were valid but scored below your review threshold (>=13/15). Feel free to update them here.

mindsdb/interfaces/agents/constants.py (1)

211-211: os.getenv("KB_MAX_INSERT_BATCH_SIZE", 50_000) uses an integer as the default, but os.getenv expects a string or None, which can cause a TypeError at runtime if the environment variable is not set.

📊 Impact Scores:

  • Production Impact: 4/5
  • Fix Specificity: 5/5
  • Urgency Impact: 3/5
  • Total Score: 12/15

🤖 AI Agent Prompt (Copy & Paste Ready):

In mindsdb/interfaces/agents/constants.py, line 211, the code uses `os.getenv("KB_MAX_INSERT_BATCH_SIZE", 50_000)` with an integer as the default value. This can cause a TypeError at runtime because os.getenv expects a string or None as the default. Change the default value to a string: `os.getenv("KB_MAX_INSERT_BATCH_SIZE", "50000")`.

@ea-rus ea-rus changed the title apply insert limit globally KB max batch size: apply insert limit globally Nov 18, 2025
Copy link
Copy Markdown
Collaborator

@StpMax StpMax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to process env var in one place - in the config.py

@ea-rus ea-rus merged commit 384ed76 into releases/25.11.0 Nov 19, 2025
34 of 35 checks passed
@ea-rus ea-rus deleted the max-insert-batch branch November 19, 2025 17:26
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 19, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants