Skip to content

chore(security): update vulnerable packages#323

Merged
Ryuk-me merged 2 commits into
datachecks:mainfrom
Ryuk-me:neeraj/dcs-1196-dcs-core-dependabot-isues
Sep 2, 2025
Merged

chore(security): update vulnerable packages#323
Ryuk-me merged 2 commits into
datachecks:mainfrom
Ryuk-me:neeraj/dcs-1196-dcs-core-dependabot-isues

Conversation

@Ryuk-me

@Ryuk-me Ryuk-me commented Sep 2, 2025

Copy link
Copy Markdown
Member

PR Type

Enhancement


Description

  • Update vulnerable Python packages to secure versions

  • Add new security-focused dependencies to Python project

  • Update JavaScript/Node.js packages for security fixes

  • Remove optional flag from elasticsearch dependency


Diagram Walkthrough

flowchart LR
  A["Security Scan"] --> B["Python Dependencies"]
  A --> C["JavaScript Dependencies"]
  B --> D["Update requests, elasticsearch"]
  B --> E["Add h11, protobuf, tornado, etc."]
  C --> F["Update babel, add braces, nanoid"]
Loading

File Walkthrough

Relevant files
Dependencies
pyproject.toml
Update Python dependencies for security                                   

pyproject.toml

  • Update requests from ^2.31.0 to ^2.32.4
  • Update elasticsearch from ^7.17.3 to ^9.1.0 and remove optional flag
  • Add new dependencies: h11, protobuf, tornado, setuptools, urllib3,
    jinja2
+8/-2     
package.json
Update JavaScript dependencies for security                           

ui/package.json

  • Add new Babel runtime packages for security
  • Add braces, cross-spawn, micromatch, nanoid dependencies
  • Add serialize-javascript and tmp packages
+9/-0     

Summary by CodeRabbit

  • Chores

    • Updated backend and UI dependencies to current versions, improving stability and maintainability.
    • Upgraded core HTTP and search libraries to enhance compatibility with newer services.
    • Added supporting libraries to streamline builds and development workflows.
  • Bug Fixes

    • Improved reliability in network interactions via updated dependency stack.
  • Performance

    • Minor build and runtime optimizations from refreshed dependency set.
  • Security

    • Incorporates upstream security patches through dependency upgrades.

@coderabbitai

coderabbitai Bot commented Sep 2, 2025

Copy link
Copy Markdown

Walkthrough

Dependency updates only. Python backend: version bumps for requests and elasticsearch, plus added libraries. UI frontend: added several JS dependencies. No source code, scripts, or public API declarations changed.

Changes

Cohort / File(s) Summary of Changes
Backend dependencies (Python)
pyproject.toml
Bumped requests ^2.31.0 → ^2.32.4; elasticsearch ^7.17.3 → ^9.1.0. Added h11 ^0.16.0, protobuf ^5.29.5, tornado ^6.5, setuptools ^78.1.1, urllib3 ^2.5.0, jinja2 ^3.1.6.
Frontend dependencies (UI)
ui/package.json
Added @babel/helpers ^7.26.10, @babel/runtime ^7.26.10, @babel/runtime-corejs3 ^7.26.10, braces ^3.0.3, cross-spawn ^7.0.5, micromatch ^4.0.8, nanoid ^3.3.8, serialize-javascript ^6.0.2, tmp ^0.2.4.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

I hop through fields of version vines,
Pluck fresh deps in tidy lines. 🥕
requests and ES, a sprightly pair—
new friends join the burrow’s lair.
Frontend twinkles, Babel bright,
Micromatch maps the starry night.
Ship it swift—my whiskers twitch with delight!

✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@qodo-code-review

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 Security concerns

Supply-chain risk:
Ensure all newly added JS and Python packages are vetted. In package.json, adding serialize-javascript is safe at ^6.0.2, but verify no older transitive versions are pulled. Confirm urllib3 (^2.5.0) aligns with requests ^2.32.4 to avoid downgrades. Removing optional=true for elasticsearch increases the attack surface for consumers who don't need it.

⚡ Recommended focus areas for review

Breaking Change Risk

Upgrading elasticsearch to ^9.1.0 and removing the optional flag may introduce API incompatibilities and force-install the client for all users. Confirm runtime compatibility (auth, transport, response handling) and whether the dependency should remain optional.

elasticsearch = "^9.1.0"
pymysql = { version="^1.1.0", optional=true, extras=["rsa"] }
Dependency Scope

Adding setuptools, urllib3, jinja2, tornado, h11, and protobuf to main dependencies can bloat install size and create transitive version constraints. Verify these are required at runtime (not only for build/dev) and pin ranges compatible with Python 3.9–3.12.

h11 = "^0.16.0"
protobuf = "^5.29.5"
tornado = "^6.5"
setuptools = "^78.1.1"
urllib3 = "^2.5.0"
jinja2 = "^3.1.6"
Version Hygiene

Newly added packages mix caret ranges with older ecosystem versions; check for compatibility with the existing Webpack/Jest/Babel toolchain and remove the stray newline in the nanoid version string to avoid JSON or install issues.

"micromatch": "^4.0.8",
"nanoid": "^3.3.8\n",
"plotly.js-cartesian-dist-min": "^2.9.0",
"react": "^18.2.0",
"react-app-polyfill": "^1.0.6",
"react-dom": "^18.2.0",
"react-router-dom": "^6.16.0",
"serialize-javascript": "^6.0.2",
"style-loader": "^3.3.3",
"tmp": "^0.2.4",
"ts-loader": "^9.5.1",
"webpack": "^5.94.0"

@Ryuk-me Ryuk-me changed the title chore(security): update vulnerable packeges chore(security): update vulnerable packages Sep 2, 2025
@qodo-code-review

qodo-code-review Bot commented Sep 2, 2025

Copy link
Copy Markdown

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
High-level
Keep elasticsearch optional; avoid transitive pins

Making elasticsearch a mandatory dependency and jumping from 7.x to 9.x is a
breaking, cross-cutting change; keep it optional (as an extra) and confirm code
compatibility/migration for the 9.x client. Additionally, avoid top-level
pinning of transitive packages like urllib3, protobuf, tornado, and jinja2
unless they are directly used—these constraints can trigger widespread
dependency conflicts. If security is the goal, prefer upgrading primary
libraries and letting the resolver select compatible transitive versions or gate
them behind optional extras.

Examples:

pyproject.toml [40-55]
elasticsearch = "^9.1.0"
pymysql = { version="^1.1.0", optional=true, extras=["rsa"] }
snowflake-sqlalchemy = { version="^1.5.3", optional=true }
pyodbc = { version="^5.0.1", optional=true }
psycopg2-binary = {version = "^2.9.9", optional = true}
pyspark = { version="^3.2.1", optional=true }
oracledb = { version=">=1.1.1,<3.0.0", optional=true }
ibm-db-sa = {version = "^0.4.1", optional = true}
ibm-db = {version = "^3.2.3", optional = true}
sqlalchemy-sybase = {version = "^2.0.0", optional = true}

 ... (clipped 6 lines)

Solution Walkthrough:

Before:

# pyproject.toml
[tool.poetry.dependencies]
...
# elasticsearch was optional, now it's mandatory with a major version bump
elasticsearch = "^9.1.0"
...
# Transitive dependencies are pinned at the top level
h11 = "^0.16.0"
protobuf = "^5.29.5"
tornado = "^6.5"
setuptools = "^78.1.1"
urllib3 = "^2.5.0"
jinja2 = "^3.1.6"
...

After:

# pyproject.toml
[tool.poetry.dependencies]
...
# Keep elasticsearch optional to avoid breaking changes for users
# who don't need it. The version bump should be carefully tested.
elasticsearch = { version="^9.1.0", optional=true }
...
# Remove transitive dependencies. Let the dependency resolver
# manage their versions based on primary dependencies.
# (h11, protobuf, tornado, etc. are removed from here)
...
Suggestion importance[1-10]: 9

__

Why: This suggestion correctly identifies two critical issues: making elasticsearch a mandatory dependency with a major version bump is a significant breaking change, and explicitly pinning transitive dependencies is poor practice that can cause future conflicts.

High
Possible issue
Remove stray newline in version

Remove the embedded newline escape from the version string. Leaving it can break
semver parsing and cause install failures. Use a clean version string.

ui/package.json [37]

-"nanoid": "^3.3.8\n",
+"nanoid": "^3.3.8",
  • Apply / Chat
Suggestion importance[1-10]: 9

__

Why: The suggestion correctly identifies a newline character typo in the nanoid version string within package.json, which would likely cause dependency installation to fail.

High
General
Keep Elasticsearch as optional
Suggestion Impact:The commit changed the elasticsearch dependency specification to include version and optional=true, making it optional as suggested.

code diff:

-elasticsearch = "^9.1.0"
+elasticsearch = { version="^9.1.0", optional=true }

Restore the optional flag to avoid forcing an Elasticsearch client install for
users who don't need it. This preserves previous install behavior and prevents
unintended breakage.

pyproject.toml [40]

-elasticsearch = "^9.1.0"
+elasticsearch = { version="^9.1.0", optional=true }

[Suggestion processed]

Suggestion importance[1-10]: 7

__

Why: The suggestion correctly identifies that the PR makes elasticsearch a mandatory dependency, whereas it was previously optional, which is a significant and likely unintended change to the project's installation requirements.

Medium
  • Update

@Ryuk-me Ryuk-me merged commit 0631d48 into datachecks:main Sep 2, 2025
4 of 7 checks passed
@Ryuk-me Ryuk-me deleted the neeraj/dcs-1196-dcs-core-dependabot-isues branch September 2, 2025 09:56

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (5)
pyproject.toml (1)

50-55: Remove unnecessary runtime dependencies.

  • setuptools → move to [build-system] requires.
  • urllib3 → use as a dev dependency only; remove from runtime deps.
  • h11, protobuf, tornado, jinja2 → no direct imports in production code; remove pins and, if needed, manage them via constraints or re-declare only when actually used.
ui/package.json (4)

7-9: Align @babel packages to avoid subtle build/runtime mismatches.

Core is 7.9.0 while helpers/runtime are 7.26.10. Either bump @babel/core to 7.26.x or pin helpers/runtime near 7.9.x.

Two options:

A) Bump core:

-"@babel/core": "7.9.0",
+"@babel/core": "^7.26.10",

B) Pin runtimes closer to core:

-"@babel/helpers": "^7.26.10",
-"@babel/runtime": "^7.26.10",
-"@babel/runtime-corejs3": "^7.26.10",
+"@babel/helpers": "^7.9.0",
+"@babel/runtime": "^7.9.0",
+"@babel/runtime-corejs3": "^7.9.0",

43-43: Move serialize-javascript to devDependencies
serialize-javascript isn’t imported by any application code and is only pulled in by build tooling; remove it from dependencies and add it under devDependencies in ui/package.json.


23-23: Move cross-spawn to devDependencies: it isn’t imported at runtime in the ui/ codebase (no require/import or crossSpawn usage) and is only needed for scripting/build.


45-45: Move “tmp” to devDependencies
No imports, requires, or runtime references to tmp were found in ui/**/*; it’s only needed at build time.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 965059d and c4d081e.

⛔ Files ignored due to path filters (2)
  • poetry.lock is excluded by !**/*.lock
  • ui/yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (2)
  • pyproject.toml (3 hunks)
  • ui/package.json (3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Pytest Unit Tests with all supported python versions (3.12)
🔇 Additional comments (3)
pyproject.toml (2)

30-30: Bump to requests ^2.32.4 verified: pyproject.toml pins requests (^2.32.4) and urllib3 (^2.5.0); poetry.lock resolves to requests-2.32.5, urllib3-2.5.0 and charset_normalizer-3.4.1, with no conflicting constraints. Minimal import smoke test imports requests, urllib3 and charset_normalizer without errors.


40-40: Confirm Elasticsearch 9.1 upgrade intent and reconcile optional flag in pyproject.toml

  • Upgrading from 7.x→9.x is breaking; verify client initialization (http_auth vs basic_auth), exception types, helpers, and transport options in
    • dcs_core/integrations/databases/elasticsearch.py (lines 16–47)
    • examples/data_generator/datasource.py (lines 26–29, 188–191)
  • PR description states “remove optional flag,” but in pyproject.toml:
    • elasticsearch = { version="^9.1.0", optional=true } (line 40)
    • listed under [tool.poetry.extras] (line 85) and in "all" (line 97)
      Either update the description or remove optional=true and clean up extras.

Option A – keep optional and fix summary:

-# PR summary says elasticsearch optional flag removed
+# PR summary should reflect: elasticsearch remains optional

Option B – make it non-optional and clean extras:

-elasticsearch = { version="^9.1.0", optional=true }
+elasticsearch = { version="^9.1.0" }

And in [tool.poetry.extras]:

- elasticsearch = ["elasticsearch"]
- all = [
-     "elasticsearch",
-     ...
- ]
+ # remove "elasticsearch" from extras and from "all"
ui/package.json (1)

1-132: Installation errors need resolution
The yarn install --check-files step failed with a postinstall error in core-js-pure. Refresh the lockfile, commit it alongside the packageManager field, then verify that both

cd ui
yarn install --check-files
yarn audit --groups dependencies

complete without errors.

Comment thread ui/package.json
"@typescript-eslint/eslint-plugin": "^2.10.0",
"@typescript-eslint/parser": "^2.10.0",
"babel-jest": "^24.9.0",
"braces": "^3.0.3",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Confirm necessity of braces as a direct dependency.

If used only by tooling (via micromatch/webpack), keep it transitive; otherwise, retain and document usage.


🏁 Script executed:

#!/bin/bash
rg -nP '\b(braces|micromatch)\b' ui/src -g '!**/dist/**' -g '!**/build/**' -C2 || true

Length of output: 87


🏁 Script executed:

#!/bin/bash
sed -n '1,200p' ui/package.json

Length of output: 3479


Remove direct braces dependency
ui/package.json (line 21): drop "braces": "^3.0.3"—it isn’t imported in ui/src and is pulled in transitively via micromatch.

🤖 Prompt for AI Agents
In ui/package.json around line 21, the project has an unnecessary direct
dependency "braces": "^3.0.3" which is not imported from ui/src and is already
pulled in transitively via micromatch; remove the "braces" entry from
dependencies, run your package manager to update the lockfile (npm install or
yarn install / pnpm install as appropriate), commit the updated package.json and
lockfile, and run the build/test to confirm nothing breaks.

Comment thread ui/package.json
"file-loader": "^6.2.0",
"jest-watch-typeahead": "0.4.2",
"material-react-table": "^2.0.4",
"micromatch": "^4.0.8",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

❓ Verification inconclusive

micromatch as a direct dep: validate usage.

Same guidance as braces; avoid unused top-level deps.


🏁 Script executed:

#!/bin/bash
rg -nP '\bmicromatch\b' ui/src -C2 || true

Length of output: 43


Remove unused micromatch dependency
Ripgrep finds no references to micromatch in the entire ui directory—please remove it from ui/package.json (line 36).

🤖 Prompt for AI Agents
In ui/package.json around line 36, the dependency "micromatch": "^4.0.8" is
unused; remove the micromatch entry from the dependencies list in package.json,
run package manager (npm/yarn/pnpm) to update lockfile (npm install / yarn
install / pnpm install) and commit both the updated package.json and the updated
lockfile to keep dependency state consistent.

Comment thread ui/package.json
"jest-watch-typeahead": "0.4.2",
"material-react-table": "^2.0.4",
"micromatch": "^4.0.8",
"nanoid": "^3.3.8\n",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Invalid version string: stray newline in nanoid.

The value includes “\n”, which will break installs. Fix to a valid semver string.

Apply this diff:

-    "nanoid": "^3.3.8\n",
+    "nanoid": "^3.3.8",
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
"nanoid": "^3.3.8\n",
"nanoid": "^3.3.8",
🤖 Prompt for AI Agents
In ui/package.json around line 37, the nanoid dependency string contains a stray
"\n" which makes the version invalid; remove the escaped newline so the value is
a valid semver string (e.g. change "\"^3.3.8\\n\"" to "\"^3.3.8\"") and save the
file so package.json is valid JSON for installs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants