Fixes 25991: Enable https scheme support for Druid connector

[mohitjeswani01]

Description:

Fixes #25991

I worked on expanding the Druid connection JSON schema to officially support the druid+http and druid+https schemes because the connector previously lacked the ability to connect to secured endpoints.

What changes did you make?

• Surgically updated druidConnection.json to expand the druidScheme enum from ["druid"] to ["druid", "druid+http", "druid+https"].
• The Pydantic model generator automatically picks up this change and makes it available to the ingestion Python builders.
• I maintained backward compatibility by keeping the default value as "druid". Therefore, no JSON migration scripts are necessary since existing connections will default correctly and remain completely unaffected.

How did I test my changes?

• Regenerated the Python Pydantic models locally using make generate.
• Installed all the unit testing database dependencies (make install_test).
• Added robust unit tests to ingestion/tests/unit/test_source_connection.py to assert that the URL builder correctly consumes druid+http and druid+https.
• Ran pytest locally which successfully passed.

Screenshots proving local test success:

Result

Type of change:

• Improvement

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes 25991: Enable https scheme support for Druid connector
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

• I have added tests around the new logic.
• For connector/ingestion changes: I updated the documentation.

---

Summary by Gitar

• New integration tests:
  • Added ingestion/tests/integration/druid/ suite to verify connector functionality using testcontainers.
  • Implemented a druid_container fixture to manage apache/druid:30.0.0 for integration testing.

_{This will update automatically on new commits.}

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[Copilot]

Pull request overview

Expands the Druid ingestion connector’s connection schema to allow explicit druid+http and druid+https SQLAlchemy URL schemes, enabling connectivity to secured endpoints while keeping druid as the default for backward compatibility.

Changes:

• Updated druidConnection.json to extend the druidScheme enum with druid+http and druid+https.
• Extended ingestion unit tests to validate URL building for the new schemes.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
openmetadata-spec/src/main/resources/json/schema/entity/services/connections/database/druidConnection.json	Adds druid+http and druid+https to supported Druid schemes while preserving the default.
ingestion/tests/unit/test_source_connection.py	Adds assertions that Druid URL construction supports the new schemes.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[mohitjeswani01]

Hi @harshach , I have addressed all the feedback from the bots (Gitar and Copilot) by updating the unit tests to use the generated DruidScheme enum members instead of raw string literals. This ensures better type-safety and consistency across the ingestion framework.

Could you please review and kindly add the safe to test label so the CI workflows can run? Thank you for your time!🙏

[harshach]

@mohitjeswani01 since druid is available as docker, it would be good to write a integration test to make sure this works. Check the integration tests for ingestion

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[mohitjeswani01]

@mohitjeswani01 since druid is available as docker, it would be good to write a integration test to make sure this works. Check the integration tests for ingestion

thanks for informing @harshach sir👋

Added an integration test for the Druid connector as requested.

Created ingestion/tests/integration/druid/ following the
existing testcontainers pattern (same as cockroach, mongodb):

• conftest.py — spins up apache/druid:30.0.0 via testcontainers,
  waits for /status/health to return 200, creates DruidConnection
  service request
• test_metadata.py — runs MetadataWorkflow and asserts at least
  one schema is discovered from Druid's built-in system schemas
• init.py — module init

The test uses DruidScheme.druid (plain http) for container
connectivity. The druid+http and druid+https schemes are
validated in the existing unit test.

Ready for re-review! 🙏

[Copilot]

Pull request overview

Expands the Druid database connection schema to support SQLAlchemy-style driver schemes for secured endpoints (druid+http, druid+https), and updates ingestion tests to validate URL construction and basic Druid ingestion behavior.

Changes:

• Extended druidScheme enum in the Druid connection JSON schema to include druid+http and druid+https.
• Updated unit tests to assert correct URL generation for the new schemes.
• Added a Druid integration test (with dockerized Druid) to validate metadata ingestion path.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
openmetadata-spec/src/main/resources/json/schema/entity/services/connections/database/druidConnection.json	Adds druid+http / druid+https to the supported scheme enum.
ingestion/tests/unit/test_source_connection.py	Extends Druid URL unit test coverage to the new scheme variants.
ingestion/tests/integration/druid/test_metadata.py	New integration test validating Druid workflow discovers schemas.
ingestion/tests/integration/druid/conftest.py	New Druid container + service setup fixtures for integration testing.
ingestion/tests/integration/druid/init.py	Adds package marker for the new integration test folder.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[Copilot]

Pull request overview

Expands the Druid connector connection schema to support explicit druid+http and druid+https SQLAlchemy schemes, and adds tests to validate URL construction and basic end-to-end metadata ingestion against a Druid container.

Changes:

• Extend druidScheme enum in the Druid connection JSON schema to include druid+http and druid+https.
• Update unit tests to assert the Druid connection URL builder emits the new schemes correctly.
• Add a new Druid integration test suite using testcontainers to validate metadata ingestion against a running Druid instance.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
openmetadata-spec/src/main/resources/json/schema/entity/services/connections/database/druidConnection.json	Adds druid+http and druid+https to the allowed connection schemes.
ingestion/tests/unit/test_source_connection.py	Extends the Druid URL unit test to cover the new schemes.
ingestion/tests/integration/druid/conftest.py	Introduces a Druid container fixture + service creation request for integration testing.
ingestion/tests/integration/druid/test_metadata.py	Runs a metadata workflow and asserts Druid schemas are discovered.
ingestion/tests/integration/druid/init.py	Adds package marker for the new Druid integration test suite.

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[mohitjeswani01]

hi @harshach , @PubChimps sir may i get a safe to test label ? i recently resolved the merge conflicts in this pr. thanks 🙏

[gitar-bot]

Code Review ✅ Approved 1 resolved / 1 findings

Enables HTTPS scheme support for the Druid connector by replacing string literals with DruidScheme enum members in the test suite. No issues found.

✅ 1 resolved

✅ Quality: Tests use string literals instead of DruidScheme enum members

📄 ingestion/tests/unit/test_source_connection.py:657 📄 ingestion/tests/unit/test_source_connection.py:663
The new test cases at lines 657 and 663 pass scheme="druid+http" and scheme="druid+https" as raw strings, while the existing test at line 651 correctly uses scheme=DruidScheme.druid. Using enum members would be more type-safe, consistent, and would also verify the generated enum actually contains these new values.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}