Fixes #21941: Use awsSessionToken as plain str in OpenSearch connector

[Megh-Shah-08]

Describe your changes:

Fixes #21941

Summary: Ingestion was failing when using AWS temporary credentials (Access Key + Secret + Session Token) because the code was attempting to call .get_secret_value() on the awsSessionToken field.

Root Cause: In the awsCredentials.json schema, awsSessionToken is defined as a plain string (without format: password). This means the generated Pydantic model treats it as a standard Python str, which does not have the .get_secret_value() method. This resulted in an AttributeError whenever a session token was provided.

Changes:

Updated connection.py to use awsSessionToken directly as a plain string.
This change aligns the OpenSearch connector with how other AWS-based connectors (like Athena and the base AWS Client) handle this field.

How I tested:

• Rebuilt the local ingestion environment.
• Ran existing unit tests: pytest tests/unit/topology/search/test_opensearch.py (Passed).
• Verified fix with a regression script simulating an AWS IAM connection with a session token (Verified no AttributeError).
  Applied formatting via black.

Type of change:

• Bug fix
• Improvement
• New feature
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes #21941: Use awsSessionToken as plain str in OpenSearch connector
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

---

Summary by Gitar

• Regression testing:
  • Added OpenSearchConnectionTest in test_opensearch.py to verify AWS authentication with a session token.

_{This will update automatically on new commits.}

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[Copilot]

Pull request overview

Fixes ingestion failures for the OpenSearch connector when using AWS temporary credentials by treating awsSessionToken as a plain string (per the AWSCredentials schema), avoiding an invalid .get_secret_value() call.

Changes:

• Update OpenSearch AWS IAM auth to pass awsSessionToken directly as str.
• Minor formatting adjustment in Basic Auth tuple construction (no behavioral change).

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[github-actions]

🟡 Playwright Results — all passed (21 flaky)

✅ 3666 passed · ❌ 0 failed · 🟡 21 flaky · ⏭️ 89 skipped

Shard	Passed	Failed	Flaky	Skipped
🟡 Shard 1	479	0	2	4
🟡 Shard 2	651	0	2	7
🟡 Shard 3	653	0	6	1
🟡 Shard 4	628	0	6	27
🟡 Shard 5	610	0	1	42
🟡 Shard 6	645	0	4	8

🟡 21 flaky test(s) (passed on retry)

• Features/CustomizeDetailPage.spec.ts › Dashboard Data Model - customization should work (shard 1, 1 retry)
• Pages/UserCreationWithPersona.spec.ts › Create user with persona and verify on profile (shard 1, 1 retry)
• Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
• Features/Glossary/GlossaryWorkflow.spec.ts › should inherit reviewers from glossary when term is created (shard 2, 1 retry)
• Features/QueryEntity.spec.ts › Query Entity (shard 3, 1 retry)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 2 retries)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 2 retries)
• Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 2 retries)
• Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
• Pages/Customproperties-part2.spec.ts › entityReferenceList shows item count, scrollable list, no expand toggle (shard 4, 1 retry)
• Pages/DataContracts.spec.ts › Create Data Contract and validate for Container (shard 4, 1 retry)
• Pages/DataContracts.spec.ts › Create Data Contract and validate for SearchIndex (shard 4, 1 retry)
• Pages/DataContractsSemanticRules.spec.ts › Validate Owner Rule Is_Set (shard 4, 1 retry)
• Pages/DataProductAndSubdomains.spec.ts › Add tags to data product via UI (shard 4, 1 retry)
• Pages/DomainUIInteractions.spec.ts › Add expert to domain via UI (shard 4, 1 retry)
• Pages/Glossary.spec.ts › Add and Remove Assets (shard 5, 1 retry)
• Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
• Pages/Lineage/LineageRightPanel.spec.ts › Verify custom properties tab IS visible for supported type: searchIndex (shard 6, 1 retry)
• Pages/ODCSImportExport.spec.ts › Multi-object ODCS contract - object selector shows all schema objects (shard 6, 1 retry)
• Pages/Users.spec.ts › Permissions for table details page for Data Consumer (shard 6, 1 retry)

📦 Download artifacts

How to debug locally

# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

[github-actions]

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[gitar-bot]

Code Review ⚠️ Changes requested 0 resolved / 1 findings

Updates the OpenSearch connector to use awsSessionToken as a string, but replacing extend-exclude with exclude in configuration defaults inadvertently drops Black/pycln settings.

⚠️ Bug: Switching from extend-exclude to exclude drops Black/pycln defaults

📄 ingestion/pyproject.toml:235 📄 ingestion/pyproject.toml:239

Changing extend-exclude to exclude in both [tool.black] and [tool.pycln] causes these tools to lose their built-in default exclusions (.git, .tox, .mypy_cache, .pytest_cache, __pypackages__, build, dist, etc.). Only src/metadata/generated, env, and venv will be excluded.

The intent was to add env/venv to the exclusion list, but extend-exclude is the correct directive for that — it adds patterns on top of the defaults. Use extend-exclude and append the new patterns.

Suggested fix

[tool.black]
extend-exclude = "src/metadata/generated|env|venv"

[tool.pycln]
all = true
extend-exclude = "src/metadata/generated|env|venv"

🤖 Prompt for agents

Code Review: Updates the OpenSearch connector to use awsSessionToken as a string, but replacing extend-exclude with exclude in configuration defaults inadvertently drops Black/pycln settings.

1. ⚠️ Bug: Switching from extend-exclude to exclude drops Black/pycln defaults
   Files: ingestion/pyproject.toml:235, ingestion/pyproject.toml:239

   Changing `extend-exclude` to `exclude` in both `[tool.black]` and `[tool.pycln]` causes these tools to lose their built-in default exclusions (`.git`, `.tox`, `.mypy_cache`, `.pytest_cache`, `__pypackages__`, `build`, `dist`, etc.). Only `src/metadata/generated`, `env`, and `venv` will be excluded.
   
   The intent was to add `env`/`venv` to the exclusion list, but `extend-exclude` is the correct directive for that — it adds patterns on top of the defaults. Use `extend-exclude` and append the new patterns.

   Suggested fix:
   [tool.black]
   extend-exclude = "src/metadata/generated|env|venv"
   
   [tool.pycln]
   all = true
   extend-exclude = "src/metadata/generated|env|venv"

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud