Skip to content

MINOR: Fix E2Es for different Services#21407

Closed
ayush-shah wants to merge 2 commits into
mainfrom
fix-e2e-2705
Closed

MINOR: Fix E2Es for different Services#21407
ayush-shah wants to merge 2 commits into
mainfrom
fix-e2e-2705

Conversation

@ayush-shah
Copy link
Copy Markdown
Member

@ayush-shah ayush-shah commented May 27, 2025

  • Refactored import path for TestCaseDefinition in test_cli_snowflake.py.
  • Updated snowflake.yaml to use privateKey and snowflakePrivatekeyPassphrase for authentication instead of password.

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

@ayush-shah
Fix: Update Snowflake ingestion tests and configuration
ad2661d 
- Refactored import path for TestCaseDefinition in test_cli_snowflake.py.
- Updated snowflake.yaml to use privateKey and snowflakePrivatekeyPassphrase for authentication instead of password.
@ayush-shah ayush-shah requested a review from a team as a code owner May 27, 2025 07:45
@github-actions github-actions Bot added Ingestion safe to test Add this label to run secure Github workflows on PRs labels May 27, 2025
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2025

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2025 •
edited
Loading

🛡️ TRIVY SCAN RESULT 🛡️

Target: openmetadata-ingestion-base-slim:trivy (debian 12.11)

Vulnerabilities (25)

Package Vulnerability ID Severity Installed Version Fixed Version
linux-libc-dev CVE-2024-53203 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37897 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37909 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37911 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37912 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37914 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37917 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37921 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37923 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37928 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37929 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37932 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37936 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37948 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37949 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37951 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37953 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37959 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37961 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37963 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37964 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37967 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37969 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37970 🚨 HIGH 6.1.137-1 6.1.140-1
linux-libc-dev CVE-2025-37990 🚨 HIGH 6.1.137-1 6.1.140-1

🛡️ TRIVY SCAN RESULT 🛡️

Target: Java

Vulnerabilities (47)

Package Vulnerability ID Severity Installed Version Fixed Version
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.11.4 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.11.4 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.13.0 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.13.0 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4
com.google.code.gson:gson CVE-2022-25647 🚨 HIGH 2.2.4 2.8.9
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 2.5.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 2.5.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 2.5.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.3.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.3.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.3.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.7.1 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.7.1 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.7.1 3.25.5, 4.27.5, 4.28.2
com.nimbusds:nimbus-jose-jwt CVE-2023-52428 🚨 HIGH 9.8.1 9.37.2
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.11.0 2.14.0
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.8.0 2.14.0
dnsjava:dnsjava CVE-2024-25638 🚨 HIGH 2.1.7 3.6.0
io.airlift:aircompressor CVE-2024-36114 🚨 HIGH 0.21 0.27
net.minidev:json-smart CVE-2021-31684 🚨 HIGH 1.3.2 1.3.3, 2.4.4
net.minidev:json-smart CVE-2023-1370 🚨 HIGH 1.3.2 2.4.9
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.11.0 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.11.0 1.11.3
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.7.7 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.7.7 1.11.3
org.apache.derby:derby CVE-2022-46337 🔥 CRITICAL 10.14.2.0 10.14.3, 10.15.2.1, 10.16.1.2, 10.17.1.0
org.apache.hadoop:hadoop-common CVE-2022-25168 🔥 CRITICAL 3.3.2 2.10.2, 3.2.4, 3.3.3
org.apache.ivy:ivy CVE-2022-46751 🚨 HIGH 2.5.1 2.5.2
org.apache.mesos:mesos CVE-2018-1330 🚨 HIGH 1.4.3 1.6.0
org.apache.spark:spark-hive-thriftserver_2.12 CVE-2024-23945 🚨 HIGH 3.3.4 3.4.2
org.apache.thrift:libthrift CVE-2019-0205 🚨 HIGH 0.12.0 0.13.0
org.apache.thrift:libthrift CVE-2020-13949 🚨 HIGH 0.12.0 0.14.0
org.apache.zookeeper:zookeeper CVE-2023-44981 🔥 CRITICAL 3.6.2 3.7.2, 3.8.3, 3.9.1
org.eclipse.jetty:jetty-server CVE-2024-13009 🚨 HIGH 9.4.48.v20220622 9.4.57.v20241219
org.xerial.snappy:snappy-java CVE-2023-34455 🚨 HIGH 1.1.8.4 1.1.10.1
org.xerial.snappy:snappy-java CVE-2023-43642 🚨 HIGH 1.1.8.4 1.1.10.4
org.yaml:snakeyaml CVE-2022-1471 🚨 HIGH 1.31 2.0

🛡️ TRIVY SCAN RESULT 🛡️

Target: Node.js

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Python

Vulnerabilities (2)

Package Vulnerability ID Severity Installed Version Fixed Version
Werkzeug CVE-2024-34069 🚨 HIGH 2.2.3 3.0.3
setuptools CVE-2025-47273 🚨 HIGH 70.3.0 78.1.1

🛡️ TRIVY SCAN RESULT 🛡️

Target: /etc/ssl/private/ssl-cert-snakeoil.key

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/extended_sample_data.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/lineage.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data.json

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage.json

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage.yaml

No Vulnerabilities Found

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2025 •
edited
Loading

🛡️ TRIVY SCAN RESULT 🛡️

Target: openmetadata-ingestion:trivy (debian 12.9)

Vulnerabilities (6)

Package Vulnerability ID Severity Installed Version Fixed Version
libperl5.36 CVE-2024-56406 🚨 HIGH 5.36.0-7+deb12u1 5.36.0-7+deb12u2
libxslt1.1 CVE-2024-55549 🚨 HIGH 1.1.35-1 1.1.35-1+deb12u1
libxslt1.1 CVE-2025-24855 🚨 HIGH 1.1.35-1 1.1.35-1+deb12u1
perl CVE-2024-56406 🚨 HIGH 5.36.0-7+deb12u1 5.36.0-7+deb12u2
perl-base CVE-2024-56406 🚨 HIGH 5.36.0-7+deb12u1 5.36.0-7+deb12u2
perl-modules-5.36 CVE-2024-56406 🚨 HIGH 5.36.0-7+deb12u1 5.36.0-7+deb12u2

🛡️ TRIVY SCAN RESULT 🛡️

Target: Java

Vulnerabilities (47)

Package Vulnerability ID Severity Installed Version Fixed Version
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.11.4 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.11.4 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.13.0 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.13.0 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4
com.google.code.gson:gson CVE-2022-25647 🚨 HIGH 2.2.4 2.8.9
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 2.5.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 2.5.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 2.5.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.3.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.3.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.3.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.7.1 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.7.1 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.7.1 3.25.5, 4.27.5, 4.28.2
com.nimbusds:nimbus-jose-jwt CVE-2023-52428 🚨 HIGH 9.8.1 9.37.2
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.11.0 2.14.0
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.8.0 2.14.0
dnsjava:dnsjava CVE-2024-25638 🚨 HIGH 2.1.7 3.6.0
io.airlift:aircompressor CVE-2024-36114 🚨 HIGH 0.21 0.27
net.minidev:json-smart CVE-2021-31684 🚨 HIGH 1.3.2 1.3.3, 2.4.4
net.minidev:json-smart CVE-2023-1370 🚨 HIGH 1.3.2 2.4.9
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.11.0 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.11.0 1.11.3
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.7.7 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.7.7 1.11.3
org.apache.derby:derby CVE-2022-46337 🔥 CRITICAL 10.14.2.0 10.14.3, 10.15.2.1, 10.16.1.2, 10.17.1.0
org.apache.hadoop:hadoop-common CVE-2022-25168 🔥 CRITICAL 3.3.2 2.10.2, 3.2.4, 3.3.3
org.apache.ivy:ivy CVE-2022-46751 🚨 HIGH 2.5.1 2.5.2
org.apache.mesos:mesos CVE-2018-1330 🚨 HIGH 1.4.3 1.6.0
org.apache.spark:spark-hive-thriftserver_2.12 CVE-2024-23945 🚨 HIGH 3.3.4 3.4.2
org.apache.thrift:libthrift CVE-2019-0205 🚨 HIGH 0.12.0 0.13.0
org.apache.thrift:libthrift CVE-2020-13949 🚨 HIGH 0.12.0 0.14.0
org.apache.zookeeper:zookeeper CVE-2023-44981 🔥 CRITICAL 3.6.2 3.7.2, 3.8.3, 3.9.1
org.eclipse.jetty:jetty-server CVE-2024-13009 🚨 HIGH 9.4.48.v20220622 9.4.57.v20241219
org.xerial.snappy:snappy-java CVE-2023-34455 🚨 HIGH 1.1.8.4 1.1.10.1
org.xerial.snappy:snappy-java CVE-2023-43642 🚨 HIGH 1.1.8.4 1.1.10.4
org.yaml:snakeyaml CVE-2022-1471 🚨 HIGH 1.31 2.0

🛡️ TRIVY SCAN RESULT 🛡️

Target: Node.js

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Python

Vulnerabilities (7)

Package Vulnerability ID Severity Installed Version Fixed Version
Werkzeug CVE-2024-34069 🚨 HIGH 2.2.3 3.0.3
apache-airflow-providers-common-sql CVE-2025-30473 🚨 HIGH 1.21.0 1.24.1
h11 CVE-2025-43859 🔥 CRITICAL 0.14.0 0.16.0
setuptools CVE-2024-6345 🚨 HIGH 65.5.1 70.0.0
setuptools CVE-2025-47273 🚨 HIGH 65.5.1 78.1.1
setuptools CVE-2025-47273 🚨 HIGH 70.3.0 78.1.1
tornado CVE-2025-47287 🚨 HIGH 6.4.2 6.5

🛡️ TRIVY SCAN RESULT 🛡️

Target: /etc/ssl/private/ssl-cert-snakeoil.key

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis-1.8.0.0.dev0.dist-info/METADATA

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /home/airflow/openmetadata-airflow-apis/openmetadata_managed_apis.egg-info/PKG-INFO

No Vulnerabilities Found

@ayush-shah
Fix: Update Snowflake password secret in E2E tests configuration
a4cd53e 
- Changed the Snowflake password secret reference in the py-cli-e2e-tests.yml workflow from TEST_SNOWFLAKE_PASSWORD to TEST_SNOWFLAKE_PASSWORD_YAML for consistency.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 27, 2025

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

@ayush-shah ayush-shah marked this pull request as draft June 6, 2025 10:43
@ayush-shah ayush-shah closed this Jun 26, 2025
@ayush-shah ayush-shah deleted the fix-e2e-2705 branch June 28, 2025 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@akash-jain-10 akash-jain-10 Awaiting requested review from akash-jain-10 akash-jain-10 is a code owner

@harshach harshach Awaiting requested review from harshach harshach is a code owner

@tutte tutte Awaiting requested review from tutte tutte is a code owner

Copilot code review Copilot Awaiting requested review from Copilot Copilot will automatically review once the pull request is marked ready for review

Assignees

No one assigned

Labels

Ingestion safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ayush-shah