docker-tests: use docker from the system (#4478)

* docker-tests: use docker from the system

Stop using the ancient docker-compose and docker packaged in Python and
rely on the one available in the system.

* Add missing -y to docker install

* This now requires docker

* it's not additive

* Drop version from docker-compose.yml

* Move system dependencies intall to workflow

And make ms sql odbc driver optional so you can run tests locally
without accepting the EULA.

* Add changelog

* Rebuild misc.yml manually

* Apply suggestions from code review

Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>

* Apply suggestions from code review

Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>

* Apply suggestion from @xrmx

* Apply suggestions from code review

Co-authored-by: Riccardo Magliocchetti <riccardo.magliocchetti@gmail.com>

Author: xrmx

.github/workflows/generate_workflows_lib/src/generate_workflows_lib/misc.yml.j2

@@ -68,6 +68,12 @@ jobs:
       - name: Install tox
         run: pip install tox-uv
 
+      {%- if job_data == "docker-tests" %}
+
+      - name: Install Microsoft SQL ODBC driver
+        run: sudo apt update -y && ACCEPT_EULA=Y sudo apt install -y msodbcsql18 unixodbc-dev unixodbc
+      {%- endif %}
+
       - name: Run tests
         run: tox -e {{ job_data }}
       {%- if job_data == "generate-workflows" %}

.github/workflows/misc.yml

@@ -59,6 +59,9 @@ jobs:
       - name: Install tox
         run: pip install tox-uv
 
+      - name: Install Microsoft SQL ODBC driver
+        run: sudo apt update -y && ACCEPT_EULA=Y sudo apt install -y msodbcsql18 unixodbc-dev unixodbc
+
       - name: Run tests
         run: tox -e docker-tests
 

CHANGELOG.md

@@ -48,6 +48,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   ([#4360](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4360))
 - `opentelemetry-instrumentation-aiohttp-server`: Use `canonical` attribute of the `Resource` as a span name
   ([#3896](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/3896))
+- `docker-tests`: Don't require sudo, debian based distro and MS SQL ODBC driver to run locally. Instead require docker and unixodbc
+  ([#4478](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4478))
 - Refactor unit tests to allow for population of the random trace id flag in the `traceparent` header
   ([#4030](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4030))
 - `opentelemetry-instrumentation-fastapi`: Fix `FastAPI` instrumentation to correctly trace `BackgroundTasks` by wrapping their execution in a dedicated span, ensuring proper parent-child relationships and accurate trace timing

tests/opentelemetry-docker-tests/tests/check_availability.py

@@ -130,8 +130,11 @@ def check_docker_services_availability():
     check_mysql_connection()
     check_postgres_connection()
     check_redis_connection()
-    check_mssql_connection()
-    setup_mssql_db()
+
+    # make accepting EULA for ms sql odbc driver optional
+    if "ODBC Driver 18 for SQL Server" in pyodbc.drivers():
+        check_mssql_connection()
+        setup_mssql_db()
 
 
 check_docker_services_availability()

tests/opentelemetry-docker-tests/tests/docker-compose.yml

@@ -1,5 +1,3 @@
-version: '3'
-
 services:
   otmongo:
     ports:

tests/opentelemetry-docker-tests/tests/sqlalchemy_tests/test_mssql.py

@@ -3,6 +3,7 @@
 
 import os
 
+import pyodbc
 import pytest
 from sqlalchemy.exc import ProgrammingError
 
@@ -30,6 +31,10 @@
 }
 
 
+@pytest.mark.skipif(
+    "ODBC Driver 18 for SQL Server" not in pyodbc.drivers(),
+    reason="No MS SQL ODBC driver installed",
+)
 class MssqlConnectorTestCase(SQLAlchemyTestMixin):
     """TestCase for pyodbc engine"""
 

tests/opentelemetry-docker-tests/tests/test-requirements.txt

@@ -3,7 +3,6 @@ amqp==5.2.0
 asgiref==3.8.1
 async-timeout==4.0.3
 asyncpg==0.29.0
-attrs==23.2.0
 bcrypt==5.0.0
 billiard==4.2.0
 celery==5.3.6
@@ -15,21 +14,15 @@ click-didyoumean==0.3.0
 click-plugins==1.1.1
 click-repl==0.3.0
 cryptography==46.0.7
-distro==1.9.0
 Django==4.2.30
 dnspython==2.6.1
-docker==5.0.3
-docker-compose==1.29.2
-dockerpty==0.4.1
-docopt==0.6.2
 exceptiongroup==1.2.0
 flaky==3.7.0
 flask==3.1.3
 greenlet==3.0.3
 grpcio==1.63.2
 idna==3.7
 iniconfig==2.0.0
-jsonschema==3.2.0
 kombu==5.3.5
 mysql-connector-python==8.3.0
 mysqlclient==2.1.1
@@ -50,24 +43,19 @@ PyMySQL==1.1.1
 PyNaCl==1.6.2
 # prerequisite: install unixodbc
 pyodbc==5.0.1
-pyrsistent==0.20.0
 pytest==8.0.2
 pytest-celery==0.0.0
 python-dateutil==2.9.0.post0
-python-dotenv==0.21.1
 pytz==2024.1
-PyYAML==5.3.1
 redis==5.0.1
 requests==2.31.0
 six==1.16.0
 SQLAlchemy==1.4.52
-texttable==1.7.0
 tomli==2.0.1
 typing_extensions==4.12.2
 tzdata==2024.1
 urllib3==1.26.19
 vine==5.1.0
 wcwidth==0.2.13
-websocket-client==0.59.0
 wrapt==1.16.0
 zipp==3.19.1

tox.ini

@@ -1082,17 +1082,21 @@ deps =
 changedir =
   tests/opentelemetry-docker-tests/tests
 
+allowlist_externals =
+  docker
+  pytest
+
 commands_pre =
-  sh -c "sudo apt update -y && sudo ACCEPT_EULA=Y apt-get install -y msodbcsql18 unixodbc-dev unixodbc"
-  python -c "import pyodbc; print(pyodbc.drivers())"
-  docker-compose up -d
+  docker --version || echo 'Docker with compose subcommand is required' && /bin/false
+  python -c 'import pyodbc; print(pyodbc.drivers())' || echo 'You need to install unixODBC' && /bin/false
+  docker compose up -d
   python check_availability.py
 
 commands =
   pytest {posargs}
 
 commands_post =
-  docker-compose down -v
+  docker compose down -v
 
 [testenv:generate]
 deps =