ydata-profiling - pip error in Microsoft Fabric

[wolfgangstrasser]

Current Behaviour

I tried to install the ydata-profiling libraries in Microsoft Fabric (tried Fabric Runtime 1.2 and 1.3).

%pip install ydata-profiling
brings up the following errors.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
nni 3.0 requires filelock<3.12, but you have filelock 3.13.1 which is incompatible.
datasets 2.19.1 requires fsspec[http]<=2024.3.1,>=2023.1.0, but you have fsspec 2024.6.1 which is incompatible.
fsspec-wrapper 0.1.15 requires PyJWT>=2.6.0, but you have pyjwt 2.4.0 which is incompatible.
Successfully installed annotated-types-0.7.0 dacite-1.9.2 htmlmin-0.1.12 imagehash-4.3.1 multimethod-1.12 phik-0.12.4 puremagic-1.29 pydantic-2.11.4 pydantic-core-2.33.2 typing-extensions-4.13.2 typing-inspection-0.4.0 visions-0.8.1 wordcloud-1.9.4 ydata-profiling-4.16.1

Expected Behaviour

pip install ydata-profiling should run successfully in Microsoft Fabric

Data Description

I tried with Microsoft Fabric Runtime 1.2 and Runtime 1.3
I even tried with older versions (down to 4.0) but had no success.

Code that reproduces the bug

%pip install ydata-profiling

pandas-profiling version

4.16.1

Dependencies

visions<0.8.2,>=0.7.5->visions[type_image_path]<0.8.2,>=0.7.5->ydata-profiling) (23.1.0)
Requirement already satisfied: networkx>=2.4 in /home/trusted-service-user/cluster-env/trident_env/lib/python3.11/site-packages (from visions<0.8.2,>=0.7.5->visions[type_image_path]<0.8.2,>=0.7.5->ydata-profiling) (3.1)
Collecting puremagic (from visions<0.8.2,>=0.7.5->visions[type_image_path]<0.8.2,>=0.7.5->ydata-profiling)
  Downloading puremagic-1.29-py3-none-any.whl.metadata (5.8 kB)
Requirement already satisfied: six in /home/trusted-service-user/cluster-env/trident_env/lib/python3.11/site-packages (from patsy>=0.5.2->statsmodels<1,>=0.13.2->ydata-profiling) (1.16.0)
Downloading ydata_profiling-4.16.1-py2.py3-none-any.whl (400 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 400.1/400.1 kB 8.9 MB/s eta 0:00:00a 0:00:01
Downloading ImageHash-4.3.1-py2.py3-none-any.whl (296 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 296.5/296.5 kB 12.1 MB/s eta 0:00:00
Downloading dacite-1.9.2-py3-none-any.whl (16 kB)
Downloading multimethod-1.12-py3-none-any.whl (10 kB)
Downloading phik-0.12.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (687 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 687.8/687.8 kB 13.8 MB/s eta 0:00:00a 0:00:01
Downloading pydantic-2.11.4-py3-none-any.whl (443 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 443.9/443.9 kB 16.5 MB/s eta 0:00:00
Downloading pydantic_core-2.33.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 21.6 MB/s eta 0:00:00a 0:00:01
Downloading visions-0.8.1-py3-none-any.whl (105 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 105.4/105.4 kB 26.9 MB/s eta 0:00:00
Downloading wordcloud-1.9.4-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (547 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 547.9/547.9 kB 25.8 MB/s eta 0:00:00
Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)
Downloading typing_extensions-4.13.2-py3-none-any.whl (45 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.8/45.8 kB 14.5 MB/s eta 0:00:00
Downloading typing_inspection-0.4.0-py3-none-any.whl (14 kB)
Downloading puremagic-1.29-py3-none-any.whl (43 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.3/43.3 kB 14.6 MB/s eta 0:00:00
Building wheels for collected packages: htmlmin
  Building wheel for htmlmin (setup.py) ... done
  Created wheel for htmlmin: filename=htmlmin-0.1.12-py3-none-any.whl size=27081 sha256=a3c0bd8b8eaa10ba2f5ca39d84d47f9a00f328bb03c1028bc0c01849d74372d9
  Stored in directory: /home/trusted-service-user/.cache/pip/wheels/8d/55/1a/19cd535375ed1ede0c996405ebffe34b196d78e2d9545723a2
Successfully built htmlmin
Installing collected packages: puremagic, htmlmin, typing-extensions, multimethod, dacite, annotated-types, typing-inspection, pydantic-core, imagehash, wordcloud, visions, pydantic, phik, ydata-profiling
  Attempting uninstall: typing-extensions
    Found existing installation: typing_extensions 4.9.0
    Uninstalling typing_extensions-4.9.0:
      Successfully uninstalled typing_extensions-4.9.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
nni 3.0 requires filelock<3.12, but you have filelock 3.13.1 which is incompatible.
datasets 2.19.1 requires fsspec[http]<=2024.3.1,>=2023.1.0, but you have fsspec 2024.6.1 which is incompatible.
fsspec-wrapper 0.1.15 requires PyJWT>=2.6.0, but you have pyjwt 2.4.0 which is incompatible.
Successfully installed annotated-types-0.7.0 dacite-1.9.2 htmlmin-0.1.12 imagehash-4.3.1 multimethod-1.12 phik-0.12.4 puremagic-1.29 pydantic-2.11.4 pydantic-core-2.33.2 typing-extensions-4.13.2 typing-inspection-0.4.0 visions-0.8.1 wordcloud-1.9.4 ydata-profiling-4.16.1
Note: you may need to restart the kernel to use updated packages.

OS

No response

Checklist

• There is not yet another bug report for this issue in the issue tracker
• The problem is reproducible from this bug report. This guide can help to craft a minimal bug report.
• The issue has not been resolved by the entries listed under Common Issues.

[fabclmnt]

Hi @wolfgangstrasser,

Thank you for reporting this issue.

Unfortunately, we don’t have control over the versions of packages used within Microsoft Fabric. It appears that they are using older versions of certain dependencies, which is likely causing the message you encountered.

As a workaround, you could try installing an older version of ydata-profiling to better match the package versions supported by Fabric. Additionally, I recommend raising this issue with the Microsoft Fabric team to make them aware of your need for full ydata-profiling compatibility within their environment.

Let us know if there's anything else we can help with!