Create tasks sub-folder with .ttl schema.org and SHACL files + basic python validator

[leobianco]

This pull request creates the tasks sub-folder to the Croissant repository, providing (1) an extension of schema.org in croissant-tasks.ttl to define metadata for machine learning tasks, (2) SHACL shapes in croissant-tasks-shapes.ttl enforcing the format compliance, (3) A validator.py script + a simple test suite targeting both valid and invalid JSON-LD examples.

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[leobianco]

Hi,

The workflow is failing at the notebook tests, it seems that it needs to be refactored? Here is an AI-generated explanation + potential fix:

The Problem
If you look closely at the logs, the installation step fails silently (or rather, pip prints an error message but the notebook continues executing) before it reaches the cell that imports etils:

1. Permission Denied for APT:

E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Permission denied)

The notebook is trying to run !apt-get install without sudo, but the GitHub actions runner requires sudo to install system packages.

2. Invalid pip syntax:

error: invalid-egg-fragment
× The 'mlcroissant[dev]' egg fragment is invalid

Modern versions of pip dropped support for the old #egg=name[extra] syntax when installing from git repositories. Because this failed, the dependencies (including etils) were never actually installed.

3. The Final Crash:

ModuleNotFoundError: No module named 'etils'
Because the pip install in the previous cell failed, Python crashes as soon as it tries to import etils.

The Solution
You need to edit the file recipes/bounding-boxes.ipynb (and potentially any other .ipynb notebook files in the recipes/ folder that have this same installation cell).

Open the notebook, find the very first code block that looks like this:

# Install mlcroissant from the source
!apt-get install -y python3-dev graphviz libgraphviz-dev pkg-config
!pip install "git+https://github.com/${GITHUB_REPOSITORY:-mlcommons/croissant}.git@${GITHUB_HEAD_REF:-main}#subdirectory=python/mlcroissant&egg=mlcroissant[dev]"

And replace the two commands with correctly formatted versions:

# Install mlcroissant from the source
!sudo apt-get install -y python3-dev graphviz libgraphviz-dev pkg-config
!pip install "mlcroissant[dev] @ git+https://github.com/${GITHUB_REPOSITORY:-mlcommons/croissant}.git@${GITHUB_HEAD_REF:-main}#subdirectory=python/mlcroissant"

What changed:

1. Added sudo in front of apt-get so it has permissions on the test runner.
2. Changed the pip syntax to the modern "Direct URL" syntax (name[extra] @ url) instead of appending &egg=... to the end of the URL.

[benjelloun]

Very nice! Sorry for the late review.

[benjelloun]

Why not the execution or evaluation?

[leobianco]

Will add the execution spec option to the execution field of the TaskProblem, as it might be filled later for a particular TaskSolution execution.

As for the evaluation, intuitively, at least for benchmarks, evaluation should be concrete on the TaskProblem, otherwise each solution could specify its own evaluation metrics?

[benjelloun]

Agree for benchmarks, but with a bit of imagination, I could imagine some other types of tasks where the evaluation is not specified by the problem...