Add inspect function to the python package

[javiermtorres]

The inspect function has been added to the encoderfile package. The pythonize crate is used to pass the info as a (json like) nested dict. It has been manually tested, this should work:

(288-python-inspect) TowoMoz:288-python-inspect ⇒ python
Python 3.13.1 (main, Jan 14 2025, 23:31:50) [Clang 19.1.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import encoderfile as ef
>>> builder = ef.EncoderfileBuilder.from_config("test_config.yml")
>>> builder.build()
WARNING: No padding_side found in tokenizer_config.json. Using default: Right
WARNING: No pad_to_multiple_of found in tokenizer_config.json. Using default: None
WARNING: No pad_token_type_id found in tokenizer_config.json. Using default: 0
WARNING: No truncation_strategy found in tokenizer_config.json. Using default: LongestFirst
WARNING: No truncation_side found in tokenizer_config.json. Using default: Right
WARNING: No stride found in tokenizer_config.json. Using default: 0
>>> ef.inspect("my-model-2.encoderfile")
{'model_config': {'model_type': 'mozilla-ai/test-dummy-token-classifier', 'num_labels': None, 'id2label': {5: 'LABEL_5', 4: 'LABEL_4', 0: 'LABEL_0', 2: 'LABEL_2', 6: 'LABEL_6', 3: 'LABEL_3', 1: 'LABEL_1'}, 'label2id': {'LABEL_0': 0, 'LABEL_5': 5, 'LABEL_1': 1, 'LABEL_2': 2, 'LABEL_6': 6, 'LABEL_4': 4, 'LABEL_3': 3}}, 'encoderfile_config': {'name': 'my-model-2', 'version': '0.1.0', 'model_type': 'token_classification', 'transform': '--- Applies a softmax across token classification logits.\n--- Each token classification is normalized independently.\n--- \n--- Args:\n---   arr (Tensor): A tensor of shape [batch_size, n_tokens, n_labels].\n---                 The softmax is applied along the third axis (n_labels).\n---\n--- Returns:\n---   Tensor: The input tensor with softmax-normalized embeddings.\n---@param arr Tensor\n---@return Tensor\nfunction Postprocess(arr)\n    return arr:softmax(3)\nend', 'lua_libs': None}}
>>> info = ef.inspect("my-model-2.encoderfile")
>>> info['model_config']['id2label']
{2: 'LABEL_2', 4: 'LABEL_4', 3: 'LABEL_3', 1: 'LABEL_1', 0: 'LABEL_0', 6: 'LABEL_6', 5: 'LABEL_5'}

[codecov-commenter]

Codecov Report

❌ Patch coverage is 6.15385% with 61 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
encoderfile-py/src/builder.rs	0.00%	41 Missing ⚠️
encoderfile/src/common/config.rs	0.00%	17 Missing ⚠️
encoderfile/src/builder/cli/inspect.rs	66.66%	2 Missing ⚠️
encoderfile/src/builder/cli/mod.rs	0.00%	1 Missing ⚠️

Files with missing lines	Coverage Δ
encoderfile/src/common/model_config.rs	90.19% <ø> (ø)
encoderfile/src/builder/cli/mod.rs	0.00% <0.00%> (ø)
encoderfile/src/builder/cli/inspect.rs	78.57% <66.66%> (-21.43%)	⬇️
encoderfile/src/common/config.rs	42.42% <0.00%> (-45.08%)	⬇️
encoderfile-py/src/builder.rs	0.00% <0.00%> (ø)

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[besaleli]

comments below