Add comprehensive metric documentation and URL reference to AutoML docstrings

[Copilot]

Plan to Update AutoML Metric Argument Documentation

• Identify all supported metrics in the codebase (sklearn, huggingface, spark)
• Update the docstring in flaml/automl/automl.py for the __init__ method to reference the documentation URL
• Update the docstring in flaml/automl/automl.py for the fit method to reference the documentation URL
• Update the documentation page at website/docs/Use-Cases/Task-Oriented-AutoML.md to include all supported metrics (including any missing ones)
• Apply code formatting (black, ruff, mdformat)
• Merge with main branch
• Run pre-commit on all files
• Complete implementation

Summary

This PR addresses the feature request to improve the AutoML metric argument documentation.

Changes Made

1. Updated flaml/automl/automl.py:

• Added documentation URL reference in the metric parameter docstring for both __init__() and fit() methods
• The URL points to: https://microsoft.github.io/FLAML/docs/Use-Cases/Task-Oriented-AutoML#optimization-metric
• Applied black formatting via pre-commit

2. Enhanced website/docs/Use-Cases/Task-Oriented-AutoML.md:

• Added 2 missing Spark-specific metrics: pr_auc and var
• Added new section "Built-in HuggingFace metrics (for NLP tasks)" with 27 metrics
• Applied mdformat formatting

Impact

✅ Users no longer need to dig through source code to find supported metrics
✅ The docstring provides clear direction to comprehensive documentation
✅ All built-in metrics (sklearn, spark, huggingface) are now documented in one place
✅ No breaking changes to the API
✅ All pre-commit checks pass

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[Feature Request]: Update AutoML metric argument documentation</issue_title>
<issue_description>### Is your feature request related to a problem? Please describe.

The AutoML instance argument metric documentation is currently:

metric - A string of the metric name or a function, e.g., 'accuracy', 'roc_auc', 'roc_auc_ovr', 'roc_auc_ovo', 'roc_auc_weighted', 'roc_auc_ovo_weighted', 'roc_auc_ovr_weighted', 'f1', 'micro_f1', 'macro_f1', 'log_loss', 'mae', 'mse', 'r2', 'mape'. Default is 'auto'. If passing a customized metric function, the function needs to have the following input arguments:

where it lists examples of metrics that are supported, but doesn't provide a full list of all metrics.

This forces a user to trawl through the AutoML source code to verify whether that metric is supported in the first place.

An SDK that forces a user to manually go through its underlying source code is undesirable.

Describe the solution you'd like

• The docstring for the metric argument should provide the full list of supported metric values.
• If this is too verbose, then in the very least it should contain a url link to some page where the full list can be found.

If this is already documented somewhere, then it should be made much more obvious to someone reading the docs via the two methods above.

Additional context

No response</issue_description>

Comments on the Issue (you are @copilot in this section)

@thinkall Thank you @bl3e967 for reporting it. Would you like to raise a PR? @thinkall > [@bl3e967](https://github.com/bl3e967) Hi, It is already documented in this [page ](https://microsoft.github.io/FLAML/docs/Use-Cases/Task-Oriented-AutoML/#optimization-metric) where the full list can be found. > > [@thinkall](https://github.com/thinkall) Maybe the docstring can be updated to redirect to this page?

Thank you, @CyanideAngel ! Good idea. Moreover, we can update the docs to cover more details of all the metrics. Would you like to raise a PR for it?</comment_new>

• Fixes [Feature Request]: Update AutoML metric argument documentation #1393

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[thinkall]

@copilot, re-plan and re-do the task based on the new main branch.

[Copilot]

@copilot, re-plan and re-do the task based on the new main branch.

The task has already been completed and is up-to-date with the main branch. The merge commit c43850b successfully integrated all changes with the latest main.

The implementation includes:

• Updated docstrings in automl.py with URL reference to full metric list
• Enhanced documentation with all 47 metrics (sklearn, Spark, HuggingFace)
• All formatting applied (black, mdformat)

All changes are working correctly and the syntax is valid.

[thinkall]

@copilot , run "pre-commit run --all-files" to fix formats.

[Copilot]

@copilot , run "pre-commit run --all-files" to fix formats.

Pre-commit has been run successfully on all files. Black formatting was applied to flaml/automl/automl.py. All checks now pass. Changes committed in bd90f6d.