Sweep job support for pipeline component in Azure Machine Learning

[donin1129]

Related command
az ml component create -f XXX.yaml
az ml job create -f XXX.yaml

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
We are implementing a Retrieval-Augmented Generation (RAG) system using Azure Machine Learning, structured into multiple components such as:

• process-data-input
• retrieve_docs_from_search_index
• build_prompt
• query_llm
• evaluate

We want to perform hyperparameter tuning across these components. For example, we aim to optimize parameters such as:

• The number of documents retrieved from the search index
• Different prompt templates
• Various q values in LLM queries

However, AzureML Sweep Jobs currently seem to only support command components, making it impossible to tune hyperparameters across components. We tried to group our command components in to a pipeline component. But sweep job does not start correctly. It would be highly beneficial if Sweep Jobs could also support pipeline component, allowing us to bundle our components into a single pipeline component and efficiently perform hyperparameter tuning.

Describe alternatives you've considered
Our current approach involves creating a standalone command component that pulls source code from all other components. However, this:

• Defeats the purpose of separating concerns across multiple components
• Violates the Single Responsibility Principle in system design
• Increases maintenance complexity

Additional context
Extending Sweep Job support to pipeline component would greatly improve modularity, maintainability, and efficiency in hyperparameter tuning.

[yonzhan]

Thank you for opening this issue, we will look into it.