Skip to content

NAS_HPO_Algorithms.md

Latest commit

 

History

History
116 lines (75 loc) · 8.77 KB

NAS_HPO_Algorithms.md

File metadata and controls

116 lines (75 loc) · 8.77 KB
graph LR
    Algorithm_Orchestrators["Algorithm Orchestrators"]
    Search_Space_Definition["Search Space Definition"]
    Search_Strategy_Candidate_Generation["Search Strategy & Candidate Generation"]
    Candidate_Evaluation["Candidate Evaluation"]
    Configuration_Utilities["Configuration Utilities"]
    Gradient_based_Optimization_Utilities["Gradient-based Optimization Utilities"]
    Result_Selection["Result Selection"]
    Algorithm_Orchestrators -- "Initializes search by querying" --> Search_Space_Definition
    Algorithm_Orchestrators -- "Requests candidates from" --> Search_Strategy_Candidate_Generation
    Algorithm_Orchestrators -- "Drives evaluation via" --> Candidate_Evaluation
    Algorithm_Orchestrators -- "Queries" --> Result_Selection
    Algorithm_Orchestrators -- "Uses" --> Configuration_Utilities
    Search_Space_Definition -- "Provides boundaries to" --> Algorithm_Orchestrators
    Search_Space_Definition -- "Informs" --> Search_Strategy_Candidate_Generation
    Search_Strategy_Candidate_Generation -- "Proposes candidates to" --> Candidate_Evaluation
    Search_Strategy_Candidate_Generation -- "Utilizes" --> Gradient_based_Optimization_Utilities
    Search_Strategy_Candidate_Generation -- "Interacts with" --> Search_Space_Definition
    Candidate_Evaluation -- "Assesses candidates from" --> Search_Strategy_Candidate_Generation
    Candidate_Evaluation -- "Provides performance metrics to" --> Algorithm_Orchestrators
    Configuration_Utilities -- "Are used by" --> Algorithm_Orchestrators
    Configuration_Utilities -- "Are used by" --> Search_Strategy_Candidate_Generation
    Gradient_based_Optimization_Utilities -- "Are utilized by" --> Search_Strategy_Candidate_Generation
    Result_Selection -- "Provides the best result to" --> Algorithm_Orchestrators
Loading

CodeBoardingDemoContact

Details

The NAS/HPO Algorithms subsystem is the core of the project, encapsulating the various search strategies and optimization techniques. Its architecture is designed for modularity and extensibility, allowing different algorithms to be plugged in and orchestrated for automated deep learning experimentation.

Algorithm Orchestrators

These components serve as the entry points and high-level controllers for specific NAS/HPO algorithms. They initialize the search process, define the iterative search loop, and coordinate interactions between other components like candidate generation, evaluation, and result selection.

Related Classes/Methods:

Search Space Definition

This component is responsible for formally defining the search domain for architectures or hyperparameters. It provides structured representations of the valid configurations that can be explored by the search algorithms.

Related Classes/Methods:

Search Strategy & Candidate Generation

This central component implements the core search logic, including proposing new candidate architectures or hyperparameter configurations. For learning-based algorithms, it also encompasses the mechanisms for updating the internal model or policy that guides future candidate generation.

Related Classes/Methods:

Candidate Evaluation

This component is responsible for assessing the performance of generated candidates. It typically involves training and validating a model based on the proposed architecture or hyperparameters to obtain performance metrics.

Related Classes/Methods:

Configuration Utilities

This component handles the transformation and conversion between different representations of architectures or hyperparameters, ensuring compatibility across various stages of the search and evaluation process.

Related Classes/Methods:

Gradient-based Optimization Utilities

This component provides low-level mathematical operations and utilities specifically for differentiable NAS algorithms, enabling gradient-based optimization of the architecture search process.

Related Classes/Methods:

Result Selection

This component is responsible for identifying and retrieving the most performant architecture or hyperparameter set discovered during the entire search process, based on the evaluation metrics.

Related Classes/Methods:

FAQ