[DOC] add multiple pages and small corrections

Author: SimonBlanke

docs/source/_snippets/user_guide/optimizers.py

@@ -56,6 +56,18 @@ def objective(params):
 # [end:repulsing_hill_climbing]
 
 
+# [start:stochastic_hill_climbing]
+from hyperactive.opt.gfo import StochasticHillClimbing
+
+optimizer = StochasticHillClimbing(
+    search_space=search_space,
+    n_iter=100,
+    experiment=objective,
+    p_accept=0.3,  # Probability of accepting worse solutions
+)
+# [end:stochastic_hill_climbing]
+
+
 # [start:downhill_simplex]
 from hyperactive.opt.gfo import DownhillSimplexOptimizer
 

docs/source/_templates/class.rst

@@ -5,8 +5,6 @@
 
 .. autoclass:: {{ objname }}
 
-.. include:: {{module}}.{{objname}}.examples
-
 .. raw:: html
 
     <div class="clearer"></div>

docs/source/_templates/function.rst

@@ -5,8 +5,6 @@
 
 .. autofunction:: {{ objname }}
 
-.. include:: {{module}}.{{objname}}.examples
-
 .. raw:: html
 
     <div class="clearer"></div>

docs/source/examples/integrations.rst

@@ -33,8 +33,8 @@ For time series forecasting and classification with sktime:
        pip install hyperactive[sktime-integration]
 
 
-Installation
-------------
+Installing Extras
+-----------------
 
 Install integration extras as needed:
 

docs/source/faq.rst

@@ -0,0 +1,40 @@
+.. _faq:
+
+==========================
+Frequently Asked Questions
+==========================
+
+This section answers common questions about Hyperactive. For migration from v4,
+see the :ref:`user_guide_migration`.
+
+.. toctree::
+   :maxdepth: 1
+
+   faq/getting_started
+   faq/search_space
+   faq/common_issues
+   faq/advanced_usage
+   faq/integrations
+   faq/getting_help
+
+
+Overview
+--------
+
+:ref:`faq_getting_started`
+    Choosing optimizers, iteration counts, and understanding maximization.
+
+:ref:`faq_search_space`
+    Defining continuous, discrete, and mixed parameter spaces.
+
+:ref:`faq_common_issues`
+    Slow optimization, reproducibility, handling errors.
+
+:ref:`faq_advanced_usage`
+    Parallel execution, callbacks, parameter constraints.
+
+:ref:`faq_integrations`
+    Using Hyperactive with PyTorch, XGBoost, and other frameworks.
+
+:ref:`faq_getting_help`
+    Where to report bugs and get support.

docs/source/faq/advanced_usage.rst

@@ -0,0 +1,69 @@
+.. _faq_advanced_usage:
+
+==============
+Advanced Usage
+==============
+
+Can I run optimizations in parallel?
+------------------------------------
+
+Currently, Hyperactive v5 runs single optimizer instances.
+For parallel evaluation of candidates, consider using Optuna
+backend optimizers which support parallel trials:
+
+.. code-block:: python
+
+    from hyperactive.opt.optuna import TPEOptimizer
+
+    optimizer = TPEOptimizer(
+        search_space=search_space,
+        n_iter=100,
+        experiment=objective,
+        # Optuna handles parallelization
+    )
+
+
+Can I save and resume optimization?
+-----------------------------------
+
+This feature is planned but not yet available in v5. As a workaround,
+you can log results during optimization and use them as initial points
+for a new run.
+
+
+Are callbacks supported?
+------------------------
+
+User-defined callbacks during optimization are not currently supported in v5.
+The Optuna backend has internal early-stopping callbacks, but there's no
+general callback interface for tracking progress or modifying behavior during
+optimization.
+
+For progress monitoring, you can add logging inside your objective function:
+
+.. code-block:: python
+
+    iteration = 0
+
+    def objective(params):
+        global iteration
+        iteration += 1
+        score = evaluate_model(params)
+        print(f"Iteration {iteration}: score={score:.4f}")
+        return score
+
+
+How do I add constraints between parameters?
+--------------------------------------------
+
+Handle constraints in your objective function by returning a poor score
+for invalid combinations:
+
+.. code-block:: python
+
+    def objective(params):
+        # Constraint: min_samples_split must be >= min_samples_leaf
+        if params["min_samples_split"] < params["min_samples_leaf"]:
+            return -np.inf  # Invalid configuration
+
+        return evaluate_model(params)

docs/source/faq/common_issues.rst

@@ -0,0 +1,72 @@
+.. _faq_common_issues:
+
+=============
+Common Issues
+=============
+
+Why is my optimization slow?
+----------------------------
+
+**Slow objective function**: The optimizer only controls search strategy.
+If each evaluation takes a long time, consider:
+
+- Reducing cross-validation folds
+- Using a subset of training data for tuning
+- Simplifying your model during search
+
+**Large search space**: More combinations require more iterations.
+Consider reducing parameter granularity or using smarter optimizers
+like Bayesian optimization.
+
+**Too many iterations**: Start with fewer iterations and increase
+if needed.
+
+
+Why does my score vary between runs?
+------------------------------------
+
+Optimization algorithms are stochastic. To get reproducible results,
+set a random seed:
+
+.. code-block:: python
+
+    optimizer = HillClimbing(
+        search_space=search_space,
+        n_iter=100,
+        experiment=objective,
+        random_state=42,  # Set seed for reproducibility
+    )
+
+
+My objective function returns NaN or raises exceptions
+------------------------------------------------------
+
+Handle invalid configurations in your objective function:
+
+.. code-block:: python
+
+    def objective(params):
+        try:
+            score = evaluate_model(params)
+            if np.isnan(score):
+                return -np.inf  # Return worst possible score
+            return score
+        except Exception:
+            return -np.inf  # Return worst possible score on error
+
+
+How do I see what parameters were tried?
+----------------------------------------
+
+Access the search history after optimization:
+
+.. code-block:: python
+
+    best_params = optimizer.solve()
+
+    # Access results
+    print(f"Best parameters: {optimizer.best_params_}")
+    print(f"Best score: {optimizer.best_score_}")
+
+    # Full search history (if available)
+    # Check optimizer attributes for search_data or similar

docs/source/faq/getting_help.rst

@@ -0,0 +1,19 @@
+.. _faq_getting_help:
+
+============
+Getting Help
+============
+
+Where can I report bugs or request features?
+--------------------------------------------
+
+Open an issue on `GitHub <https://github.com/SimonBlanke/Hyperactive/issues>`_.
+
+
+Where can I get help?
+---------------------
+
+- Check the :ref:`examples` for code samples
+- Read the :ref:`user_guide` for detailed explanations
+- Join the `Discord <https://discord.gg/7uKdHfdcJG>`_ community
+- Search or ask on `GitHub Discussions <https://github.com/SimonBlanke/Hyperactive/discussions>`_

docs/source/faq/getting_started.rst

@@ -0,0 +1,53 @@
+.. _faq_getting_started:
+
+===============
+Getting Started
+===============
+
+Which optimizer should I use?
+-----------------------------
+
+For most problems, start with one of these recommendations:
+
+**Small search spaces (<100 combinations)**
+   Use :class:`~hyperactive.opt.gfo.GridSearch` to exhaustively evaluate all options.
+
+**General-purpose optimization**
+   :class:`~hyperactive.opt.gfo.BayesianOptimizer` works well for expensive
+   objective functions where you want to minimize evaluations.
+
+**Fast, simple problems**
+   :class:`~hyperactive.opt.gfo.HillClimbing` or
+   :class:`~hyperactive.opt.gfo.RandomSearch` are good starting points.
+
+**High-dimensional spaces**
+   Population-based methods like :class:`~hyperactive.opt.gfo.ParticleSwarmOptimizer`
+   or :class:`~hyperactive.opt.gfo.EvolutionStrategyOptimizer` handle many
+   parameters well.
+
+See :ref:`user_guide_optimizers` for detailed guidance on choosing optimizers.
+
+
+How many iterations do I need?
+------------------------------
+
+This depends on your search space size and objective function:
+
+- **Rule of thumb**: Start with ``n_iter = 10 * number_of_parameters``
+- **Expensive functions**: Use fewer iterations with Bayesian optimization
+- **Fast functions**: Use more iterations with simpler optimizers
+
+You can monitor progress and stop early if the score plateaus.
+
+
+Does Hyperactive minimize or maximize?
+--------------------------------------
+
+**Hyperactive maximizes** the objective function. If you want to minimize,
+return the negative of your metric:
+
+.. code-block:: python
+
+    def objective(params):
+        error = compute_error(params)
+        return -error  # Negate to minimize

docs/source/faq/integrations.rst

@@ -0,0 +1,64 @@
+.. _faq_integrations:
+
+============
+Integrations
+============
+
+Can I use Hyperactive with PyTorch (not Lightning)?
+---------------------------------------------------
+
+Yes, create a custom objective function:
+
+.. code-block:: python
+
+    import torch
+
+    def objective(params):
+        model = MyPyTorchModel(
+            hidden_size=params["hidden_size"],
+            dropout=params["dropout"],
+        )
+        # Train and evaluate your model
+        train_model(model, train_loader)
+        accuracy = evaluate_model(model, val_loader)
+        return accuracy
+
+
+How does Hyperactive compare to Optuna?
+---------------------------------------
+
+**Hyperactive with native GFO backend**:
+
+- Simple, unified API
+- Wide variety of optimization algorithms
+- Great for hyperparameter tuning
+
+**Hyperactive with Optuna backend**:
+
+- Access Optuna's algorithms through Hyperactive's interface
+- Combine the strengths of both libraries
+
+**Pure Optuna**:
+
+- More features (pruning, distributed, database storage)
+- Larger community and ecosystem
+- More configuration options
+
+Choose based on your needs: Hyperactive for simplicity, Optuna for
+advanced features.
+
+
+Can I use Hyperactive with other ML frameworks?
+-----------------------------------------------
+
+Yes, any framework works with custom objective functions:
+
+.. code-block:: python
+
+    # XGBoost example
+    import xgboost as xgb
+
+    def objective(params):
+        model = xgb.XGBClassifier(**params)
+        scores = cross_val_score(model, X, y, cv=3)
+        return scores.mean()