Skip to content

Commit 4917688

Browse files
SimonBlankeSimonBlanke
committed
Merge branch 'master' into feature/add-constr-opt
2 parents bf26abd + 4fe153e commit 4917688

15 files changed

Lines changed: 311 additions & 14 deletions

File tree

‎README.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -76,9 +76,9 @@ As its name suggests Hyperactive started as a hyperparameter optimization packag
7676
<br>
7777

7878
## What's new?
79+
- ### 27.08.2023 v4.5.0 add early-stopping for Optimization Strategies
7980
- ### 01.03.2023 v4.4.0 add new feature: "Optimization Strategies"
8081
- ### 18.11.2022 v4.3.0 with three new optimization algorithms (Spiral Optimization, Lipschitz Optimizer, DIRECT Optimizer)
81-
- ### 04.05.2022 v4.2.0 with support of handling Exceptions and Callbacks
8282

8383
<br>
8484

@@ -115,29 +115,29 @@ Hyperactive features a collection of optimization algorithms that can be used fo
115115
<li><a href="./examples/optimization_techniques/hill_climbing.py">Hill Climbing</a></li>
116116
<li><a href="./examples/optimization_techniques/repulsing_hill_climbing.py">Repulsing Hill Climbing</a></li>
117117
<li><a href="./examples/optimization_techniques/simulated_annealing.py">Simulated Annealing</a></li>
118-
<li>Downhill Simplex Optimizer</li>
118+
<li><a href="./examples/optimization_techniques/downhill_simplex.py">Downhill Simplex Optimizer</a></li>
119119
</ul><br>
120120
<a><b>Global Search:</b></a>
121121
<ul>
122122
<li><a href="./examples/optimization_techniques/random_search.py">Random Search</a></li>
123-
<li>Grid Search Optimizer</li>
123+
<li><a href="./examples/optimization_techniques/grid_search.py">Grid Search</a></li>
124124
<li><a href="./examples/optimization_techniques/rand_rest_hill_climbing.py">Random Restart Hill Climbing</a></li>
125125
<li><a href="./examples/optimization_techniques/random_annealing.py">Random Annealing</a> [<a href="#/./overview#experimental-algorithms">*</a>] </li>
126-
<li>Powell's Method</li>
127-
<li>Pattern Search</li>
126+
<li><a href="./examples/optimization_techniques/pattern_search.py">Powell's Method</a></li>
127+
<li><a href="./examples/optimization_techniques/powells_method.py">Pattern Search</a></li>
128128
</ul><br>
129129
<a><b>Population Methods:</b></a>
130130
<ul>
131131
<li><a href="./examples/optimization_techniques/parallel_tempering.py">Parallel Tempering</a></li>
132132
<li><a href="./examples/optimization_techniques/particle_swarm_optimization.py">Particle Swarm Optimizer</li>
133-
<li>Spiral Optimization</li>
133+
<li><a href="./examples/optimization_techniques/spiral_optimization.py">Spiral Optimization</li>
134134
<li><a href="./examples/optimization_techniques/evolution_strategy.py">Evolution Strategy</a></li>
135135
</ul><br>
136136
<a><b>Sequential Methods:</b></a>
137137
<ul>
138138
<li><a href="./examples/optimization_techniques/bayesian_optimization.py">Bayesian Optimization</a></li>
139-
<li>Lipschitz Optimization</li>
140-
<li>Direct Algorithm</li>
139+
<li><a href="./examples/optimization_techniques/lipschitz_optimization.py">Lipschitz Optimization</a></li>
140+
<li><a href="./examples/optimization_techniques/direct_algorithm.py">Direct Algorithm</a></li>
141141
<li><a href="./examples/optimization_techniques/tpe.py">Tree of Parzen Estimators</a></li>
142142
<li><a href="./examples/optimization_techniques/forest_optimization.py">Forest Optimizer</a>
143143
[<a href="#/./overview#references">dto</a>] </li>
@@ -939,7 +939,7 @@ Each of the following optimizer classes can be initialized and passed to the "ad
939939
</details>
940940

941941
<details>
942-
<summary><b>v4.5.0</b> </summary>
942+
<summary><b>v4.5.0</b> :heavy_check_mark: </summary>
943943

944944
- [x] add early stopping feature to custom optimization strategies
945945
- [x] display additional outputs from objective-function in results in command-line

‎examples/optimization_techniques/direct_algorithm.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import DirectAlgorithm
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = DirectAlgorithm()
20+
21+
22+
hyper = Hyperactive()
23+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
24+
hyper.run()

‎examples/optimization_techniques/downhill_simplex.py‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import DownhillSimplexOptimizer
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = DownhillSimplexOptimizer(
20+
alpha=1.2,
21+
gamma=1.1,
22+
beta=0.8,
23+
sigma=1,
24+
)
25+
26+
27+
hyper = Hyperactive()
28+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
29+
hyper.run()

‎examples/optimization_techniques/grid_search.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import GridSearchOptimizer
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = GridSearchOptimizer(
20+
step_size=3,
21+
)
22+
23+
24+
hyper = Hyperactive()
25+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
26+
hyper.run()

‎examples/optimization_techniques/lipschitz_optimization.py‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import LipschitzOptimizer
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = LipschitzOptimizer(
20+
sampling={"random": 100000},
21+
)
22+
23+
hyper = Hyperactive()
24+
hyper.add_search(sphere_function, search_space, n_iter=100, optimizer=opt)
25+
hyper.run()

‎examples/optimization_techniques/pattern_search.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import PatternSearch
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = PatternSearch(
20+
n_positions=2,
21+
pattern_size=0.5,
22+
reduction=0.99,
23+
)
24+
25+
26+
hyper = Hyperactive()
27+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
28+
hyper.run()

‎examples/optimization_techniques/powells_method.py‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import PowellsMethod
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = PowellsMethod(
20+
iters_p_dim=20,
21+
)
22+
23+
hyper = Hyperactive()
24+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
25+
hyper.run()

‎examples/optimization_techniques/spiral_optimization.py‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import SpiralOptimization
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-25, 10, 0.1)),
16+
"y": list(np.arange(-10, 15, 0.1)),
17+
}
18+
19+
opt = SpiralOptimization(
20+
population=15,
21+
decay_rate=0.99,
22+
)
23+
24+
hyper = Hyperactive()
25+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
26+
hyper.run()

‎examples/optimization_techniques/stochastic_hill_climbing.py‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
import numpy as np
2+
3+
from hyperactive import Hyperactive
4+
from hyperactive.optimizers import StochasticHillClimbingOptimizer
5+
6+
7+
def sphere_function(para):
8+
x = para["x"]
9+
y = para["y"]
10+
11+
return -(x * x + y * y)
12+
13+
14+
search_space = {
15+
"x": list(np.arange(-10, 10, 0.1)),
16+
"y": list(np.arange(-10, 10, 0.1)),
17+
}
18+
19+
opt = StochasticHillClimbingOptimizer(
20+
epsilon=0.01,
21+
n_neighbours=5,
22+
distribution="laplace",
23+
p_accept=0.05,
24+
)
25+
26+
hyper = Hyperactive()
27+
hyper.add_search(sphere_function, search_space, n_iter=1500, optimizer=opt)
28+
hyper.run()

‎hyperactive/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
# Email: simon.blanke@yahoo.com
33
# License: MIT License
44

5-
__version__ = "4.4.3"
5+
__version__ = "4.5.0"
66
__license__ = "MIT"
77

88

0 commit comments

Comments
 (0)