Use optimization from GADMA
GADMA provides different optimizations algorithms with one interface that could be used to optimize different functions.
GADMA has gadma.optimizers module with different global, local optimizers and their combinations. Here we will try to use them for find optimums of different functions.
All optimizations have two base options: * log_transform - indicates if search space of optimization is log-transformed. * maximize - indicates what problem this optimizer solves: maximization or minimization.
Base pipeline
To run optimization one should prepare function f that takes vector x of parameters values as first argument, list of variables that corresponds to x and name of optimization algorithm.
Function to optimize
For this example we will use Rosenbrook function as an example:
\[f(x) = \sum_{i=1}^{n-1} \left[ 100(x_{i+1} - x_i^2)^2 + (1- x_i)^2\right]\]
The optimum of such function is in point \((1, 1, ..., 1)\).
We will use existing realization of Rosenbrook function in scipy:
[1]:
from scipy.optimize import rosen
f = rosen
Now let us create variables for our Rosenbrook function. There is additional example about different variables in GADMA API in examples/api_examples/variables_example.ipynb or in the documentation.
[2]:
from gadma import *
var1 = ContinuousVariable('var1', [-1, 2])
var2 = ContinuousVariable('var2', [0, 10])
var3 = PopulationSizeVariable('var3') # it actually suits this example by value of its bound.
Global optimizations
GADMA has basic algorithm of global optimization - genetic algorithm. There is also Bayesian optimization but it is not stable anough to use now. So let us take our f and optimize it with GA. First of all get our optimizer:
[3]:
opt1 = get_global_optimizer("Genetic_algorithm")
opt1.maximize = False # we have minimization problem
[4]:
help(opt1.optimize)
Help on method optimize in module gadma.optimizers.genetic_algorithm:
optimize(f, variables, args=(), num_init=50, X_init=None, Y_init=None, linear_constrain=None, maxiter=None, maxeval=None, verbose=0, callback=None, report_file=None, eval_file=None, save_file=None, restore_file=None, restore_points_only=False, restore_x_transform=None) method of gadma.optimizers.genetic_algorithm.GeneticAlgorithm instance
Return best values of `variables` that minimizes/maximizes
the function `f`.
:param f: function to minimize/maximize. The usage must be the
following: f(x, *args), where x is list of values.
:param variables: list of variables (instances of
:class:`gadma.Variable` class) of the function.
:param X_init: list of initial values.
:param Y_init: value of function `f` on initial values from `X_init`.
:param args: arguments of function `f`.
:param maxiter: maximum number of genetic algorithm's generations.
:param maxeval: maximum number of function evaluations.
:param callback: callback to call after each generation.
It will be called as callback(x, y), where x, y -
best_solution of generation and its fitness.
So we will set our function and variables and run optimization by:
[5]:
f = rosen
variables = [var1, var2, var3]
# first run for 5 iterations of genetic algorithm and print all output
res = opt1.optimize(f, variables, verbose=1, maxiter=10)
Generation #0.
Current generation of solutions:
N Value of fitness function Solution
0 56.574674 (var1=-8.02e-01, var2=0.11707, var3=0.5128) r
1 67.648121 (var1=0.13811, var2=0.78656, var3=0.90075) r
2 73.174806 (var1=0.90363, var2=0.98986, var3=0.1422) r
3 87.813760 (var1=0.56828, var2=0.00362, var3=0.8743) r
4 114.322055 (var1=0.96019, var2=1.05319, var3=0.0481) r
5 130.260656 (var1=0.50258, var2=0.82626, var3=1.66796) r
6 172.393181 (var1=0.76879, var2=1.41752, var3=3.02846) r
7 230.069293 (var1=0.03052, var2=1.2144, var3=2.37939) r
8 1681.749298 (var1=1.5708, var2=2.14935, var3=0.53315) r
9 2022.846855 (var1=-8.85e-01, var2=2.30096, var3=1.06656) r
Current mean mutation rate: 0.200000
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 56.57467403543714
Solution: (var1=-8.02e-01, var2=0.11707, var3=0.5128) r
Generation #1.
Current generation of solutions:
N Value of fitness function Solution
0 56.574674 (var1=-8.02e-01, var2=0.11707, var3=0.5128) r
1 67.648121 (var1=0.13811, var2=0.78656, var3=0.90075) r
2 67.648121 (var1=0.13811, var2=0.78656, var3=0.90075) c
3 69.529045 (var1=0.90363, var2=0.98986, var3=0.16426) m
4 83.673058 (var1=0.76879, var2=1.41752, var3=2.39843) m
5 172.393181 (var1=0.76879, var2=1.41752, var3=3.02846) c
6 214.039213 (var1=0.03052, var2=1.38965, var3=2.37939) m
7 1681.749298 (var1=1.5708, var2=2.14935, var3=0.53315) c
8 7754.096662 (var1=0.60962, var2=2.94208, var3=0.23596) r
9 13542.056084 (var1=0.09119, var2=1.7776, var3=14.66097) r
Current mean mutation rate: 0.240000
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 56.57467403543714
Solution: (var1=-8.02e-01, var2=0.11707, var3=0.5128) r
Mean time: 0.007 sec.
Generation #2.
Current generation of solutions:
N Value of fitness function Solution
0 15.104148 (var1=0.90363, var2=0.73737, var3=0.16426) m
1 48.206888 (var1=0.76879, var2=0.11707, var3=0.5128) c
2 56.574674 (var1=-8.02e-01, var2=0.11707, var3=0.5128) r
3 67.648121 (var1=0.13811, var2=0.78656, var3=0.90075) r
4 69.916279 (var1=-8.02e-01, var2=0.11707, var3=0.63218) m
5 191.433031 (var1=0.76879, var2=1.41752, var3=0.90075) c
6 599.249314 (var1=-4.41e-01, var2=1.61298, var3=0.61294) r
7 722.800104 (var1=0.76879, var2=1.0936, var3=3.83694) mm
8 7948.366158 (var1=0.13811, var2=2.94208, var3=0.23596) c
9 305868.434960 (var1=0.06174, var2=7.46726, var3=0.96438) r
Current mean mutation rate: 0.288000
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 15.10414752105835
Solution: (var1=0.90363, var2=0.73737, var3=0.16426) m
Mean time: 0.006 sec.
Generation #3.
Current generation of solutions:
N Value of fitness function Solution
0 15.104148 (var1=0.90363, var2=0.73737, var3=0.16426) m
1 39.359484 (var1=-4.41e-01, var2=0.11707, var3=0.61294) c
2 48.206888 (var1=0.76879, var2=0.11707, var3=0.5128) c
3 48.206888 (var1=0.76879, var2=0.11707, var3=0.5128) c
4 49.279849 (var1=0.76879, var2=0.10812, var3=0.5128) m
5 124.926305 (var1=0.13811, var2=1.0936, var3=0.90075) c
6 234.544529 (var1=0.76879, var2=1.41752, var3=0.7209) m
7 8029.610396 (var1=0.13811, var2=2.94208, var3=0.18785) m
8 180909.408864 (var1=0.45318, var2=6.89186, var3=5.49729) r
9 469334.638352 (var1=0.13088, var2=8.25583, var3=0.15192) r
Current mean mutation rate: 0.275168
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 15.10414752105835
Solution: (var1=0.90363, var2=0.73737, var3=0.16426) m
Mean time: 0.005 sec.
Generation #4.
Current generation of solutions:
N Value of fitness function Solution
0 9.008701 (var1=0.90363, var2=0.56035, var3=0.16426) m
1 15.104148 (var1=0.90363, var2=0.73737, var3=0.16426) m
2 35.253067 (var1=0.76879, var2=0.11707, var3=0.35947) m
3 39.359484 (var1=-4.41e-01, var2=0.11707, var3=0.61294) c
4 48.206888 (var1=0.76879, var2=0.11707, var3=0.5128) c
5 150.202214 (var1=0.13811, var2=1.0936, var3=0.61294) c
6 362.490609 (var1=0.13811, var2=1.41752, var3=0.7209) c
7 458.283642 (var1=0.27063, var2=0.40782, var3=2.27868) r
8 7913.137784 (var1=0.13811, var2=2.94208, var3=0.2569) m
9 26221.189961 (var1=-1.02e-01, var2=5.14986, var3=11.1714) r
Current mean mutation rate: 0.330201
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 9.008700592809172
Solution: (var1=0.90363, var2=0.56035, var3=0.16426) m
Mean time: 0.005 sec.
Generation #5.
Current generation of solutions:
N Value of fitness function Solution
0 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) m
1 9.008701 (var1=0.90363, var2=0.56035, var3=0.16426) m
2 9.008701 (var1=0.90363, var2=0.56035, var3=0.16426) c
3 15.104148 (var1=0.90363, var2=0.73737, var3=0.16426) m
4 38.320585 (var1=-4.41e-01, var2=0.13988, var3=0.61294) m
5 71.990308 (var1=0.76879, var2=1.0936, var3=0.5128) c
6 443.675322 (var1=0.08627, var2=1.36533, var3=0.2569) mm
7 4654.010086 (var1=1.6115, var2=2.65169, var3=0.21191) r
8 7374.023998 (var1=1.07976, var2=3.34865, var3=2.91163) r
9 25456.311743 (var1=0.90363, var2=5.14986, var3=11.1714) c
Current mean mutation rate: 0.396241
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 2.5378326176371835
Solution: (var1=0.73636, var2=0.56035, var3=0.16426) m
Mean time: 0.005 sec.
Generation #6.
Current generation of solutions:
N Value of fitness function Solution
0 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) m
1 7.515823 (var1=0.90363, var2=0.56035, var3=0.22743) m
2 9.008701 (var1=0.90363, var2=0.56035, var3=0.16426) m
3 17.915913 (var1=-4.41e-01, var2=0.56035, var3=0.16426) c
4 39.108874 (var1=1.07976, var2=0.56035, var3=0.16426) c
5 39.878190 (var1=0.76879, var2=1.0936, var3=0.8144) m
6 45.474861 (var1=0.76879, var2=0.13988, var3=0.5128) c
7 67.896375 (var1=-4.41e-01, var2=0.13988, var3=0.82446) m
8 76.898545 (var1=-1.48e-01, var2=0.74295, var3=0.06687) r
9 407910.264188 (var1=0.88308, var2=7.97531, var3=0.14823) r
Current mean mutation rate: 0.378586
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 2.5378326176371835
Solution: (var1=0.73636, var2=0.56035, var3=0.16426) m
Mean time: 0.004 sec.
Generation #7.
Current generation of solutions:
N Value of fitness function Solution
0 1.214084 (var1=0.73636, var2=0.56035, var3=0.21816) m
1 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) m
2 2.583082 (var1=0.76879, var2=0.56035, var3=0.16426) c
3 7.412472 (var1=-6.40e-01, var2=0.56035, var3=0.16426) m
4 7.515823 (var1=0.90363, var2=0.56035, var3=0.22743) m
5 7.515823 (var1=0.90363, var2=0.56035, var3=0.22743) c
6 9.335244 (var1=-5.93e-01, var2=0.56035, var3=0.16426) m
7 41.731136 (var1=-4.41e-01, var2=0.56035, var3=0.82446) c
8 1510.723283 (var1=0.78144, var2=0.03964, var3=3.84494) r
9 4155.124561 (var1=-2.53e-02, var2=2.48096, var3=0.20816) r
Current mean mutation rate: 0.454303
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 1.2140835589750745
Solution: (var1=0.73636, var2=0.56035, var3=0.21816) m
Mean time: 0.004 sec.
Generation #8.
Current generation of solutions:
N Value of fitness function Solution
0 1.214084 (var1=0.73636, var2=0.56035, var3=0.21816) m
1 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) m
2 7.515823 (var1=0.90363, var2=0.56035, var3=0.22743) c
3 7.842367 (var1=-5.93e-01, var2=0.56035, var3=0.22743) c
4 15.883830 (var1=0.44482, var2=0.56035, var3=0.16426) m
5 20.640181 (var1=0.37193, var2=0.56035, var3=0.16426) m
6 56.952460 (var1=-6.72e-01, var2=0.97386, var3=1.46674) r
7 1478.888527 (var1=0.31412, var2=0.03964, var3=3.84494) m
8 4207.538350 (var1=-2.53e-02, var2=2.48096, var3=0.16426) c
9 882830.189226 (var1=-5.77e-01, var2=9.72183, var3=1.02946) r
Current mean mutation rate: 0.434061
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 1.2140835589750745
Solution: (var1=0.73636, var2=0.56035, var3=0.21816) m
Mean time: 0.004 sec.
Generation #9.
Current generation of solutions:
N Value of fitness function Solution
0 1.214084 (var1=0.73636, var2=0.56035, var3=0.21816) m
1 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) m
2 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) c
3 5.092744 (var1=0.46282, var2=0.42174, var3=0.21816) mm
4 15.883830 (var1=0.44482, var2=0.56035, var3=0.16426) c
5 53.310934 (var1=0.37193, var2=0.71254, var3=0.06204) mm
6 62.353679 (var1=-6.72e-01, var2=1.2224, var3=1.46674) m
7 4207.538350 (var1=-2.53e-02, var2=2.48096, var3=0.16426) c
8 127440.740649 (var1=1.37133, var2=5.98973, var3=0.41879) r
9 494962.340131 (var1=0.03034, var2=8.43732, var3=1.34653) r
Current mean mutation rate: 0.414720
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 1.2140835589750745
Solution: (var1=0.73636, var2=0.56035, var3=0.21816) m
Mean time: 0.004 sec.
Generation #10.
Current generation of solutions:
N Value of fitness function Solution
0 0.555263 (var1=0.73636, var2=0.56035, var3=0.26304) m
1 1.214084 (var1=0.73636, var2=0.56035, var3=0.21816) m
2 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) m
3 2.537833 (var1=0.73636, var2=0.56035, var3=0.16426) c
4 3.674001 (var1=0.54404, var2=0.42643, var3=0.06204) mm
5 6.272040 (var1=0.46282, var2=0.42174, var3=0.06204) c
6 16.359379 (var1=0.41742, var2=0.56035, var3=0.21816) m
7 327.262053 (var1=-2.53e-02, var2=1.2224, var3=0.16426) c
8 24507.166510 (var1=-8.40e-01, var2=4.37933, var3=3.96577) r
9 403191.326218 (var1=0.39847, var2=8.0792, var3=2.27597) r
Current mean mutation rate: 0.497664
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 0.5552629843855047
Solution: (var1=0.73636, var2=0.56035, var3=0.26304) m
Mean time: 0.004 sec.
[6]:
# now run for 1000 iterations and print every 100 iteration
# It may converge faster than 1000 iterations
res = opt1.optimize(f, variables, verbose=100, maxiter=1000)
Generation #0.
Current generation of solutions:
N Value of fitness function Solution
0 10.552293 (var1=-8.09e-01, var2=0.48978, var3=0.44764) r
1 30.387173 (var1=-6.87e-01, var2=0.41098, var3=0.68686) r
2 33.131647 (var1=0.08459, var2=0.47698, var3=0.54289) r
3 174.058054 (var1=-2.18e-01, var2=1.09656, var3=0.41154) r
4 231.890043 (var1=1.39326, var2=1.40142, var3=0.54117) r
5 392.481091 (var1=0.05274, var2=1.58042, var3=1.30463) r
6 517.699804 (var1=0.33108, var2=1.94534, var3=2.44504) r
7 584.194038 (var1=1.0545, var2=0.91638, var3=3.24881) r
8 645.364608 (var1=-9.81e-01, var2=2.46343, var3=4.03448) r
9 1056.283990 (var1=-6.82e-01, var2=1.77675, var3=0.18879) r
Current mean mutation rate: 0.200000
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 10.552293186197584
Solution: (var1=-8.09e-01, var2=0.48978, var3=0.44764) r
Generation #100.
Current generation of solutions:
N Value of fitness function Solution
0 0.061651 (var1=0.88493, var2=0.78108, var3=0.61094) m
1 0.063339 (var1=0.88493, var2=0.78108, var3=0.61428) m
2 0.063339 (var1=0.88493, var2=0.78108, var3=0.61428) c
3 0.063339 (var1=0.88493, var2=0.78108, var3=0.61428) c
4 0.090993 (var1=0.88493, var2=0.78108, var3=0.62723) c
5 0.302246 (var1=0.9028, var2=0.77055, var3=0.61428) m
6 373.547100 (var1=1.62753, var2=3.01363, var3=7.1957) c
7 7707.724174 (var1=0.87255, var2=3.01363, var3=0.59881) m
8 271046.472139 (var1=-6.66e-01, var2=7.21326, var3=0.41509) r
9 512406.489146 (var1=0.33401, var2=8.60858, var3=3.03519) r
Current mean mutation rate: 0.020477
Current mean number of params to change during mutation: 1
--Best solution by value of fitness function--
Value of fitness: 0.06333856222983994
Solution: (var1=0.88493, var2=0.78108, var3=0.61428) m
Mean time: 0.004 sec.
So the result we got:
[7]:
print(f"Full result info:\n{res}")
print(f"Best values of parameters:\t{res.x}")
print(f"Best value of function:\t{res.y}")
print(f"Optimal valies of parameters:\t[1, 1, 1]")
print(f"Optimum value of function:\t{f([1, 1, 1])}")
Full result info:
status: 0
success: True
message: CONVERGENCE: NO IMPROVEMENT DURING 100 ITERATIONS
x: [0.8849317473156754 0.7810785929158809 0.6142801591640911]
y: 0.06333856222983994
n_eval: 1783
n_iter: 180
Best values of parameters: [0.8849317473156754 0.7810785929158809 0.6142801591640911]
Best value of function: 0.06333856222983994
Optimal valies of parameters: [1, 1, 1]
Optimum value of function: 0.0
So our result is close but not ideal. To find better solution we could use local optimizations.
Local optimizations
GADMA provides several local optimizations (local optimizations work with continuous variables only):
[8]:
print("GADMA has following local optimizers:")
for opt in all_local_optimizers():
print(opt.id)
GADMA has following local optimizers:
optimize
optimize_log_fmin
optimize_log
optimize_powell
None
optimize_lbfgsb
optimize_log_lbfgsb
optimize_log_powell
optimize_fmin
Names were taken from dadi and moments software. Here are some descriptions:
optimize - BFGS.
optimize_lbfgsb - L-BFGS-B.
optimize_powell - Powell’s method.
optimize_fmin - Neadler-Mead algorithm.
None - no optimization.
If there is log in name then this log transform for search space is using. All these methods could be got by get_local_optimizer by one of its names. For example Powell’s method with log_transform could be got by:
[9]:
opt2 = get_local_optimizer("Powell_log")
# or
opt2 = get_local_optimizer("optimize_log_powell")
print(f"Optimizer is log transformed: {opt._log_trasform}")
Optimizer is log transformed: False
So the optimize function is almost the same except new x0 argument for first approximation of best solution. Let us use result from global optimizer:
[10]:
help(opt2.optimize)
Help on method optimize in module gadma.optimizers.local_optimizer:
optimize(f, variables, x0, args=(), options={}, linear_constrain=None, maxiter=None, maxeval=None, verbose=0, callback=None, eval_file=None, report_file=None, save_file=None, restore_file=None, restore_points_only=False, restore_x_transform=None) method of gadma.optimizers.local_optimizer.ManuallyConstrOptimizer instance
Run optimization of local search algorithm.
:param f: Target function to optimize.
:type f: func
:param variables: Variables of `f` which values should be optimized.
:type variables: :class:`gadma.utils.VariablePool`
:param x0: Initial point to start optimization.
:type x0: list
:param args: Additional arguments of target function.
:type args: tuple
:param options: Additional options kwargs for optimization.
:type options: dict
:param maxiter: Maximum number of iterations to run.
:type maxiter: int
:param maxeval: Maximum number of evaluations to run. If None then run
until converge.
:type maxeval: int
:param verbose: Verbosity of the output. If 0 then no reports.
:type verbose: int
:param callback: Callback to run after each iteration of optimization.
Should be called as `callback(x, y)`
:type callback: function
:param report_file: File to save report. Check option `verbose`.
:type report_file: str
:param eval_file: File to save all evaluations of the function `f`.
:type eval_file: str
:param save_file: File to save information during optimization for its
reconstruction.
:type save_file: str
:param restore_file: File to restore previous run.
:type restore_file: str
:param restore_points_only: Restore point/points from previous run and
run optimization from them once more. If
False then previous run will be resumed.
:type restore_points_only: bool
:param restore_x_transform: Restore points but transform them before
usage in this run.
:type restore_x_transform: function
[11]:
opt2.optimize(f, variables, x0=res.x, verbose=1)
1 0.06333856222983994 (var1=0.88493, var2=0.78108, var3=0.61428)
2 57.0227960616551 (var1=0.17547, var2=0.78108, var3=0.61428)
3 30.96696404229835 (var1=0.47698, var2=0.78108, var3=0.61428)
4 81.13972107200483 (var1=1.29657, var2=0.78108, var3=0.61428)
5 7.438547050163425 (var1=0.71467, var2=0.78108, var3=0.61428)
6 7.198651372348043 (var1=1.02394, var2=0.78108, var3=0.61428)
7 0.29295430196062716 (var1=0.85667, var2=0.78108, var3=0.61428)
8 0.06720632937571171 (var1=0.88055, var2=0.78108, var3=0.61428)
9 0.06315252292607579 (var1=0.88425, var2=0.78108, var3=0.61428)
10 0.06315005999083027 (var1=0.88415, var2=0.78108, var3=0.61428)
11 0.06315006214326982 (var1=0.88416, var2=0.78108, var3=0.61428)
12 0.06315009580883257 (var1=0.88415, var2=0.78108, var3=0.61428)
13 1697.2887647241073 (var1=0.88415, var2=2.12319, var3=0.61428)
14 74.86616894871793 (var1=0.88415, var2=0.15488, var3=0.61428)
15 32.46086956077491 (var1=0.88415, var2=0.421, var3=0.61428)
16 61.54420179234234 (var1=0.88415, var2=1.14441, var3=0.61428)
17 5.673506040486336 (var1=0.88415, var2=0.64839, var3=0.61428)
18 5.61345551991403 (var1=0.88415, var2=0.90377, var3=0.61428)
19 0.17060254969790106 (var1=0.88415, var2=0.76584, var3=0.61428)
20 0.062429944243381066 (var1=0.88415, var2=0.78149, var3=0.61428)
21 0.060596795921373126 (var1=0.88415, var2=0.78391, var3=0.61428)
22 0.060592711044827614 (var1=0.88415, var2=0.7838, var3=0.61428)
23 0.060592850594461796 (var1=0.88415, var2=0.78378, var3=0.61428)
24 0.06059308840816433 (var1=0.88415, var2=0.78383, var3=0.61428)
25 111.45551711704908 (var1=0.88415, var2=0.7838, var3=1.66979)
26 24.320568755524313 (var1=0.88415, var2=0.7838, var3=0.1218)
27 8.08363424457279 (var1=0.88415, var2=0.7838, var3=0.3311)
28 8.221410496823118 (var1=0.88415, var2=0.7838, var3=0.90002)
29 0.5443490198370673 (var1=0.88415, var2=0.7838, var3=0.5448)
30 0.16797640077573167 (var1=0.88415, var2=0.7838, var3=0.64712)
31 0.06097784125681071 (var1=0.88415, var2=0.7838, var3=0.61238)
32 0.0605925093321193 (var1=0.88415, var2=0.7838, var3=0.6143)
33 0.060592241914231765 (var1=0.88415, var2=0.7838, var3=0.61435)
34 0.06059224193454777 (var1=0.88415, var2=0.7838, var3=0.61435)
35 0.06059224198829155 (var1=0.88415, var2=0.7838, var3=0.61435)
36 0.06477847578610435 (var1=0.88338, var2=0.78654, var3=0.61442)
37 57.43796088434369 (var1=0.17532, var2=0.7838, var3=0.61435)
38 31.31144654021446 (var1=0.47656, var2=0.7838, var3=0.61435)
39 80.11682054591802 (var1=1.29543, var2=0.7838, var3=0.61435)
40 7.529066819296717 (var1=0.71537, var2=0.7838, var3=0.61435)
41 6.953463422328221 (var1=1.02303, var2=0.7838, var3=0.61435)
42 0.2861168269898842 (var1=0.85847, var2=0.7838, var3=0.61435)
43 0.06428830228071211 (var1=0.88193, var2=0.7838, var3=0.61435)
44 0.059858770096483535 (var1=0.88587, var2=0.7838, var3=0.61435)
45 0.05984883224732027 (var1=0.8857, var2=0.7838, var3=0.61435)
46 0.059848853851534975 (var1=0.88568, var2=0.7838, var3=0.61435)
47 0.05984896091797328 (var1=0.88571, var2=0.7838, var3=0.61435)
48 1723.1480270347665 (var1=0.8857, var2=2.1306, var3=0.61435)
49 75.12838038498057 (var1=0.8857, var2=0.15542, var3=0.61435)
50 32.4478916541783 (var1=0.8857, var2=0.42247, var3=0.61435)
51 62.90870988417057 (var1=0.8857, var2=1.1484, var3=0.61435)
52 5.677105812022754 (var1=0.8857, var2=0.64932, var3=0.61435)
53 10.76225018492341 (var1=0.8857, var2=0.9479, var3=0.61435)
54 0.23815276213761033 (var1=0.8857, var2=0.76171, var3=0.61435)
55 0.06679229760215485 (var1=0.8857, var2=0.78915, var3=0.61435)
56 0.05962353034984764 (var1=0.8857, var2=0.78452, var3=0.61435)
57 0.05962009441218935 (var1=0.8857, var2=0.78461, var3=0.61435)
58 0.059620092457041 (var1=0.8857, var2=0.78462, var3=0.61435)
59 0.0596201365019107 (var1=0.8857, var2=0.78463, var3=0.61435)
60 111.22398421222725 (var1=0.8857, var2=0.78462, var3=1.66997)
61 24.44422045435629 (var1=0.8857, var2=0.78462, var3=0.12182)
62 8.1529970137971 (var1=0.8857, var2=0.78462, var3=0.33114)
63 8.153102754728097 (var1=0.8857, var2=0.78462, var3=0.90012)
64 0.5449507960340264 (var1=0.8857, var2=0.78462, var3=0.54595)
65 0.17316022247937884 (var1=0.8857, var2=0.78462, var3=0.64935)
66 0.05984867609681263 (var1=0.8857, var2=0.78462, var3=0.61365)
67 0.05945731427272312 (var1=0.8857, var2=0.78462, var3=0.61554)
68 0.059456620763198215 (var1=0.8857, var2=0.78462, var3=0.61563)
69 0.0594566313363329 (var1=0.8857, var2=0.78462, var3=0.61562)
70 0.05945664307016195 (var1=0.8857, var2=0.78462, var3=0.61564)
71 0.059063473594041196 (var1=0.88724, var2=0.78544, var3=0.61691)
72 0.060025755543307475 (var1=0.88975, var2=0.78676, var3=0.619)
73 0.05919613277444269 (var1=0.8882, var2=0.78594, var3=0.61771)
74 0.05912536477717941 (var1=0.88665, var2=0.78512, var3=0.61642)
75 0.059063245028626477 (var1=0.88728, var2=0.78546, var3=0.61694)
76 0.05906328025043809 (var1=0.8873, var2=0.78546, var3=0.61696)
77 0.05906328875914428 (var1=0.88726, var2=0.78545, var3=0.61693)
78 112.43780824198534 (var1=0.88728, var2=0.78546, var3=1.67703)
79 24.522673831410383 (var1=0.88728, var2=0.78546, var3=0.12233)
80 8.147679210348716 (var1=0.88728, var2=0.78546, var3=0.33253)
81 8.295021601686166 (var1=0.88728, var2=0.78546, var3=0.90392)
82 0.5469231486347989 (var1=0.88728, var2=0.78546, var3=0.54709)
83 0.1670055102659087 (var1=0.88728, var2=0.78546, var3=0.64979)
84 0.05945183240537135 (var1=0.88728, var2=0.78546, var3=0.61497)
85 0.059063493869065425 (var1=0.88728, var2=0.78546, var3=0.61689)
86 0.05906324305841605 (var1=0.88728, var2=0.78546, var3=0.61694)
87 0.059063243057883776 (var1=0.88728, var2=0.78546, var3=0.61694)
88 0.05906324305808255 (var1=0.88728, var2=0.78546, var3=0.61694)
89 1736.6340318745927 (var1=0.88728, var2=2.13509, var3=0.61694)
90 75.73447222260324 (var1=0.88728, var2=0.15575, var3=0.61694)
91 32.74624162552199 (var1=0.88728, var2=0.42336, var3=0.61694)
92 63.300390902270294 (var1=0.88728, var2=1.15082, var3=0.61694)
93 5.73241554589938 (var1=0.88728, var2=0.65086, var3=0.61694)
94 5.868540865051781 (var1=0.88728, var2=0.90883, var3=0.61694)
95 0.17267609444484244 (var1=0.88728, var2=0.76836, var3=0.61694)
96 0.0633672845134948 (var1=0.88728, var2=0.79028, var3=0.61694)
97 0.05861470068420056 (var1=0.88728, var2=0.78653, var3=0.61694)
98 0.05861360319190697 (var1=0.88728, var2=0.78659, var3=0.61694)
99 0.058613610873836944 (var1=0.88728, var2=0.7866, var3=0.61694)
100 0.05861368491724064 (var1=0.88728, var2=0.78658, var3=0.61694)
101 0.05856301882253134 (var1=0.88887, var2=0.78742, var3=0.61826)
102 0.060175002362247695 (var1=0.89144, var2=0.78878, var3=0.62039)
103 0.05892977872248001 (var1=0.88985, var2=0.78794, var3=0.61907)
104 0.058488811326223474 (var1=0.88826, var2=0.7871, var3=0.61775)
105 0.05848767237150018 (var1=0.88818, var2=0.78706, var3=0.61768)
106 0.05848767951529672 (var1=0.88817, var2=0.78706, var3=0.61768)
107 0.05848769059051995 (var1=0.88819, var2=0.78706, var3=0.61769)
108 0.05896518413306298 (var1=0.89066, var2=0.78951, var3=0.61975)
109 112.32848044627256 (var1=0.88818, var2=0.78706, var3=1.67904)
110 24.757505038921366 (var1=0.88818, var2=0.78706, var3=0.12248)
111 8.268109455453654 (var1=0.88818, var2=0.78706, var3=0.33293)
112 8.211725247493176 (var1=0.88818, var2=0.78706, var3=0.90501)
113 0.5495939855289741 (var1=0.88818, var2=0.78706, var3=0.54936)
114 0.07402408431425862 (var1=0.88818, var2=0.78706, var3=0.60687)
115 0.05826076360286656 (var1=0.88818, var2=0.78706, var3=0.62041)
116 0.05817110676174364 (var1=0.88818, var2=0.78706, var3=0.61947)
117 0.05817111276118113 (var1=0.88818, var2=0.78706, var3=0.61945)
118 0.05817116478022161 (var1=0.88818, var2=0.78706, var3=0.61949)
119 1750.1244261308098 (var1=0.88818, var2=2.13945, var3=0.61947)
120 76.18369080408024 (var1=0.88818, var2=0.15606, var3=0.61947)
121 32.95574130393726 (var1=0.88818, var2=0.42423, var3=0.61947)
122 63.765495290045735 (var1=0.88818, var2=1.15317, var3=0.61947)
123 5.773729543554591 (var1=0.88818, var2=0.65214, var3=0.61947)
124 5.909974377233213 (var1=0.88818, var2=0.91069, var3=0.61947)
125 0.17280038509770868 (var1=0.88818, var2=0.7699, var3=0.61947)
126 0.06252638612231508 (var1=0.88818, var2=0.79189, var3=0.61947)
127 0.0577278138226938 (var1=0.88818, var2=0.78813, var3=0.61947)
128 0.05772669444797826 (var1=0.88818, var2=0.78818, var3=0.61947)
129 0.05772670194809414 (var1=0.88818, var2=0.78819, var3=0.61947)
130 0.05772677531194169 (var1=0.88818, var2=0.78817, var3=0.61947)
131 0.057680860205617926 (var1=0.88976, var2=0.78902, var3=0.62079)
132 0.05930735532496334 (var1=0.89234, var2=0.79038, var3=0.62294)
133 0.05805215976192045 (var1=0.89075, var2=0.78954, var3=0.62161)
134 0.057604461431079154 (var1=0.88916, var2=0.7887, var3=0.62029)
135 0.05760303690579306 (var1=0.88906, var2=0.78865, var3=0.62021)
136 0.05760304366471021 (var1=0.88906, var2=0.78865, var3=0.6202)
137 0.05760305505631843 (var1=0.88907, var2=0.78866, var3=0.62022)
138 0.05672352078886539 (var1=0.88995, var2=0.79025, var3=0.62275)
139 0.05531136491723874 (var1=0.89139, var2=0.79284, var3=0.62687)
140 0.01851480494447806 (var1=1.05464, var2=1.11413, var3=1.24593)
141 1.3248253739057012 (var1=1.38445, var2=1.93199, var3=3.78598)
142 0.19410156895811675 (var1=1.17015, var2=1.37484, var3=1.90486)
143 0.0007382340401204149 (var1=0.98903, var2=0.97836, var3=0.9584)
144 0.0005111821003906816 (var1=1.00462, var2=1.00982, var3=1.02165)
145 0.00028872178968469646 (var1=0.998, var2=0.99641, var3=0.99443)
146 0.00029328027575736403 (var1=0.99915, var2=0.99874, var3=0.99913)
147 0.00029847649862004455 (var1=0.99685, var2=0.99408, var3=0.98974)
148 292.51357321737976 (var1=0.998, var2=0.99641, var3=2.70313)
149 63.30496906025734 (var1=0.998, var2=0.99641, var3=0.19718)
150 20.869239570401955 (var1=0.998, var2=0.99641, var3=0.536)
151 21.545255018733265 (var1=0.998, var2=0.99641, var3=1.457)
152 1.2638805200183942 (var1=0.998, var2=0.99641, var3=0.88041)
153 0.267533544551759 (var1=0.998, var2=0.99641, var3=1.04455)
154 0.001030686537574667 (var1=0.998, var2=0.99641, var3=0.98967)
155 3.322842370271053e-05 (var1=0.998, var2=0.99641, var3=0.99279)
156 3.305789680351054e-05 (var1=0.998, var2=0.99641, var3=0.99282)
157 3.307545539151939e-05 (var1=0.998, var2=0.99641, var3=0.99284)
158 3.30915493889246e-05 (var1=0.998, var2=0.99641, var3=0.99281)
159 0.00044779686099664484 (var1=0.99978, var2=0.99747, var3=0.99494)
160 0.002011627398846074 (var1=0.99512, var2=0.99469, var3=0.9894)
161 0.0004051169337430322 (var1=0.9969, var2=0.99575, var3=0.99152)
162 4.244799744744943e-05 (var1=0.99868, var2=0.99681, var3=0.99363)
163 1.457480567793838e-05 (var1=0.99831, var2=0.99659, var3=0.99319)
164 1.4576318867608422e-05 (var1=0.99831, var2=0.99659, var3=0.99319)
165 1.4576985361126131e-05 (var1=0.9983, var2=0.99659, var3=0.99318)
166 0.08778115430611688 (var1=1.12063, var2=1.25912, var3=1.59244)
167 0.13227805726800443 (var1=0.82802, var2=0.68266, var3=0.46268)
168 0.024507770420730733 (var1=0.92948, var2=0.86249, var3=0.74185)
169 0.010448996313275331 (var1=1.04337, var2=1.0897, var3=1.18945)
170 5.247198857653113e-05 (var1=0.9968, var2=0.99356, var3=0.98709)
171 6.047176104281367e-07 (var1=0.99975, var2=0.9995, var3=0.99905)
172 4.003149273302758e-07 (var1=0.99995, var2=0.99991, var3=0.99988)
173 4.006708748625715e-07 (var1=0.99993, var2=0.99988, var3=0.99982)
174 4.028249275713102e-07 (var1=0.99997, var2=0.99994, var3=0.99995)
175 0.09034161973753463 (var1=1.12466, var2=1.26776, var3=1.61198)
176 295.20087927767787 (var1=0.99995, var2=0.99991, var3=2.71796)
177 64.24923769822183 (var1=0.99995, var2=0.99991, var3=0.19826)
178 21.24116216005215 (var1=0.99995, var2=0.99991, var3=0.53894)
179 21.63821082419558 (var1=0.99995, var2=0.99991, var3=1.46499)
180 1.2813263193573465 (var1=0.99995, var2=0.99991, var3=0.88663)
181 0.2830800729077699 (var1=0.99995, var2=0.99991, var3=1.05303)
182 0.0010202903275264585 (var1=0.99995, var2=0.99991, var3=0.99663)
183 6.608147129021295e-07 (var1=0.99995, var2=0.99991, var3=0.99974)
184 2.405086189856941e-08 (var1=0.99995, var2=0.99991, var3=0.99982)
185 2.4085128118948434e-08 (var1=0.99995, var2=0.99991, var3=0.99982)
186 2.4091779817398097e-08 (var1=0.99995, var2=0.99991, var3=0.99982)
187 0.0006296031361739891 (var1=1.00174, var2=1.00098, var3=1.00196)
188 0.0016545317341091841 (var1=0.99706, var2=0.99819, var3=0.99638)
189 0.00024488938686373585 (var1=0.99885, var2=0.99925, var3=0.9985)
190 9.015174042643673e-05 (var1=1.00063, var2=1.00032, var3=1.00064)
191 9.076294545495014e-09 (var1=0.99996, var2=0.99992, var3=0.99983)
192 8.892000789206292e-09 (var1=0.99996, var2=0.99992, var3=0.99983)
193 8.893221737973971e-09 (var1=0.99996, var2=0.99992, var3=0.99983)
194 8.893805841189655e-09 (var1=0.99996, var2=0.99992, var3=0.99983)
195 0.09052842677379169 (var1=1.12248, var2=1.26332, var3=1.6031)
196 0.13037718486674457 (var1=0.82939, var2=0.68494, var3=0.46578)
197 0.023509618590357294 (var1=0.93102, var2=0.86537, var3=0.74681)
198 0.01128089562627626 (var1=1.04509, var2=1.09333, var3=1.19741)
199 5.4687011937690806e-05 (var1=0.99678, var2=0.9935, var3=0.98691)
200 1.0717809758906074e-07 (var1=0.99986, var2=0.99971, var3=0.99942)
201 2.7414864920312316e-10 (var1=1.0, var2=1.0, var3=1.0)
202 2.72233770917516e-10 (var1=1.0, var2=1.0, var3=1.0)
203 2.7307177564616513e-10 (var1=1.0, var2=1.0, var3=0.99999)
204 3.598630733510889e-07 (var1=1.00005, var2=1.00008, var3=1.00011)
205 295.2476321588907 (var1=1.0, var2=1.0, var3=2.71827)
206 64.2734624764209 (var1=1.0, var2=1.0, var3=0.19829)
207 21.25151230799889 (var1=1.0, var2=1.0, var3=0.539)
208 21.637583838450993 (var1=1.0, var2=1.0, var3=1.46516)
209 1.2817534302316456 (var1=1.0, var2=1.0, var3=0.88678)
210 0.2836401834492792 (var1=1.0, var2=1.0, var3=1.05325)
211 0.0010209746860975568 (var1=1.0, var2=1.0, var3=0.9968)
212 6.612685071833725e-07 (var1=1.0, var2=1.0, var3=0.99991)
213 1.8833859866696548e-11 (var1=1.0, var2=1.0, var3=0.99999)
214 1.7203557956609677e-11 (var1=1.0, var2=1.0, var3=0.99999)
215 1.7228871318367522e-11 (var1=1.0, var2=1.0, var3=0.99999)
216 1.7229318930444673e-11 (var1=1.0, var2=1.0, var3=0.99999)
217 0.0006357636213793796 (var1=1.00179, var2=1.00106, var3=1.00213)
218 0.0016451068932795663 (var1=0.99711, var2=0.99828, var3=0.99655)
219 0.00024118676237828036 (var1=0.9989, var2=0.99934, var3=0.99868)
220 9.245592746599701e-05 (var1=1.00068, var2=1.0004, var3=1.00081)
221 1.9582478409023648e-10 (var1=1.0, var2=1.0, var3=0.99999)
222 7.81705816475785e-12 (var1=1.0, var2=1.0, var3=1.0)
223 7.817087268252697e-12 (var1=1.0, var2=1.0, var3=1.0)
224 7.818888380554825e-12 (var1=1.0, var2=1.0, var3=0.99999)
225 0.09059627672437778 (var1=1.12253, var2=1.26343, var3=1.60336)
226 0.13033068226674646 (var1=0.82943, var2=0.68499, var3=0.46585)
227 0.023485387713827674 (var1=0.93106, var2=0.86544, var3=0.74693)
228 0.011301706200024126 (var1=1.04514, var2=1.09342, var3=1.1976)
229 5.4744450773146535e-05 (var1=0.99678, var2=0.99349, var3=0.98691)
230 1.048126506077899e-07 (var1=0.99986, var2=0.99972, var3=0.99943)
231 1.3705323106380515e-12 (var1=1.0, var2=1.0, var3=1.0)
232 2.3250474719700296e-13 (var1=1.0, var2=1.0, var3=1.0)
233 2.3325033560159503e-13 (var1=1.0, var2=1.0, var3=1.0)
234 2.332766288103868e-13 (var1=1.0, var2=1.0, var3=1.0)
235 2.413558204834738e-10 (var1=1.0, var2=1.0, var3=1.0)
236 295.24919611727694 (var1=1.0, var2=1.0, var3=2.71828)
237 64.27416756075355 (var1=1.0, var2=1.0, var3=0.19829)
238 21.251806194757773 (var1=1.0, var2=1.0, var3=0.539)
239 21.6375932535148 (var1=1.0, var2=1.0, var3=1.46516)
240 1.2817660354998646 (var1=1.0, var2=1.0, var3=0.88678)
241 0.28365506656692285 (var1=1.0, var2=1.0, var3=1.05326)
242 0.0010209942933596897 (var1=1.0, var2=1.0, var3=0.9968)
243 6.618943552007046e-07 (var1=1.0, var2=1.0, var3=0.99992)
244 1.714803008646205e-12 (var1=1.0, var2=1.0, var3=1.0)
245 1.712009571683522e-14 (var1=1.0, var2=1.0, var3=1.0)
246 1.714189774194186e-14 (var1=1.0, var2=1.0, var3=1.0)
247 1.714326063106875e-14 (var1=1.0, var2=1.0, var3=1.0)
248 0.0006359166932032684 (var1=1.00179, var2=1.00107, var3=1.00214)
249 0.0016448762646223418 (var1=0.99711, var2=0.99828, var3=0.99655)
250 0.00024109618308367493 (var1=0.9989, var2=0.99934, var3=0.99868)
251 9.251342440500468e-05 (var1=1.00068, var2=1.00041, var3=1.00081)
252 1.881037727283135e-10 (var1=1.0, var2=1.0, var3=1.0)
253 6.839741561293433e-15 (var1=1.0, var2=1.0, var3=1.0)
254 6.691625214211994e-15 (var1=1.0, var2=1.0, var3=1.0)
255 6.692668301616802e-15 (var1=1.0, var2=1.0, var3=1.0)
256 6.692668865547536e-15 (var1=1.0, var2=1.0, var3=1.0)
257 0.09059829142686093 (var1=1.12253, var2=1.26343, var3=1.60337)
258 0.13032930278479304 (var1=0.82943, var2=0.68499, var3=0.46585)
259 0.023484668957449988 (var1=0.93106, var2=0.86544, var3=0.74693)
260 0.011302324140043405 (var1=1.04514, var2=1.09342, var3=1.19761)
261 5.474616000254614e-05 (var1=0.99678, var2=0.99349, var3=0.98691)
262 1.0475054954015859e-07 (var1=0.99986, var2=0.99972, var3=0.99943)
263 1.1182355382286338e-12 (var1=1.0, var2=1.0, var3=1.0)
264 1.9574993633866026e-16 (var1=1.0, var2=1.0, var3=1.0)
265 1.9617737704165209e-16 (var1=1.0, var2=1.0, var3=1.0)
266 1.9662581007496488e-16 (var1=1.0, var2=1.0, var3=1.0)
267 2.063159267500598e-13 (var1=1.0, var2=1.0, var3=1.0)
268 295.2492426794891 (var1=1.0, var2=1.0, var3=2.71828)
269 64.27418829249157 (var1=1.0, var2=1.0, var3=0.19829)
270 21.25181481505033 (var1=1.0, var2=1.0, var3=0.539)
271 21.637593608860218 (var1=1.0, var2=1.0, var3=1.46516)
272 1.2817664066346002 (var1=1.0, var2=1.0, var3=0.88678)
273 0.28365550013003127 (var1=1.0, var2=1.0, var3=1.05326)
274 0.0010209948674819377 (var1=1.0, var2=1.0, var3=0.9968)
275 6.619130314300672e-07 (var1=1.0, var2=1.0, var3=0.99992)
276 1.6996742017436959e-12 (var1=1.0, var2=1.0, var3=1.0)
277 1.2854270001320846e-17 (var1=1.0, var2=1.0, var3=1.0)
278 1.2884591669368082e-17 (var1=1.0, var2=1.0, var3=1.0)
279 1.293512947234887e-17 (var1=1.0, var2=1.0, var3=1.0)
280 0.0006359218265768335 (var1=1.00179, var2=1.00107, var3=1.00214)
281 0.001644868477120546 (var1=0.99711, var2=0.99828, var3=0.99655)
282 0.0002410931333813359 (var1=0.9989, var2=0.99934, var3=0.99868)
283 9.251535602681285e-05 (var1=1.00068, var2=1.00041, var3=1.00082)
284 1.8810007825574122e-10 (var1=1.0, var2=1.0, var3=1.0)
285 1.5234099467936265e-16 (var1=1.0, var2=1.0, var3=1.0)
286 6.34333011831363e-18 (var1=1.0, var2=1.0, var3=1.0)
287 6.343993140325057e-18 (var1=1.0, var2=1.0, var3=1.0)
288 6.3439955715871806e-18 (var1=1.0, var2=1.0, var3=1.0)
289 0.09059835029273014 (var1=1.12253, var2=1.26343, var3=1.60337)
290 0.1303292624863931 (var1=0.82943, var2=0.68499, var3=0.46585)
291 0.02348464795908789 (var1=0.93106, var2=0.86544, var3=0.74693)
292 0.011302342194715972 (var1=1.04514, var2=1.09342, var3=1.19761)
293 5.474620994414608e-05 (var1=0.99678, var2=0.99349, var3=0.98691)
294 1.0474874198035864e-07 (var1=0.99986, var2=0.99972, var3=0.99943)
295 1.1174586408173103e-12 (var1=1.0, var2=1.0, var3=1.0)
296 1.6704792755897435e-19 (var1=1.0, var2=1.0, var3=1.0)
297 1.4457346156911087e-19 (var1=1.0, var2=1.0, var3=1.0)
298 1.4533302908425346e-19 (var1=1.0, var2=1.0, var3=1.0)
299 1.4533329734864395e-19 (var1=1.0, var2=1.0, var3=1.0)
300 1.7505695095377394e-16 (var1=1.0, var2=1.0, var3=1.0)
301 295.24924416324774 (var1=1.0, var2=1.0, var3=2.71828)
302 64.27418892490746 (var1=1.0, var2=1.0, var3=0.19829)
303 21.251815075710496 (var1=1.0, var2=1.0, var3=0.539)
304 21.63759362832723 (var1=1.0, var2=1.0, var3=1.46516)
305 1.2817664180110544 (var1=1.0, var2=1.0, var3=0.88678)
306 0.28365551291022767 (var1=1.0, var2=1.0, var3=1.05326)
307 0.0010209948846769965 (var1=1.0, var2=1.0, var3=0.9968)
308 6.619135775871371e-07 (var1=1.0, var2=1.0, var3=0.99992)
309 1.6997191534108103e-12 (var1=1.0, var2=1.0, var3=1.0)
310 1.3221232417780401e-20 (var1=1.0, var2=1.0, var3=1.0)
311 1.3021801724926641e-20 (var1=1.0, var2=1.0, var3=1.0)
312 3.421220155407809e-20 (var1=1.0, var2=1.0, var3=1.0)
313 0.0006359219546091228 (var1=1.00179, var2=1.00107, var3=1.00214)
314 0.0016448682851094847 (var1=0.99711, var2=0.99828, var3=0.99655)
315 0.00024109305784479078 (var1=0.9989, var2=0.99934, var3=0.99868)
316 9.251540407921042e-05 (var1=1.00068, var2=1.00041, var3=1.00082)
317 1.881001464435445e-10 (var1=1.0, var2=1.0, var3=1.0)
318 1.4595201158471513e-16 (var1=1.0, var2=1.0, var3=1.0)
319 6.812575045440988e-21 (var1=1.0, var2=1.0, var3=1.0)
320 6.813533484533279e-21 (var1=1.0, var2=1.0, var3=1.0)
321 6.813529307699675e-21 (var1=1.0, var2=1.0, var3=1.0)
322 0.09059835213825823 (var1=1.12253, var2=1.26343, var3=1.60337)
323 0.1303292612360932 (var1=0.82943, var2=0.68499, var3=0.46585)
324 0.02348464730608157 (var1=0.93106, var2=0.86544, var3=0.74693)
325 0.011302342758519915 (var1=1.04514, var2=1.09342, var3=1.19761)
326 5.474621151934561e-05 (var1=0.99678, var2=0.99349, var3=0.98691)
327 1.0474868570966901e-07 (var1=0.99986, var2=0.99972, var3=0.99943)
328 1.1174403840269652e-12 (var1=1.0, var2=1.0, var3=1.0)
329 2.6344066574629286e-20 (var1=1.0, var2=1.0, var3=1.0)
330 3.767150344268938e-21 (var1=1.0, var2=1.0, var3=1.0)
331 3.777366692784683e-21 (var1=1.0, var2=1.0, var3=1.0)
332 3.777556682636742e-21 (var1=1.0, var2=1.0, var3=1.0)
333 6.779352922309785e-20 (var1=1.0, var2=1.0, var3=1.0)
334 3.340124824826803e-19 (var1=1.0, var2=1.0, var3=1.0)
335 6.66127581578252e-20 (var1=1.0, var2=1.0, var3=1.0)
336 4.0460026566678415e-21 (var1=1.0, var2=1.0, var3=1.0)
337 1.6971913708814172e-22 (var1=1.0, var2=1.0, var3=1.0)
338 1.7018458084874403e-22 (var1=1.0, var2=1.0, var3=1.0)
339 1.7010072527964568e-22 (var1=1.0, var2=1.0, var3=1.0)
340 0.09059835217479639 (var1=1.12253, var2=1.26343, var3=1.60337)
341 0.13032926120480973 (var1=0.82943, var2=0.68499, var3=0.46585)
342 0.023484647290527287 (var1=0.93106, var2=0.86544, var3=0.74693)
343 0.011302342770775936 (var1=1.04514, var2=1.09342, var3=1.19761)
344 5.47462115460117e-05 (var1=0.99678, var2=0.99349, var3=0.98691)
345 1.0474868440348028e-07 (var1=0.99986, var2=0.99972, var3=0.99943)
346 1.1174399698632204e-12 (var1=1.0, var2=1.0, var3=1.0)
347 2.2748930710625645e-20 (var1=1.0, var2=1.0, var3=1.0)
348 1.7683514883528712e-22 (var1=1.0, var2=1.0, var3=1.0)
349 1.768246035522473e-22 (var1=1.0, var2=1.0, var3=1.0)
350 0.0006359219593141911 (var1=1.00179, var2=1.00107, var3=1.00214)
351 0.0016448682779338206 (var1=0.99711, var2=0.99828, var3=0.99655)
352 0.0002410930550405004 (var1=0.9989, var2=0.99934, var3=0.99868)
353 9.251540585186086e-05 (var1=1.00068, var2=1.00041, var3=1.00082)
354 1.8810014915764005e-10 (var1=1.0, var2=1.0, var3=1.0)
355 1.459432722432434e-16 (var1=1.0, var2=1.0, var3=1.0)
356 9.902838342434328e-24 (var1=1.0, var2=1.0, var3=1.0)
357 1.0037148715473513e-23 (var1=1.0, var2=1.0, var3=1.0)
358 1.0067668252456365e-23 (var1=1.0, var2=1.0, var3=1.0)
359 3.2445830103515833e-21 (var1=1.0, var2=1.0, var3=1.0)
360 1.0015054030053625e-20 (var1=1.0, var2=1.0, var3=1.0)
361 1.6082870759412662e-21 (var1=1.0, var2=1.0, var3=1.0)
362 3.961907619967029e-22 (var1=1.0, var2=1.0, var3=1.0)
363 7.552815123722756e-25 (var1=1.0, var2=1.0, var3=1.0)
364 7.5964687140654235e-25 (var1=1.0, var2=1.0, var3=1.0)
365 7.631397502796347e-25 (var1=1.0, var2=1.0, var3=1.0)
[11]:
status: 0
success: True
message: Optimization terminated successfully.
x: [1. 1. 1.]
y: 7.552815123722756e-25
n_eval: 365
n_iter: 11
And we found our [1, 1, 1] solution!
Combination of global and local optimizers
There is a special class for combination of global and local optimizers - gadma.optimizers.GlobalOptimizerAndLocalOptimizer:
[12]:
opt1 = get_global_optimizer("Genetic_algorithm")
opt2 = get_local_optimizer("BFGS_log")
opt = GlobalOptimizerAndLocalOptimizer(opt1, opt2)
[13]:
help(opt.optimize)
Help on method optimize in module gadma.optimizers.combinations:
optimize(f, variables, args=(), global_num_init=50, X_init=None, Y_init=None, local_options={}, linear_constrain=None, global_maxiter=None, local_maxiter=None, global_maxeval=None, local_maxeval=None, verbose=0, callback=None, eval_file=None, report_file=None, save_file=None, restore_file=None, restore_points_only=False, global_x_transform=None, local_x_transform=None) method of gadma.optimizers.combinations.GlobalOptimizerAndLocalOptimizer instance
:param f: Objective function.
:type f: func
:param variables: List of objective function variables.
:type variables: list of class:`gadma.utils.VariablesPool`
:param args: Arguments of `f`.
:type args: tuple
:param global_num_init: Number of initial points for global optimizer.
:type global_num_init: int
:param X_init: List of initial vectors.
:type X_init: list
:param Y_init: List of values of target function on points of `X_init`.
:type Y_init: list
:param local_options: Options for local optimizer.
:type local_options: dict
:param linear_constrain: Linear constrain on variables.
:type linear_constrain: :class:`gadma.optimizers.LinearConstrain`
:param global_maxiter: Maximum number of global optimizer iterations
to run.
:type global_maxiter: int
:param global_maxeval: Maximum number of function evaluation during
global optimization.
:type global_maxeval: int
:param local_maxiter: Maximum number of local optimizer iterations
to run.
:type local_maxiter: int
:param local_maxeval: Maximum number of function evaluation during
local optimization.
:type local_maxeval: int
:param verbose: Varbosity of reports. If 0 then no output.
:type verbose: int
:param callback: callback to run after each iteration of both
optimizers.
:type callback: func
:param eval_file: File to save of objective function evaluations.
:type eval_file: str
:param report_file: File to save report each `verbose` iteration. If
None and `verbose` > 0 then report will be printed
to stdout.
:type report_file: str
:param save_file: File to save information during the run.
:type save_file: str
:param restore_file: File to restore previous run that was saved by
:meth:`save` method.
:type restore_file: str
:param restore_points_only: Restore run last results and run again from
it.
:type restore_points_only: bool
:param global_x_transform: Transformation of vectors after restore
before run of global optimizer.
:type global_x_transform: func
:param local_x_transform: Transformation of vectors after restore
before run of local optimizer.
:type local_x_transform: bool
[14]:
f = rosen
# Lets add discrete variable - it will be optimized by global optimization and fixed during local.
var4 = DiscreteVariable('var4', [-1, 1])
variables = [var1, var2, var3, var4]
res = opt.optimize(f, variables, global_maxiter=1000, local_maxiter=50)
print(res)
status: 2
success: False
message: GLOBAL OPTIMIZATION: CONVERGENCE: NO IMPROVEMENT DURING 100 ITERATIONS; LOCAL OPTIMIZATION: Desired error not necessarily achieved due to precision loss.
x: [0.999999992799917 0.999999997047826 0.9999999907298661 1]
y: 4.8759147986991476e-14
n_eval: 2775
n_iter: 304
So this class is much simplier to use.
[ ]: