Difference between revisions of "Optimizer"

From SUMOwiki
Jump to navigationJump to search
 
(3 intermediate revisions by one other user not shown)
Line 1: Line 1:
The SUMO Toolbox contains a whole hierarchy of optimizers that can be used by several components (for instance the [[SampleSelector#OptimizeCriterion| OptimizeCriterion]] or the [[ModelBuilder#OptimizeModelBuilder|OptimizerModelBuilder]]).
+
The SUMO Toolbox contains a whole hierarchy of optimizers that can be used by several components (for instance the [[Config:SampleSelector#expectedImprovement|EGO algorithm]] or the [[Config:AdaptiveModelBuilder#krigingoptim|OptimizerModelBuilder]]).
  
 
A list of available optimizers is found in <code>src/matlab/tools/Optimization/optimizers</code>. Note that some optimizers need some external code available in the extension pack (or the Matlab Optimization toolbox).
 
A list of available optimizers is found in <code>src/matlab/tools/Optimization/optimizers</code>. Note that some optimizers need some external code available in the extension pack (or the Matlab Optimization toolbox).
  
A couple of example configurations are listed below. These tags can be used in supported components as stated above. When a particular option is not given a sane default will be used. Full information about the several options can be found in the optimizer source codee, implementation and corresponding papers.
+
A couple of '''example configurations''' are found [[Config:Optimizer|here]]. These tags can be used in supported components as stated above. When a particular option is not given a sane default will be used. Full information about the several options can be found in the optimizer source code, implementation and corresponding papers.
 
 
=== CMAESOptimizer ===
 
Evolution Strategy with Covariance Matrix Adaptation (CMA-ES)
 
<source lang="xml">
 
<Optimizer type="CMAESOptimizer">
 
<Option key="maxFunEvals" value="Inf" />
 
<Option key="stopFitness" value="10" />
 
<Option key="maxIterations" value="1000" />
 
<Option key="TolFun" value="10" />
 
<Option key="sigma" value="-1" />
 
</Optimizer>
 
</source>
 
 
 
=== DifferentialEvolution ===
 
Differential Evolution.
 
<source lang="xml">
 
<Optimizer type="DifferentialEvolution">
 
<Option key="strategy" value="3" />
 
<Option key="weight" value="0.85" />
 
<Option key="crossover" value="1" />
 
<Option key="populationSize" value="30" />
 
<Option key="maxIterations" value="50" />
 
</Optimizer>
 
</source>
 
 
 
=== DirectOptimizer ===
 
The DIviding RECtangles (DIRECT) optimizer of Donald D. R. Jones.
 
<source lang="xml">
 
<Optimizer type="DirectOptimizer">
 
<Option key="eps" value="1e-4" />
 
<Option key="maxevals" value="200" />
 
<Option key="maxits" value="100" />
 
<Option key="maxdeep" value="100" />
 
<Option key="showits" value="false" />
 
<Option key="penalty" value="25000" />
 
</Optimizer>
 
</source>
 
 
 
=== MatlabGA ===
 
Matlab Genetic Algorithm
 
<source lang="xml">
 
<Optimizer type="MatlabGA">
 
<Option key="Generations" value="1000" />
 
<Option key="EliteCount" value="2" />
 
<Option key="CrossoverFraction" value="0.8" />
 
<Option key="PopulationSize" value="20" />
 
<Option key="MigrationInterval" value="20" />
 
<Option key="MigrationFraction" value="0.1" />
 
<Option key="Vectorize" value="on" />
 
</Optimizer>
 
</source>
 
 
 
=== MatlabOptimizer ===
 
Wrapper around fmincon and fminunc functions of Matlab Optimization toolbox
 
<source lang="xml">
 
<Optimizer type="MatlabOptimizer">
 
<Option key="maxIterations" value="1000" />
 
<Option key="maxFunEvals" value="100" />
 
<Option key="largeScale" value="off" />
 
<Option key="diagnostics" value="off" />
 
<Option key="functionTolerance" value="1e-4" />
 
</Optimizer>
 
</source>
 
 
 
=== MatlabPatternSearch ===
 
Matlab Pattern search (patternsearch function of Matlab Direct Search toolbox).
 
<source lang="xml">
 
<Optimizer type="MatlabPatternSearch">
 
<Option key="maxIterations" value="500" />
 
<Option key="maxFunEvals" value="100" />
 
<Option key="timeLimit" value="Inf" />
 
<Option key="meshTolerance" value="1e-4" />
 
<Option key="functionTolerance" value="1e-4" />
 
<Option key="maxMeshSize" value="Inf" />
 
<Option key="cache" value="on" />
 
<Option key="cacheTolerance" value="1e-6" />
 
</Optimizer>
 
</source>
 
 
 
=== MatlabSimAnnealing ===
 
Matlab Simulated Annealing.
 
<source lang="xml">
 
<Optimizer type="MatlabSimAnnealing">
 
<Option key="AnnealingFcn" value="@annealingfast" />
 
<Option key="TemperatureFcn" value="@temperatureexp" />
 
<Option key="AcceptanceFcn" value="@acceptancesa" />
 
<Option key="TolFun" value="1.0000e-6" />
 
<Option key="StallIterLimit" value="500" />
 
<Option key="MaxFunEvals" value="1000" />
 
<Option key="TimeLimit" value="Inf" />
 
<Option key="MaxIter" value="Inf" />
 
<Option key="ObjectiveLimit" value="-Inf" />
 
</Optimizer>
 
</source>
 
 
 
=== MatlabPSOt ===
 
Particle Swarm Optimization (needs PSOt toolbox).
 
<source lang="xml">
 
<Optimizer type="PSOtOptimizer">
 
<Option key="maxiters" value="200" />
 
<Option key="popSize" value="24" />
 
<Option key="acc_const1" value="2" />
 
<Option key="acc_const2" value="2" />
 
<Option key="initialInertiaWeight" value="0.9" />
 
<Option key="finalInertiaWeight" value="0.4" />
 
<Option key="epochInertia" value="150" />
 
<Option key="minGlobalError" value="1e-25" />
 
<Option key="gradientTermination" value="150" />
 
<Option key="errorGoal" value="NaN" />
 
<Option key="typePSO" value="0" />
 
<Option key="seedPSO" value="0" />
 
<Option key="mv" value="4" />
 
</Optimizer>
 
</source>
 
  
 
=== Add a custom optimizer ===
 
=== Add a custom optimizer ===
In order to implement a new optimization algorithm one should only create one new class in <code>src/matlab/tools/Optimization/optimizers</code>. The class should be derived from the base class Optimizer and MUST implement at least the following functions (for example implementations, please see the provided optimizers):
+
In order to implement a new optimization algorithm one should only create one new class in <code>src/matlab/tools/Optimization/optimizers</code>. The class should be derived from the base class Optimizer and MUST implement at least the following functions:
 
* A constructor accepting one parameter (the configuration xml tag) OR a varargin
 
* A constructor accepting one parameter (the configuration xml tag) OR a varargin
* [this, x, fval] = optimize(this, arg )
+
* [this, x, fval] = optimize(this, arg)
Accepts a function handle to optimize OR a SUMO model that needs to be optimized. x and fval are the resulting optimum(s) and associated function value(s).
+
** Accepts a function handle OR a SUMO model to optimize. x and fval are the resulting optimum(s) and associated function value(s).
 
In addition the following methods COULD be implemented if appropriate:
 
In addition the following methods COULD be implemented if appropriate:
* When implementing a population-based optimization algorithm: size = getPopulationSize(this)
+
* size = getPopulationSize(this)
* When the algorithm supports constraints: this = setInputConstraints( this, con )
+
** When implementing a population-based optimization algorithm
 +
* this = setInputConstraints( this, con )
 +
** When the algorithm supports constraints
  
It is advised to copy a simple existing optimizer and work from there. For instance, DifferentialEvolution is a good starting point. The optimizer can be used like:
+
It is advised to copy a simple existing optimizer and work from there. For example implementations, please see the provided optimizers. For instance, DifferentialEvolution is a good starting point. The optimizer can be used like:
 
<source lang="xml">
 
<source lang="xml">
 
<Optimizer type="CLASSNAME">
 
<Optimizer type="CLASSNAME">

Latest revision as of 17:33, 30 January 2012

The SUMO Toolbox contains a whole hierarchy of optimizers that can be used by several components (for instance the EGO algorithm or the OptimizerModelBuilder).

A list of available optimizers is found in src/matlab/tools/Optimization/optimizers. Note that some optimizers need some external code available in the extension pack (or the Matlab Optimization toolbox).

A couple of example configurations are found here. These tags can be used in supported components as stated above. When a particular option is not given a sane default will be used. Full information about the several options can be found in the optimizer source code, implementation and corresponding papers.

Add a custom optimizer

In order to implement a new optimization algorithm one should only create one new class in src/matlab/tools/Optimization/optimizers. The class should be derived from the base class Optimizer and MUST implement at least the following functions:

  • A constructor accepting one parameter (the configuration xml tag) OR a varargin
  • [this, x, fval] = optimize(this, arg)
    • Accepts a function handle OR a SUMO model to optimize. x and fval are the resulting optimum(s) and associated function value(s).

In addition the following methods COULD be implemented if appropriate:

  • size = getPopulationSize(this)
    • When implementing a population-based optimization algorithm
  • this = setInputConstraints( this, con )
    • When the algorithm supports constraints

It is advised to copy a simple existing optimizer and work from there. For example implementations, please see the provided optimizers. For instance, DifferentialEvolution is a good starting point. The optimizer can be used like:

<Optimizer type="CLASSNAME">
	<Option key="option1" value="1" />
	<Option key="option2" value="2" />
	...
</Optimizer>