Difference between revisions of "FAQ"
| Line 80: | Line 80: | ||
=== I sometimes get flat models when using rational functions === | === I sometimes get flat models when using rational functions === | ||
First make sure the model is indeed flat, and does not just appear so on the plot. You can verify this by looking at the output axis range and making sure it's within reasonable bounds. When there are poles in the model, the axis range is sometimes stretched to make it possible to plot the high values around the pole, causing the rest of the model to appear flat. If the model contains poles, refer to the next question for the solution. | First make sure the model is indeed flat, and does not just appear so on the plot. You can verify this by looking at the output axis range and making sure it's within reasonable bounds. When there are poles in the model, the axis range is sometimes stretched to make it possible to plot the high values around the pole, causing the rest of the model to appear flat. If the model contains poles, refer to the next question for the solution. | ||
| + | |||
The PolynomialModel tries to do a least squares fit, based on which monomials are allowed in numerator and denominator. We have experienced that some models just find a flat model as the best least squares fit. There are two causes for this: | The PolynomialModel tries to do a least squares fit, based on which monomials are allowed in numerator and denominator. We have experienced that some models just find a flat model as the best least squares fit. There are two causes for this: | ||
# The number of sample points is few, and the model parameters (as explained [[Model types explained#PolynomialModel|here]] and [[Adaptive Model Builders|here]]) force the model to use only a very small set of degrees of freedom. The solution in this case is to increase the minimum percentage bound in the xxxPolynomialInterface section of your configuration file: change the percentageBounds option to "60,100", "80,100", or even "100,100". A setting of "100,100" will force the polynomial models to always exactly interpolate. However, note that this does not scale very well with the number of samples. If, after increasing the percentage bounds you still get weird, spiky, models you simply need more samples or you should switch to a different model type. | # The number of sample points is few, and the model parameters (as explained [[Model types explained#PolynomialModel|here]] and [[Adaptive Model Builders|here]]) force the model to use only a very small set of degrees of freedom. The solution in this case is to increase the minimum percentage bound in the xxxPolynomialInterface section of your configuration file: change the percentageBounds option to "60,100", "80,100", or even "100,100". A setting of "100,100" will force the polynomial models to always exactly interpolate. However, note that this does not scale very well with the number of samples. If, after increasing the percentage bounds you still get weird, spiky, models you simply need more samples or you should switch to a different model type. | ||
# Another possibility is that given a set of monomial degrees, the flat function is just the best possible least squares fit. In that case you simply need to wait for more samples. | # Another possibility is that given a set of monomial degrees, the flat function is just the best possible least squares fit. In that case you simply need to wait for more samples. | ||
| − | |||
| − | |||
| − | |||
=== When using rational functions I sometimes get 'spikes' (poles) in my model === | === When using rational functions I sometimes get 'spikes' (poles) in my model === | ||
| Line 92: | Line 90: | ||
When the denominator polynomial of a rational model has zeros inside the domain, the model will tend to infinity near these points. In most cases these models will only be recognized as being `the best' for a short period of time. As more samples get selected these models get replaced by better ones and the spikes should disappear. | When the denominator polynomial of a rational model has zeros inside the domain, the model will tend to infinity near these points. In most cases these models will only be recognized as being `the best' for a short period of time. As more samples get selected these models get replaced by better ones and the spikes should disappear. | ||
| − | + | However, since even the Cross-Validation measure (which is the most accurate one available) is not capable of detecting all poles, it is possible that a model which does contain some poles will be considered the best model by the toolbox. This may or may not be an issue, depending on what you want to use the model for. If it doesn't matter that the model is very inaccurate at one particular, small spot (near the pole), you can use the model with the pole and it should perform properly. | |
| + | |||
| + | However, if the model should have a reasonable error on the entire domain, several methods are available to reduce the chance of getting poles or remove the possibility altogether. The possible solutions are: | ||
* Simply wait for more data, usually spikes disappear but not always. | * Simply wait for more data, usually spikes disappear but not always. | ||
| − | * If you roughly know the output range your data should have, an easy way is to use the Min-Max Measure together with your current measure (Crossvalidation by default). This will cause models whose response falls outside the min-max bounds to be penalized extra, thus spikes should disappear | + | * If you roughly know the output range your data should have, an easy way to elimiante poles is to use the Min-Max Measure together with your current measure (Crossvalidation by default). This will cause models whose response falls outside the min-max bounds to be penalized extra, thus spikes should disappear. See the [[MinMaxMeasure]] and [[Combining measures]] pages for more information. |
| − | * Use a different model type (RBF, ANN, SVM,...), spikes are a typical problem of rational functions | + | * Use a different model type (RBF, ANN, SVM,...), spikes are a typical problem of rational functions. |
* Try using the RationalPoleSampleSelector, it was designed to get rid of this problem more quickly, but it only selects one sample at the time and therefore probably needs updating. | * Try using the RationalPoleSampleSelector, it was designed to get rid of this problem more quickly, but it only selects one sample at the time and therefore probably needs updating. | ||
| + | |||
| + | |||
| + | === There is no noise in my data yet the rational functions don't interpolate === | ||
| + | See the first question. | ||
=== When loading a model from disk I get "Warning: Class ':all:' is an unknown object class. Object 'model' of this class has been converted to a structure." === | === When loading a model from disk I get "Warning: Class ':all:' is an unknown object class. Object 'model' of this class has been converted to a structure." === | ||
Revision as of 12:55, 27 June 2007
General
What is a global surrogate model?
A global surrogate model is a mathematical model that mimics the behavior of a computationally expensive simulation code over the complete parameter space as accurately as possible, using as little data points as possible. So note that optimization is not the primary goal, although it can be done as a post-processing step. Global surrogate models are useful for:
- design space exploration, to get a feel of how the different parameters behave
- sensitivity analysis
- what-if analysis
- ...
In addition they are a cheap way to model large scale systems, multiple global surrogate models can be chained together in a model cascade.
The M3-Toolbox is primarily concerned with global surrogate modeling, though surrogate driven optimization is supported as well.
What about surrogate driven optimization?
See the About#What is it used for page.
How can I stay up to date with the latest news
To stay up to date with the latest news and releases, we also recommend subscribing to our mailinglist here. Traffic will be kept to a minimum and you can unsubscribe at any time.
Installation and Configuration
Upgrading
How do I upgrade to a newer version?
Delete your old toolbox directory and replace it by the new one.
Using
I want to model my own problem
See the Adding an example page.
I want to contribute some data/patch/documentation/...
See the Contributing page.
How do I interface with the toolbox?
See the Data format page.
Can I give a model created by the toolbox to somebody else?
Why are the Neural Networks so slow?
You are probably using the CrossValidation measure. CrossValidation is used by default if you have not defined a measure yourself. Since you need to train them, neural nets will always be slower than the other models. Using crossvalidation will slow things down much much more (5-times slower by default). Therefore, when using one of the neural network model types, please use a different measure such as TestSamples or SampleError. See the comments in default.xml for examples.
How can I speed things up?
There are a number of things you can do to speed things up:
- If you have a multi-core/multi-cpu machine, set the threadCount variable in LocalSampleEvaluator equal to the number of cores/CPUs
- Disable some, or all of the profilers
- Turn off the plotting of models in ContextConfig
- Upgrade to Matlab 7.4 or later which has better multi-threaded support
How do I turn off adaptive sampling (run the toolbox for a fixed set of samples)
You can switch off adaptive sample selection if you simply do not specify a <SampleSelector> tag in your configuration file. In this case all the available data will be used and only adaptive modeling will be done. This is useful if you just want to see what the best model is you can get for a fixed dataset. This only works with datasets.
How do I change the error function (relative error, RMS, ...)?
The <Measure> tag specifies the algorithm to use to assign models a score, e.g., cross validation. It is also possible to specify which error function to use, in the measure. Say you want to use cross validation with the maximum absolute error, then you would put:
<Measure type="CrossValidation" target="0.001" errorFcn="maxAbsoluteError"/>
On the other hand, if you wanted to use the TestSamples measure with a relative root-mean-square error you would put:
<Measure type="TestSamples" target="0.001" errorFcn="relativeRms"/>
The default error function is "absoluteRms". These error functions can be found in the src/matlab/tools directory. You are free to modify them and add your own.
What regular expressions can I use to filter Profilers
See the syntax here.
Troubleshooting
I have a problem and I want to report it
See the Reporting problems page.
I sometimes get flat models when using rational functions
First make sure the model is indeed flat, and does not just appear so on the plot. You can verify this by looking at the output axis range and making sure it's within reasonable bounds. When there are poles in the model, the axis range is sometimes stretched to make it possible to plot the high values around the pole, causing the rest of the model to appear flat. If the model contains poles, refer to the next question for the solution.
The PolynomialModel tries to do a least squares fit, based on which monomials are allowed in numerator and denominator. We have experienced that some models just find a flat model as the best least squares fit. There are two causes for this:
- The number of sample points is few, and the model parameters (as explained here and here) force the model to use only a very small set of degrees of freedom. The solution in this case is to increase the minimum percentage bound in the xxxPolynomialInterface section of your configuration file: change the percentageBounds option to "60,100", "80,100", or even "100,100". A setting of "100,100" will force the polynomial models to always exactly interpolate. However, note that this does not scale very well with the number of samples. If, after increasing the percentage bounds you still get weird, spiky, models you simply need more samples or you should switch to a different model type.
- Another possibility is that given a set of monomial degrees, the flat function is just the best possible least squares fit. In that case you simply need to wait for more samples.
When using rational functions I sometimes get 'spikes' (poles) in my model
When the denominator polynomial of a rational model has zeros inside the domain, the model will tend to infinity near these points. In most cases these models will only be recognized as being `the best' for a short period of time. As more samples get selected these models get replaced by better ones and the spikes should disappear.
However, since even the Cross-Validation measure (which is the most accurate one available) is not capable of detecting all poles, it is possible that a model which does contain some poles will be considered the best model by the toolbox. This may or may not be an issue, depending on what you want to use the model for. If it doesn't matter that the model is very inaccurate at one particular, small spot (near the pole), you can use the model with the pole and it should perform properly.
However, if the model should have a reasonable error on the entire domain, several methods are available to reduce the chance of getting poles or remove the possibility altogether. The possible solutions are:
- Simply wait for more data, usually spikes disappear but not always.
- If you roughly know the output range your data should have, an easy way to elimiante poles is to use the Min-Max Measure together with your current measure (Crossvalidation by default). This will cause models whose response falls outside the min-max bounds to be penalized extra, thus spikes should disappear. See the MinMaxMeasure and Combining measures pages for more information.
- Use a different model type (RBF, ANN, SVM,...), spikes are a typical problem of rational functions.
- Try using the RationalPoleSampleSelector, it was designed to get rid of this problem more quickly, but it only selects one sample at the time and therefore probably needs updating.
There is no noise in my data yet the rational functions don't interpolate
See the first question.
When loading a model from disk I get "Warning: Class ':all:' is an unknown object class. Object 'model' of this class has been converted to a structure."
You are trying to load a model file without the toolbox in your matlab path. Make sure the toolbox is in your matlab path. In short: Start matlab, run <M3-Toolbox-directory>/startup.m (to ensure the toolbox is in your path) and then try to load your model.
When loading a model from disk I get something like "Warning: Element(s) of array 'foo' do not match the current constructor definition for class 'bar'. The element(s) have been converted to structures."
This is a silly bug in the code. If you are using a toolbox version prior to 4.0 please upgrade to the latest release or latest snapshot, your problems should be fixed. If you are using a current version (>= 4.0) please report it as a bug.
