Difference between revisions of "About"

From SUMOwiki
Jump to navigationJump to search
 
Line 1: Line 1:
General toolbox information, port content from the COMS site
+
== History ==
 +
In 2004, research within the COMS group was focused on developing efficient, adaptive and accurate algorithms for polynomial and rational modeling of linear time-invariant (LTI) systems. This work resulted in a set of Matlab scripts that were used as a testing ground for new ideas and techniques. Research progressed, and with time these scripts were re-worked and refactored into one coherent Matlab toolbox, tentatively named the Multivariate MetaModeling (M3) Toolbox.
 +
 
 +
 
 +
== Design Goals ==
 +
During research into multivariate metamodeling techniques and algorithms it became clear that there was room for an adaptive tool that integrated different surrogate modeling approaches and did not tie the user down to one particular set of problems or techniques. More concretely, we were unable to find evidence of any projects that integrated:
 +
 
 +
# Building standalone global metamodels
 +
# Support for different model types, different modeling algorithms, ... (adaptive modeling)
 +
# Sequential design
 +
# Distributed computing
 +
# Usable implementation in software
 +
 
 +
This gave rise to a number of design goals that served as the guidelines for the design of the M3-toolbox. These goals are:
 +
 
 +
# Development of a fully automated, adaptive metamodel construction algorithm. Given a simulation model, the software should produce a metamodel with as little user interaction as possible ("one button approach").
 +
# There is no such thing as a "one-size-fits-all", different problems need to be modeled differently and require different levels of process knowledge. Therefore the software should be modular and extensible but not be too cumbersome to use or configure (sensible defaults).
 +
# The toolbox should minimize the required prior knowledge of the system to be modeled.
 +
# The algorithm should minimize the number of required samples in order to come to an acceptable metamodel.
 +
# The algorithm should terminate only when the predefined accuracy (set by the user) has been reached or the maximum number of iterations has been exceeded.
 +
 
 +
== Features ==
 +
 
 +
{| class="wikitable" style="text-align:left" border="0" cellpadding="5" cellspacing="0"
 +
! Latest Version
 +
| 3.3
 +
|-
 +
! Implementation Language
 +
| Matlab, Java
 +
|-
 +
! Minimum Requirements
 +
| Matlab v7.2, Works on every Matlab compatible OS (see [[System Requirements]])
 +
|-
 +
! Supported data sources*
 +
| Local executable/script, matlab script, dataset (txt file)
 +
|-
 +
! Supported data types
 +
| Supports multi-dimensional inputs and outputs. Outputs can be any combination of real/complex.
 +
|-
 +
! Configuration
 +
| Extensively configurable through one main XML configuration file.
 +
|-
 +
! Flexibility
 +
| Virtually every component of the modeling process can be configured, replaced or extended by a user specific, custom implementation
 +
|-
 +
! Predefined accuracy
 +
| The toolbox will run until the user required accuracy has been reached (on the selected measures), the maximum number of samples has been exceeded or a timeout has occurred
 +
|-
 +
! Model Types*
 +
| Out of the box support for:
 +
    * Polynomial/Rational functions
 +
    * Feedforward Neural Networks
 +
    * Radial Basis Functions
 +
    * RBF Neural Networks
 +
    * Kriging Models (2 implementations with one based on The DACE toolbox)
 +
    * Spline Models
 +
    * Support Vector Machines (SVM)
 +
    ** Least Squares SVM (based on LS-SVMlab)
 +
    ** epsilon-SVM (based on LIBSVM)
 +
    ** nu-SVM (based on LIBSVM)
 +
|-
 +
! Modeling algorithms*
 +
| Pattern Search, Simulated Annealing, Genetic Algorithm, BGFS, ...
 +
|-
 +
! Sampling algorithms*
 +
| Random, error based, density based, gradient based
 +
|-
 +
! Experimental design*
 +
| Latin Hypercube Sampling, random, based on a dataset, full factorial, adaptive (by doing a preliminary 1D screening in each dimension)
 +
|-
 +
! Model selection measures*
 +
| Validation set, cross-validation, comparison on a grid, AIC
 +
|-
 +
! Sample Evaluation*
 +
| Sequentially on the local machine or in parallel on a cluster/grid
 +
|-
 +
! Supported distributed middlewares*
 +
| Sun Grid Engine, LCG Grid middleware (both accessed through a SSH accessible frontnode), A Parameter Sweep Tool (APST) (for on the local LAN)
 +
|-
 +
! Logging
 +
| Extensive (configurable) logging (to file and console) to enable close monitoring of the modeling process
 +
|-
 +
! Profiling*
 +
| Extensive profiling framework for easy gathering (and plotting) of modeling metrics
 +
|-
 +
! Easy tracking of modeling progress
 +
| Automatic storing of best models and their plots. Ability to automatically generate a movie of the sequence of plots.
 +
|-
 +
! Available test problems*
 +
| Out of the box support for various built-in functions (Ackley, Camel Back, Goldstein-Price, ...) and datasets (Abalone, FishLength, ...) from various application domains. Including a number of datasets (and some simulation code) from electronics. In total over 50 examples are available.
 +
|}
 +
 
 +
<nowiki>*</nowiki> Custom implementations can easily be added
 +
 
 +
== Screenshots ==
 +
 
 +
== Documentation ==
 +
 
 +
== Developers ==
 +
 
 +
== References ==

Revision as of 22:50, 10 May 2007

History

In 2004, research within the COMS group was focused on developing efficient, adaptive and accurate algorithms for polynomial and rational modeling of linear time-invariant (LTI) systems. This work resulted in a set of Matlab scripts that were used as a testing ground for new ideas and techniques. Research progressed, and with time these scripts were re-worked and refactored into one coherent Matlab toolbox, tentatively named the Multivariate MetaModeling (M3) Toolbox.


Design Goals

During research into multivariate metamodeling techniques and algorithms it became clear that there was room for an adaptive tool that integrated different surrogate modeling approaches and did not tie the user down to one particular set of problems or techniques. More concretely, we were unable to find evidence of any projects that integrated:

  1. Building standalone global metamodels
  2. Support for different model types, different modeling algorithms, ... (adaptive modeling)
  3. Sequential design
  4. Distributed computing
  5. Usable implementation in software

This gave rise to a number of design goals that served as the guidelines for the design of the M3-toolbox. These goals are:

  1. Development of a fully automated, adaptive metamodel construction algorithm. Given a simulation model, the software should produce a metamodel with as little user interaction as possible ("one button approach").
  2. There is no such thing as a "one-size-fits-all", different problems need to be modeled differently and require different levels of process knowledge. Therefore the software should be modular and extensible but not be too cumbersome to use or configure (sensible defaults).
  3. The toolbox should minimize the required prior knowledge of the system to be modeled.
  4. The algorithm should minimize the number of required samples in order to come to an acceptable metamodel.
  5. The algorithm should terminate only when the predefined accuracy (set by the user) has been reached or the maximum number of iterations has been exceeded.

Features

Latest Version 3.3
Implementation Language Matlab, Java
Minimum Requirements Matlab v7.2, Works on every Matlab compatible OS (see System Requirements)
Supported data sources* Local executable/script, matlab script, dataset (txt file)
Supported data types Supports multi-dimensional inputs and outputs. Outputs can be any combination of real/complex.
Configuration Extensively configurable through one main XML configuration file.
Flexibility Virtually every component of the modeling process can be configured, replaced or extended by a user specific, custom implementation
Predefined accuracy The toolbox will run until the user required accuracy has been reached (on the selected measures), the maximum number of samples has been exceeded or a timeout has occurred
Model Types* Out of the box support for:
   * Polynomial/Rational functions
   * Feedforward Neural Networks
   * Radial Basis Functions
   * RBF Neural Networks
   * Kriging Models (2 implementations with one based on The DACE toolbox)
   * Spline Models
   * Support Vector Machines (SVM)
   ** Least Squares SVM (based on LS-SVMlab)
   ** epsilon-SVM (based on LIBSVM)
   ** nu-SVM (based on LIBSVM)
Modeling algorithms* Pattern Search, Simulated Annealing, Genetic Algorithm, BGFS, ...
Sampling algorithms* Random, error based, density based, gradient based
Experimental design* Latin Hypercube Sampling, random, based on a dataset, full factorial, adaptive (by doing a preliminary 1D screening in each dimension)
Model selection measures* Validation set, cross-validation, comparison on a grid, AIC
Sample Evaluation* Sequentially on the local machine or in parallel on a cluster/grid
Supported distributed middlewares* Sun Grid Engine, LCG Grid middleware (both accessed through a SSH accessible frontnode), A Parameter Sweep Tool (APST) (for on the local LAN)
Logging Extensive (configurable) logging (to file and console) to enable close monitoring of the modeling process
Profiling* Extensive profiling framework for easy gathering (and plotting) of modeling metrics
Easy tracking of modeling progress Automatic storing of best models and their plots. Ability to automatically generate a movie of the sequence of plots.
Available test problems* Out of the box support for various built-in functions (Ackley, Camel Back, Goldstein-Price, ...) and datasets (Abalone, FishLength, ...) from various application domains. Including a number of datasets (and some simulation code) from electronics. In total over 50 examples are available.

* Custom implementations can easily be added

Screenshots

Documentation

Developers

References