Model Visualization GUI

The Model Visualization GUI included in the SUMO Toolbox has several possibilities for exploring models with any number of inputs and outputs. If you want to learn more about a model, find areas of interest in a model's domain, create customized screenshots and movies or view information about a model, this GUI will help you reach your goal.

You can browse a model's dimensions interactively using different visualization techniques and customize the appearance of the plot. Those customizations can be retained for generating a movie over one of the input dimensions not shown in the plot window.

The Model Information window lists useful information about a model like its description, validation scores and expression. It plots error statistics against its training samples and any validation dataset or function you can provide.

Visualizing a Model

The main component of the Model Visualization GUI is the Visualization Console, a console for interactively browsing a model's input and output domain. You can search and study areas of interest, find interesting relations between input parameters and their effect on the model's outputs. The GUI allows you to create fully customized screenshots for using at a presentation, on your website or elsewhere.

Examples

Let us first take a look at some screenshots of plots created using the Model Visualization GUI.

Getting Started

For working with a model saved on disk, you can start the visualization GUI by entering the command guiPlotModel at the Matlab command line. If the command is not found, verify that you have added the toolbox to your path with its startup command. The load from file dialog will appear. After selecting a *.mat file, the model it contains will be loaded and the visualization controls present themselves. Take a look at the command line options for other ways to start the GUI, or execute help guiPlotModel to get an explanation of the options at command line.

The screenshot below shows a console on a Windows platform.

Every edit box can be reset to its default value by leaving it empty instead of entering a new value. You can enter any valid Matlab expression in the value fields of the input and output controls, like pi, 5/12 or even 2^(pi*2-4/3).

Model Input Controls

Each of the model's inputs can be controlled with one line of the input controls, as shown in the screenshot below.

1. Label: the name of the input parameter, shown as a label on the axis representing it in the plot window;
2. Current value: slice value used when the input is not plotted;
3. Minimum value: lower bound for the axis representing the input;
4. Slider: can be used to change the current value when the input is not plotted;
5. Maximum value: upper bound for the axis representing the input;
6. X radio button: select this radio button to plot the input on the X-axis of the plot;
7. Y radio button: select this radio button to plot the input on the Y-axis of the plot.

One input can only be shown on one axis. This constraint is enforced by the GUI. E.g. when you associate to the X-axis the input that is currently shown on the Y-axis, the new Y-axis will become the topmost free input. When using the edit fields, keep in mind that the lower bound (minimum) of an input must always be smaller than the upper bound (maximum). The GUI enforces this by adjusting the other boundary when you enter a value that breaks the said constraint. Likewise, the current value always lies between the boundaries and is adjusted automatically. When an input's upper bound equals its lower bound, you won't be able to change the value and the slider will disappear to indicate this.

Model Output Controls

This subsection explains how you can control the model's outputs. The screenshot below shows the part of the console that allows you to do this.

1. Select the output to plot and edit the label of the current output.
2. Select how to handle complex outputs.
3. Clip the output axis.
4. Checkbox for plotting all outputs: when you check this box, all the model's outputs will be shown at the same time, in separate plot windows. All changes made in the console will be applied simultaneously to all plots. You can still switch between outputs, but only to change their labels.

Plot Customization Controls

These options provide a way to change the visual appearance of the plot.

• Different plot types are available: choose between plotting 1 input, 2 inputs with a 3D surface or 2 inputs with a contour plot.
• You can set the size of the evaluation grid used for plotting the model in the Mesh size field. If plotting is slow, you can try lowering the mesh size to get a less accurate but faster visualization.
• A model is built using training samples. These points are shown on the plot when the Plot points checkbox is enabled. However, when not all input dimensions are shown in the plot window, not all samples are plotted. Because most of the samples don't have the same fixed slice values as specified with the input controls, their output value will most likely be completely different from the output value of the model, and showing this wide range of output values will distort the plot if you're not using output axis clipping. Instead, the training samples are filtered according to their distance from the current slice value of the input dimensions not shown in the plot. You can set the maximum distance of points to be plotted, by entering a maximum deviation percentage in the popup/edit box, or selecting a predefined percentage from the list. The deviation percentage is relative to the size of the interval between the minimum and maximum in each dimension not shown in the plot window. E.g. if you select 10 % while the single dimension not shown on the plot, has an interval size 2 and value val, all training samples with a value in that dimension between [val-0.2,val+0.2] will be plotted. If all dimensions are visible, the popup/edit box is disabled and all points are plotted. When not all dimensions are shown in the plot window, the size of the plotted sample points can be set to reflect their distance from the current slice with the Scale points checkbox. The closer the point is to the slice values of the dimensions not shown in the plot window, the larger it will be shown. Scaling is done linearly between the exact slice value and the maximum deviation specified. If you disable scaling, all points will be plotted with the same size.
• In 2D plot mode, you can show the edges of a contour plot below the surface plot by checking the Countour lines checkbox.
• The Plot model checkbox can be disabled if you only want to plot the samples and hide the model.
• A Fix camera checkbox is provided to keep the camera settings in consecutive plots. When you enable this option, you override Matlab's default behaviour of always returning the camera to its default position when a figure is plotted. This is a very important feature if you're exploring a specific part of the plotted figure, or if you simply want to look at the model from a custom position. The camera will always return to its default position when you change the plot type, regardless of the Fix camera setting.
• You can add lighting to the scene in 2D plot mode. Checking the Lighting option adds a light at a default position in the scene. While the option is enabled, you can add, delete or modify lights in the plot window using the standard Matlab plot edit tools. The modifications will be retained in successive plots.
• By default, all plots use the jet colormap, which varies from dark blue to dark red. You can also use a grayscale colormap, e.g. if you want to print a screenshot with a black and white printer.
• Finally, a checkbox is provided to set a logarithmic scale for the output axis.

Actions

All actions can be accessed from the menu bar, in either the File menu or the Actions menu. Four of those actions are also available via buttons at the bottom of the console.

Through the file menu you can load a model from file by choosing Open. The dialog will open in the directory containing the current model, if it exists. The Reload item allows you to reload the current model from disk. Reloading is especially useful when you are viewing the current best model during a toolbox simulator run. Finally, you can also close the console through this menu.

All other menu items can be found in the Actions menu. A listing of the available actions:

• Open the Create Movie and Model Info GUI's for the current model.
• Quickly start an optimization of the current model in Matlab's optimization toolbox.
• Take a look at the sample distribution plot, a scatter plot of the training samples across all pairs of input dimensions and box plots for every input dimension separately.
• View a Slice Plot of the current model, retaining all compatible customizations. Most importantly, bounds, labels and lighting options are taken from the console. This Slice Plot can be useful for getting an overall idea of what the model looks like. The number of slices shown can be changed through the Command Line Options.
• Reset all controls to their default values.

The menu items have Alt+<Letter> keyboard shortcuts that can be used while navigating through the menus. Press the Alt key to underline the different letters that can be used as shortcuts. Some actions also have Ctrl+<Letter> keyboard shortcuts, also shown in the menus. The Ctrl shortcuts can be accessed without having to open the menu all the time.

Title Bar

The title bar shows the handles of figures associated with the console and the model type. If the model was loaded from disk the file name will also be visible in the title bar.

Command Line Options

Instead of relying on the automatically generated defaults, you can use parameters to specify your own defaults. guiPlotModel has the following (optional) parameters:

1. The first parameter is the model to be plotted in the figure window. If you already loaded the model in memory, or simply don't have it saved on disk, you can pass it as a parameter. This disables the reload function, unless you also specify a modelFilename field in the options parameter, containing the path to the model on disk.
2. The second parameter represents the index of the output to plot.
3. The third parameter is a structure containing input settings: bounds and values for each input and indices of the inputs to plot. The defaults can be retrieved using settings = defaultInputSettings(model).
4. A structure containing the plot customization options can be passed as the fourth parameter. Retrieve the defaults using [dummy options] = guiPlotModel. You don't have to specify all fields when using this parameter. Fields that you don't provide will be set to their default value. With these options you can specify some additional settings that can't be changed through the GUI:
   • The slices field specifies the number of slices shown in a Slice Plot.
   • The postFunction field can hold a function handle that must be called after each plot. The function should have 1 parameter: the handle to the plot figure. This enables you to easily add an overlay to the plot.
   • You can turn on elevation labels on contour plots by settings elevationLabels to true.

Automatically Reusing an Existing Console

A model is equal to another model if it has the same number of inputs and outputs, and all input and output labels are the same. When you try to open a console for a model that is equal to a model already shown in a console, no new console shall be opened. Instead, the new model will be inserted in the existing console, while the current settings are retained. This is quite useful if you want to look at different instantiations of the same model without having to redo the plot customizations.

Creating a Movie

The Create Movie GUI allows you to bring your model to life by varying over (a part of) the domain of a single input parameter. That way, you get an overview of its influence on the model's output. You can create a fluid movie by generating many slices and setting a high frame rate. With a lower frame rate you can simply watch an arbitrary number of slices in succession.

Examples

Here you can take a look at some movies created with the Create Movie GUI.

Getting Started

You can access the Create Movie GUI from the Visualization Console, using the Create movie button at the bottom or in the Actions menu. All customizations applied in the console will be used for the movie. The Create Movie GUI can also be accessed from command line. You won't be able to use any other Matlab GUI component while using the Create Movie GUI. E.g. if you want to further customize the plot in the Visualization Console, press cancel in the Create Movie GUI to return to it.

The screenshot of the Create Movie GUI shown below, gives an overview of the available options.

Movie Controls

• You can select which input to vary over time with the topmost popup box. All inputs that are not set to be plotted are available. The slices (frames) will be generated over the entire range of the selected input, as specified in the Visualization Console, command line input settings or model defaults. The first slice will have the value of the lower bound, the last one will be equal to the upper bound.
• On the second line you can see two checkboxes. The first one specifies if the output axis should be clipped to the maximum and minimum output values found in all slices. Keep this checkbox selected if you want to prevent that the axis scale changes during the movie, which causes irregular jumps in the consecutive plots. However, enabling this option also means creating the movie will take longer, because the correct bounds can only be known after all slices have been generated. The slices will be reloaded from disk and modified to the custom output bounds. To avoid this second pass and prevent the output axis scale from changing during the movie, you can enable output clipping in advance. Using the Visualization Console, you can explore the model to get an idea of a good set of bounds to use. Enable the 'Clip to checkbox found at the bottom left of the Visualization Console to clip the output.
• By default, the title of each slice plot in the movie holds the current slice value of the input varying over time. This option can be controlled with the Show slice in title checkbox.
• Specify how many slices should be generated in the Slice count field.
• To the right of the slices field you can enter the number of frames per second to be shown in the movie. Fractions can be used, e.g. 0.2 fps means each slice will be shown for five seconds. You can enter a number yourself or select one from the popup box.
• Movies are created using the default Matlab codec by default. On Linux platforms, this is an uncompressed AVI format. On other platforms, the default Matlab codec does support compression. The compression quality can be set using the popup/edit box to the right of the compression popup box. If you have the Java Media Framework installed and in your Matlab class path, you can also create QuickTime movies, but that codec does not allow you to specify a compression quality.
• The Output file field shows the path to the file where the movie shall be created. You can enter a file path manually or use the browse button. The correct file extension for the selected codec is automatically appended to the file path.
• At the bottom of the window 3 buttons are located. The first one resets all fields in the window to their default value. The middle button starts the movie generation process. The rightmost button closes the window. Upon pressing the Create movie button, the current settings (except for the selected input dimension) are stored in memory, for the duration of the current Matlab session. The next time you open the Create Movie GUI, the saved settings will be restored. This is a handy feature if you want to create several movies using the same settings.

Command Line Options

Instead of starting the Create Movie GUI through the Visualization Console, you can also start it directly from command line using the Matlab command guiCreateMovie. Without parameters, a load from file dialog will appear so you can select a model stored on disk. Below are the optional parameters to the command, in order of appearance:

1. The first parameter is the model to be plotted in the figure window. If you already loaded the model in memory, or simply don't have it saved on disk, you can pass it as a parameter.
2. The second parameter represents the index of the output to plot.
3. The third parameter is a structure containing input settings: bounds and values for each input and indices of the inputs to plot. The defaults can be retrieved using settings = defaultInputSettings(model).
4. A structure containing the plot customization options can be passed as the fourth parameter. Retrieve the defaults using [dummy1 dummy2 options] = quickPlotModel. You don't have to specify all fields when using this parameter. Fields that you don't provide will be set to their default value.
5. If you want to plot the slices of the movie in a specific figure handle, pass the handle as the fifth parameter. Otherwise the current figure will be used.

The first four parameters are in the same order as used to start the Visualization Console from command line. As with the Visualization Console, the command help guiCreateMovie will provide a brief description of the function and its parameters, in the correct order.

Reusing the Generated Slices

Next to the actual movie file, a subdirectory slices shall be created containing all slices used in the movie, in png format. If you enabled the Fix output axis option, every png file will also have a corresponding Matlab figure. The slices subdirectory will be erased when starting a new movie generation process in the same directory.

You might want to create several movies with different output formats or frame rates. This process can be speeded up by using the Matlab command images2movie. It allows you to create a movie from an existing directory of images. These are the parameters to the function:

1. path to the directory containing the images;
2. file path to the location where the movie should be saved (file extension defines the video codec);
3. file extension of the images, in this case png;
4. number of frames per second;
5. compression quality percentage.

Creating a Movie from Command Line

Instead of creating a movie using the GUI, you can use the genMovie command to create one from command line. The parameters to the command are the same as the parameters to the Create Movie GUI with the addition of a sixth optional parameter containing the movie options. The default movie options can be retrieved with the command options = genMovie. Like elsewhere, the options structures don't have to be complete. Fields that you don't provide will be set to their default value. You can run help genMovie for more information at command line.

Since the command doesn't use a GUI, the first parameter is no longer optional if you want to create a movie. You have to pass a model already present in the Matlab workspace as the first parameter. See #Loading a Model from Disk for an easy way to load a model into the Matlab workspace.

Viewing Model Information

The Model Information GUI gives you a quick overview of a model. It shows error statistics, scores, description, dimensions and more. Some information fields are specific to the selected output, others are specific to the model only. You can select which output's information you want to see. Additionally, you can specify a custom validation set or function to compute error statistics. You can compare your custom statistics to the training error statistics.

All information fields contain selectable text, so you can easily copy the information you want to the clipboard.

Examples

Here are some screenshots that should give you an idea what you can do with the Model Information GUI tool.

Getting Started

The Model Information GUI can quickly be started from the Visualization Console, by clicking the button at the bottom of the console or through the Actions menu. It can also be accessed from the command line.

Controls

The controls, visible in the screenshot below, are positioned in the top right corner of the Model Information GUI.

• You can select which output's error statistics, error plots and expression must be shown using a popup box.
• External validation data can be selected from another popup box. All matrices of doubles and function handles in the workspace are listed. You must select a valid matrix or function handle. If the variable selected is invalid, an error message will be shown in a dialog box. Select the empty matrix, [], to disable external validation. Validation data provided at command line is listed as cmd line. See #Using External Validation Data for more information.
• When using a function handle for external validation, an evaluation grid will be constructed for you. The maximum number of points to use in the evaluation grid can be specified. The points will be linearly spaced along the model's input dimension ranges.
• Finally, a checkbox allows you to use logarithmic scaling for the Y axis in the error plots.

Using External Validation Data

External validation can be achieved in two ways. You provide a complete dataset of expected values or a handle of a function that computes these expected values. The dataset must be a matrix with a column for every input and output, in the correct order. The first columns must represent the input values, the last columns must represent the corresponding output values. The function handle must take an evaluation grid as its single input parameter and return a matrix with a column for each of the model's outputs. The return matrix must specify the expected output values corresponding to the input values specified in the input matrix.

The validation matrix or function should be present in the Matlab workspace, ideally before starting the Model Information GUI. If you want to use validation settings added to the workspace after you started the GUI, you first have to reset the GUI using Ctrl+R or by clicking Reset in the Actions menu.

Actions

In addition to the controls available in the top right corner of the GUI, some actions are available through the menu. The File menu enables you to open another model in the current Model Information GUI, reload the current model and close the GUI window. Through the actions menu, you can reset the GUI controls and fields to their default values. It also allows you to export the model to a Matlab M-file for using it outside the SUMO Toolbox.

Like in the Visualization Console, the menu items have Alt+<Letter> keyboard shortcuts that can be used while navigating through the menus. Press the Alt key to underline the different letters that can be used as shortcuts. Some actions also have Ctrl+<Letter> keyboard shortcuts, also shown in the menus. The Ctrl shortcuts can be accessed without having to open the menu all the time.

Error plots

A screenshot showing error plots for both training samples and using a custom validation matrix.

On the left side of the Model Information GUI, plots representing the error statistics for the currently selected output are presented. The top four plots represent the error in the training samples. If you have specified valid validation data, another four plots will be shown, representing the error using the validation data.

Both absolute and relative errors are shown. The plots on the left show the absolute error, the plots on the right show the relative error. Two plot types are used for showing the errors: a plot showing the highest validation error against the number of points that remain below this error and a histogram.

Information Fields

All information fields are presented on the right hand side of the GUI, below the controls. Here you can look at a screenshot of those fields alone.

• At the top left of the screenshot you can see a field specifying the number of inputs and outputs the model has. Next to it stands the number of free parameters used for constructing the model.

• A general model description is presented in the Description field.

• The middle fields contain error statistics for the current output. At the left the statistics measured against the training data are shown. The right field shows statistics for the custom validation set or function you can specify. If no custom validation data is set, this field will be empty.

• The field at the bottom left contains the list of scores used while generating the model, for every output. Next to it you can see the expression used inside the model for calculating the value of the current output. This expression can become quite large for difficult models, so you will have to use the scroll bar attached to the field for seeing it completely.

Command Line Options

Instead of starting the Model Information GUI through the Visualization Console, you can also start it directly from command line using the Matlab command guiModelInfo. Without parameters, a load from file dialog will appear so you can select a model stored on disk. Below are the optional parameters to the command, in order of appearance:

1. The model whose information you want to view. If you already loaded the model in memory, or simply don't have it saved on disk, you can pass it as a parameter.
2. The second parameter is the index of the output of which output specific information must be shown.
3. You can pass validation data directly as the third command line parameter. You can always return to the validation data specified at command line by selecting cmd line in the validation popup box. See #Using External Validation Data for more information.
4. Specify the path to the file containing the model to be able to reload it, even when you passed it as a command line parameter. Leave this parameter empty if you don't have a file containing the model on disk.
5. The last parameter is a small options structure. The defaults can be retrieved using [dummy options] = guiPlotModelErrors. You don't have to specify all fields when using this parameter. Fields that you don't provide will be set to their default value.

The first two parameters are in the same order as used to start the Visualization Console from command line. As with the Visualization Console, the command help guiModelInfo will provide a brief description of the function and its parameters, in the correct order.

Loading a Model from Disk

The GUI comes with an easy to use command for loading a model from disk into the Matlab workspace: guiLoadModel. It provides the load from file dialogs for the three GUI components explained above, but you can also use it separately. You can specify a starting path for the dialog as an optional parameter. The function returns the loaded model as the first return parameter, and the name of the file containing the model as the second return parameter. Run help guiLoadModel to get a short description of the function and its parameters at command line.