While Sensitivity Analysis examines how the ranking of alternatives responds to changes in input weights, Uncertainty Analysis focuses on the overall uncertainty in the inputs, including the Analysis Matrix itself. This tool is particularly useful for users to explore different scenarios among the alternatives, without assuming that the criteria values for each option are fixed, as is the case in Sensitivity Analysis. This procedure can be applied both to the single ranking of a user (single weights vector) and to include all the different opinions (multiple weight vectors) derived from a participatory decision-making process.
The steps to perform Uncertainty Analysis in mDSS are as follows:
- Create and download the simulation file (.zip Batch File), which contains the list of perturbed Analysis Matrices and other necessary files.
- Upload the generated .zip file back into mDSS to use the built-in Batch Run procedure and to download a new .zip file containing the Batch outcome.
- Analyze the outcome using the Group Decision section of mDSS or the CART algorithm (see below the dedicated paragraph).
Notably, the third step is not mandatory. The user can simply download the Batch outcome File — which also includes a .csv file — and conduct the analysis independently using the generated data and their preferred statistical methods.
Create the Batch File
To create the Batch File, after obtaining the final rankings from the SAW, OWA, or WOWA methods, the user should navigate to the “mDSS add-ons” section in the Main Menu and select the Batch File creation option that fits their needs. For example, a user who wants to create a Batch File for the OWA or WOWA methods working on an MCA project can simply click on the menu option “OWA/WOWA Method – create batch files for MCA.”
Notably, to generate the Batch File, mDSS requires uploading certain files that can be saved during different phases of the mDSS session. Users should ensure they have previously downloaded these files to enable a smooth batch file creation process. The list of these files includes:
- The Analysis Matrix (AM)
- The .wgt file(s) for the weights (SAW and OWA/WOWA, depending on the selected batch option)
- The Value Functions implemented to generate the Evaluation Matrix (EM)
After clicking on the desired Batch option in the “mDSS add-ons” menu, mDSS will prompt the user to upload the Analysis Matrix (AM). Then, a window similar to the one displayed in Figure 48 will appear.
The table in the upper left represents the Analysis Matrix (AM) that the user uploaded in the previous step. To the right of the AM is the Uncertainty Matrix (UM). This matrix is central to the batch creation process, as it guides mDSS in determining the range of relative variation to apply to the criteria values during the batch process. Specifically, mDSS generates a set number of perturbed Analysis Matrices based on the information provided in the Uncertainty Matrix (UM).
To create these new matrices, mDSS randomly selects a value for each criterion and option within a range defined by the relative variation specified in the Uncertainty Matrix, using a uniform distribution. For example, in the default case shown in Figure 48, where each cell of the UM is set to 10, mDSS generates a list of AMs where each cell is defined by a random number within the criterion’s range of variation (+/- 10%). This variation can be easily adjusted by clicking on the pencil icon under the “Edit” column.
Additionally, users can upload or download previously generated Uncertainty Matrices using the “Load UM” and “Save UM” buttons. The number of perturbed Analysis Matrices, i.e., the number of scenarios to be generated, is determined by the user in the cells below the tables, next to the label “Number of Analysis Matrices in the batch file.”
In the last part of the window the user must upload the following:
- The .wgt vector(s) of the SAW weights
- The .wgt vector(s) of the OWA/WOWA weights (if required by the procedure)
- The value functions file (s)
Notably, the user can choose to upload only one file for each of the three columns, corresponding to performing the UA on the outcome of a single individual, or upload multiple files for each column (e.g., multiple SAW weights, multiple OWA weights, multiple value function files). This is useful for incorporating the different opinions of a participatory process into the Uncertainty Analysis.
Once all the files are uploaded, the user simply needs to click on “Save in batch file” to download the batch .zip file. This file contains the perturbed AMs and should be uploaded again into mDSS in the next phase of the UA process.
Run the Batch File
After creating the batch file in the first step, the user must select “mDSS add-ons” again and click on “Batch Run.” Then, by clicking on “Load input batch file” in the opened window, the user should upload the batch file generated and automatically downloaded in the previous step. A window similar to the one reported in Figure 49 will appear.
Notably, the user should first select which type of batch file she has uploaded (SAW, OWA, or WOWA). The user can also choose to include the SAW/OWA vector and the AM values in the final .zip outcome file. While this step does not influence the outcome, it simply adds additional files to the generated .zip file.
After a quick check by the software, if everything is correct, the “Start” button at the bottom of the window will appear. By clicking on it, the .zip file containing the UA outcomes will be downloaded. The file includes:
- A list of .opt files representing the different outcomes across the various scenarios. Notably, this type of file can be read by mDSS in the “Group Decision” section.
- A .csv and a .xlsx file containing a detailed description of the outcomes across the scenarios, including information on the values of the weights, criteria, and other relevant data.
These files — particularly the .csv and .xlsx — can be used by the user to perform statistical and mathematical analyses independently on other platforms. If the user would like general guidance on how to approach this, the next paragraph will present two possible methods to follow.
Analyze the Batch Outcome
In this paragraph, we propose two methods for analyzing the results obtained from the Batch Run. The first is the simplest and roughest approach, based on the “Group Decision” section of mDSS. The second involves applying the CART algorithm — a powerful machine learning tool — to the obtained results through the implementation of an .R script.
“Group Decision” Analysis
The idea behind this method is to simply upload the .xlsx file (or the .opt files) obtained from the .zip outcome into the “Group Decision” section of mDSS and apply the group decision methods currently implemented in the software. Each ranking obtained from a different criteria-weight scenario is treated as the “vote” of an individual, and these preferences are aggregated using the available methods (Condorcet, Borda, and Extended Borda).
This method is not a statistical analysis; however, despite its relatively lower technical complexity, it represents a simple and effective option for extracting useful insights from the mDSS batch output.
CART Algorithm
The CART (Classification and Regression Trees) algorithm is a machine learning method used to build decision trees for both classification (categorical outcome) and regression (continuous outcome) tasks. CART works by recursively splitting the dataset into smaller subsets based on input feature values. At each split, it selects the feature and threshold that best separate the data according to a certain criterion (such as Gini impurity for classification or mean squared error for regression). This process continues until the data in each subset is as “pure” as possible, meaning most or all instances in a subset belong to the same class or have similar values. For further information, please refer to the Methods Handbook.
It is easy to apply this machine learning method to a .csv Batch Run file using the following R script. To do so, simply download the provided R code, update the file path at the beginning of the script to point to your .csv file downloaded from mDSS, and then execute the script.