Multiple criteria decision analysis (MCA) has become an indispensable tool for dealing with complex and unstructured decision problems in environmental and natural resource management which involve a number of conflicting objectives and a variety of stakeholders. Despite their popularity, choosing which of the many multicriteria methods to use is tricky. Different methods may yield different results and therefore the decision may depend on the method selected. In this paper we review a number of experiments conducted to compare the results of different MCA methods when applied to the same decision problem. I compare and critically examine the hypotheses postulated and the results obtained from these experiments. Despite the equivocal results yielded in some experiments, many authors recommend applying two or more MCA methods, especially in the case of unstructured decision problems. This multimethod MCA application may be regarded as a type of validation which is more extensive than standard sensitivity analysis and which enables the decisionmaker to review the preferences and judgments previously elicited by a single method.
Get full access to this article
View all access options for this article.
References
1.
BellM LHobbsB FElliottE MEllisH.RobinsonZ., 2000, “An evaluation of multicriteria decision-making methods in integrated assessment of climate policy”Lecture Notes in Economics and Mathematical Systems 487: Research and Practice in Multiple Criteria Decision Making Eds HaimesY YSteuerR E, (Springer, Berlin) 228–237.
2.
BellaA.DucksteinL.SzidarovszkyF., 1996, “A multicriterion analysis of the water allocation conflict in the upper Rio Grande basin”Applied Mathematics and Computation77245–265.
3.
BeltonV., 1986, “A comparison of the analytic hierarchy process and a simple multi-attribute value function”European Journal of Operational Research267–21.
4.
BuchananJ T, 1994, “An experimental evaluation of interactive MCDM methods and the decision making process”Journal of Operation Research Society451050–1059.
5.
CornerJ LBuchananJ T, 1997, “Capturing decision maker preference: Experimental comparison of decision analysis and MCDM techniques”European Journal of Operational Research9885–97.
6.
CornerJ LCornerP D, 1995, “Characteristics of decisions in decision analysis practice”Journal of the Operational Research Society46304–314.
7.
EdwardsW., 1977, “How to use multiattribute utility measurement for social decision-making”IEEE Transactions on Systems, Man and Cybernetics1326–340.
8.
FantinoE., 1998, “Behaviour analysis and decision making”Journal of the Experimental Analysis of Behaviour6935–364.
9.
ForgionneG A, 2000, “Decision making support system effectiveness: The process to outcome link”Information, Knowledge, System Management2169–188.
10.
GoicoecheaA.StakhivE.LiF., 1992, “Experimental evaluation of multiple criteria decision models for application to water resource”Water Resource Bulletin2889–102.
11.
HarteJ MKoeleP., 1995, “A comparison of different methods for the elicitation of attribute weights: Structural modelling, process tracing, and self-reports”Organisational Behavior and Human Decision Processes6449–64.
12.
HobbsB F, 1986, “What can we learn from experiments in multiobjective decision analysis?”Transactions on Systems, Man and Cybernetics16384–396.
13.
HobbsB FHornG T F, 1997, “Building public confidence in energy planning: A multimethod MCDM approach to demand-side planning at BC Gas”Energy Policy25357–375.
14.
HobbsB FMeierP M, 1994, “Multicriteria methods for resource planning: An experimental comparison”IEEE Transactions on Power Systems91811–1817.
15.
HobbsB FChankongV.HamadehW.StakhivE Z, 1992, “Does choice of multicriteria method matter? An experiment in water resource planning”Water Resources Research2381767–1779.
16.
HokkanenJ.SalminenP., 1997, “ELECTRE III and IV decision aids in an environmental problem”Journal of Multi-Criteria Decision Analysis6215–226.
17.
JiaJ.FischerG W, 1993, “Evaluating multiattribute decision quality: A simulation study”, paper presented at the ORSA/TIMS Joint National Meeting Chicago, IL.
18.
JiaJ.FisherG WDyerJ S, 1997, “Attribute weighting methods and decision quality in the presence of response error: A simulation study”Journal of Behavioral Decision Making1185–106.
19.
JiangH.EastmanJ R, 2000, “Application of fuzzy measures in multi-criteria evaluation in GIS”International Journal of Geographic Information Science14173–184.
20.
JoubertA RLeimanA.DeklerkH MKatuaS.AggenbachL C, 1997, “Fynbos (fine bush) vegetation and the supply of water: A comparison of multi-criteria decision analysis and cost–benefit analysis”Ecological Economics22123–140.
21.
LaiS KHopkinsL D, 1988, “Experimental assessment of multiattribute evaluation technique”, paper presented at ORSA/TIMS Joint National Meeting, Denver, CO; copy available from Professor Hopkins, Department of Urban and Regional Planning, University of Illinois, Urbana-Champaign, IL.
22.
LarichevO I, 1992, “Cognitive validity in design of decision-aiding techniques”Journal of Multicriteria Decision Analysis1127–138.
MahmoudM RGarciaL A, 2000, “Comparison of different multicriteria evaluation methods for the Red Bluff Diversion Dam”Environmental Modelling and Software15471–478.
25.
OlsonD LMoshkovichH MSchellenbergerR.MechitovA I, 1995, “Consistency and accuracy in decision aid: Experiments with four multiattribute systems”Decision Science26723–748.
26.
OzelkanE CDucksteinL., 1996, “Analysing water resources alternatives and handling criteria by multi criterion decision techniques”Journal of Environmental Management4869–96.
27.
PöyhönenM.HämäläinenR P, 2001, “On the convergence of multiattribute weighting methods”European Journal of Operational Research129569–585.
28.
QureshiM EHarrisonS R, 2001, “A decision support process to compare riparian revegetation options in Scheu Creek catchment in North Queensland”Journal of Environmental Management62101–112.
RajuK SDucksteinL.ArondelC., 2000, “Multicriterion analysis for sustainable water resource planning: A case study in Spain”Water Resource Management14435–456.
31.
SalminenP.HokkanenJ.LahdelmaR., 1998, “Comparing multicriteria methods in the context of environmental problems”European Journal of Operational Research104485–496.
32.
SchneeweißC., 1991Planung 1, Systemanalytische und entscheidungstheoretische Grundlagen [Planning 1, system-analytical and decision-theoretical foundation] (Springer, Berlin).
33.
TecleA., 1992, “Selecting a multicriterion decision technique for watershed resources management”Water Resources Bulletin28129–140.
34.
TimmermansD.VlekC.HandrickxL., 1989, “An experimental study of the effectiveness of multidimensional decision making methods”, in Improving Decision Making in Organisations Eds LocettA GIsletG. (Springer, Heidelberg) pp12–23.
35.
VanderpootenD., 1992, “Three basic conceptions underlying multiple criteria interactive procedures”, in Multiple Criteria Decision Making Eds GoicoecheaA.DucksteinL.ZiontsS. (Springer, Berlin) pp441–448.
36.
Van der VeerenR J H MLorenzC M, 2002, “Integrated economic–ecological analysis and evaluation of management strategies on nutrient abatement in the Rhine basin”Journal of Environmental Management66361–376.
37.
Von WinterfeldtD.EdwardsW., 1986Decision Analysis and Behavioral Research (Cambridge University Press, Cambridge).
38.
ZanakisS HSolomonA.WishartN.DublishS., 1998, “Multi-attribute decision making: A simulation comparison of selected methods”European Journal of Operational Research107507–529.
